TMMi-P_Syll2020 TMMi - Test Maturity Model Integration Professional Questions and Answers

In the Test Design and Execution process area of TMMi Level 2, there are four specific goals. The relevant goals for this question are SG 2 (Perform Test Implementation) and SG 3 (Perform Test Execution):

SG 2 Perform Test Implementationincludes practices such as developing and prioritising test procedures and creating specific test data. Therefore, the specific practice that aligns with this goal is"Create specific test data" (III).

SG 3 Perform Test Executioninvolves executing the tests and logging the outcomes. One of the specific practices here is"Write test log" (II), which documents the execution data in a structured manner.

Thus, the correct pair is SG 2 withIII (Create specific test data)and SG 3 withII (Write test log)​​.

The generic practice"Establish a defined process"addresses the need to tailor a managed process from the organization’s set of standard processes. This practice is part ofGeneric Goal 3 (GG 3), which aims to ensure that processes are standardized and defined across the organization. By tailoring processes based on established guidelines, organizations can create specific implementations that fit their projects while maintaining consistency and control across the organization.

TMMi References:

In TMMi, establishing a defined process involves tailoring standard processes to suit specific project needs, ensuring the institutionalization of best practices across projects​​.

Formal assessments, in contrast to informal ones, are conducted with more rigor, and the results can lead to an official certification of maturity level. A formal assessment report includes a validated maturity level rating, making it distinct from informal assessments, which are used only for indicative purposes and do not result in an official rating. Thus, only formal assessments provide an assessment report with a valid maturity level rating​​.

With a 40% achievement in the Test Planning process area, the rating would be "Partly achieved." According to TMMi, a process area is rated "Partly achieved" when the percentage of achievement is between 15% and 50%. This indicates that some practices have been implemented, but significant improvement is still required to meet the full requirements of the process area.

The specific practice"Define product risk categories and parameters"is part of theTest Policy and Strategyprocess area in TMMi. This practice involves identifying and categorizing the product risks to guide testing efforts and establish a clear understanding of potential product issues. Product risk categories help in determining the focus areas for testing, ensuring that testing is prioritized based on business and technical risks.

TMMi References:

The practice of defining product risk categories is directly linked to theTest Policy and Strategyprocess area at TMMiLevel 2, where risk-based testing approaches are established​​.

Required componentsdescribe what an organizationmustcomply with to satisfy a process area in the TMMi model. These components includespecific goalsandgeneric goals, and they form the basis for determining whether a process area has been successfully implemented during assessments.Expected componentsprovide guidance on how to achieve the required goals, but they are not mandatory.

TMMi References:

TMMi definesrequired componentsas those that organizations must achieve to satisfy a process area​​.

As part of theProduct Quality Evaluationprocess area (4.2), the specific goal"Project Goals for Product Quality and their Priorities are Established"includes the practice ofidentifying product quality needs. This involves understanding and defining what quality means for the product, determining quality characteristics (such as reliability, usability, etc.), and establishing priorities for these characteristics based on project goals.

Other practices such asestablishing test process measuresandmeasuring product quality quantitativelycome later in the lifecycle or are part of other specific goals.

TMMi References:

InProduct Quality EvaluationatTMMi Level 4, identifying product quality needs is critical for setting up measurable and prioritized goals for product quality​​.

The statement thatSpecific Goal 1of theTest Measurement (4.1)process area covers the communication of test measurement results after analysis isFALSE. Specific Goal 1 focuses on aligningtest measurement and analysis activities, which involves setting measurement objectives, specifying the measures to be taken, and establishing procedures for data collection, storage, and analysis. However, the communication of results comes underSpecific Goal 2, which deals with the analysis and reporting of test measurement results.

TMMi References:

Test MeasurementatTMMi Level 4includes defining measures, data collection, and storage as part of aligning test measurement and analysis activities, while communicating results is part of a different goal​​.

In an Agile context, theSCRUM teammodel is a typical format for a test organisation. Testing is integrated into the cross-functional SCRUM team, with testers working alongside developers and product owners throughout the sprint. This ensures that testing is a continuous activity, rather than a separate phase. The test organisation in Agile environments focuses on collaboration and team-based activities, rather than a separate or independent test group​​.

The absence of amaster test plan spanning multiple test levelspoints to a weakness in theTest Lifecycle and Integrationprocess area, which is aTMMi Level 3process area. This process area focuses on ensuring that testing is integrated throughout the development lifecycle and includes the development of comprehensive test plans (including master test plans) that cover multiple test levels and align with the overall testing strategy.

TMMi References:

TheTest Lifecycle and Integrationprocess area atTMMi Level 3addresses the need for comprehensive and integrated test planning across all test levels​​.

"Provide Necessary Test Training" is part of theTest Training Programprocess area, not theTest Organizationprocess area. Test Organization focuses on establishing a structured group responsible for testing, defining roles, career paths, and implementing process improvements. Therefore, providing test training is not a specific goal of Test Organization but rather falls under the Test Training Program process area.

TheAdvanced Reviewsprocess area, which is part ofTMMi Level 4, is concerned with measuring product quality early in the lifecycle. It aims to enhance thetest strategyandtest approachby aligningstatic testing(reviews and inspections) withdynamic testing(execution of test cases). The results and data from static testing are used to optimize the dynamic testing approach, making the overall testing process more efficient and effective.

TMMi References:

Advanced Reviewsat TMMi Level 4 focuses on integrating static and dynamic testing approaches to optimize test strategy and product quality evaluation​​.

This statement is not true regarding a staged model. In astaged model, such as TMMi, organisations must follow a pre-defined path through maturity levels, and the process areas are organised in a fixed order that must be achieved sequentially. Each maturity level forms the foundation for the next, meaning that an organisation cannot achieve process areas at different capability levels independently, as is possible in acontinuousmodel like CMMI. The staged model is designed to ensure that each process area is fully implemented and institutionalised before progressing to the next maturity level.

In theActing phaseof the IDEAL model, the focus is on the implementation of the proposed solutions. Activities such as refining the solution and implementing the solution are key components of this phase. These actions are based on the plan developed in the earlier phases, and they are designed to apply the solutions to achieve the desired improvements. Specifically:

Refine solutions (i)refers to improving or adjusting the solution based on feedback from pilots or initial tests.

Implement Solution (iii)refers to the full implementation of the refined solution across the test process or organisation​​.

In an Agile context, some practices from TMMi levels 4 and 5 might be considered less relevant or adjusted to fit the Agile methodology. Specifically, at TMMi level 5, testing with statistical methods may indeed be less relevant. In Agile projects, operational profiles or usage models, which are essential to perform statistically valid testing, are often not utilised. Agile methodologies focus more on incremental development and continuous feedback loops, which do not typically rely on statistical sampling methods. Therefore, the statement that "Agile projects normally do not use operational profiles or usage models of a product on which to base statistically valid inferences to help create a representative sample of tests, thus the TMMi level 5 specific goal 'Testing is performed using Statistical Methods' is considered not relevant in an Agile context" is true​.

In an Agile context, the emphasis is typically placed on lightweight, flexible documentation. Test documentation should be "just enough" to support the test process, and excessive documentation may be seen as unnecessary overhead. Therefore, calling formore rigorous and thorough documentationisnotin line with Agile principles. Instead, Agile favours working software and collaboration over comprehensive documentation, supported by mentoring and just-in-time guidance rather than formalised documentation​.

In an Agile context, testing is often a shared responsibility among team members. This means that improvement ideas related to testing can come from any team member, not just dedicated testers. Agile encourages cross-functional collaboration, and because of the continuous feedback loops, all team members contribute to the refinement of both development and testing processes. This distributed responsibility fosters a more dynamic environment for test process improvements, where insights are drawn from diverse team roles.

An indirect benefit of a test improvement programme is improved staff motivation. While direct benefits, such as shortened lead time or higher defect detection rates, are focused on measurable improvements, indirect benefits include enhanced morale and engagement of the staff involved. A well-structured test improvement programme can lead to better recognition of the testing function within the organisation, leading to increased satisfaction and motivation among testing professionals.

This statement is false because the CMMI does not have defined levels specifically for testing maturity. The CMMI model is more general and focuseson overall process improvement across various disciplines, including but not limited to testing. CMMI includes practices like verification and validation but does not specifically address a structured path to testing maturity, unlike TMMi, which defines specific levels for testing process maturity. Therefore, statement D is incorrect.

In the TMMi framework, aSpecific Goalis unique to a particular process area and represents the essential outcomes that must be achieved for that process area. In contrast, aGeneric Goalapplies across multiple process areas and ensures that the processes are institutionalised, managed, and aligned with broader organisational objectives. Generic Goals help maintain consistency and effectiveness of processes across the organisation, while Specific Goals focus on the particular needs of an individual process area.

Agile organisations often focus on rapid iterations and minimal documentation, which can lead to the omission or underuse of structured testing practices.TMMiserves as a valuable guide for Agile teams by highlightingcritical testing practicesthat might be overlooked, ensuring that essential quality processes such as risk analysis, peer reviews, and test planning are not neglected​​​.

This reminder ensures that teams maintain a disciplined approach to testing while still benefiting from Agile flexibility.

The three TMMi Level 5 process areas are as follows:

Defect Prevention(A): The key specific goal here is todetermine common causes of defects. This process involves analysing defect patterns and establishing actions to prevent similar defects in the future.

Quality Control(B): This area focuses onevaluating new testing technologies to determine their impact on the testing process. Statistical methods help predict product quality, and these evaluations drive improvements in the test process.

Test Process Optimization(C): The goal here is toestablish a statistically controlled test process, where baselines are set, and improvements are implemented in an orderly manner​​​.

The process areaAdvanced Reviewsbelongs to TMMi Level 4 (Measured). At this maturity level, the focus is on quantitatively measuring and managing product quality, and advanced review techniques are employed to integrate peer reviews (static testing) with dynamic testing. These advanced reviews play a key role in ensuring that product quality is assessed early in the lifecycle and that testing is driven by measurable goals.

In a staged model like TMMi, an organisation follows a predefined improvement path that includes a set of process areas organised by maturity levels. Each level builds upon the previous one, and the organisation progresses in a step-by-step manner through these maturity levels to improve its processes. The staged model is well-structured and prescriptive, offering a clear improvement roadmap for the organisation.

CMMI version 2 recognises the importance of testing in software development and includes two dedicated practice areas:VerificationandValidation. These areas specifically addressthe processes required to ensure that a system or component meets its specified requirements and that the final product is fit for its intended use. While CMMI does not provide a testing-specific maturity model, these practice areas contribute to the overall quality management framework, and organisations adopting CMMI can leverage these processes to improve their testing practices. The TMMi framework complements CMMI by providing more detailed guidelines for testing.

CMMI and TMMi complement each other in many ways. While CMMI focuses on improving broader software development processes,TMMispecifically addresses testing. Implementing CMMI provides value to test organizationsby enabling a structured and continuous improvement approach that can incorporate TMMi as part of the overall process.

CMMI’s continuous representation allows organizations to implement specific process areas related to testing (like verification and validation) and integrate TMMi practices forcontinuous improvementin their testing processes. Organizations followingTMMimay also refer to CMMI practices for complementary process improvements in areas such as configuration management, planning, and performance measurement.

TMMi References:

TMMi outlines how CMMI’s process areas, such asVerification and Validation, provide critical value for testing organizations​​.

CMMI’s structure supports continuous process improvement that aligns well with TMMi’s structured approach to test process maturity​.

Sprint retrospectives are held at the end of every sprint (not every 2-3 iterations) to evaluate the team's performance and identify opportunities for improvement. Retrospectives are a core Agile practice, but they are not considered peer reviews, which typically involve reviewing work products such as code or design documents. Peer reviews focus on product quality, while retrospectives focus more on team performance and process improvements.

ISO/IEC 15504, which is known for its process assessment model, was not one of the sources used during the development of the TMMi model. The TMMi model was developed using inputs from sources such as the Capability Maturity Model Integration (CMMI), international testing standards (like those from ISO and IEEE), and industry testing best practices surveys. While the ISO/IEC 15504 model has relevance to general process assessments, it was not specifically referenced in the creation of the TMMi.

When implementing theTest Policy and Strategyprocess area in TMMi, one of the specific goals you would be targeting is toEstablish Test Performance Indicators. These indicators are essential for measuring the effectiveness of testing processes and for assessing whether the organization's testing goals, as defined by the test policy, are being met.

Other goals, such as performing product risk assessments or establishing a test organization, are related to different process areas in the TMMi model.

TMMi References:

TheTest Policy and Strategyprocess area in TMMi includes the specific goal of establishingTest Performance Indicatorsto measure and track testing performance​​.

The specific practice "Define a coordinated approach that coordinates both static and dynamic testing" is part of theAdvanced Reviewsprocess area at TMMi Level 4. The goal of this process area is to integrate static and dynamic testing, particularly through peer reviews, which helps optimise both defect detection and product quality management throughout the lifecycle​​​.