Many organizations are planning to transition their framework for process improvement efforts from the Capability Maturity Model^® for Software (SW-CMM^®) to the Capability Maturity Model^® Integration^SM (CMMI^SM). The CMMI is generally consistent with the guidelines of the primary external industry benchmarks for Earned Value Management (EVM), including the Electrical Industries Association (EIA) Standard EIA-748-A, "Earned Value Management Systems" [1] (EVM Standard). However, the CMMI is more stringent than the EVM standard regarding objective measurement and more focused on requirements.

Those organizations that already use EVM can develop efficient process improvement plans and minimize transition costs, including appraisal costs, if they utilize the relationships between the CMMI and external industry benchmarks, and address the gaps between their EVM practices and CMMI goals.

Other organizations should consider implementing EVM as a process improvement. In the CMMI context, EVM is a process thread that crosses many discipline boundaries and is critical to effective process integration. Consequently, implementation of EVM during the transition will be more efficient if it is part of an overall plan to improve and integrate processes.

In 1997, the Office of the Undersecretary of Defense, Acquisition, Technology and Logistics (OUSD) joined with the National Defense Industrial Association to initiate the CMMI project to integrate process improvement models that would build on the success of the Software Engineering Institute's (SEI) SW-CMM. The SW-CMM began as the SEI's answer to a challenge by the Air Force to find a set of key questions about a company's software processes that would guide their selection of the most competent or mature software developer [2].

At the November 2001 CMMI Technology and User Group Conference, Dr. Nancy Spruill, Director of Acquisition Resources and Analysis, OUSD, discussed future plans for the Department of Defense (DoD) to move toward the CMMI for greater breadth of coverage for software intensive systems.

Joe Jarzombek, Deputy Director for Software Intensive Systems, OUSD asked the Aerospace Industries Association to comment on several questions about the current policy, which mandated SW-CMM compliance as a requirement for bidders on specified programs. This included commenting on the rationale for the DoD to allow CMMI to join the SW-CMM as an equivalent model in comparing the capabilities of potential suppliers.

Jarzombek also encouraged the insertion of more EVM content into the CMMI to amplify existing specific practices or elaborations to the existing generic practices. Jarzombek stated in an e-mail, "However, to really have an impact on the way organizations do business, EVM might need to be introduced as part of the 'required' or 'expected' parts of the model/assessment ... I hope you can help us get more EVM into CMMI" [3].

The CMMI has increased the emphasis on integrated, quantitative project management, the core of the EVM, including the following:

• Managing requirements development and the technical solution for the total system to include systems engineering, not just software engineering.
• Integrating a project's cost, schedule, and technical planning parameters.
• Establishing precise, quantifiable measures of progress toward meeting an organization's measurement objectives.

There are three significant differences between the CMMI and the SW-CMM: the process areas emphasized for process improvement, the component structure, and the specific guidance. The comparisons in the following paragraphs are limited to those differences that are relevant to implementing the EVM during process improvement. All references are based on the CMMI-Systems Engineering/Software Engineering/Integrated Product and Process Development (SE/SW/IPPD) model.

The CMMI has three process areas that emphasize quantitative and integrated project management, whereas the SW-CMM does not. These include measurement and analysis (MA), project monitoring and control (PMC), and integrated project management (IPM).

The most significant difference between the CMMI and the SW-CMM, with regard to EVM, is MA. As described by two members of the CMMI product team, "Measurement has been elevated to the status of a separate process area ... MA provides a focus area and foundation for the various applications of measurement in project management and process improvement activities. The process area provides greater consistency and understanding with respect to the practice of measurement" [4].

Another product team member, D. Zubrow, of the SEI, identified 16 measurement-related process areas, including six within project management: project planning, PMC, supplier agreement management, IPM, risk management, and quantitative project management. The integration of MA activities into project processes supports the following:

• Objective planning and estimating.
• Tracking actual performance against established plans and objectives.
• Identifying and resolving process-related issues [5].

The PMC process area complements MA in its emphasis on quantitative project management. PMC SP 1.1 requires monitoring of actual performance against project planning parameters and identifying significant deviations. The SW-CMM discusses only quantitative process management. It does not include project planning parameters or the management practices of comparing actual values to a plan and identifying deviations.

In the IPM process area, the CMMI requires, in SP 1.4, that the project be managed using integrated plans. The SW-CMM does not address integrated plans.

The CMMI also differs from the SW-CMM in its component structure. The CMMI components include specific practices, subpractices, and typical work products (TWP). A specific practice describes the activities expected to result in achievement of the associated specific goal of a process area. Subpractices are detailed descriptions that provide guidance for interpreting specific practices or generic practices. TWPs are informative model components that provide example outputs from a specific or generic practice.

Although the EVM is not cited as a specific goal or a specific practice in the CMMI, the implementation and institutionalization of the EVM, consistent with external industry standards, provides evidence of achieving 15 specific practices within five of the 24 process areas in the CMMI. There are 458 specific practices in the CMMI.

In the following discussion and mappings, guidance and information is provided to identify those specific practices, subpractices, and TWPs that are consistent with the institutionalization of EVM. For organizations that seek external industry benchmarks as a source of measurement objectives, there are three benchmarks that address the EVM:

• EVM Standard.
• Practical Software and Systems Measurement: A Foundation for Objective Project Management (PSM) [6].
• Project Management Institute (PMI) Guide to the Project Management Body of Knowledge (PMBOK^®) [7].

Both the EVM Standard and the PMBOK provide guidance for integrated project management and for using EV as a derived measure. The PSM provides an excellent source of base measures for EV measures.

A sample of specific practices that map to the EVM Standard is provided in Tables 1 through 4. They show the commonality between the typical work products and subpractices in the CMMI and the corresponding guidelines in the EVM Standard. For example, the sub-practice in Table 1 requires that the project plan integrate with other plans. This sub-practice meets the intent of the EVM Standard guideline that requires integration of planning, scheduling, and budgeting.

Table 1: Integrated Project Management PA Mapped to EVM Standard

IPM Specific Practice	IPM TWP, Subpractice	EVM Std. Guideline #
1.3 Integrate Plans	Integrate other plans with project plan.	2.1.c Provide for integration of planning, scheduling, budgeting, work authorization, cost processes... and WBS.

Table 2 shows that a subpractice in requirements development - identify technical performance measures - is consistent with a component of the EVM Standard guideline: Identify technical performance goals to measure progress. Table 3 shows that a specific practice to specify measures maps to the EVM Standard guideline regarding measures of progress. Table 4 maps specific practices in PMC to the corresponding EVM Standard guidelines.

The importance of using this type of table is discussed in the "guidance for process improvement and appraisal" section later in this article.

Table 2: Requirements Development PA Mapped to EVM Standard

RD Specific Practice	RD TWP, Subpractice	EVM Std. Guideline #
3.3 Analyze Requirements	Identify technical performance measures that will be tracked.	2.2.b Identify physical products, milestones, technical performance goals... measure progress.

Table 3: Measurement & Analysis PA Mapped to EVM Standard

MA Specific Practice	MA TWP, Subpractice	EVM Std. Guideline #
1.2 Specify measures: Precise, quantifiable	Base measures Derived measures (EV, SPI) Operational definitions.	2.2.b Identify physical products, milestones, technical performance goals... measure progress.

Table 4: Project Monitoring & Control PA Mapped to EVM Standard

PMC Specific Practice	PMC TWP, Subpractice	EVM Guideline #
1.1 Monitor project planning parameters	Progress vs. schedule Cost Attributes of work products and tasks.	2.4.a Compare EV with time-phased budget and actual cost.
2.2 Take corrective actions	Determine actions needed.	2.4.e. Managerial actions 2.4.f. Estimate at completion 2.5.e. Changes to Performance Measurement Baseline

The Specific Goal 1 of MA is to align measurement and analysis activities. The CMMI, and other external industry benchmarks, stresses the importance of identifying measurement objectives. In the CMMI, MA has improved coverage of goals and practices to establish measurement objectives (SP 1.1) and to establish precise, quantifiable measures that meet the objectives (SP 1.2). Per SP 1.1, as follows: Establish and maintain measurement objectives that are derived from identified information needs and objectives ... sources of information needs and objectives include the following:

• Established management objectives.
• Project plans.
• Monitoring of project performance.
• External industry benchmarks.

For most businesses, established management objectives include achieving financial performance goals and customer satisfaction. These objectives are supported, at the project level, by achieving the project's cost, schedule, and technical objectives. The project objectives are incorporated into the project plans, and then project performance is monitored against the plans. Consequently, the specification of EVM is consistent with the above sources.

Specific Goal 1 of requirements management (RM) is to manage requirements. To achieve this goal, the project's plans, activities and work products should be reviewed for consistency with the requirements. Consistency is required for both the initial requirements and for changes to the requirements baseline SP1.5.

RM also refers to the PMC process area for information about tracking and controlling the activities and work products that are based on the requirements. In specifying the measures to be used to monitor progress towards the project's cost, schedule, and technical objectives, it is recommended that the practices of RM, MA, project planning, and PMC be closely related. It is recommended that the base measures for EVM be focused on the status of requirements from definition through verification.

RM, SP 1.4 requires that bi-directional traceability of requirements be maintained as follows: "When the requirements are managed well ... traceability can be established from the source requirement to its lower level requirements and ... back. Requirements traceability can also cover the relationships to other entities such as intermediate and final work products, changes in design documentation, test plans, and work tasks."

P. Baxter describes the relationship of measurement-to-requirements traceability. Baxter examines the application of the measurement process in providing visibility into the RM process. He says, "The measurement process provides an independent mechanism for quantifying the activities performed in the other essential project functions ... The measurement process greatly enhances the control and monitoring of requirements engineering by quantifying the status and progress of requirements-related activities ... Popular measurement selection techniques include ... PSM ... a systematic means for selecting specific metrics that support the defined needs of management" [8].

The importance of a requirements baseline is also discussed in Practical Software Measurement, Performance-Based Earned Value^SM (PBEV^SM) [9]. "Establishing a time-phased requirements baseline against which progress can be consistently measured is the most important EVM step. It drives the ... budget ... and the schedule. The technical requirements also establish the criteria for completing tasks ... Of equal importance are a disciplined requirements traceability process and a requirements traceability database."

An excellent reference on both requirements management and objective measurement is R. Young's book on effective requirements practices. He discusses a common misconception about earned value: "Its major weakness for software effort is its inability to assess technical progress (percent complete) accurately and objectively" [10]. This is based on a misunderstanding of earned value, which is a derived measure. Therefore, its effectiveness as a measure of schedule progress is a function of its underlying base measures:

• The selected measure of schedule progress for completion of an activity.
• The budget value allocated to the activity.

The quality and utility of earned value for PMC is highly dependent on the base measures of schedule progress of the requirements-related activities and work products.

Specific Goal 3 of requirements development is to analyze and validate requirements. One of the actions is to determine which key requirements will be used to track technical progress and to develop technical performance measures (TPM). EVM also requires the identification of technical performance goals (Table 2).

To achieve the specific goals cited above, it is recommended that the highest priority be given to measures of progress towards meeting a project's system requirements. The focus of planning and measurement should be on those work products and components of work products that are on the path toward satisfying system requirements.

A work product is any artifact produced by a process. The recommendation to focus on the requirements' work products is consistent with SP 3.3 in the requirements development process area. That specific practice, analyze requirements, includes the subpractice, "Identify key requirements that have a strong influence on cost, schedule, functionality, risk, or performance."

During the process of specifying effective measures of technical and schedule performance, it is recommended that, as a process improvement, some traditional measures of schedule performance and earned value be reassessed to determine if they address the measurement objectives that are derived from current information needs and objectives and be abandoned if they fail the assessment. Retaining unneeded measures increases project costs without providing management value.

An objective of PBEV is to exclude activities from progress measurement that do not produce work products. Although all activities require resources and scheduling, a project includes many activities that do not produce work products. Examples of these are coordination meetings, design reviews, and project status reviews. If during the meeting or review, a design or a test procedure is approved, the approval is the significant event because the approval is normally a criterion for completing the related work product. In order to minimize the costs of maintaining an earned value system, it is recommended that only those activities that result in work products be the base measures for earned value. All other activities can be level of effort.

PSM, one of the external industry benchmarks, has principles for identifying, collecting, and tracking project measures that produce work products. Examples of performance-based measures for earned value, from PSM, include functional requirements status, component status, test status, and increment content-functions [11].

PBEV is cost-effective because it limits the number of activities to be planned, discretely measured, and monitored to those that result in typical work products such as those in the CMMI. The work products within the requirements development and technical solution process areas receive the largest budget allocations. However, the work products of RM, validation, verification, and MA are also emphasized because of their roles in controlling the project. A sample of typical work products in the CMMI follows.

Requirements Development TWP:

• Customer requirements.
• Derived requirements.
• Product requirements.
• Product-component requirements.
• Interface requirements.
• Functional architectures.
• Activity diagrams and use cases.
• Object-oriented analyses with services identified.
• Key requirements.
• Technical performance measures.
• Records of analysis methods and results.
• Results of requirements validation.

• Product component operational concepts, scenarios, and environments.
• Use cases.
• Documented relationships between requirements and product components.
• Product architectures.
• Product-component designs.
• Technical Data Packages.
• Allocated requirements.
• Product component descriptions.
• Key product characteristics.
• Required physical characteristics and constraints.
• Interface requirements.
• Material requirements.
• Verification criteria used to ensure requirements have been achieved.
• Conditions of use (environments) and operating/usage scenarios, modes, and states for operations, support, training, and verifications throughout the life cycle.
• Comprehensive interface.
• Interface design specifications.
• Interface control documents.
• Implemented design.
• Product support documentation (training materials, users manual, maintenance manual, online help.)
• Requirements Management TWP.
• Requirements traceability matrix.
• Validation TWP.
• Validation results.
• Verification TWP.
• Exit and entry criteria for work products.
• Verification results.
• Measurement and Analysis TWP.
• Specifications of base and derived measures.

In preparing for CMMI appraisals and planning process improvements, it is recommended that an organization's processes and practices be mapped to the CMMI and the selected external industry benchmarks.

For organizations that already use EVM or that will implement EVM, the costs of process improvement using the CMMI, including appraisal costs and time can be minimized. M. Phillips of the SEI advises, "The appraised organization should provide sufficient evidence to the appraisal team to enable the team to quickly verify and validate the specific practices rather than to take the additional time to discover the targeted practices" [12].

I. Minnich, a CMM appraiser, advises, "Also, shift as much work as possible away from the 'on-site' portion of the appraisal and complete it beforehand. The pre-work should include mapping the organization's processes to CMMI. The better data your appraisal team starts with, the less time it will take the team to complete" [13].

To organizations that are using the EVM, it is recommended that artifacts be prepared based on the referenced tables. They should map existing procedures and practices to the CMMI and to the selected external industry benchmarks. Next they analyze the gaps and begin process improvement. The artifacts can be used in planning the appraisals and shown as evidence of institutionalizing the practices and achieving specific goals.

Organizations that are implementing the EVM as a process improvement should follow a similar process. They should identify and close gaps between the organization's existing practices, the practices of the CMMI, and the external industry standards.

The tables that were discussed earlier are building blocks for the artifacts that should be prepared. The artifacts will be complete when the organization's processes and practices are mapped to the tables.

Organizations that are compliant with the EVM Standard may need to identify and implement process improvements in order to achieve the related CMMI-specific goals. For example, there are two process areas have specific practices and information components that do not have a strong relationship to the EVM Standard, MA and RM.

As discussed earlier, MA SP 1.2 (Table 3) is more specific than the related guidelines in the EVM Standard with regard to focusing on defining objective measures of progress. Regarding objective measurement, MA SP 1.2 requires that precise and quantifiable measures be specified and that the base measures are obtained by direct measurement. SP 1.2 also includes a typical work product, specifications of base and derived measures (such as earned value), and a subpractice to specify operational definitions for the measures. The subsequent SP 1.3 requires specification of how measurement data will be obtained and stored. In comparison, the EVM Standard does not require objective progress measurement. The relevant EVM guideline 2.2.(a) requires only identification of a physical product, milestones and technical performance goals, or other indicators to measure progress. In the discussion of discrete effort, the EVM standard sets a far weaker standard, that management assessment may include the use of metrics for work measurement.

In the CMMI, the purpose of RM is to manage the requirements of the project's products and product-components and to identify inconsistencies between those requirements and the project's plans, activities and work products. In comparison, the EVM Standard addresses only work requirements, not product requirements. It does not discuss the work, plans, budgets, or schedules in relation to the product requirements.

The other process areas that provide a framework for process improvement of an organization's EVM practices are process and product quality assurance, requirements development, risk management, and organizational training.

It is recommended that the process improvement plan include objectives to develop documentation of process descriptions, standards, and procedures that provide evidence of achieving the specific goals of the CMMI process areas that were previously cited.

It is further recommended that process improvement include plans to incorporate the PBEV objectives. PBEV is consistent with the CMMI and can be used as a framework for reducing the cost of implementing the EVM as a process improvement during transition to the CMMI. Compared with the EVM Standard, PBEV is consistently focused on measuring the status of requirements and technical performance.

The process improvements and other recommendations discussed above will support the objectives to achieve higher capability and maturity in the CMMI process areas, to institutionalize a cost-effective EVM, and to integrate the EVM with the organization's systems engineering and enterprise-wide processes. Quantitative project management with the effective use of the EVM will increase the likelihood of achieving a project,s cost, schedule, and technical objectives.

1. Electrical Industries Association. Earned Value Management Systems (EIA-748-A). Washington, D.C., 2002.
2. Phillips, M., and S. Shrum. "Creating an Integrated CMM for Systems and Software Engineering." CrossTalk, Sept. 2000: 26.
3. Jarzombek, J. "DCMA Presentation of SIS Initiatives." E-mail to Paul Solomon. 9 Aug. 2001.
4. Lucero, S. "Up Close with Lt. Col (Ret.) Joe Jarzombek, Bruce Allgood." CrossTalk July 2000: 4.
5. Zubrow, D. "Putting 'M' in the Model: Measurement and CMMI^SM." Software Technology Conference. Salt Lake City, 2 May 2001.
6. U.S. Department of Defense and U.S. Army. Practical Software and Systems Measurement: A Foundation for Objective Project Management Ver. 4.0b. Oct. 2000. Available at www.psmsc.com.
7. Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK^® Guide) 2000 Ed. 2000.
8. Baxter, P. "Focusing the Measurement Process on Requirements: Integrating DOORS^® and MetricCenter^®." White paper. Available at www.distributive.com.
9. Solomon, P. "Practical Software Measurement, Performance-Based Earned Value." CrossTalk Sept. 2001: 26.
10. Young, R. Effective Requirements Practices. Addison Wesley, 2001: 256.
11. Solomon, P. "Practical Software Measurement, Performance-Based Earned Value." CrossTalk Sept. 2001: 28.
12. Phillips, M. "CMMI Version 1.1: What Has Changed?" CrossTalk Feb. 2002: 5.
13. Minnich, I. "CMMI Appraisal Methodologies: Choosing What is Right for You." CrossTalk Feb. 2002: 8.

Paul Solomon is the EVM-focal point for the B-2 Spirit Bomber and Global Hawk programs of Northrop Grumman Corp., Integrated Systems. He is also a visiting scientist at the Software Engineering Institute at Carnegie Mellon University. Solomon was on the team that wrote the EVM Standard and received the Department of Defense David Packard Excellence in Acquisition Award. He is on the Project Management Institute team that is writing the Practice Standard for EVM. Solomon has advised the Software Program Managers Network and presented concepts in this article at the 2002 Software Engineering Process Group and Software Technology Conferences. He is a Project Management Professional and has a bachelor's of arts degree and masters in business administration from Dartmouth College.

Dept. TD21/4B
Palmdale, CA 93550
Phone: (661)540-0618
Fax: (661)266-5648
E-mail: solompa@mail.northgrum.com

SM CMMI and CMM Integration are service marks of Carnegie Mellon University.

Performance-Based Earned Value and PBEV are service marks of Paul J. Solomon.