The President’s Office of Management and Budget (OMB) has asked federal agencies to use a project management discipline known as EVM as a strategy to avoid costly IT failures. Other than within the Department of Defense (DoD), EVM is not well understood by federal agencies [1]. OMB issued its EVM policy guidelines in two memos issued in August 2004 and August 2005. In addition to requiring federal agencies and their contractors to use EVM for managing all major IT projects, the OMB established new reporting requirements. Agencies must include EVM data when they submit Exhibit 300s, documents in which they present their business cases for major IT projects. The OMB requires agencies to use EVM to calculate and report each project’s estimated total cost and completion date.

While EVM is an effective and useful project management tool, there are constraints within the organizational environment of the federal government that impede a smooth implementation of EVM.

In December 2005, the federal CIO Council released “A Framework for Developing EVMS Policy for IT Projects” [2] to assist agencies in developing their EVM policies as required by OMB Memorandum M-05-23 [3] for major IT projects. The guidance states the following:

EVM is a project management control tool allowing visibility into technical, cost and schedule planning, performance, and progress for major IT projects. EVM not only encourages contractors to use effective internal cost and schedule management control systems, but also provides the manager with timely and consistent cost, schedule, and progress data. The implementation of an EVMS ensures that cost, schedule, and technical aspects of the contract are truly integrated and estimated, and actual progress of the project can be identified. [4]

Program managers (PMs) should manage project cost and schedule performance measurements as integrated elements and not as separate entities. If the budget spend plan shows the project over-spending and the project schedule shows milestones slipping, the PM may know they might be in trouble but may have no way to make a quantitative assessment of how bad the trouble is. EVMS solves this problem by providing an accurate picture of spending and accomplishments related to a baseline plan. This enables the PM to quickly form conclusions about the project team’s staffing levels and productivity, as well as giving insight into areas of the work breakdown structure where the problems occur.

EVM compares the following three pieces of information:

1. How much work you planned to have accomplished until now (in dollars or hours) is called the Planned Value (PV).
2. How much you have actually spent until now (in dollars or hours) is called Actual Cost (AC).
3. The value, in terms of your baseline budget, of the work accomplished until now (in dollars or hours) is called the Earned Value (EV).

The first two pieces of data are compared to the EV in terms of differences resulting in variances and ratios resulting in performance indexes.

Basic EVM calculations involve differences or ratios with respect to EV:

1. The difference between EV and your plan (PV) is Schedule Variance (SV). SV = EV - PV.
2. The difference between EV and your spending (AC) is Cost Variance (CV). CV = EV - AC.
3. The ratio of EV to plan (PV) is your Schedule Performance Index (SPI). SPI = EV/PV.
4. The ratio of EV to cost (AC) is your Cost Performance Index (CPI). CPI = EV/AC.

Positive variance is favorable and negative is unfavorable. Having an EVM performance index that is greater than 1 is favorable, and less than 1 is unfavorable.

CPI is a reading on productivity and SPI is a reading on progress. If there is good productivity and slow progress, then the project is understaffed. If there is low productivity, then either the project has too much unplanned work or the project manager may have estimated poorly and the project has more work content than previously thought.

That is the essence of EVM; the rest are details.

The DoD has been using cost and schedule controls on aerospace and defense projects since the mid-60s. In the Office of the Under Secretary of Defense (Acquisition Technology and Logistics) policy memorandum dated March 7, 2005, the DoD revised its EVM policy to streamline, improve, and increase consistency in EVM implementation and application [5]. The DoD requirement for EVM applies to cost or incentive contracts, subcontracts, intra-government work agreements, and other agreements that meet the dollar thresholds prescribed (see Table 1). This memorandum requires the Table’s application thresholds (total contract value including planned options in then-year dollars [TY$]).

Table 1: DoD EVM Thresholds and Requirements

Although EVM is not required on contracts, subcontracts, intra-government work agreements, and other agreements valued at less than $20 million (total contract value including planned options) and/or less than 12 months in duration including options, PMs have the discretion to implement an EVMS. If implemented, the PM is required to conduct a cost-benefit analysis. The purpose of the cost-benefit analysis is to explain the rationale for the decision to require cost/schedule visibility in the contract and to substantiate that the benefits to the government outweigh the associated costs. If the value of a contract is expected to surpass $20 million or last longer than 12 months, acquisition guidelines suggest that the PM should consider imposing an EVM requirement on the contract.

The Defense Acquisition Guidebook discourages the application of EVM on firm-fixed price contracts, subcontracts, intra-government work agreements, and other agreements regardless of dollar value [6]. If knowledge by both parties requires access to cost/schedule data, the first action is to re-examine the contract type (e.g., fixed price incentive). However, in extraordinary cases where cost/schedule visibility is required and cannot be obtained using other means, the PM is required in accordance with acquisition guidelines to obtain a waiver for individual contracts from the Milestone Decision Authority (MDA)¹. In these cases, the PM is required to conduct a business case analysis that includes rationale for why a cost or fixed price incentive contract was not the proper contracting vehicle. When appropriate, the business case analysis should be included in the acquisition approach section of the program acquisition strategy (see Figure 1).

Figure 1: Decision Process for EVM Application

However, the Federal Acquisition Regulation (FAR) council issued a rule in July 2006 that went into effect in August 2006 and gave federal agencies broad discretion in determining when and how to use EVM [7]. The council noted that agencies have significant discretion in determining the size and complexity of projects that meet the criteria for a major acquisition set by the agency [8]. While the council determined agencies could set their own dollar thresholds under this new rule, they also stated, it is not appropriate to exclude certain contract types from EVMS requirements in the FAR. In accordance with OMB Circular A-11, Part 7, EVMS is required for major acquisitions for development regardless of contract type [8]. The DoD allows exemptions. The Defense Federal Acquisition Regulations Supplement (DFARS) has its own proposed rule on EVM. That rule was published in January 2006 and was open for public comment until late March [9]. The DFARS proposed rule, which would be subordinate to the FAR rule, is now under review.

As federal agencies learn how to apply the principles of EVM to manage IT projects, they have encountered obstacles and challenges. Some of the challenges are related to the suitability of EVM itself to IT projects. Some of these challenges are cultural in nature as most experts say EVM cannot help agencies that cannot accept bad news. The experts say EVM is most valuable if agencies use it to help people learn from their mistakes rather than to punish them. Nevertheless, the fear of punishment must be addressed. Change management becomes a difficult aspect of bringing EVM into an agency in order to keep employees involved and ensure appropriate use of the EVM system. Some of the challenges are related to putting the infrastructure in place to support the EVMS process. It takes a lot of commitment and effort to get tools and systems in place and integrate them with other existing systems to generate the data and timely reports required.

In theory, the EVMS enables PMs to track money spent on a project as well as measure the work accomplished against that cost and the schedule in a near real-time status. However, this theory does not always translate well in an actual project management setting. Speculations as to why EVMS may be ineffective include the following ideas:

• Lack of senior management support.
• Little understanding of EVM methodology as it pertains to software versus hardware and the accompanying belief that IT projects are not measurable and therefore EV cannot be applied to those projects [1].
• The need for employees trained in the concepts and methodologies.
• The perception that EVM is burdensome and somewhat costly to implement [10].
• The perceived questionable cost benefit of applying EVM to already budgeted IT projects [11].
• The perception that EVM measures the quantity but not the quality of work performed [1].
• EVMs underlying assumption that problems derive from poor project execution rather than inadequate project planning [1].

Depending on how deeply ingrained these perceptions are and the knowledge of the workforce on EVMS concepts, the PM will have to address these cultural issues.

In applying EVM, having a realistic baseline is critical. However, in the federal arena, there are several systemic realities that can introduce errors to the baseline from the very beginning.

The process for creating the initial funding estimate can introduce errors. Developing a baseline budget is usually dependent on having experience with previous projects of similar type, size, and scope. In today’s DoD IT environment in which we are trying to develop architectures with complementary infostructures²that support net-centric operations, we are venturing into uncharted territory in which we are pursuing project objectives that have not been achieved before in terms of technology, size, and scope. Furthermore, the initial budgetary funding estimate is usually based on well intentioned, but nevertheless best guess assumptions about how much change or rework is likely to occur as requirements are clarified during the design phase and hence how much cost and schedule risk is associated with the new program. This budgetary estimate is then overly constrained years too early in the Planning, Programming, Budgeting, and Execution (PPBE) process to secure adequate funding. In the PPBE process, requirements are identified years before a budget is prepared and submitted. These requirements are expressed in the Future Year Defense Program. The planning cycle is shown in Table 2.

Table 2: Planning Cycle

Given the rapidly changing environment of technology, the estimates are often too low. Because the budget is constrained by the PPBE process, the program is already in potential jeopardy before arriving in the request for proposal (RFP) stage.

The budgeting issue often escalates during the RFP stage. Contractors oftentimes base their proposal estimates using historical actuals with inflation factors built in for time and manpower. However, since not reporting overtime is a common problem, these so called actuals often do not reflect all hours truly expended on difficult tasks in past projects. Furthermore, the accuracy of the historical data is also dependent on whether or not progress was tracked and reported on a daily basis and many organizations are challenged by the lack of an automated time-reporting system. All of which can result in underestimating the duration of tasks that are then used to generate the project cost estimate.

If the contractor bases their proposal on the budget allowance knowing that it cannot be met, then they may be relying on making up the shortfalls later on in their negotiations for requirements changes. The negative implications of underbidding based on a budget allowance can further be compounded after the contract is awarded. In this case, the contractor may implement the program at the funded/proposed amount and then reduce all the budgets by 10 to 15 percent to create a management reserve in time and budget to allow for the unexpected. Ideally, the contractor would have accelerated the schedule to create a schedule reserve and shorten the length of the program to fund the budget reserve. However, contractors are often unable to accelerate the schedule because sufficient funding (or personnel) is unavailable to support an earlier schedule.

Program execution comes with many challenges with technology, staffing, schedules and budgets. One of the biggest challenges is to react to changes, such as requirements creep, without losing control of the program.

When changes or problems surface that require a modification to the schedule, adjusting the program baseline can pose a real challenge. There are several reasons for this. For the contractor, a change request must be internally coordinated to check feasibility. It is often a challenge to gather all the required people to analyze the impact of the change within the project and across other projects to determine if the staffing available can support the revised plans. Once the contractor has developed a new baseline with the change, the contractual process will create a lag. It can take weeks or months to get a change request approved. Scope and schedule changes often require coordinating handoffs between several organizations before agreeing on new delivery dates. Since it is unusual to stop a program while you are re-baselining, the project team continues to track against the original baseline that is in reality obsolete while the approval process is taking place. Continuing to track against an obsolete plan impedes effective management because true priorities become lost which can then lead to the misallocation of resources.

In order to validate that the program can be completed within the contract requirements, EVM system descriptions require a schedule. The larger programs typically require a schedule with logical ties between the tasks (often a critical path method schedule) as opposed to the simpler schedule without links between the tasks. The PM can base the program network schedule on the resources available or assume that management will get the resources in time for the program to be successful. Neither approach is ideal since the non-availability of the right resources can delay the program and even a schedule that uses resource allocations to determine the durations makes assumptions about the availability of resources needed in the future. In addition, program contract leaders have to develop program plans around the available funding constraints in terms of both amount and timing of cash flow. Lack of sufficient funding prevents contractors from developing schedules optimized for best cost, schedule, or resource availability and thereby results in a more inefficient plan.

At the Strategic Level
If the true intent of OMB’s implementation of EVM is to better manage IT projects’ cost, schedule, and performance to maximize benefit for the taxpayer dollar, then the entire operating environment in which EVM is implemented will need transformation as well. EVM can track progress of a project, but it cannot solve the underlying systemic issues that created an underfunded/underbid project budget that inadvertently puts the project at high risk from the outset.

Agencies can take steps to practice more realistic project portfolio management in order to identify duplication of effort in attempts to gain desired capabilities. This would promote cross leveling to enable more adequate funding of projects that remain in the portfolio. Agencies can also leverage system engineering techniques to divide the project into smaller parts to enable a more agile response to changes.

Because agencies must meet 90 percent of agency goals for cost, schedule, and performance in order to achieve green on the President’s Management Agenda scorecard, there will be cultural pressure to view the EVM process as a contractual requirement that is administered with audit-like rigor by the review teams. Accommodating extensive government certification reviews, collecting and arraying data in prescribed categories, and preparing detailed reports requires time, effort, and cost to the government and can draw some of contractor engineering resources away from program execution. Efforts should be made to counter this cultural pressure to ensure that tailored EVM requirements remain tailored and do not become overly cumbersome. The Government Accounting Office (GAO) found that commercial firms that use EV systems produce reports more frequently, more quickly, and in less detail than traditionally found in the DoD [12]. The focus should be on the information the EVMS is communicating and less on the presentation itself. This will enable program management to identify the areas that need program management attention and develop corrective actions needed to achieve program success.

At the Organizational and Project Levels
All levels of program management (oversight, management, and execution) need to understand the principles of EV, but more importantly they need to understand human behavior. A good PM will anticipate reluctance and will prepare to employ savvy political strategies to solicit buy-in. By emphasizing that the EVMS is aimed at improving the overall program progression to successful completion on time and on budget and by understanding the impacts of their behavior, management can positively influence the team to do their best. However, the PM must be realistic and cannot deny the challenges of implementation with the team.

In overcoming those challenges of implementation, training is key. This training must include the oversight authorities. This will help them understand and trust the signals that the EVMS is sending so they respond in a timely manner when issues are raised such that the situation does not become unrecoverable. Training in the concepts of EV should include practice in developing project schedules with different styles of work breakdown structures (execution oriented vs. product oriented) to demonstrate which orientation style works best when changes occur and what level of detail to plan. Too much detail makes the system burdensome – too little and it lacks credibility. In developing these work breakdown structures, project members should acknowledge and anticipate that later life-cycle activities, such as testing, will have different cost/schedule variances than earlier life-cycle activities. The goals of effective program execution need to be emphasized during the training. Questions to answer during the training sessions might include the following:

1. If work is tracked at a high level, can the details be used to sum up to the total? If not, how can the high-level work completed be assessed?
2. How will the plan work in practice?
3. What happens if the plan changes?

EVM requires truth telling [13]. It often involves reporting contentious facts, delivering bad news, and sharing difficult feedback. Some areas in which interpersonal relations training may help the team are the following:

1. Communications.
2. Leadership.
3. Active Followership.
4. Emotional Intelligence.

Once the team has been trained, then the PM must create and diligently maintain a culture of execution. Strategies to accomplish this might include the following:

• Celebrating success. People respond to positive reinforcement. When the team’s efforts result in a big win for the project, celebrate! Celebrating success builds team spirit and encourages repeat performance. Make sure the success is public knowledge. Share it with the entire organization, if possible. Making public heroes out of those responsible for the success is likely to encourage others to strive for their chance in the limelight.
• Being a role model. The PM must personify the principles of the EVM process in every interaction. If you do not practice what you preach, how can you expect it of your team? Speaking candidly, insisting upon realistic information, focusing on results and being actively involved in the success of the project makes the PM a great role model for the project team. When the team submits undesirable, but realistic numbers, the PM must remain cool. Candid communication and realistic information are cornerstones of an EVMS. PMs must remain calm and then try to find out what happened and how the project can be brought back on track. The PM must be accessible and available to the team on a regular basis.
• Encouraging appropriate behavior. EVM in action is not always easy. Speaking candidly and using realistic data comes with risk. The PM can encourage the team to do so by praising them when they are bold enough to present realistic numbers, even when doing so makes them and the project look bad. Acknowledge them for speaking their minds and communicating project challenges.

Organizational leaders and PMs must keep in mind that what you measure is important – what you pay attention to and focus on tends to get reinforced whether or not it enhances project progress. Counting missed milestones and focusing on the negative can result in an overemphasis on doing tasks on time against a plan that may be out of date. If the program only measures missed milestones instead of keeping the emphasis on the final goals (focusing on the positive), near term tasks may receive a disproportionately high priority to increase the number of tasks completed. However, the best approach for the long term may have been to miss some near-term tasks to take advantage of resources available now but which another program will require at the same time if the tasks are performed in the order of the original plan. If innovation and creativity are stifled because of a culture that punishes managers who have EV variances, the goal of improved program execution by implementing EVM will not be met.

One sociological definition of technology is a set of standardized operations which yield predetermined results [14]. The more likely predetermined results occur, the stronger the technology. Developing software does not have the predictability of outcome as manufacturing processes do. Many people would argue that developing software programs and information systems is just as much art as it is science. While style guides can be implemented to maintain some consistency, two different programmers can still approach the same requirement very differently obtaining the same results via very different coding paths. Predicting, replicating, and standardizing those thought patterns that created those coding paths is very difficult. As such, applying EVM can be more challenging in IT projects. Nevertheless, most experts today agree that EVM is suitable for managing major IT projects [1].

With this in mind, PMs should carefully consider exercising their discretion in applying EVM. Ultimately, the decision will largely be subjective based on how the cost-benefit analysis is conducted. Therefore, the value the PM gets will be determined by how EVM is implemented, taking care to avoid unnecessary cost drivers, such as the following:

• Lengthy systems descriptions of EVMS [15].
• Written variance analysis at the control account level [15].
• Over-specified work breakdown structure [15].
• Over- or under-compensating for in-evitable planning errors [16].

Fleming and Koppelman offer sound advice in the June 2006 issue of Cross-Talk on how to pragmatically obtain the benefits of EVM using simple EV without overtaxing the project team [17].

Tracking project progress should be a continuous activity where data is collected as the activity occurs. Thus when EVM is optional, PMs should seek as close to real-time data as possible directly from the contractor in whatever format the contractor uses, as long as the format remains consistent and the data is accurate and verifiable.

The following set of questions can assist a PM in developing a tracking and measurement program:

• What visibility do you have in terms of resources, time, and cost?
• What can you track and measure? How often can you do it?
• Who sets the standards for performance? How realistic are they? How clear are they? Do these standards contribute to project goal achievement?
• How often do you need to report and to whom? How long does it take to prepare the reports?
• What performance variance is acceptable? At what level of variance is action required?
• What rewards and penalties are available?
• What is the criticality of the system being developed?
• What is the critical path for the system being developed to be operational?

A good PM knows metrics are just one of many tools of the project management tool set. When a healthy balance and perspective is maintained by using EVM as a management tool rather than a financial report card that supersedes all other tools, the benefits of EVM become more apparent.

Given the imperfect world in which we operate in the DoD and the federal government, can EVM by itself achieve the goal of avoiding costly IT failures? Probably not. EVM will not prevent requirements creep or contractors underbidding projects based on budget allowances, or poorly planned projects. However, by managing its adoption through cultural modifications and training, it is a step in the right direction.

1. Olsen, Florence. “EVM Sounds the Alarm: Smart Agencies Let Earned Value Management Lead Them to Better Program Management.” Feder- al Computer Week 16 Mar. 2006 <www. fcw.com/article92541-03-13-06Print>.
2. CIO. A Framework for Developing Earned Value Management Systems (EVMS) Policy for Information Tech-nology (IT) Projects . Washington: CIO Office, 2005 <www.cio.gov/ documents/Framework_for_Devel oping_EVMS_Policy_ 12-5-05.pdf>.
3. Evans, Karen. Improving Information Technology Project Planning and Execution . Washington: Office of Management and Budget, 2005.
4. Federal CIO Council. A Framework for Developing Earned Value Manage-ment Systems Policy for Information Technology Projects . Washington: CIO Office, 2005 <www.cio.gov/ documents/Framework_for_ Developing_EVMS_Policy_12-5 -05.pdf>.
5. Wynn, Michael. Revision to DoD Earned Value Management Policy . Washington: Undersecretary of Defense for Acquisition, Technology, and Logistics, 2005 <www.acq.osd. mil/pm/currentpolicy/EVM%20 Policy%20letter%203-7-05.pdf>.
6. DoD. Defense Acquisition Guide-book . U.S. DoD, 2006 <http:// akss.dau.mil/dag/Guidebook/IG_ c11.3.1.asp>.
7. Rogin, Josh. “No One-Size-Fits-All Approach for EVM.” Federal Com-puter Week 10 July 2006 <www.fcw. com/article95183-07-10-06-Print>.
8. Federal Register 5 July 2006 <http:// a257.g.akamaitech.net/7/257/2422/ 01jan20061800/edocket.access.gpo. gov/2006/06-5966.htm>.
9. Rogin, Josh. “FAR Gives Agencies Wiggle Room.” Federal Computer Week 5 July 2006 <www.fcw.com/ article95156-07-05-06-Web>.
10. Thormeyer, Rob. “Agencies Struggle With EVM as OMB Deadline Approaches.” Government Computer News 3 Nov. 2005 <www.gcn.com/ vol1_no1/daily-updates/37486-1. html>.
11. Mosquera, Mary. “What’s the Future of Your IT Project?” Government Computer News 22 Aug. 2005 <www. gcn.com/24_24/news/36698-1. html>.
12. United States. GAO. Significant Changes Underway in DoD’s Earned Value Management Process . GAO/ NSIAD-97-108. Washington: U.S. General Accounting Office, May 1997 <www.gao.gov/archive/1997/ ns97108.pdf>.
13. Harris, Lynn, and Jeff Arnold. “Truth Telling: Confronting the Reality of Lack of Candor Inside Organiza-tions.” In Transition 2006.
14. Freeman, David. “People, Technology, and the Environment.” Colorado State University, Fall 1998 Semester.
15. Christensen, David. S. “The Costs and Benefits of the Earned Value Management Process.” Acquisition Quarterly Fall (1998): 373-385 <www. suu.edu/faculty/christensend/cbaev ms.pdf>.
16. ASK Process, Inc. “Using Earned Value Part II: Tracking Software Projects.” Ask Process (2003) <www. askprocess.com/Articles/UsingEV2. pdf#search=%22Using%20Earned%20Value%20Part%20II%3A%20Tracking%20Software%20Projects%22>.
17. Fleming, Quentin W., and Joel M. Koppelman. “Start with ‘Simple’ Earned Value on All Your Projects.”June 2006. <www. stsc.hill.af.mil/CrossTalk/2006/06/0606FlemingKoppelman.html>.

1. The MDA’s primary responsibility is to make decisions on whether the programs should be initiated and whether they should proceed through the various phases of the acquisition life cycle. At each major decision point, the MDA must determine whether the program, or a key increment of the program, should be terminated, modified, or approved to proceed.
2. Infostructure is an Army Network Enterprise Technology Command term used to describe the enterprise-managed IT infrastructure that is part of the Global Information Grid.

LTC Nanette Patton is currently serving as a program analyst in the Office of the Surgeon General (OTSG). She recently completed a one-year fellowship at the RAND Corporation. Patton has eight years experience as a health services systems manager and is Level III DoD-Acquisition certified in Information Technology and Level II certified in Program Management. She has a master’s degree in human resources management from Chapman University, and a Master of Business Administration in computer information systems from Colorado State University.

OTSG
PA&E Manpower
5109 Leesburg Pike
Skyline 5, RM 598
Falls Church, VA 22041
E-mail: nanette.patton@us.army.mil

Allan Shechet is president of Savvy Services Incorporated, a project management consulting company and has more than 20 years experience with EVM systems in Aerospace and IT companies. He has a master’s degree in organization development from Fielding Graduate University, is certified as a Project Management Professional by the Project Management Institute (PMI), and is an active volunteer for the Los Angeles chapter of PMI.

Savvy Services
2239 Linnington AVE
Los Angeles, CA 90064
Phone: (310) 720-2522
E-mail: allan@savvyservices.net