Software professionals, especially those working in the Department of Defense environment, face a somewhat bewildering array of relevant standards and best practices. As awareness and penetration of Lean Six Sigma in this environment have increased significantly over the last several years, we find many organizations struggling to understand and leverage the relationships between Lean Six Sigma and several other approaches to software process improvement, including CMMI^®. Specifically, the following are questions we hear quite frequently:

• We are re already doing CMMI^®; does it make sense to do Six Sigma as well?
• We are already doing Six Sigma; does it make sense to do CMMI^® as well?
• We are probably CMMI^® (staged) level 2 or 3; does it make sense to do Six Sigma before we get to level 4?
• Management wants us to get to level 5 as soon as possible; can Six Sigma help us get there quicker, or will it slow us down?
• We are just getting started on process improvement; should we do Six Sigma, CMMI^®, or both? At the same time, or one or the other first?

One the authors (Williams) attended a 1984 seminar that featured both W. Edwards Deming and Joseph Juran. Someone from the audience asked, Should we follow Deming’s teachings or those of Juran? Deming, in his inimitable, gruff voice responded, Pick either…just do something. Certainly do something remains sound advice in the context of Six Sigma and CMMI^®, but certainly it is not an either – or proposition.

Williams provided CMMI^® training and performed Class C Appraisals in three different organizations that were well into Six Sigma deployment. Black Belts, Green Belts, and Champions had been trained and all staff had some form of Six Sigma orientation. The results across the board were significantly different than the average initial appraisal. A significant number of processes were already documented and used, as opposed to the usual blank stares when procedures / templates are requested. All three had results more typical of a second or third round of appraisals than the normal results.

Especially noticeable was the difference in the quantitatively based Process Areas (PA), e.g. Measurement and Analysis (MA) at Staged Level 2 and the Level 4 and 5 Process Areas covering Quantitative Process Management (QPM) and Organizational Process Performance (OPP). In most initial Class C Appraisals, we do not even bother to look at Levels 4 – 5, sometimes not even Level 3. In all three cases, plans called for reviewing only Levels 2 and 3. As the appraisals progressed, the results were so startling that the Levels 4 – 5 were also reviewed.

• Measurement and Analysis Process Area – All goals and practices compliant with no improvement opportunities.
• Level 4 Process Areas (2) – All but two goals satisfied. All practices largely compliant with three improvement opportunities.
• Level 5 Process Areas (2) – All but two goals satisfied. All practices largely / partially compliant with two improvement opportunities.
• Follow-up accomplishments – The Group completed a Level 2 Appraisal as planned for calendar-year objectives. The Level 3 Appraisal was successful nine months later. Plans were halted when the organization was acquired by a large multi-national organization.

• Measurement and Analysis Process Area – All goals and practices compliant with one improvement opportunity.
• Level 4 Process Areas (2) – All but two goals satisfied. All practices largely / partially compliant with five improvement opportunities.
• Level 5 Process Areas (2) – All but one goal satisfied. All practices largely compliant with two improvement opportunities.
• Follow-up accomplishments – The Group was appraised at Level 3 three months later and is planning for a Level 5 Appraisal this quarter.

• Measurement and Analysis Process Area – All goals satisfied and practices compliant with one minor improvement opportunity.
• Level 4 Process Areas (2) – All but one goal satisfied. All practices largely compliant with three improvement opportunities.
• Level 5 Process Areas (2) – All goals and practices satisfied.
• Follow-up accomplishments – Group was appraised at Level 3 one month later and Level 5 six months following that event

All three case study organizations shared other attributes as well, including the following:

• All processes and procedures contained measurements, root cause analysis, and continuous improvement follow-up.
• All organizational results exceeded the data published by the Software Engineering Institute benchmarking activities.
• All levels of management and staff used the measurements on a daily basis.

The above results are phenomenal for first time Class C Appraisals. Upon further investigation, all stakeholders agreed that the Six Sigma tools and Six Sigma mindset had contributed greatly to such unparalleled results.

In our view, these cases and other industry experience answer in the affirmative four of the questions posed at the outset of this article:

• We are already doing Six Sigma; does it make sense to do CMMI^® as well? – Clearly each of the cases above felt there was additional value to be gained from engaging CMMI^® in addition to Six Sigma.
• We are probably CMMI^® (staged) level 2 or 3; does it make sense to do Six Sigma before we get to level 4? – None of these organizations had reached level 4, but all had realized benefits from Six Sigma at lower levels.
• Management wants us to get to level 5 as soon as possible; can Six Sigma help us get there quicker, or will it slow us down? – Case 3 above, as well as other results reported in the literature , clearly demonstrates Six Sigma can reduce the time needed to move to higher maturity levels.
• We are already doing CMMI^® ;does it make sense to do Six Sigma as well? – In her article Using Six Sigma in Software Development in News@SEI Lauren Heinz wrote the following:

Each year at Lockheed Martin, corporate management challenges its Integrated Systems & Solutions business unit (IS&S) to reduce total costs. Each year, IS&S uses Six Sigma tools to make it happen. Six Sigma has resulted in significant cost savings, said Lynn Penn, director of quality systems and process management at IS&S. It’s a structured approach that provides more than a checklist—it shows you what’s coming next, lets you look at data from different views, and gives you a big picture of your practices for making decisions.

Lockheed Martin is part of a growing number of organizations using Six Sigma to improve software quality and cycle time, reduce defects in products and services, and increase customer satisfaction. As Six Sigma evolves from an improvement framework for the manufacturing sector to one that can be applied across all levels of an enterprise, the SEI is looking at ways that Six Sigma has benefited software and systems development.

Answering the final question necessarily moves into the realm of opinion – the following is ours:

• We are just getting started on process improvement; should we do Six Sigma, CMMI^®, or both? At the same time, or one or the other first? – Our experience convinces us that most organizations will get measurable business results more quickly with Six Sigma than with CMMI^®, but ultimately will need the additional insights available from CMMI^® to realize maximum benefit. Government contracting organizations will in most cases need to do both in parallel - CMMI^® to satisfy government bid requirements and Six Sigma to ensure tight and near-term linkage to measurable business results. Even when CMMI^® comes first, level 4 and 5 will invariably lead organizations to something that is very like Six Sigma, even if by another name.

In the remainder of this article describes a little about Lean Six Sigma and its relationship to various aspects of CMMI^®. Part Two examines the DMAIC roadmap and part three examines the Design for Lean Sigma Six/ Define, Measure, Analyze, Design, Validate (DfLSS)/DMADV) roadmap.

Lean Six Sigma consists of the following three distinct roadmaps or methodologies:

• DMAIC – Generally used to improve existing products and processes.
• DfLSS (often described by the DMADV) – Generally used to develop new products or processes.
• A deployment process that includes defined roles such as Champions, Black Belts, and Green Belts, a formal approach to Lean Six Sigma project selection based on quantified benefits, training, and certification.

This article will discuss one view of the relationship of the DMAIC and DfLSS roadmaps to the CMMI^®. While it is beyond the scope of this article to deal specifically with Lean, it may be useful to note that while Lean tends to focus on cycle-time reduction and elimination of delays, and Six Sigma tends to focus on prevention and remediation of defects (in the broadest sense, including cost overruns, schedule delays, etc.), they are in fact highly synergistic and have come to be fully integrated within the Lean Six Sigma framework.

Interestingly, all three of the case studies involved organizations whose Six Sigma training and focus had been strictly DMAIC. All three believe that their CMMI^® results as well as the organizational measurements would have been significantly better if their Six Sigma training had included DfLSS, with more focus on the front end do it right the first time versus the DMAIC fix and improve approaches.

As illustrated in Table 1, the Continuous CMMI^® consists of four process categories, each containing several process areas. Process Areas primary connections to DMAIC, DfLSS, and Lean Six Sigma Deployment are indicated in Table 1 but are not meant to suggest that these relationships are exclusive, as there can be many to many relationships in particular circumstances. The most significant connections of the roadmaps to selected Process Area are discussed next.

Table 1: Lean Six Sigma and CMMI^®:

As measurement is central to both Six Sigma and CMMI^®, an understanding of how they relate in the context of the MA Process Area is central to envisioning how they can be used in combination. In the MA context we will introduce the DMAIC roadmap. The DfLSS/DMADV roadmap will be introduced later in the context of the Engineering Process Areas.

The purpose of MA is to provide management information necessary to implement monitoring and control of various required processes, and as such it has many relationships to other process areas (as does Lean Six Sigma).

Figure 1: MA Process Area

As with other Process Areas (PAs) of the CMMI^®, the emphasis is on what should be done, not how to do it. That’s where Lean Six Sigma makes a strong connection to MA (and to many of the other PAs) – it supplies a specific and proven answer to the how question – a performance-based methodology for applying measurement and analysis to problem solving and project management. Note that there is a subtle difference in perspective here between DMAIC and MA – the MA diagram above does not necessarily imply a sequence, whereas DMAIC definitely does – hence in practice there may be considerable back and forth between activities related to measurement objectives and those pertaining to measurement results.

Throughout this discussion we acknowledge the following two points of view:

• That of the software project being executed, which clearly has a certain set of needs related to measurement and analysis, with primary emphasis on getting the current project done on budget, on schedule, and with required quality.
• That of those concerned with process improvement, with emphasis on understanding and improving process efficiency and effectiveness.

While complementary and overlapping, each point of view has its own needs. In this portion of the article we consider application of the DMAIC flavor of Lean Six Sigma in the process improvement context of CMMI^® MA. The DfLSS flavor of Six Sigma, addressed later in the article, typically aligns more closely with the project execution point of view and illustrates other connections between CMMI^® and Lean Six Sigma.

This first MA process area step aligns very closely with the Define Phase (Figure 2) in a Lean Six Sigma DMAIC project that has been chartered to improve a specific aspect of performance. Here we notice the first important distinction and value-add that comes from the conjunction of Lean Six Sigma and CMMI^®. Lean Six Sigma places primary emphasis on understanding and managing performance (outcomes) while CMMI^® (often in practice if not in principle) places more emphasis on maturity level. Many within the SEI have expressed concern about over-emphasis on maturity rating.

This concern was one of the factors that motivated the selection of Measurement for Performance Driven Improvement (authors’ emphasis) as the title for new SEI courses recently introduced. Certainly maturity level is important for government contracting organizations, but is not sufficient by itself to quantitatively demonstrate improved outcomes in terms of cost, quality, or cycle time.

The Lean Six Sigma roadmap provides a very specific approach to establishing the overall objectives and identifying potential measures for an improvement project (using DMAIC) as illustrated in figure 2. Define activities also support realization of Generic Practice (GP) 4.1. Establish Quality Objectives.

Figure 2: DMAIC Define
(Click on image above to show full-size version in pop-up window.)

The DfLSS/DMADV roadmap provides similar guidance relating to product objectives and measures.

These second and third steps in the MA process area align quite closely with the Measure Phase of DMAIC (Figure 3). Again, the Lean Six Sigma roadmap provides detailed guidance for how to conduct these activities. M1 and M2 relate to Establishing Objectives and Specifying Measures, while M3 involves actually collecting the measures and perhaps placing them in a measurement repository.

Figure 3: DMAIC Measure
(Click on image above to show full-size version in pop-up window.)

The Analyze Phase (Figure 4) of DMAIC encompasses the activities implied by the Analyze Measurement Data step of the MA process area. Lean Six Sigma training includes instruction in selection and application of appropriate statistical tools, including criteria for determining which tools and methods are most applicable to a particular situation. Again, the roadmap provides detailed guidance on how to proceed.

As with all methodologies, brain engaged is necessary to success. For convenience these roadmaps are presented as though sequential, while in practice iteration is both common and appropriate.

Figure 4: DMAIC Analyze
(Click on image above to show full-size version in pop-up window.)

The fourth step in DMAIC, illustrated in Figure 5, parallels Support Process Areas (Causal Analysis and Resolution [CAR], Quantitative Process Management [QPM]) and Process Management PAs (Organization Process Performance [OPP], Organizational Innovation and Deployment [OID]). Analyze also relates to GP 5.2, Correct Common Cause of Problems.

The structure of the CMMI^® separates MA, where data is collected and analyzed, from the other process areas (CAR, QPM, etc.) that use the measures and analyses to define and implement improvements. In this respect DMAIC is structurally, although not substantively, different from CMMI^®. DMAIC envisions a continuous flow of activities from problem definition through solution and implementation performed by the same team, illustrating a distinction between CMMI^® what (defined by a series of Process Areas) and LSS how (defined by a project roadmap such as DMAIC).

If the primary goal of an improvement initiative is to create organization infrastructure and institutional capability, as the SEI intended to be the case for government contracting organizations for which the CMMI^® was originally designed, then the separation of MA from the various types of improvement activities clearly makes sense. MA focuses on the creation of measurement infrastructure, while DMAIC is typically more narrowly focused on time-limited resolution of a specific problem. While different in approach, the end result over time is essentially the same.

Figure 5: DMAIC Improve
(Click on image above to show full-size version in pop-up window.)

The Control phase of a Lean Six Sigma DMAIC project is in one sense a bridge back to the project point of view mentioned earlier. Control may be understood to align most closely with CMMI^® Generic Practices (GP) 2.8 and 3.2:

GP 2.8 – Monitor and Control - Monitor and control the process against the plan for performing the process and take appropriate corrective action.

GP 3.2 – Collect Improvement Information - Collect work products, measures, measurement results, and improvement information derived from planning and performing the process to support the future use and improvement of the organization’s processes and process assets.

Figure 6: DMAIC Control
(Click on image above to show full-size version in pop-up window.)

While DfLSS can have important implications for all process categories, the most obvious and significant impacts will be on the engineering category.

Requirements Development (RD) has three goals: develop customer requirements; develop product requirements, including characterization of operational concepts and use scenarios (allocating customer requirements to system elements such as hardware, software, people); and analyze and validate requirements (ensuring they are necessary and sufficient and appropriately balance stakeholder needs and constraints).

The goal of the Requirements Management (REQM) process area is to manage requirements in order to ensure consistency with plans and work products. REQM entails five specific practices, several of which have direct connections to DfLSS.

• Understand the requirements.
• Obtain a commitment from those who will implement the requirements.
• Manage changes to the requirements, including change history.
• Provide bi-directional traceability between requirements and related work products.
• Identify inconsistencies between the requirements and project plans and work products.

These process areas connect most directly to the Define and Measure phases of the DfLSS DMADV roadmap illustrated below.

Figure 7: DfLSS Define
(Click on image above to show full-size version in pop-up window.)

Step D2 in the DfLSS/DMADV roadmap explicitly addresses elicitation of customer requirements, including careful distinction of needs specifically identified by the customer and context data that frequently implies needs that are easily missed – the latent requirements. Context data describe the customer environment and often provide insight that clarifies the why behind the requirements and thereby facilitates the understanding required for effective requirements management.

Step D3 focuses attention on the critical to quality (CTQ) requirements that will drive success, distinguishing them from the routine many. In this step we also associate measures with the CTQs, thereby gaining additional insight that facilitates understanding, clarifies ambiguities, and establishes the framework necessary for early acceptance test planning. As test planning will often find as many defects as does testing, the definition of measures at this early stage leads to significant improvements in the quality of the requirements.

Figure 8 – DfLSS Measure
(Click on image above to show full-size version in pop-up window.)

Step M1 deals with prioritization issues and facilitates balancing of customer needs and constraints in a quantified and reviewable manner that can be revisited whenever requirements changes occur.

Step M2 includes Y to x flow-down, a technique that identifies the lower level factors that must be controlled or optimized in order to satisfy the CTQ requirements – incorporating one of the central concepts of Lean Six Sigma, Y=f(x); i.e., Y, an outcome typically not directly controllable, is driven by or is a function of one or more x's or factors that are controllable and must be carefully addressed. (For an illustration of this technique in a DMAIC context, see http://software.isixsigma.com/library/content/c030528a.asp)

Step M3 entails planning for and collecting data relevant to measurement success in meeting the requirements of the customer.

The Analyze and Design/Build phases (figures 9 and 10) of DfLSS relate to the third PA within the Engineering category, Technical Solution (TS). This PA has three goals – goal 1, Selecting Product-Component Solutions, aligns most directly with the Analyze phase, while goal 2, Develop the Design, and goal 3, Implement the Product Design, align with the Design/Build phase of the DfLSS/DMADV roadmap.

Figure 9: DfLSS Analyze
(Click on image above to show full-size version in pop-up window.)

During step A1 we use measures and data to develop a statistically sound understanding of the transfer functions that connect relevant x's to Ys of interest, in other words, determining which factors are the most important and quantifying the strength of the relationships that exist. This approach guides selection of alternative solution concepts or architectures.

During step A2 we generate alternative concepts and use tools such as the concept selection scorecard to guide our evaluation and selection among the feature and architecture alternatives identified. The concept selection scorecard, an evolution of Pugh Concept Selection, is a technique for evaluating feature set alternatives value vs. cost in financial terms. This step produces a Product Strategy that articulates the concept selected and the feature set to be provided.

Step A3 evaluates alternatives with respect to Implementation Strategy² . This assessment evaluates the business value impact of schedule/cost/quality alternatives – e.g., which is best in this situation – a fast delivery at higher cost and lower quality, a later delivery date with higher quality and lower cost, or delivering less functionality as a compromise?

Developing and implementing the design, including Technical Solution and Product Integration, aligns most directly with the DfLSS/DMADV Design/Build phase; here we are primarily in the realm of traditional software engineering practices and DfLSS typically plays a smaller role, often related to optimization of certain aspects of performance.

Figure 10: DfLSS Design-Build
(Click on image above to show full-size version in pop-up window.)

The final two process areas within the Engineering category, Verification and Validation (V&V) align most directly with the Verify Phase (Figure 11) of the DfLSS/DMADV roadmap, noting that certain validation activities are on-going throughout the life cycle during Define, Measure, and Analyze.

Figure 11: DfLSS Verify
(Click on image above to show full-size version in pop-up window.)

Lean Six Sigma is being used successfully in conjunction with CMMI^® in many organizations. Both are powerful in themselves and highly synergistic in combination. Lean Six Sigma and CMMI^® are powerful, broad in scope, and multi-faceted. There are many valid ways to understand them and describe their synergy. We invite readers to participate in the http://software.isixsigma.com discussion forum to share perspectives and insights into these and other connections between Six Sigma and software/IT industry best practices and standards.

Readers are also referred to Technical Report CMU/SEI-2004-TR-018, www.sei.cmu.edu/publications/documents/04.reports/04tr018.html for a discussion of Six Sigma as an enabler of technology transition.

This article is based, in part, on work done in creation of two new SEI courses ("Measurement for Performance Driven Improvement" 1 and 2) dealing with integration of Six Sigma and CMMI^®. Special thanks go to Dave Zubrow, Jeannine Siviy, Bill Florac, and Robert Stoddard of the SEI and David Hallowell of Six Sigma Advantage for their many contributions and insights.

1. Using Six Sigma to Accelerate the Adoption of CMMI for Optimal Results http://www.sei.cmu.edu/sema/presentations/siviy/optimal/
2. Applying Six Sigma to Software Implementation Projects http://software.isixsigma.com/library/content/c040915b.asp

Term	Definition	For more information
AHP	Analytical Hierarchy Process – a pair-wise prioritization technique developed by Thomas Saaty	http://thequalityportal.com/q_ahp.htm
ANOVA	A statistical technique known as "Analysis of Variance" (an unfortunate misnomer)	http://www.statsoft.com/textbook/stathome.html (a comprehensive stats reference available free on the web – printed versions can be purchased)
CMMI®	Capability Maturity Model Integrate	http://sei.cmu.edu
Cp, Cpk	Measures of process capability	http://isixsigma.com
CTQ	Critical to Quality – the "essential few" customer requirements	http://software.isixsigma.com
DfLSS	Design for Lean Six Sigma	http://software.isixsigma.com
DMADV	Design, Measure, Analyze, Design-Build, Verify; a Six Sigma product/process design methodology	http://software.isixsigma.com
DMAIC	Define, Measure, Analyze, Improve, Control; a Six Sigma product/process improvement methodology	http://software.isixsigma.com
DOE	Design of Experiments	http://www.isixsigma.com/dictionary/Design_of_Experiments_-_DOE-41.htm
DPU, DPMO	Defects per Unit, Defects per million opportunities	http://isixsigma.com
FMEA	Failure Modes and Effects Analysis	http://www.fmeainfocentre.com/
GP	CMMI® Generic Practice	http://sei.cmu.edu
GRPI	"Goals-Roles-Processes-Interpersonal Relationships"	http://isixsigma.com
IPPD	Integrated Product and Process Development	http://www.sei.cmu.edu/cmmi/presentations/sepg01.presentations/ippd/
KJ	An affinitization and prioritization technique developed by the anthropologist Jiro Kawakita (Kawakita, Jiro)	http://www.uie.com/articles/kj_technique/
LSS	Lean Six Sigma	http://software.isixsigma.com
PA	CMMI® Process Area – e.g., QPM (Quantitative Process Management)	http://sei.cmu.edu
QFD	Quality Function Deployment	http://www.qfdi.org/
SIPOC	"Supplier-Input-Process-Output-Customer" – a form of process map	http://isixsigma.com
TRIZ	"TIPS" is the acronym for "Theory of Inventive Problem Solving," and "TRIZ" is the acronym for the same phrase in Russian. TRIZ was developed by Genrich Altshuller and his colleagues in the former USSR starting in 1946, and is now being developed and practiced throughout the world.	http://www.triz-journal.com/whatistriz_orig.htm
VoB	Voice of the Business	http://isixsigma.com
VoC	Voice of the Customer	143,000 Google search results - key ideas: know who your customers are (they're not homogenous); find out what they want/need by asking, not assuming; finding unstated (latent) needs is part of the job
"x"	An independent variable; a factor that drives an outcome; controllable; a leading indicator	http://software.isixsigma.com
"Y"	A dependent variable; and outcome; typically not directly controllable; a lagging indicator	http://software.isixsigma.com

Gary A. Gack is a founding partner of Six Sigma Advantage, Inc., based in Rockland, Massachusetts, USA. He has an MBA from the Wharton School, is an ASQ-certified software quality engineer and a Six Sigma Black Belt. During his 40-year career in the software and IT industry, he has managed a variety of large-scale software projects and has consulted with dozens of Fortune 1000 firms. Mr. Gack co-authored Six Sigma Advantage's Black Belt, Green Belt and foundation curriculum.

He participated as a Research Affiliate and Visiting Scientist in a collaboration with the SEI Software Engineering Measurement and Analysis (SEMA) group and others in creation of two new courses ("Measurement for Process Driven Improvement [MPDI]-1", which has a Lean Six Sigma process improvement ("DMAIC") focus and "MPDI-2" with a focus on more advanced aspects of Six Sigma, including elements of Design for Lean Six Sigma). The SEI and Licensed Partners are now offering these courses.

For more information on these courses, see http://www.sei.cmu.edu/products/courses/p49.html and http://www.sei.cmu.edu/products/courses/p56.html

E-mail: ggack@sixsigma-advantage.com

Karl D. Williams is a Principal Consultant for Six Sigma Advantage. He has trained over 17,000 people in CMM, CMMI, Six Sigma and software skills. Karl was formerly a Director at Motorola and, more recently, a Senior Vice President of Process Design for Bank of America. He is a Master Black Belt, SEI CMMI Trainer and Lead Appraiser, and has performed over 150 CMM, CMMI, QSR and customized assessments at over 100 organizations in 19 countries. He has published over 70 articles and wrote a book entitled Continuous Improvement & Reengineering…A Better Way.

E-mail: kwilliams@sixsigma-advantage.com