Starting with the Appraisal Requirements for Capability Maturity Model^® (CMM^®) Integration^SM (CMMI^SM) (ARC) Version 1.0 [1], the authors of the CMMI product suite laid out the requirements for three classes of appraisal methods. This is important because it recognizes that an organization can get benefits from internal appraisals at various levels of resource expenditures.

Of course this has always been true, but the ARC formalizes the three classes by mapping requirements to them, which provides a consistency and standardization that has not been available with any of the CMMI predecessor models. It also allows organizations the freedom to develop an appraisal methodology that works best for their organization, and once mapped to the ARC appraisal classes, the results of any appraisal can be easily benchmarked against other appraisals from the same class.

The characteristics of the CMMI appraisal classes are summarized in Table 1 (see page 8). Class A describes a full appraisal, usually performed by a team of six to 10 people, primarily drawn from inside the organization being appraised. A class A appraisal is expected to be the most accurate, designed to maximize buy-in from the appraisal participants, and leaves the organization with the best understanding of issues that need to be fixed and strengths that should be shared. The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) describes a class A appraisal method [2].


Table 1: Characteristics of CMMI Appraisal Classes[1]
(Click on image above to show full-size version in pop-up window.)

Class B describes a smaller scale appraisal methodology, sometimes called a mini-appraisal or a pre-appraisal. A class B appraisal can be accomplished with a smaller team of expert appraisers over a reduced number of days. It can be used as a substitute for a full appraisal or to spot check the organization between full appraisals.

Class C describes the least intensive appraisal methodology, sometimes called a micro-appraisal or questionnaire-based appraisal. A class C appraisal can be used to get a rough idea of the current state of the practice within an organization.

The length of time needed to complete an appraisal can be significantly more for a class A appraisal than for a class B or C appraisal. Many factors contribute to the time needed to complete an appraisal. Some examples are the size and complexity of the organization, the number of process areas and capability levels or maturity levels covered, the size of the appraisal team, the training and experience level of the appraisal team, and the amount and rigor of evidence review.

The goal for completing the on-site portion of a full class A or SCAMPI appraisal is roughly two weeks. The majority of this time will be spent gathering and evaluating evidence to determine appropriate coverage of the model's practices. A rough estimate for a class B appraisal is one week. This may be accomplished through less rigorous evidence collection and review, perhaps by relying more on interviews or "spot checking" for practice compliance. Class C appraisals may be only hours long and are likely to be based on questions and answers, with little examination of evidence.

Another difference between appraisal classes is the expected training or experience level of the lead appraiser. As with the Software CMM lead appraisers in the past, the Software Engineering Institute (SEI) is responsible for training and authorizing lead appraisers for SCAMPI appraisals for the CMMI. This does not mean that others can not lead an appraisal, only that if you want or need "official" appraisal results you must use a SEI authorized SCAMPI lead appraiser.

There are no plans currently to authorize appraisers for class B or C lead appraisers. It is up to the individual organization planning an appraisal to determine the qualifications needed by the lead appraiser to meet the needs of the organization's appraisal.

An organization can undoubtedly benefit from each of the types of appraisal methods, and will probably develop an appraisal tool kit that contains aspects of all three. For example, an organization may develop the following:

• A questionnaire or checklist (class C) to be used quarterly to "remind" everyone of the processes that should be followed.
• A mini-appraisal (class B) that will be performed internally every year to determine the current state of the practice.
• A full appraisal (class A) that will be performed by an outside source every two to three years or as needed for contract procurement.

Preparation for the appraisal always plays a big part in its success. As with all of the predecessors to CMMI, the definition of the scope of the organization is probably the one decision that most affects the time to complete the appraisal itself. Since CMMI can be used to evaluate the activities associated with systems engineering, software engineering, integrated product and process development (IPPD), and acquisition, be aware that the broader the scope of the organization the more people will be involved in the scope of the appraisal. A broader organization, or an organization now looking at including more "disciplines," takes more time to appraise. This has always been true, but many appraisal sponsors may not be aware of the obvious correlation.

Another way to lessen the pain of an appraisal is to shift as much work as possible away from the "on-site" portion of the appraisal and complete it beforehand. There are many variations of this. Suggestions for appraisal pre-work include mapping the organization's processes to CMMI, gathering and/or reviewing evidence, distributing and completing CMMI-based questionnaires, and developing interview questions for use during the appraisal. The better the data your appraisal team starts with, the less time it will take the team to complete.

Probably the biggest contributor to the success of your CMMI appraisal will be in providing your appraisal participants with the proper level of education, especially if they have some preconceived notions based on the use of predecessor models. The appraisal sponsors need to have realistic expectations concerning the scope of the organization, the number of appraisal participants and their areas of expertise, and the use of each of the appraisal classes. The appraisal team and the supporting staff responsible for the appraisal prework need to understand the CMMI, the requirements and methodology for the appropriate class of appraisal, and how to map or translate the work being performed in the organization to the CMMI. The remainder of the appraisal participants most likely will not need any special CMMI training.

Thorough planning and pre-work of a CMMI appraisal may be more important than ever before, especially if your organization is planning to broaden its definition of organization or include additional disciplines and activities. Setting expectations, educating participants, and mapping terminology are key to the success of an appraisal. Developing an appraisal tool kit, including the different appraisal classes will allow your organization to meet its process improvement needs in an efficient and effective manner.

Note: All these documents are available on the Web at www.sei.cmu.edu.

1. CMMI Product Development Team. Appraisal Requirements for CMMI (ARC), Version 1.0. CMU/SEI-2000-TR-011, Aug. 2000.
2. Standard CMMI Appraisal Method for Process Improvement: Method Description, Version 1.0. CMU/SEI-2000-TR-009, Oct. 2000.
3. CMMI Product Development Team, Capability Maturity Model Integration for Systems Engineering/Software Engineering/Integrated Product and Process Development, Version 1.1. Continuous Representation. CMU/SEI-2000-TR-019, Aug. 2000.

Ilene Minnich is a SECAT LLC founding partner and has worked as a process improvement and appraisal consultant to more than 70 organizations worldwide. She is a part-time visiting scientist at the Software Engineering Institute (SEI) and works with the Capability Maturity Model^® (CMM^®) Integration^SM training team. She is an experienced CMM author, appraiser, and trainer. Previously at Hughes Aircraft Company, Minnich was responsible for the Software-CMM deployment and conducting Systems Engineering-CMM appraisals at all company-wide systems engineering organizations. Minnich has a bachelor's of science degree from Lehigh University and a master's degree from the University of Southern California.

SECAT LLC
14742 Beach Blvd. #405
La Mirada, CA 90638
Phone: (714) 449-0423
Fax: (714) 449-0256
E-mail: iminnich@secat.com