Lewis Gray
Lewis Gray
This article describes ISO/IEC 12207. It shows how the standard solves some of the problems with DOD-STD-2167A that motivated the development of MIL-STD-498. It points out that ISO/IEC 12207 and MIL-STD-498 are similar in many important ways, and yet they are very different in their scope.
According to Dr. Raghu Singh, editor of ISO/IEC 12207, this international standard on software lifecycle processes [1] was created to establish a common international framework to acquire, supply, develop, operate, and maintain software. ISO/IEC 12207 was proposed in 1988 and published in August 1995. Singh also managed the Department of Defense project that developed MIL-STD-498 [2] during the same period. Therefore, it is not surprising that the two standards are alike in some important ways.
Maj. George A. Newberry [3] describes how MIL-STD-498 solves problems discovered when using DOD-STD-2167A [4]. First, I will list some of these problems, focusing on problems that especially affect the use of modern software development methods such as objectoriented development (OOD) and Rapid Application Development (RAD). Then, I will show how ISO/IEC 12207 solves some of these same problems with DOD-STD-2167A that motivated the development of MIL-STD-498.
This article updates and expands a section of an earlier paper presented at the Eighth Annual Software Technology Conference April 24, 1996 [5].
Why is There MIL-STD-498?
As early as 1989, there was deep concern among military software developers that DOD-STD-2167A was not well suited to use with object-oriented methods. This was one of several problems with DOD-STD-2167A that motivated developing a new standard. Jane Radatz [6] and Newberry [3] review some of the others. MIL-STD-498 was intended to correct these problems.
| Table 1: Issues with DOD-STD-2167A that are related to OOD and RAD | |
|---|---|
| Issue 1: | Perceived preference for waterfall development model. |
| Issue 2: | Compatability with incremental and evolutionary development models. |
| Issue 3: | Dependence on formal reviews and audits. |
| Issue 4: | Compatability with Ada and object-oriented methods. |
| Issue 5: | Distinction between requirements and design. |
| Issue 6: | Emphasis on preparing documents. |
| Issue 7: | Use of CASE Tools. |
Table 1 contains what I consider to be the most important issues with DOD-STD-2167A that were related to OOD and RAD.
The characteristics of MIL-STD-498 that result from solving these problems in the older standard make the new standard easily compatible with modern software development methods. Although ISO/IEC 12207 was developed independently of MIL-STD-498, it solves many of these same problems also. Let's look at the problems individually to see how.
Figure 1. Technical Contents of ISO/IEC 12207.
Figure 2. Views of Software Lifecycle Processes.
What Does ISO/IEC 12207 Do?
ISO/IEC 12207 describes the major component processes of a complete software lifecycle, their interfaces with one another, and the highlevel relations that govern their interactions. Figure 1 (from Singh [1]) summarizes its contents. Figure 2 (from Singh [1]) shows several different points of view of the processes. As Figure 2 suggests, acquirers and suppliers see the contract view of software development. The supply process begins when a contractual relationship to supply software is formed between an acquirer and a supplier. Depending upon the terms of the contract, the supply process may employ the development process to develop new software, the operation process to provide software operation services, or the maintenance process to repair and improve the software. The figure also shows that operators and users, developers and maintainers, and others have their own distinctive views of the lifecycle.
Figure 1 shows that organizations acquire software through projects that participate in contracts (usually) and carry out the activities and tasks associated with acquisition, supply, development, operation, and maintenance. In doing this, projects carry out processes for joint reviews, audits, quality assurance, and verification and validation. They also execute processes of documentation, configuration management, problem resolution, and tailoring from time to time. Meanwhile, the organization manages the project, providing the necessary development infrastructure and training for project members, and improving the project's software process. These component processes and their relationships in Figure 1 provide a highlevel summary of the technical content of ISO/IEC 12207.
| Project planning and oversight | Establishing a software development environment |
| System requirements analysis | System design |
| Software requirements analysis | Software design |
| Software implementation and unit testing | Unit integration and testing |
| CSCI qualification testing | CSCI/HWCI integration and testing |
| System qualification testing | Preparing for software use |
| Preparing for software transition | Software configuration management |
| Software product evaluation | Software quality assurance |
| Corrective action | Joint technical and management reviews |
| Risk management | Software management indicators |
| Security and privacy | Subcontractor management |
| Interface with software IV & V agents | Coordination with associate developers |
| Improvement of project processes | |
| Table 2. Major and Other Activities in MIL-STD-498. | |
|---|---|
To compare the scope of ISO/IEC 12207 to that of MIL-STD-498, compare Table 2 to Figure 2. The following major activities in MIL-STD-498 (Table 2) correspond to processes in ISO/IEC 12207 (Figure 2):
Also, there are activities in MIL-STD-498 that correspond to the documentation and improvement processes in ISO/IEC 12207. All of the following major activities in MIL-STD-498 correspond to only a single process in ISO/IEC 12207, the development process:
As the comparison shows, there is a striking difference between the scope of ISO/IEC 12207 and the scope of MIL-STD-498. There are no activities in MIL-STD-498 that correspond specifically to the acquisition process, the supply process, the operation process, the maintenance process, or the training process in ISO/IEC 12207. This is a tremendous difference between the two standards.
Another big difference between ISO/IEC 12207 and MIL-STD-498 is that the requirements in the international standard are at a much more general level than those in the military standard. In fact, ISO/IEC 12207 says in paragraph 1.5 that it "describes the architecture of the software lifecycle processes but does not specify the details of how to implement or perform the activities and tasks included in the processes." Here is an example: MIL-STD-498 and its DIDs contain 117 pages of engineering and data requirements related to MIL-STD-498 activities eight through 18 above that correspond to the development process in ISO/IEC 12207 the international standard contains less than seven pages of requirements on that process.
Unlike MIL-STD-498, ISO/IEC 12207 has no DIDs associated with it (yet). In most cases, the requirements in MIL-STD-498 are compatible with the corresponding requirements in ISO/IEC 12207, just more detailed. Like MIL-STD-498, ISO/IEC 12207 is not intended to be a do-it-yourself guide on how to develop software. It was written for trained, skilled software developers, software development managers, and software acquirers.
The "Waterfall" Bias
ISO/IEC 12207 responds to Issue 1 and Issue 2 (see Table 1), as does MIL-STD-498. However, the two standards in Table 1 do differ in their development activities. As with DOD-STD-2167A, the development process described in ISO/IEC 12207 consists of activities that bundle together tasks that would have their own separate activities if the standard were MIL-STD-498. The 12 engineering activities in ISO/IEC 12207 are listed below. In the list that follows, eval indicates a bundled product evaluation task, revs indicates a bundled joint review task, cm indicates a bundled software configuration management task, user docs indicates a bundled task from preparations for software use, and test indicates a bundled testing task:
Singh has described the ISO/IEC 12207 approach as an implementation of a plan, do, check, act (PDCA) cycle. The intent is that the output of an engineering task should be checked by the engineer before it becomes input to the next task. This checking should occur in addition to (not as a replacement for) product and process evaluations that are performed or supervised by software quality assurance. Singh argues that engineers often are the best judges of where high risks exist in their own engineering solutions. The PDCA approach was chosen with the intent of adding their evaluations to the other product and process evaluations required by earlier standards such as DOD-STD-2167A.
Despite its bundling together of tasks of different kinds, the activities of the ISO/IEC 12207 development process have the independence from one another that the activities in MIL-STD-498 have. As in MIL-STD-498, they are not ordered in a waterfall sequence and, in my opinion, there are no requirements in the international standard that dictate which of them must be executed first and which next. In fact, ISO/IEC 12207 says explicitly, in paragraph 5.3.1.1, in language similar to that in MIL-STD-498, "these activities and tasks may overlap or interact and may be performed iteratively or recursively."
In paragraph 1.5, ISO/IEC 12207 states that, "this international standard does not prescribe a specific lifecycle model or software development method." In language similar to that in MIL-STD-498, paragraph 5.3.1.1 states that unless the contract stipulates one, "the developer shall define or select a software lifecycle model appropriate to the scope, magnitude, and complexity of the project. The activities and tasks of the development process shall be selected and mapped onto the lifecycle model." The intent and effect of the language in the international standard are to provide flexibility in ordering activities, and to choose development models to avoid the waterfall bias of other standards.
Reviews and Audits
DOD-STD-2167A depends upon MIL-STD-1521B reviews as gates to exit from most of its major activities. For example, a system design review (SDR) closes system requirements analysis and design, and a software specification review (SSR) closes software requirements analysis (this is Issue 3). Unfortunately, the requirements in MIL-STD-1521B were not revised when DOD-STD-2167A superseded the earlier, less flexible DOD-STD-2167. Certain outofdate requirements in MIL-STD-1521B, such as paragraphs 30.1, 40.1, and 50.1.2 that link software development activities in waterfall relationships, were incorporated by reference into DOD-STD-2167A, with the result that they reinforced the links between DOD-STD-2167A's major activities and contributed to the waterfall bias of the earlier standard.
Unfortunately, MIL-STD-1521B reviews and audits are complicated, formal affairs that require elaborate preparation and completion. They are often called "dog and pony" shows, referring to both the tone of the meetings and to the frequent preoccupation of the meetings with superficial outward characteristics of the software products to be reviewed rather than with the complicated, subtle, engineering decisions that caused the products to be how they are. By contrast, MIL-STD-498 calls for smaller, more frequent, less elaborate joint reviews of interim software work products by developers and acquirers as the work products are developed. The acquirer and the developer must decide when such reviews are warranted.
By their nature, joint reviews of the type described in MIL-STD-498 are easier to schedule and carry out than the formal reviews defined by MIL-STD-1521B. This is a major advantage for some projects. RAD loses its reason for being when projects are slowed down or stopped for weeks by each formal review, which happens under the combination of DOD-STD-2167A and MIL-STD-1521B. Imagine trying to develop the equivalent of 110,000 lines of COBOL code in 13 weeks (according to James Martin's [7] RAD timetable) while completing system design review (SDR), software specification review (SSR), preliminary design review (PDR), critical design review (CDR), test readiness review (TRR), and two configuration audits (FCA, PCA) in accordance with MIL-STD-1521B. The MIL-STD-498 approach to joint reviews avoids this type of problem.
Like MIL-STD-498, ISO/IEC 12207 avoids "dog and pony" shows and defines two categories of joint reviews, joint technical reviews and joint management reviews. They are very similar to the reviews defined by the military standard. Audits are defined as a separate process in ISO/IEC 12207. Between them, joint technical reviews, joint management reviews, and audits, they accomplish everything that MIL-STD-498's review and audits accomplish.
Nonhierarchical Software Designs
In 1990, ACM SIGAda [8] documented many problems with the software design requirements in DOD-STD-2167A and its predecessor DOD-STD-2167. The problems were discovered first by Ada developers doing OOD (this is Issue 4). The core cause of the problems was that DOD-STD-2167A and its predecessor DOD-STD-2167 required software developers to decompose their CSCIs into computer software components and low-level units that were hierarchically related to one another. This position was a formal requirement in paragraph 4.2 of the earlier standard. In DOD-STD-2167A, it was understood through reference to the diagram in the standard that showed a CSCI decomposition (in DOD-STD-2167A the diagram is found in Figure 3). Figure 3 in this article shows the problem diagram, and indicates that the decomposition requirements in the DOD-STD-2167A caused a serious translation problem: regardless of how components resulting from the real work of software design are related to one another (for example, as distributed processes, or through data abstraction and object-oriented organization), the software design must be presented in a software design document (SDD) as though the components are related to one another as Figure 3 in the standard described. This forced developers to translate their design representations into a structure of components whose relationships resembled the figure.
Figure 3. Software Organizational Structure Translation
Problem.
Often, components of object-oriented designs are not hierarchically related to one another. Also, it often happens that classes participate in one set of relationships (classification relationships, shown by a class diagram) and modules in another (assembly relationships, shown in module diagrams). So no single hierarchical structure describes both. Object-oriented designs often have to be translated into a form consistent with the DOD-STD-2167A diagram. This step is time-consuming and error-prone. It often obscures the real engineering decisions that produced the design. The frequent need for translation caused several design methodologists (for example, Grady Booch at Rational) to develop mappings between the designs produced by their methods and the elements of DOD-STD-2167A's design diagram. Translation and mapping are wasted efforts because they are not needed by any of the categories of software development activities in DOD-STD-2167A, as I argued in Lewis Gray [10].
The problems have mostly gone away for new military software development because MIL-STD-498 dropped the offending requirements in the earlier standards. ISO/IEC 12207 also avoids the bad requirements in DOD-STD-2167 and DOD-STD-2167A. Although the approach to software design in ISO/IEC 12207 is somewhat different from that in MIL-STD-498, the effect on the problems related to Issue 4 is the same: they have gone away. The international standard requires developers to create software components and to record their design. The requirements in the standard are deliberately simple and brief. Good designers who know nothing about this requirement will already do this, regardless of what design method the designers use. The approach in ISO/IEC 12207 is compatible with any design method and any relationship between design elements, as is the approach in MIL-STD-498.
Requirements vs. Design
It has been common in software development practice to say that requirements are descriptions of what the software does; the design describes how the software does it. This distinction has always been somewhat confusing, and the confusion has caused rework and at least a few engineering change proposals that would not have been needed otherwise (this is Issue 5).
Unlike MIL-STD-498, ISO/IEC 12207 does not attempt to clarify the difference between requirements and design. Although some development methods, such as OOD and RAD, tend to blur together requirements analysis and design activities, and although it is critical to keep track of requirements in contractual situations, requirements can be well tracked by projects that use these methodologies under ISO/IEC 12207, although probably not as easily as under MIL-STD-498.
Documentation and CASE Tools
Many software developers and acquirers would agree that the typical types and quantity of documentation required by military software development projects are not right (this is Issue 6). Also, both developers and acquirers say that they want to automate more of the software development process, but both saw DOD-STD-2167A as a barrier because the standard emphasizes the production and delivery of many paper documents (Issue 7).
RAD projects use practices designed to achieve quick development times and high productivity through heavy use of sophisticated, integrated CASE tools. These projects are extremely vulnerable to excessive documentation requirements.
ISO/IEC 12207 responds to these concerns:
This is how ISO/IEC 12207 responds to these concerns: first, paragraph 1.5 of the standard states its intent not to describe data items when it says "this international standard is not intended to prescribe the name, format, or explicit content of the documentation to be produced." ISO/IEC 12207 has no DIDs. Since the standard has no DIDs, acquirers are less likely than with earlier standards like DOD-STD-2167A to accidentally incorporate excessive documentation requirements in contracts.
Second, ISO/IEC 12207 explains in paragraph 4.1.1.2 and paragraph 6.1 that, "the documentation process is a process for recording information produced by a life cycle process or activity." The information to be recorded includes that needed by "all concerned such as managers, engineers, and users ..." Implementation of the ISO/IEC 12207 development process, as described in paragraph 5.3.1, includes documenting (recording) the outputs of all of the activities and tasks of the process. Thus, the results of the engineering work must be recorded. This requirement is not in any way conditional upon whether paper documents or other documents must be delivered to the acquirer.
Third, paragraph 1.5 of ISO/IEC 12207 says that, "this international standard, however, does not imply that such documents be developed or packaged separately or combined in some fashion." Paragraph 6.1.3.1 says that, "production and distribution of documents may use paper, electronic, or other media." As I showed above, paragraphs 4.1.1.2 and 6.1 describe the documenting process simply as a process of recording information with no indication that the information must be recorded as a traditional document rather than as data in a CASE tool, for example. Nevertheless, the bulk of the documentation process requirements within paragraph 6.1 and its subparagraphs suggest that the authors had traditional documents in mind. For example, paragraph 6.1.2.1 says
"Each identified document shall be designed in accordance with applicable documentation standards for format, content description, page numbering, figure/table placement, proprietary/security marking, packaging, and other presentation items."
In my opinion, the requirements in ISO/IEC 12207 simply allow developers to use data in CASE tool storage or format to document their work but do not explicitly encourage this, as the requirements in MIL-STD-498 do.
Fourth, in paragraph 6.6, ISO/IEC 12207 defines the joint review process flexibly as "a process for evaluating the status and products of an activity of a project as appropriate. Joint reviews are at both the project management and technical levels and are held throughout the life of the contract." Here, the standard allows for reviewing the results of planning and engineering work in their natural, technical form.
ISO/IEC 12207's treatment of documentation and CASE tools is more suitable for most development methods than requirements in older standards like DOD-STD-2167A. ISO/IEC 12207 has no required deliverable data. ISO/IEC 12207 is a suitable standard for RAD where sophisticated integrated CASE tools play a critical role. In its untailored form, it poses less of a risk of creating accidental, excessive documentation requirements in contracts than does MIL-STD-498.
Conclusion
By early 1997, DoD will replace MIL-STD-498 with a commercial software standard now under joint development by the EIA and the IEEE, with Raghu Singh as IEEE co-chair. The planned commercial standard, tentatively labeled EIA/IEEE US 12207, Software Life Cycle Processes, at the time this article was submitted, will combine engineering and data requirements from MIL-STD-498 with additional requirements from ISO/IEC 12207 on software lifecycle processes like acquisition, operation, and maintenance that are not covered in the military standard. Drawing on its parent standards, US 12207 is planned to offer similar improvements over DOD-STD-2167A to those in MIL-STD-498 and ISO/IEC 12207.
About the Author
Lewis Gray is president of Ada PROS, Inc. He coaches software process improvement and consults on and teaches MIL-STD-498 and ISO/IEC 12207. He is a member of the planning committee that is developing EIA/IEEE US 12207, Software Life Cycle Processes, based on ISO/IEC 12207 to replace MIL-STD-498. He was a member of the joint EIA, NSIA, and AIA working group on the development of MIL-STD-498. He has led and participated in military software development and in process improvement efforts in large and small companies. Prior to founding Ada PROS, he held key management or technical positions at TRW, GTE, and INTELLI-MAC, Inc. Gray holds a doctorate in the philosophy of science from Indiana University.
Lewis Gray, Ph.D.
President, Ada PROS, Inc.
12224 Grassy Hill Court
Fairfax, VA 22033-2819
Voice: 703-591-5247
Fax: 703-591-5005
E-mail: adapros@aol.com
References
1. Singh, Raghu, "Introduction to International Standard ISO/IEC 12207: Software Life Cycle Processes," tutorial slides, Oct. 2, 1995.
2. U.S. Department of Defense, "MIL-STD-498, Software Development and Documentation," Naval Publications and Forms Center, Philadelphia, Dec. 5, 1994.
3. Newberry, Maj. George A., "Changes from DOD-STD-2167A to MIL-STD-498," Crosstalk, STSC, Hill Air Force Base, Utah, April 1995, pp. 4-7.
4. U.S. Department of Defense, "DOD-STD-2167A, Defense System Software Development," Naval Publications and Forms Center, Philadelphia, Feb. 29, 1988.
5. Gray, Lewis, "Using MIL-STD-498 and ISO/IEC 12207 for OOD and RAD," Proceedings of the Eighth Annual Software Technology Conference, Salt Lake City, Utah, April 1996.
6. Radatz, Jane, Myrna Olson, and Stuart Campbell, "MIL-STD-498," Crosstalk, STSC, Hill Air Force Base, Utah, February 1995, pp.2-5, 28.
7. Martin, James, Rapid Application Development, Macmillan, New York, 1991.
8. SIGAda (Special Interest Group on Ada), Gray, Lewis ed., "Implementing the DOD-STD-2167 and DOD-STD-2167A Software Organizational Structure in Ada," SIGAda Software Development Standards and Ada Working Group, Association for Computing Machinery, New York, August 1990.
9. Shaw, Mary and David Garlan, Software Architecture: Perspectives on an Emerging Discipline, Prentice Hall, Upper Saddle River, N.J., 1996.
10. Gray, Lewis, "No One Needs DOD-STD-2167A's CSCs and CSUs," Proceedings of the 1993 Washington Ada Symposium, Association for Computing Machinery, New York, June 1993, pp. 125-132.