Quantifying the Debate: Ada vs. C++

Donald J. Reifer

Software developers continue to debate the virtues of C++ vs. Ada. In an attempt to provide a fair assessment, this is one consultant's analysis of cost, reliability, and maintainability metrics for similar commercial and military systems. Although this assessment favors C and C++ in some areas, Ada holds up well as a choice for weapon systems and airborne and spaceborne avionics.

After I left government service, many people called me for an update to the hard data I used to periodically publish the cost, reliability, and maintainability of systems developed with Ada, C, C++, and other software engineering technologies [1, 2]. Because these requests have value, I have taken the time to analyze the data I continue to get from the many organizations I helped during the past decade to institute a metrics data collection program.

Quantifying the Experience

Many firms insert a post-mortem review into their process to make sense out of the data they have collected during a software development. At this review, facilitators do two things. First, they validate that the cost, defect, and other metrics data collected truly reflect what went on during the development. For example, cost figures are analyzed to determine whether either casual overtime or part-time labor was included in the raw data. Both are important because their absence could inflate the productivity norms used by the firm to estimate future jobs. Second, the facilitators capture the lessons learned. These are commonly captured at a team meeting by asking the following question: "If you had the job to do over again and a genie appeared who would grant you three wishes, what would you ask for?"

Firms correct the cost, defect, and other metrics data they have gathered based upon the results of the post-mortem review. They then normalize the data so that it adheres to the conventions they have set and use it to develop the norms they use to assess performance of other jobs, validate bids, and calibrate their cost models. For example, size as measured by lines of code or function points needs to be adjusted to compensate for differences in languages or development methodologies. Standards are used for this purpose.

Looking at the Hard Numbers and Indicators

The data I will share with you in the following tables have been normalized according to a published set of conventions [3]:

Table 1. Summarizing the Database (1996) - highlights some of the characteristics of the 190 software development projects upon which the findings of this article are based. It is important to note that none of the projects in the database are over three years old.

Table 1. Summarizing the Database (1996).

Table 2. Dollar Cost per Delivered Source Line of Code (1995) - summarizes the average costs in terms of dollars and lines of code by domain for software developed in Ada83, Ada95, C, C++, on 3GL like COBOL or FORTRAN, and for the domain norm. The norm in this case represents over 1,500 projects, none of which is over seven years old.

Table 2. Dollar Cost per Delivered Source Line of Code (1995).

Table 3. Post-delivery Errors per KSLOC (1996) - shows the reliability in terms of errors and thousand lines of code for different languages and the norm for software after it was delivered to the customer (system test, end user, etc.).

Table 3. Post-Delivery Errors per KSLOC (Thousand Source Lines of Code) (1996).

Table 4. Reliability: Weeks Until Next Major Repair Incident (1996) - identifies the maintainability in terms of weeks until a major error was reported for different languages and the norm after it was delivered to the customer.

Table 4. Reliability: Weeks Until Next Major Repair Incident (1996).
Note: Bigger numbers are better.

In addition to the hard data, it may be instructive to look at the differences in strategy being pursued between the commercial and the military sectors. Both are in the midst of a paradigm change. However, as Table 5 illustrates, the military sector seems to be moving from one of a kind to commercial-off-the-shelf (COTS)-based systems while the commercial sector is pushing ahead with generative strategies. Perhaps the Perry memorandum, which called for the use of COTS as the department's highest priority, is the root cause of this tendency.

Table 5. Investigating the Paradigm Shift (1996).

Each of the strategies called out in Table 5 is briefly explained in Figure 1. Please note that the table identifies primary strategies for a domain. Recognize that most strategies are compatible, and many can be used in conjunction with one another. For example, many product line strategies are architecture-centric and focus on the use of COTS. However, they just as easily could emphasize generative approaches as they attempt to upgrade a migration system for use in a client-server environment without any architectural factors being considered.

Architecture-centric strategy - the system is architected to satisfy user needs and to take advantage of reuse opportunities upfront. The architecture is then populated with the parts needed to satisfy the needs of the family of systems or product line of interest.

COTS-based strategy - systems are designed to exploit the use of commercial-off-the-shelf components to the maximum degree possible. Users may have to take a 90 percent solution to conserve time and effort involved in system development.

Generative strategy - applications are primarily generated automatically from high-level specifications for the system.

Unique systems - each system is one of a kind. Opportunities for reuse and prototyping may be taken advantage of in an opportunistic manner.

Product line strategy - families of systems are designed to share assets within a domain or line of business of interest. Such designs could be based on architectural, management, or market considerations.

Rapid development strategy - systems are fielded incrementally with each release adding features and fixing problems experienced in the previous version. The goal is to get the basic product to market quickly. Then, efforts may be mounted to optimize it.

Spiral development strategy - systems are built in a risk adverse manner in a series of increments called spirals [5].

Figure 1. Alternative Software Development Paradigms.

Some will say that the Department of Defense (DoD) is already pursuing reuse that is domain-specific, architecture-centric, and systematic. Although such a vision exists [5], such movement is happening only in isolated cases. As suggested by both the General Accounting Office [6] and the Defense Science Board [7], I and others are hopeful that the department will take greater advantage of the opportunities for architecture-centric reuse, and fund the software reuse initiative to pursue realization of their strategy [8] and that of the services and agencies.

Understanding What the Data Means

The tendencies of the data are revealing. They show systems continue to get bigger and more complex. They show that software costs are decreasing at about 6 to 8 percent annually, and schedules are getting longer. Finally, they show that quality levels remain stable even as more and more COTS and reusable software are employed to build the products.

The most interesting observations relate to comparisons between like military and commercial developments. The data suggests that costs and schedules for commercial projects are about 30 to 50 percent less than those for military systems. This difference can be explained by the amount of red tape associated with DoD's acquisition regulations and procurement practices. Unfortunately, when you look at the trends associated with the data, this gap has not narrowed during the past five years. This is surprising because DoD has been pushing hard to cut costs by streamlining acquisitions and adopting best commercial practices. Based upon the data collected to date, the results of this push have not yet positively impacted the bottom line.

Conclusions

The cost, quality, and other metrics data displayed in Tables 1 through 4 permit me to make the following three conclusions, none of which will make me popular with my many friends in the Ada community:

Use of Ada, C, and C++ is preferable. The hard data provides evidence that the use of either Ada or C/C++ results in cost and quality advantages over 3GL's.

The jury is still out on Ada95. There are still too few users to determine if Ada95 will deliver its promised benefits. Those who use the language to develop products complain that poor compilers and the lack of tools and bindings make their jobs difficult.

C and C++ have become more cost-effective. The costs and quality advantages Ada has enjoyed over C and C++ have almost disappeared as these languages have become more widely used. The only application domain where Ada currently seems to hold a cost advantage is airborne and spaceborne avionics.

The softer data shown in Table 5 and defined in Figure 1 permits me to add the following two conclusions about current paradigm shifts:

Use of COTS and architecture are becoming the in things. Many organizations are making a paradigm shift aimed at developing an architectural infrastructure that permits engineers to take better advantage of COTS and third party software.

Generative strategies are becoming popular within the commercial community. Use of application generators (especially those that rely on some form of inference engine) is increasing as commercial systems become architected to be more standards based.

These results should not surprise anyone. The commercial community has been pursuing a COTS-based strategy for years. Most commercial firms do not develop or sustain payroll, logistics, or other applications they can buy off the shelf. Instead, they focus their attention on systems that permit them to lever their core competencies to gain a competitive advantage (mining firms develop mining software, oil firms build exploration systems, etc.).

Summary

I have shared with you the latest data I have been able to assemble relative to the continuing Ada vs. C++ controversy. I have tried to be fair in my assessment and point to hard data instead of opinions to resolve the debate over which language is better. The data suggests that both languages can be used with advantage. Ada, with its tasking and other real-time features, can provide needed support to those who develop software for weapons systems. C and C++ can be used when programmers need to interface to open environments or take a product to market quickly.

Ada95 still holds great promise. Perhaps I will be able to show its merits the next time I publish business case statistics. Unfortunately, such data currently does not exist.

References
1. Reifer, Donald J., "Ada - The Business Case," presentation at the Washington Ada Symposium, available from author, June 30, 1993.
2. Reifer, Donald J., "Ada - A True Success Story," in Defense Systems Management College Management of Software Acquisition Course Notes, March 15, 1995.
3. Reifer, Donald J., Asset-R and SoftCost-R Reference Manual, available from Resources Calculations, Inc., Englewood, Colo., 1995.
4. Defense Information Systems Agency, DoD Software Reuse Initiative Vision and Strategy, Sept. 30, 1995.
5. Boehm, Barry W., "A Spiral Model of Software Development and Enhancement," Computer, IEEE Computer Society, May 1988, pp. 61-72.
6. Department of Defense, DoD Software Reuse Initiative, Report to Congress, July 15, 1992.
7. Office of the Under Secretary of Defense for Acquisition and Technology, Report of the Defense Science Board Task Force on Acquiring Defense Software Commercially, June 1994.
8. Defense Information Systems Agency, DoD Software Reuse Initiative Strategic Plan, June 16, 1995.

About the Author

Donald J. Reifer is a consultant who specializes in change management with Reifer Consultants, Inc. in Torrance, Calif. He has almost 30 years experience in managing software and putting software technology to work in Fortune 500 firms. From 1993 to 1995, he was the chief of the Ada Joint Program Office, technical adviser to the Center for Software, and chief of the DoD Software Reuse Initiative under an Intergovernmental Personnel Act assignment with the Defense Information Systems Agency. Currently, Reifer helps several international clients insert product line and software architecture concepts into their software operations.

Donald J. Reifer, President
Reifer Consultants, Inc.
P.O. Box 4046 Torrance, CA 90510
E-mail:d.reifer@ieee.org
Voice and Fax: 310-530-4493