The F/A-18 Hornet is the Navy's premier strike fighter, which now forms the core of the Navy's air warfare capability. As older aircraft are phased out of inventory, and the newest variant the F/A-18E/F is phased into the fleet, combat aircraft on the Navy's carrier decks will consist almost exclusively of F/A-18s. It is truly the heart of naval carrier aviation. The F/A-18 also serves as the primary fighter with seven U.S. military allies.

Success in today's air combat arena is a function of many variables. One of the most important is aircraft mission systems and their interface with the aircrew, especially in an era of exponential improvements in digital technology. The F/A-18 Advanced Weapons Laboratory (AWL) delivers these improved warfighting capabilities to the fleet.

As a full life-cycle activity, the F/A-18 AWL provides mission-system-engineering support for F/A-18E/F, as well as lifecycle support for out-of-production F/A-18A/B/C/D aircraft. The AWL coordinates F/A-18 system upgrades and enhancements and provides systems engineering for F/A-18 hardware and software. It accomplishes every aspect of the life cycle of the system configuration sets (SCS), including the software design for the mission computers and the stores management system. For the E/F aircraft, the AWL acts as system engineers and performs test activity; their teammate The Boeing Corporation is the design agent. Additionally the AWL manages a wide range of avionics and weapon systems developments, weapons integration, and foreign military products.

The F/A-18 AWL develops its own simulation laboratories, test equipment, and flight instrumentation; it generates and manages aircraft modification proposals and flight clearances. In its six integration and simulation laboratories, the AWL performs detailed subsystem and integration tests. The F/A-18 AWL and their Boeing teammates are Software Engineering Institute Capability Maturity Model^® (CMM^®) Level 4 software facilities. The AWL is well on its way to Level 5.

"The developers' transition to CMM Level 4 has resulted in reduced rework and reduced costs of test points," says Gary Kessler, Naval Air System Command representative. "The fleet is ecstatic."

Functioning as part of a greater F/A-18 Integrated Product Team (IPT), the people of the F/A-18 AWL are a Navy/industry team whose major contractors are The Boeing Corporation, Raytheon, and many other prime and support contractors. From technical leadership to business and financial management, they provide progressive, experienced management expertise for all levels of programs across a wide variety of disciplines.

Scope of the Project
During the top five contest award period of January 2000 to June 2001, the AWL delivered to the operational testers (VX-9) a major software block upgrade called the 15C SCS. This was approximately a $120-million effort that incorporated more than one hundred requirements. Here are just a few of the major products implemented in the SCS: the Joint StandOff Weapon, the AIM-9X Sidewinder, the Joint Helmet Mounted Cueing System, the Multifunctional Information Distribution System, the Digital Communication System, and the requirements from six foreign military sales customers.

"The 15C SCS effort was long and complex," says Boeing Block Captain Doug Garrette. The project began in the first quarter of 1997. The initial plan consisted of three builds with 61 USN statements of requirements (SORs) and 14 Foreign Military Sales SORs, he says. It grew to four builds and picked up 59 impact statements (additional requirements).

"The SCS involved the integration of three new weapons, five new avionics systems and a new aircraft configuration (A+)," says Garrette. Each of these programs was driven by their own schedules and needs, he adds. "15C had to be flexible and react to the dependencies that were brought on by these parallel activities. It was through the dedicated effort of the combined USN/Boeing team that commitments were met."

Watts S. Humphrey, a Top 5 judge noted the vast scope of the project. "While the technology appears to be relatively standard, at least for the set of best projects, the size, complexity, and number of systems involved does represent a significant technical challenge in itself."

In addition, the team was not co-located but came from different organizations, says Barry Douglas, Advanced Weapons Laboratory, IPT lead. "But that didn't matter," he says. "The team pulled together from the beginning, overcame development difficulties posed by their separation, and produced a successful product.

The aircraft has more than 10 million words of code in more than 40 different processors. Each aircraft type has two distinct configurations. The major differences include the stores management computer (Q-9 or AYK-22), multiplex bus architectures (either five or six), radars (APG-65 or APG-73), two variants of the AYK-14 mission computer, and various other minor differences. The airframes different processors are programmed in eight variants of assembly language, and in Ada, C, PL/M-86, and Jovial. The software development environment also uses Fortran, Ada, and C.

The majority of the effort was in the two mission computers, stores management set, and radar. The software development environment has more than 4 million source lines of code (SLOC) in unique software. The documentation contained the complete set of logistics elements that include the following: aircrew publications, maintenance publications, training, trainer updates, technical directives, and mission planning module software.

Methods to Ensure Quality
The mission computer software team's effort was larger and more complex than most members had ever experienced, notes Kim Brestal, Boeing software lead. "The task included implementation of an extraordinary number of requirements representing new weapons, new aircraft systems and a new aircraft configuration.

"The biggest challenge, by far, was providing for efficient use of critical mission computer resources to allow for successful implementation of all the requirements," says Brestal. "An MC resource team was formed to devise and implement risk mitigation plans for each affected resource."

Truly this project was large and complex agrees Capers Jones, a Top 5 judge. "The combination of low rates of delivered defects and high levels of customer satisfaction indicates this project was very well planned and managed." Jones cites the AWL's processes as a key to their success. "The project was produced by a SEI CMM Level 4 organization, and demonstrates the value of the higher CMM levels."

To achieve this quality goal, the AWL team performed the following:

• Achieved a CMM Level 4 and aggressively started moving to Level 5.
• Used the Capability Maturity Model^®- Integrated^SM to assess organizational maturity and process area capability. Established priorities for improvement and methods to implement these improvements.
• Published, updated, and distributed a strategic plan that defines basic core beliefs, visions, and mission.
• Tested jointly with the Operational T&E Squadron throughout the verification phase of 15C. This gave them an early look at the product and gave the AWL earlier insight into operational problems in the product.
• Published an F/A-18 AWL Management and Systems Engineering Process Manual to systematically identify and apply leverage to areas of weakness and expand on what they do right.
• Maintained and improved its system-configuration review board process to obtain a very solid, well thought out, and adequately funded set of requirements.
• Improved on and used a comprehensive set of metrics. An example of the numerous metrics used is the indicator used to indicate software maturity level. At 0.12 software anomaly reports per test hour, the software is ready for operational test.

• Reduced cycle time from 56 months to 38 months.
• Reduced schedule slips from 12 months to on time.
• Decreased rework rate from 20:1 to 3:1.
• Decreased regression testing from 70 percent to 20 percent.
• Decreased redundant testing from 100 percent to 10 percent.
• Improved test efficiency from 0.42 to 1.6 test points closed per hour of test time.

• Defect density was very low, 3.8 defects per KSLOC -- down from 13.5.
• Productivity in the design phase was 3.45 man-hours per SLOC -- down from 15.7.
• Design phase cost was $200 per SLOC -- down from $725.
• Life-cycle cost was $400 per SLOC -- down from $1,170.
• The number of test flights was 0.6 flights per KSLOC -- down from 3.1.

"This is a very large, real-time operational system that has made significant improvement in cost, schedule, and quality," says Jack Ferguson, a Top 5 judge.

Accomplishments Are Applauded
For software of this size and complexity, the AWL feels this is one of the top software projects in the government for total life-cycle costs, quality, schedule, and performance. It says this is especially commendable considering this is "real time" processing in an extremely mission critical system that pushes the technology envelope, and that requires absolute safety of flight.

If the high cost of flight test vs. the commercial process of free "beta testing" is factored out, this software is a bargain in any commercial market, says Douglas. "The overall cost and quality statistics for this level of effort are truly outstanding, but the improvement during the past 10 years is truly phenomenal."


The F/A-18 Advanced Weapons Lab located at China Lake, Calif
(Click on image above to show full-size version in pop-up window.)
Top Photo: A VX-9 F/A-18 Aircraft Over the China Lake, Calif., range.
Bottom Photo: The F/A-18 Advanced Weapons Lab located at China Lake, Calif.

^SMCapability Maturity Model-Integrated and CMMI are service marks of Carnegie Mellon University.