(Editor's Note: Copyright © M.J.Hillelsohn, 1996.)
Software engineering literature reports improvement in the definition of the sound software development processes and the adoption of tools and practices that support successful software development projects. This improvement is true for a segment of the maturing software development industry, but there are still many software developers and customers who are operating in a less structured development environment.
In the course of observing and working with several software development contracts, primarily with government customers and private industry contractors, some observations seem consistent across the contracts. Contracts that started out with everyone excited about the prospect of developing a significant piece of application software ended up with finger pointing and excuses. Invariably, things happen during the contract that require decisions that have long-term implications for the success of the effort. If the decisions are made with appropriate considerations in mind, the implications for the end product are more likely to be positive. The following considerations are proposed as a positive influence on the quality of contract activities, products, and services.
Everything Cannot Be Held Constant
You can have either fixed quality, schedule, or cost but not all three. In this context, the quality dimension that is most affected is the amount of functionality in a product or scope of a service. Schedule is time to complete a task or phase. Cost includes both fiscal and nonmonetary resources such as personnel, facilities, and technology. Most projects start optimistically and amicably. A contract for a specified dollar amount, to be expended over a defined time frame, for a useful end product is awarded. The customer has asked for everything they thought they wanted, and the developer just promised to deliver the product on time and within budget. Everyone is euphoric. Once the contract is underway, however, things happen: resources get used in unanticipated ways, internal dates get slipped for valid reasons, and the product gets better defined as it moves from theory to reality. In a well-managed project, the customer and developer are communicating about these occurrences, and both realize, at the time, that they are necessary and ultimately will result in a better product. But then, a major contractual milestone arrives, and the customer has expectations, based on the developer's proposal, about what will be delivered on that milestone and how much it will cost to make it happen. Invariably, as things occurred, the contract had not been modified and trade-offs had not been extensively explored. This situation is a recipe for the customer-developer relationship taking an extreme turn toward the adversarial.
The customer has to prioritize the three variables of product quality, cost, and schedule for the developer so that appropriate trade-off analysis can take place. With the analysis in hand, the customer can make an informed decision about the impact on quality, cost, and schedule of an occurrence. The decision may not be the same for every aspect of the product and at every time in the life of the project. Some aspects of the product may be so critical to mission success that they cannot be compromised, while other aspects may be more fault tolerant and can be added in later releases to keep the project on schedule or within budget. Funding for the fiscal year may be exhausted, and the remaining funds must be used to get a scaled-down version of the product operable, rather than the optimal version, to meet real budget constraints. Other aspects of the customer operation may be scheduled to start based upon the delivery of the product on a specific date, and if they cannot start, bad things would happen to the customer's business, so the product may have to be modified and more resources expended to meet the schedule. These and other situations are realistic reasons for the customer to choose either product quality, cost, or schedule at a point in time in the project as the most important compliance factor for the developer. The customer should not be afraid to ask the developer questions such as
This approach will lead to far more realistic estimates than telling the developer what product they will have to deliver at a specified time for a limited budget. The underlying assumption for holding everything constant is that the developer is nothing more than a pair of hands and brings no expertise or experience to the program. If the developer has an established metrics program, the estimates for producing a product or delivering a service will be more reliable since they are based on data rather than intuition. It is the nature of humans and projects that, although they try, they cannot deliver well on all three dimensions, if all three are constrained. Consequently, they do not deliver well on product quality, cost, or schedule.
Drop-Dead Dates Are Usually Deadly
The earlier in the development process that the dates were generated, the more likely they will be deadly. In major software procurements, the schedule is often generated during the proposal-writing activity where the developer is likely to be optimistic in schedule estimation. If the proposer is not optimistic, there is a good chance the contract will be won by the firm that proposed a shorter schedule. Often, the end dates for the project are determined years in advance when the procurement is first scoped by the customer. The end dates then become part of the solicitation, and contractors shoehorn their proposed schedule into the solicitation's dates to be compliant. If they are not compliant with the solicitation schedule, they cannot win the job. And if the proposal is compliant, they often cannot perform the task within the time frame. Extended work weeks and fragile, unrealistic assumptions are two of the techniques for writing proposed schedules that will surely slip. Those companies that maintain a database of level of effort required to produce some unit of software (lines of code, function points) can make more realistic estimates, but they are still faced with solicitation compliance.
Most of us can only conceptualize and not actualize the scope of a major software effort. We can, however, project more accurately when we limit the scope of the software product. Further, after each phase of a systematic approach, we can make better estimates of what it takes to accomplish the next phase. If we translate this situation into contractual terms, the best way to contract for software development, for both the customer and the developer, is through iterative schedules in a task order environment. It also may be advantageous to both parties to use cost-plus during early stages and fixed price task orders later when the product is better defined.
If Interim Dates Slide the Impact on the Final Delivery Must Be Assessed
Fixed end dates, especially on long projects, lead to frantic activity at the end of the project for tasks such as testing, training, and user documentation. When there were little hiccups at the beginning of the project, both developer and customer agreed that some early deliverable dates could slip, and the time would be made up later. They forgot that the deliverables are not just a contractual item; they are needed outputs of parts of an overall development process. The development process consists of activities linked together by dependencies on analysis and outputs from earlier activities. Most activities in a software development process are on the critical path--an activity later in the process can neither commence nor be completed until earlier work is satisfactorily completed.
Activities may have one of three relationships to each other. They may be dependent, independent, or related.
When a project gets into schedule problems, and all of a sudden those dates that were so far in the future are occurring next month, there is a tendency to ignore the true relationships among tasks and redefine them as independent, i.e., "You can start to code before the design is complete" and "Go ahead and write the user manual from the specifications; that's how the system will look to the user." The results are poor quality products that require extensive rework. The old adage that "there is never time to do it right the first time, but always time to redo it" seems to be especially targeted for software development projects. But projects cannot go on forever. There have to be firm delivery dates or the products will be continuously tweaked and reworked and never be fielded.
Program management tools can assist project planners on both the developer and the customer staffs to analyze the consequences of early schedule slippage on final delivery dates. These tools generally allow management to do "what-if" scenarios to assess the effect of slippage, adding more resources, reducing scope, and manipulating other project variables. The relationships that are established between project activities and the duration of each activity should be held constant as the effects are studied. For example, the original estimate for training development was that it would take 20 hours of development time for each hour of instruction. We are now at a point in the project when we are considering the training, but the application coding has taken longer than originally anticipated, so the training development window is smaller. It then is not reasonable to reduce the development ratio to 10-to-1, but it may be reasonable to add another instructional designer at an appropriate time in the analysis and design phase.
Project management must constantly be aware of the affect of interim decisions on end dates so that schedule crises can be averted or mitigated. Nothing is free, and to repeatedly push project staff to meet dates that are now unreasonable because of earlier slippage only results in a demotivated staff and poor products. There are times in any project when everybody has to work especially hard to meet a deliverable date because some unforeseen events created a crisis, but if this is the exception rather than the rule, the project staff's spirit will generally rise to the occasion and accomplish the task.
Do Not Rely on Heroes
People who do the best work in the company tend to be the people with the most work to accomplish. The reward for doing a good job is getting more work dumped on you, while the people who are not so reliable often are underutilized. Management relies on the heroes. There are likely to be multiple outcomes, few of them are good for the project, when individuals are overloaded with work. In the best of these scenarios, the hero is promoted to supervise the poorer performers. Sometimes, the good work habits rub off, and the poor performers improve. But most often, the project has lost a valuable technical resource and now has a novice manager in a critical position. Even heroes, when faced with overwhelming workloads, tend to shortcut sound software development practices such as commenting and documenting code. They are too busy for design reviews and walk-throughs, and nobody else is qualified to inspect their code. The result is an overworked, frustrated employee who will probably seek a better working environment elsewhere. When the hero leaves the company, the project is in a lurch and requires extensive time and effort to overcome the sudden departure.
Those projects that use well-defined software development processes implemented by interdisciplinary teams can avoid the disruption caused by the loss of key personnel. In many instances, the use of the development team concept can preclude the overwork and stressful situation that leads to attrition. Heroes may be the team lead, if their skills and temperament are appropriate, thus acquiring leadership skills in a focused, cooperative environment. A properly amassed team will provide different perspectives on the final product and avoid the myopia that can occur with a single developer. Also, because the entire team is responsible for the end product and works together throughout the development lifecycle, there is less finger pointing and more cooperation. To whom is the programmer going to hand off the product for acceptance testing? Another member of the team? No one on the team can give up ownership of the product until the team as a whole has completed its task. In the process of working together, cross-training, regarding the application, naturally occurs. Thus, people's jobs are more interesting, and attrition is reduced. And when people leave, others are ready to fill the vacancy.
Generate Designs Before Starting Code
"Stop producing all this paper and start doing some real programming." These words are uttered by program managers at design reviews when they are panicked and have no confidence in the value of a systematic approach to software development. Time spent in the front end of the lifecycle has the biggest payoff for the development effort, since errors found prior to code are the easiest and cheapest to fix.
All systematic approaches to solving any problem have five basic steps: analyze, design, develop, evaluate, and implement. Different lifecycles and disciplines may use different terminology, but regardless of what they are called, they all basically follow these five steps. When a project is planned, the amount of time scheduled for each step is conjecture. Development usually has the preponderance of the allocated time in the typical software project. However, most of the problems that arise in projects seem to be the result of inadequate time spent in analysis and design. If the assumptions made during product conceptualization, i.e., in the proposal, are not updated based on detailed analysis of the user, working environment, external factors, or operational and technical realities, when customers takes delivery of the product, they reject it because it will not work appropriately in their environment. Juran's definition of quality as "fitness for use" by the customer must be met if the product is to be accepted. Detailed front-end analysis and continuous customer interaction increases the probability of success. The "rush to code" ensures that problems will occur in the final product that will require returning to the analysis and design stages to rectify the situation.
Successful analysis is not an armchair activity. Analysts must get out of their offices and have regular contact with practitioners. The development team is often prone to technical arrogance, where they have so much ego involvement with the product that they do not want to have input from someone who doesn't understand the new product. Meanwhile, most products are evolutionary, and there are people in the field who have experience with the process being automated and can provide valuable insights into the workings of the application. Early and ongoing involvement by end users also gives the development team some ownership of the product from its inception. When they are in their workplace they will talk about the product, even during the analysis phase, and become the local authority among their peers. As stakeholders in the product, they will make every effort to ensure the success of the product when it is fielded in their workplace. In fielding an Automated Instructional Development System, during an unsuccessful implementation, the developer's staff was on site to support and operate the system. At a later successful implementation of the identical system, the developer stayed across the street in a hotel while the end user's staff operated the system. The primary difference between the two implementations was the level of user involvement during all five phases of the development lifecycle. The involvement of end users, as appropriate, throughout the development process is a key element in transitioning to the new system during the operational phase of the project.
Use the Office Walls to Communicate
Communication is one of the key characteristics of successful organizations. You don't know from where the next idea will come, and most ideas are reactions and synthesis of other ideas. Secrecy and intrigue belong in spy novels and covert operations, not in most software development activities. Even in the most classified projects, most project participants have a need to know design options so they can make valuable contributions to the success of the development effort. There are different perspectives on the product from different people in different parts of the organization. They can only have input if it is actively solicited.
Input during analysis and design is called suggestions. Input after test and evaluation is called criticism (constructive or otherwise). Suggestions usually are appreciated and can be fostered on a project by giving people the opportunity to make them. One technique is to post design diagrams, storyboards, and other representations on the walls in public areas of the office, and encourage people to provide suggestions directly on the posted data. This also gives the customer an opportunity to see the product evolve when they visit the facility and have input during early stages. If it is not feasible to hang the data on the walls, other techniques such as expanded, frequent design review meetings and project electronic bulletin boards can provide opportunities for communication. It is important that the project team solicits information during analysis and does not wait until the end to find out if they built the right thing in the best way.
Technical support activities such as product assurance, testing, training, customer support, documentation, and others should be involved in the analysis and design stages to anticipate potential implementation problems and design the solutions into the product. For example, quality assurance (QA) organizations have learned that they cannot inspect quality into products at the end of the development cycle. To be successful, QA cannot be the quality police, but rather facilitators who raise concerns about quality characteristics of the end product during the early stages of the lifecycle. Other organizational elements need to have similar early input to create a responsive and useful software product.
Know When You Are Finished
We all have stories about the never ending software product or project: It is 90 percent complete; only one more enhancement to add and it will be perfect; the user would like one less screen in the interface; the documentation will begin tomorrow. Apparently, small changes that seem to go on forever and sometimes have major impact on quality, cost, and schedule are regularly undergoing development and test. There is always a point where we have to stop tweaking the product and deliver this version. The key to knowing when to deliver is to define the end product at the beginning of the effort and continuously monitor the team's focus on the goal.
All systems approaches start with an analysis phase. The Shewart cycle starts with planning. One of the first things to be done during analysis is to define the end product of the task. The list of requirements from the solicitation is often not sufficient to define the form, fit, and function of the end product. Techniques such as Joint Application Development, storyboarding, and prototyping facilitate the definition of the dimensions of the end product. This process must be highly interactive between the customer and the developer. If the customer expects the end product to be a luxury car but only paid for an economy model, this is the time to discover and resolve the discrepancy. The earlier it is resolved, the less wasted effort and rework will occur during design and development phases.
As the product is developed, the developers have more ideas on how to improve its look and function. As the customer tries out prototypes and talks with developers and field users, new capabilities and useful features come to light. Requirements that were interpreted one way during analysis are reinterpreted as more people are exposed to the immature product. Constant formal, e.g., lifecycle reviews, walk-throughs, and informal communication between customer and developer is the only way to monitor the product evolution process and maintain a consistent image of the end product.
Quality, cost, and schedule ramifications of changes in the end product definition must be explored and resolved. Changes in the definition will occur. The "I know it when I see it" answer to "Is this the right thing?" is a real answer. Even when we define standards and product characteristics in the beginning of an activity, changes will take place. Successful projects manage the change thereby reducing the risk of never being done with the product. In the contracting world, technical people must involve the contracting officers whenever definition changes occur so that the developer's legal obligations stay current. Everyone on the project has to know the up-to-date definition of the end product, so they know when it is certifiably ready to be delivered, without any more bells and whistles.
Contractual Requirements Are Open to Interpretation
The solicitation and proposal become the contractual source documents for project performance (solicitation usually has precedence). A good proposal has addressed all the "shalls" in the solicitation, and the customer acknowledges the satisfactory response by awarding the job. If contract award and periodic program management reviews are the only times the developer and customer interact, the project is doomed to failure as divergent views of the real program requirements evolve. The contractual requirements are a starting point for ongoing communication between developer and customer. If the customer and user are not the same entity, the user must also be included in defining what the contractual requirements mean in an operational setting. All parties must work together to decompose them into real performance specifications for software engineers to use as a basis for product design and development.
The factors that affect the likelihood of satisfactorily meeting the contractual requirements should be assessed. When the risk of failure has been mitigated to an acceptable level, work should proceed on meeting the requirement(s). Performing risk assessment at the requirements level allows areas of the project to proceed while other areas may be delayed because risks related to a specific requirement are too great. Risks in a software development project usually involve availability of appropriate resources, technology, or information and operational concerns. Anticipating threats and vulnerabilities and managing risk mitigation strategies is critical to successful project completion. If a task order approach to contract management is used, as requirements are clarified and risks associated with them are reduced, task orders can be written to proceed with the development associated with the requirement(s).
Upper Management Should Lead
It is essential for corporate management to lead and not manage all the minutiae associated with a software project. In putting out the fires that occur on a regular basis in a development environment, it is easy to lose sight of the big picture, and it is the responsibility of corporate management to maintain that perspective. Companies need to establish rewards and organizational incentives for teams that excel. High-quality outputs are, to a large extent, the result of a corporate attitude. Everybody reports to someone. People want to be responsive to the person to whom they report and from whom they get recognition and rewards. Nobody likes to be dumped on, and everyone likes praise for a job well done. People respond to the things their bosses reward and ignore those things that are not priorities for their supervisors. If performance reviews emphasize first-time quality and reduction of rework, people will strive to accomplish those goals. These attitudes and practices have to permeate the organizational culture, especially upper management. Upper management's reward structure needs to reflect corporate values.
Leadership is not an abdication of management responsibility for the project, but is knowing when to be intimately involved and when to let project management deal with day-to-day issues. The output of program management tools are one device by which upper management monitors a project. Sometimes we forget, however, that a tool works for the tool user, and not vice versa. One of the strengths of computer-based tools is the ease with which data can be revised to reflect new realities on the project. The project management tool output should be revisited when each new task order is issued to assess impact and update schedules and resource allocations.
Companies Cannot Stay in Business if They Lose Money
As software development projects drag on, cost overruns occur. Money for people's salaries and other resources has to come from somewhere. If the cost overrun is covered by money from profits, the company starts to lose one of its reasons for existing. A company that has shareholders must make more profit by doing its business rather than putting its money in the bank and getting interest, or the shareholders will lose patience with corporate management. If the overrun is covered by overhead charges, the company's overhead rate goes up, and it will not be cost competitive on future solicitations. In a fixed price contract, the risk is greatest for the developer, and in a cost plus contract, the customer assumes some of the immediate risk. Even in a cost plus environment, although some of the immediate overruns are covered by the customer, the developer jeopardizes its chances for future business by getting a reputation for performance history of overruns.
Underbidding jobs is a common way companies get themselves into a precarious fiscal position. Companies have to avoid the situation where the good news is that they won the job, and the bad news is now they have to do the work for what they bid. Jobs may be underbid by understaffing, by proposing too many junior (lower priced) and not enough senior (higher paid) staff or not enough overall staff, estimating an excessive amount of uncompensated overtime, reducing task durations to unworkable levels, creating unrealistic rate structures, trying to skimp on technology upgrades and needed resources, ignoring cycle times and activity relationships during scheduling, and seat-of-the-pants estimates for what it takes to do the job. "Buying in" to a new customer or business area to build a new relationship or capability may, on occasion, be a sound strategic decision for a business, but it has to be done carefully with risk mitigation tactics in place. Most companies that bet on the outcome do not make up their losses on follow-on work or modifications to the contract because they are locked into the fallacious rate structure they bid to win the job. Also, the operational lifecycle of applications software is getting short so that a developer cannot build a product and expect to have a long-term relationship with the customer without being able to satisfactorily compete on recompetes for their own system. The recompete is usually not an opportunity to increase the bid rate because the customer expects and budgets for only slight increases in rates.
The effects of underbidding on staff dynamics also are dramatic. People generally like to be productive and do a good job in which they can take pride. An underbid development effort usually manifests itself in excessive overtime, constant "crunch" mode, minimum time spent in the analysis stage, chaotic development processes, few product assurance policies and practices, and lots of stress. The words "no communication" and "frustrated" are heard in the halls, and the result often is a high attrition rate, with its attendant problems.
Customers also have to be aware that they are dealing with a business, and that nothing is free. The same customer that wouldn't think of asking the local supermarket for a free head of lettuce often asks for changes to the contract and expects no cost increases or scope reductions. The job was bid at a certain price that assured the customer of getting the product they asked for in the solicitation and the developer of making a fair profit. In well-managed development activities where there is a lot of developer-customer interaction, the good relationship itself can lead to "requirements creep" where the changes are small and subtle, and the developer, who is eager to please the customer, says "yes" too often. The response usually should be "yes and these are the risks and cost and schedule implications of doing that. Now you (customer) decide how we should proceed." Customers then have the option of changing the current contract or saving the change for a future release of the software. Once the contract is in place, it is in the customer's best interest not to jeopardize the solvency of the developer by recognizing the business implications of program decisions.
Customers Do Not Last if They Are Not Satisfied
A business is built on a solid foundation of satisfied customers. The developer's business plan must include repeat business from existing customers. To achieve growth with existing customers, the developer's product(s) must have all the attributes of high-quality products. Based on David Garvin's Managing Quality, 1988, quality definitions, (transcendent, product based, user based, manufacturing based and value based), the customer must perceive
From these definitions, it is clear that a developer policy of "minimum compliance" with contract requirements will not result in a satisfied customer. It should be equally clear, however, that the notion of customer satisfaction is not a blank check for the customer to require things of the developer that are beyond the scope of their formal agreement. There are high-quality luxury cars and high-quality economy cars, so the customer should be satisfied with the product if it meets all the quality dimensions for the product, i.e., type of car, they ordered.
Customer Satisfaction Is Not a Constant
It will vary over time as the developer produces the product. It is incumbent on the developer to regularly determine how well the customer is being satisfied. This can be accomplished formally through periodic customer satisfaction surveys and informally through ongoing interaction. The elimination of surprises in a development environment is an important factor in keeping everyone satisfied. A healthy development environment includes constant customer involvement in product definition and development status. The customer and developer are in a collaborative relationship to make the program a success. Collaboration is based on each one performing professionally within the scope of their respective roles and responsibilities.
The project's customer contacts are people who report to their own bosses, shareholders, and other internal and external groups. Customers regularly provide status information to their supervisors and other elements within their organization for planning and budget purposes. Poor status information undermines the customer's planning activities. The developer's job can be thought of as getting their customer contact a good performance appraisal within the customer's organization. The implementation stage of a software application, for example, usually requires extensive coordination throughout the customer organization, and the reputation of the project's customer contact is on the line. Schedule slippage at this point may have career implications for the customer contact and fiscal implications for the customer organization. The developer has to reduce the customer's exposure by constantly providing honest status information. An honest and open developer-customer relationship in both directions will ultimately lead to a better product and a satisfied customer. Like any other relationship, long-term business relationships are based on mutual trust and benefit.
Successful development projects deal with these considerations in a proactive manner. The project culture emphasizes commitment, communication, effective management, empowerment, well-defined standards and processes, and customer orientation. The customer and the developer are bound together by the common goal of delivering a high-quality product for a fair price within a reasonable time frame. If either forgets that they both want to end up in the same place, the project is doomed to problems and contention.
Michael J. Hillelsohn
The Orkand Corporation
7799 Leesburg Pike
Falls Church, VA 22043-2499
Voice: 703-610-4607
Fax: 703-610-4230
E-mail:
mhillelsohn@orkand.com
About the Author
Michael J. Hillelsohn has been involved in the development of automated systems in a contract environment for more than 25 years. He is currently director of training and chairs the Software Engineering Process Group and the Quality Council at The Orkand Corporation. The Orkand Corporation is a leading provider of information technology services to federal government and private sector clients.