Critical information assets (systems, networks, and sensitive data) can be compromised by malicious or inadvertent actions despite an organization's best efforts. System and network administrators are on the firing line and even when they know what to do, they often do not have the time to take action; operational day-to-day concerns and the need to keep systems functioning take priority over securing those systems.

Administrators must choose how to protect assets. But when managers cannot prioritize critical assets and threats (as part of a business strategy for managing information security risk), then the protections an administrator offers will be arbitrary at best. Unfortunately, managers often fail to understand that securing assets is an ongoing process. They do not consider this factor when allocating administrator time and resources.

Most system and network administrators learned from their peers how to protect and secure systems, not by consulting published procedures that serve as de facto standards accepted by the administrator community - no such standards currently exist. Administrators are sorely in need of a structure for organizing, prioritizing, and selecting security practices that is easy to understand, describe, justify, and implement.

The Security Knowledge in Practice (SKiP^SM)¹ method was developed to organize security practices published on the Computer Emergency Response Team (CERT®)/CERT Coordination Center®² Web site into a more process-based approach, departing from the more common problem-based approach. SKiP defines a cyclical process for establishing and sustaining the security of critical information assets such as the following:

• Systems running mission critical applications.
• Network infrastructure, including routers, hubs, and switches.
• Subsystems or sub-networks such as those providing E-mail services, Web content production and delivery services, and perimeter protection services.
• A network architecture and topology.
• Sensitive or proprietary information such as mission logistics, customer data, or financial projections.
• Computers installed at home.

SKiP defines seven steps, the actions for each, and a well-defined order for executing these steps. For a less experienced administrator, SKiP provides a road map of the practices necessary to build and sustain the security of an information asset. For the more experienced administrator, SKiP provides an ordered arrangement and a checklist of practices, allowing an administrator to identify gaps in the tasks they are already performing. Once an administrator understands SkiP steps, they are repeated for the life of the asset.

Figure 1 shows the steps that compose SKiP. In this article, we discuss how to apply SKiP to securing a computer system that runs a mission critical application in a production environment.

Security Knowledge in Practice

The SKiP method, the CERT Security Practices, and all tools discussed in this article are publicly available.

Vendors sell systems configured so their customers will be eager to buy. Often, systems are general purpose, that is, full-featured with most of the software enabled for ease of use. They are meant to satisfy everyone's needs and, perhaps, some they did not realize they had. Such systems frequently contain the following:

• Services that are not needed and often insecurely configured.
• Little or no protection on access to data objects such as files and directories.
• Ease-of-use features often provided at the expense of security.
• Vulnerabilities that intruders can use to break into systems³.

In today's marketplace, a vendor provides an administrator with a product containing the operating system and an assortment of software applications and tools. While some are needed for the system to function, others are provided to accommodate the demand for an all-purpose box. Unfortunately, this one-size-fits-all mentality will continue as long as we continue to purchase a vendor's products instead of demanding they produce a more secure product. Once customers start voting with their dollars, the responsibility for securely configuring systems will shift from administrators to vendors.

Until then, the system administrator's role is to identify the tasks that will be performed by the system and determine the necessary (minimum essential) functions to meet the organization's goals, eliminating those that are unnecessary. Securing a system is challenging, especially for a novice administrator. As a result, it is often considered unnecessary, low priority, or virtually impossible. The SKiP method helps an administrator make this security task more orderly and manageable.

Harden/Secure configures a system to meet an organization's security requirements, retaining only those services and features needed to address specific business needs. By removing unnecessary functionality, an administrator begins to harden the system. However, simply limiting functionality is not enough; administrators need to configure remaining functions correctly (such as installing necessary patches) to sustain a stable and secure configuration.

This step strengthens a system against known attacks by eliminating vulnerabilities and other weaknesses commonly used by intruders. The practices performed during this step may change over time to address new attacks and vulnerabilities.

Here are some examples of Harden/Secure practices:

• Install only the minimum essential operating system features. Disable and remove unneeded software. The fewer the services and software on a system, the harder it is to access that system through available means. For example, remove the FTP server and client if the system is not expected to need or provide FTP service.
• Install all applicable patches that correct known deficiencies and vulnerabilities.
• Install the most secure, current versions of system applications.
• Replace applications that contain known vulnerabilities. For example, on a UNIX system, remove telnet and the Berkeley rcommands⁴ and replace them with SSH, the Secure Shell.
• Install tools needed to operate a system securely. Examples are tools that scan for viruses, characterize a system's behavior (Tripwire)⁵, and perform secure administration (SSH)⁶.
• Remove all privileged and lenient (too weak or open) object access controls. This follows the deny first, then allow principle. Grant privileges and access only as needed. Remove all default accounts.

If at all possible, we recommend performing harden/secure on a system that is not attached to any network. This minimizes opportunities to compromise the system while it is being built. Also install the operating system, patches, and tools from removable media such as Zip disks or CD-ROMs.

The CERT/CC provides a checklist on securing UNIX-based systems⁷ and SANS provides guides for Linux, Windows 2000, Windows NT, and Solaris⁸. An administrator needs to frequently check these resources and others because harden/secure practices change over time.

"The CERT Guide to System and Network Security Practices" [1] provides a detailed description of the practices necessary to harden/secure a general-purpose server (chapter 2), a public Web server (chapter 3), and a firewall system (chapter 4). It also has detailed descriptions of the practices involved in its steps: prepare, detect, respond, and improve. The CERT guide is a compendium of six reports in the CERT Security Improvement Module series. These reports are publicly available⁹. The four steps in the CERT guide are described in the following sections.

One of the essential concepts behind the Prepare Step is that undiscovered vulnerabilities exist. This requires an administrator to be able to recognize when previously unknown vulnerabilities start to be exploited. To support such recognition, it is important to characterize a system, understand its normal behavior in an operational setting, and be able to determine departures from normal.

Characterization entails examining a system's operation and performance under normal conditions and recording expected behavior as the system's known baseline state. This baseline state contains information (also called attributes) about expected changes at the network, system, process, user, file, directory, and hardware levels. Once a trusted baseline state of system attributes is captured, an administrator compares attributes of an executing system to the baseline to learn if something has changed and then judges whether or not the change is acceptable and expected.

One way to think about the distinction between harden/secure practices and the characterization practices in the Prepare Step is that hardening attempts to solve known problems by applying known solutions, whereas characterization helps to identify new problems and formulate new solutions. In using a characterization baseline for comparison, problems can be identified through anomaly-based detection techniques, that is, departures from normal behavior so that new solutions can be formulated and applied. After completing characterization, an administrator knows the following:

• The expected changes in files, directories, and the operating system.
• The expected list of processes, when they run, by whom, and what resources they consume
• The expected network traffic consumed and produced.
• The expected hardware inventory on the system.

Characterization needs to be performed periodically so that an administrator always has an accurate, up-to-date characterization baseline with which to compare. Characterization information must be recorded and stored securely so it can serve as a reliable basis for comparison. Use WORM (Write Once, Read Many) media such as CD-ROM. We also recommend using cryptographic checksums and digital signatures like those provided with PGP¹⁰ to help ensure characterization baseline integrity. Additional practices in the Prepare Step include the following:

• Instigating the development of policies and procedures to do the following:
• Identify critical assets, threats to those assets, and possible response actions.
• Determine the priority and sequence of detection and response actions.
• Specify the authority to act when an intrusion is detected.
• Form and operate a computer security incident response team or equivalent capability.
• Define what data to collect, where and when to collect it, and the means for its review and protection.
• Assign necessary roles and responsibility.
• Ensure users are adequately trained.
• Ensure your organization is legally compliant with all laws and regulations.
• Manage data collection mechanisms (such as logging and monitoring tools) and the outputs they produce. Enable as much system logging as possible to provide an audit trail of all system activities. This information aids an administrator in understanding what happened when an incident occurs.
• Understanding, selecting, configuring, installing, and maintaining tools for intrusion detection and response. Such mechanisms must be in place well before they need to be used.

The Detect Step occurs while monitoring a system running in a production environment (such as looking at the logs produced by a firewall system or a public Web server). An administrator does the following:

• Notices any unusual, unexpected, or suspicious behavior.
• Learns something new about the system's characteristics.
• Receives information from an external stimulus (a user report, a call from another organization, a security advisory or bulletin).

These indicate either that something needs to be analyzed further or that something on the system has changed or needs to change (a new patch needs to be applied, a new tool version needs to be installed, etc.). Analysis includes investigating unexpected behavior that may be the result of an intrusion and drawing some initial conclusions, which are further refined during the next step: respond. In other words, sufficient analysis is performed during the Detect Step to determine what action needs to be taken.

An administrator uses many of the same tools and procedures that generated the system characterization baseline to detect signs of intrusion. The difference is that characterization results from the currently executing system are compared against the trusted baseline. An administrator's task is to reconcile the differences between that baseline and the new behavior. There are two possibilities when differences occur:

1. An administrator did not or was not able to accurately characterize the system, and the discrepancies represent previously unknown but acceptable behavior. An administrator returns to the Prepare Step to update the characterization baseline, create a new CD-ROM, and update checksums and digital signatures where appropriate.
2. The difference is truly anomalous and indicates that something has been tampered with on the system. An administrator moves to the Respond Step and proceeds with those practices.

Once an intrusion is discovered, an administrator performs several practices to respond to the problem and contain it. The response can be as simple as taking little or no action - accepting the risk - to taking significant steps to contain and recover from the intrusion. Successful response means that the system is returned to the normal operational capability that existed before the compromise.

If the transition to the Respond Step results from anomalous behavior caused by an intrusion, an administrator further analyzes the effects of damage caused by the intrusion and its scope. Dealing with the effects of an intrusion may result in the insertion of new technology, practices, procedures, and personnel. The administrator's goals are as follows:

• Contain the effects as far as possible. One of the challenges administrators face in responding is deciding when they have learned enough about the intrusion so that they can take the appropriate recovery steps vs. continuing to monitor an intruder's actions in order to discover all access paths and entry points. It is a delicate balancing act. If an administrator does not discover and eliminate all intruder access paths, then it is likely that the intruder will return. However, if the intruder is allowed to roam through systems, then the damage caused to an organization's assets may be fatal. Identifying and containing the full effects of an intrusion can be a very difficult task and can take a long time.
• Work to eliminate future intruder access. Part of the analysis involves discovering how the intruder gained access. Frequently, there is a strong push to return a system to operation even if it means recovering to a previous but vulnerable and insecure state. In this event, the compromised system state is lost, and so are the indicators of how the intrusion happened. The key is to consciously decide that this is the desired course of action and to recognize the ramifications and risks associated with that decision.
• Return the system to a known, operational state while continuing to monitor and analyze. One of the goals of responding is to return the system to a usable state that is an improvement over the previous state that resulted in the intrusion. Once the system is returned to operation, it is a heightened target for intruders who may be planning to gain access through back doors they have installed. An administrator may learn more about the intruder's attack methods (and therefore the required defenses) through more detailed monitoring and analysis.

While these activities are going on, an administrator notifies all other parties that may be affected, conveying concise and accurate reports of the intruder's activities and the response actions taken to the proper party. This notification must be in concert with the organization's information dissemination policy.

An administrator must collect and protect information that may become evidence in possible legal proceedings, regardless of the organization's policy on prosecuting intruders. An organization must assume that other sites affected by the intrusion will request this information for use in their prosecutions, perhaps by subpoena.

The collection and analysis of intrusion-related information is called computer forensics. It is a growing information security discipline characterized by new tools, practices, and procedures. An administrator should track developments in this area, including the availability of new analysis approaches and tools.

If the transition to the Respond Step is the result of another external stimulus, it is addressed and then the SKiP process transitions to the Improve Step. For example, if the stimulus is the release of a vendor's patch, the administrator applies the patch, and this action constitutes completion of the Respond Step.

Respond activities take place with the system operating in a production mode. Systems may be unavailable while repairs are made.

An administrator needs a test environment to understand the nature of the intrusion. In this environment, an administrator may be able to run a quarantined version of any captured attack tools to learn which vulnerabilities the intruder used to gain access, thus ensuring that these means of access have been removed before the systems are returned to operation.

Improvement actions typically occur following the Detect or Respond Steps. Improvement actions may include the following:

• Holding a post-mortem review meeting to discuss lessons learned.
• Updating policies and procedures.
• Selecting new tools.
• Collecting business case measures, including the resources required to deal with the intrusion and its resulting impacts, such as loss of user productivity and administrator time. It is important to quantify the cost of the intrusion in order to effectively reallocate resources and better prepare for future attacks. This information often serves as the most compelling argument to convince management to allocate sufficient resources to address security issues. At a minimum, capture staff effort (hours, weeks) and capital investments.

The Improve Step takes place when the system is operating in a production mode, though we strongly recommend installing and executing any new tools in a test environment before deploying them.

Any changes made during Detect, Respond, and Improve Steps are factored into the system's characterization baseline by returning to the Prepare Step. Note that this repetition is different from the initial one because the practices are performed when the system is operating in a production mode.

This article described the seven steps in the SKiP method and the practices comprising each step:

• Vendors provide systems that are general-purpose and need to be handcrafted to meet an organization's needs.
• Harden/secure the system against known problems.
• Prepare the system so the administrator will be able to spot anomalies that may indicate the occurrence of unknown problems.
• Detect those anomalies and other changes.
• Respond to them when they occur.
• Improve the practices and procedures after updating the systems.
• Repeat the process as long as the organization needs to protect the information assets on the system and the system itself. SKiP should be followed until the system is retired.

1. Allen, Julia. The CERT Guide to System and Network Security Practices. Boston: Addison-Wesley, 2001.
2. Allen, Julia, et al. "Improving the Security of Networked Systems." CrossTalk 13. 10 (Oct. 2000).
3. 3. Allen, Julia. "CERT System and Network Security Practices." 5TH National Colloquium for Information Systems Security Education, 22-24 May 2001. George Mason University www.cert.org/archive/pdf/NCISSE_practices.pdf.

1. SkiP is a service mark of Carnegie Mellon University. See also www.cert.org/security-improvement/skip.html.
2. CERT and CERT Coordination Center are registered in the U.S. Patent and Trademark Of-fice. See also www.cert.org/security-improvement.
3. Refer to Congressional testimony provided by the CERT Program Director Rich Pethia for details on the vulnerability of today's technology. Available at: www.cert.org/congressional_testimony/Pethia_testimony_Sep26.html.
4. rpc, rdate, rdist, remsch, rlogin, rpcinfo, rsh, rksh, rup, ruptime, rusers, rwho
5. www.tripwiresecurity.com. A public domain version of Tripwire is available at: ftp.cerias.purdue.edu/pub/tools/unix/ids/tripwire.
6. www.openssh.org.
7. www.cert.org/tech_tips/AUSCERT_checklist2.0.html.
8. www.sansstore.org/Templates/frmTemplateK.asp?SubFolderID=22SearchYN=N.
9. www.cert.org/security-improvement.
10. www.pgp.com. Public domain versions of PGP are available.

Lawrence Rogers is a senior member of the technical staff within the Networked Systems Survivability Program at the Software Engineering Institute. He develops and delivers information security education and training courses for senior executives and system and network administrators. Rogers has 25 years of technical experience in systems development and administration. He has a bachelor's of science degree in systems analysis from Miami University and a master's degree in computer engineering from Case Western Reserve University.

Carnegie Mellon University
Software Engineering Institute
Pittsburgh, PA 15213
Phone: (412) 268-8042
Fax: (412) 268-7966
E-mail: lrr@cert.org

Julia Allen is a senior member of the technical staff within the Networked Systems Survivability Program at the Software Engineering Institute. She is engaged in developing and transitioning security improvement practices for network-based systems. Allen has more than 25 years of managerial and technical experience in software engineering. She has a bachelor's of science degree in computer science from the University of Michigan and a master's degree in electrical en-gineering from the University of Southern California.

Carnegie Mellon University
Software Engineering Institute
Pittsburgh, PA 15213
Phone: (412) 268-7995
Fax: (412) 268-7966
E-mail: jha@cert.org