Home

   
 

Title: Intrusion Detection System (IDS)

Author: SpyHat

1. Executive Summary:

Intrusion detection systems, or IDSs, have become an important component in the Security Officer's toolbox. However, many security experts are still in the dark about IDS, unsure about what IDS tools do, how to use them, or why they must. This report will offer a brief overview of intrusion detection systems, including: a description of what IDSs are, the functions they serve, the two primary types of IDS, and the different methods of intrusion detection that they may employ.


2. Introduction to Intrusion Detection System:

On December 7 th 1999, two programs were published on Packetstorm [ 1] , which changed the face network security.  These utilities allowed a systematic, coordinated, denial of service attack to be launched by thousands of systems simultaneously.  Many web servers crumbled when hit by the flood of over 20 billion packets/second generated by the attack.  However, the most amazing thing about these tools is that they were automated .  The attacks came from thousands of locations around the planet, with absolutely no one behind them, and no one to be held accountable.  Such is the power of distributed technology.

Firewalls alone can no longer withstand the increasingly sophisticated and organized attacks we see on a daily basis.  What measures are available to counter these attacks?  What are their weaknesses?  How can they be improved on ?  These questions are considered in this distributed approach to network security.  The paper assumes a basic understanding of network protocols


2.1 What is distributed network security?

Distributed systems are not a new concept.  Distributed computing has been around since the 70's, when networks became increasingly integrated.  Distributed security is however, a concept in its infancy.  Only within the past few years have vender such as Internet Security Systems stepped forward to create network security systems that span entire networks.  This technology has come to be known as Intrusion Detection Systems, or IDS. ( Innella , Paul, 2001)

Note that distributed network security is not synonymous with IDS.  An ID is just the biggest and most common type of distributed network security.

Intrusion detection is the process of monitoring computers or networks for unauthorized entrance, activity, or file modification. IDS can also be used to monitor network traffic, thereby detecting if a system is being targeted by a network attack such as a denial of service attack. There are two basic types of intrusion detection: host-based and network-based. Each has a distinct approach to monitoring and securing data, and each has distinct advantages and disadvantages. In short, host-based IDSs examine data held on individual computers that serve as hosts, while network-based IDSs examine data exchanged between computers. ( Innella , Paul, 2001)

An intrusion detection system (IDS) inspects all inbound and outbound network activity and identifies suspicious patterns that may indicate a network or system attack from someone attempting to break into or compromise a system ( Innella , Paul, 2001)

Traditional IDS classification schemes put most system into two distinct camps: misuse detection models and anomaly-based detection models. Sometimes, a distinction is made between misuse and intrusion detection. The term intrusion is used to describe attacks from the outside ; whereas, misuse is used to describe an attack that originates from the internal network. Intrusion Detection Systems are like a burglar alarm for computer network. They detect unauthorized access attempts. They are the first line of defense for the computer systems. ( Innella , Paul, 2001)


This report focuses on two implementation of the misuse detection model:

• Network Based IDS

• Host Based IDS

There are many other intrusion detection models, but are less popular. However, most modern-day IDS implementation can be grouped into one of these categories:

2.2 Network Based IDS:

NIDS devices are raw packet-parsing engines glorified on steroids. They capture network traffic and compare the traffic with a set of known attack patterns or signatures. NIDS devise compare these signatures every single packet that they see, in hope of catching intruders in the act. NIDS devise can be deployed passively, without requiring major modification to system or network. (Anonymous, 2001)


2.3 Host Based IDS:

These systems vary from vender to vender, but they are usually system centric in their analysis. Most host-based IDSs will have components that parse system logs and watch user logins and processes. Some of the more advanced systems will even have built in capabilities to catch Trojan code deployment. They require the installation of program on the systems they protect. (Anonymous, 2001)


2.4 Anomaly Based IDS:

Anomaly Based system are a bit more obscure, and are often times referred to as more of a “concept” than an actual model. The philosophy behind anomaly based approaches is to understand the patterns of users and traffic on the network, and find deviations in those patterns. For example, a user who normally log in Saturday through Thursday but is now logging in at 3 am on a Friday might be flagged as a potential problem by an anomaly IDS. In theory, anomaly-based IDS could detect that something was wrong without specifically what the source of the problem was. ( Aurobindo Sundaram , 2001)

Anomaly detection systems are also computationally expensive because of the overhead of keeping track of, and possibly updating several system profile metrics. A block diagram of a typical anomaly detection system is shown in Figure 1. ( Aurobindo Sundaram , 2001)

Fig. 1 ( Aurobindo Sundaram , 2001)


2.5 Misuse Detection:

The concept behind misuse detection schemes is that there are ways to represent attacks in the form of a pattern or a signature so that even variations of the same attack can be detected . This means that these systems are not unlike virus detection systems they can detect many or all known attack patterns, but they are of little use for as yet unknown attack methods. An interesting point to note is that anomaly detection systems try to detect the complement of "bad" behavior. Misuse detection systems try to recognize known "bad" behavior. The main issues in misuse detection systems are how to write a signature that encompasses all possible variations of the pertinent attack, and how to write signatures that do not also match non-intrusive activity. A block diagram of a typical misuse detection system is shown in Figure 2 below. ( Aurobindo Sundaram , 2001)


Fig. 2 ( Aurobindo Sundaram , 2001)

The most common IDS type, commercial and deployed, are HIDS and NIDS models. Although working models of anomaly-based IDS exist, they are rarely deployed outside of government and academic. ( Aurobindo Sundaram , 2001)


3.Who Should Be Using an IDS:

Although an IDS technology is certainly attractive, before sinking any time into IDS research. You should first ask whether IDS makes sense for your organization. If, for example, an organization is lacking basic security fundamentals such as firewalls, system OS lockdown procedure, or virus protection, an IDS deployment shouldn't take priority over those efforts. An IDS should be installed only after other facts of the information security strategy have already been initiated, or to solve specific situation or shortcomings. For example, if a new e-commerce initiative is launched that you simply cannot secure adequately, an IDS might help to keep a sharper eye on it. Moreover, some people use IDSs as a validation tool for their firewall rulesets, but if your network is a chaotic collage of vulnerabilities, IDS will simply help you to become the master of the obvious. You'll already have problem that an IDS certainly won't fix. Remember, modern day IDSs are still, for the most part, reactive devices, they won't fix your problems. (Anonymous, 2001)


4. Network Based IDS:     

As opposed to monitoring the activities that take place on a particular network, Network-based intrusion detection analyzes data packets that travel over the actual network. These packets are examined and sometimes compared with empirical data to verify their nature : malicious or benign. Because they are responsible for monitoring a network, rather than a single host, Network-based intrusion detection systems (NIDS) tend to be more distributed than host-based IDS. Software, or appliance hardware in some cases, resides in one or more systems connected to a network, and is used to analyze data such as network packets. Instead of analyzing information that originates and resides on a computer, network-based IDS uses techniques like “packet-sniffing” to pull data from TCP/IP or other protocol packets traveling along the network. (IDS Group, 1999)

This surveillance of the connections between computers makes network-based IDS great at detecting access attempts from outside the trusted network. In general, network-based systems are best at detecting the following activities:

• Unauthorized outsider access: When an unauthorized user logs in successfully, or attempts to log in, they are best tracked with host-based IDS. However, detecting the unauthorized user before their log on attempt is best accomplished with network-based IDS.

• Bandwidth theft/denial of service: These attacks from outside the network single out network resources for abuse or overload. The packets that initiate/carry these attacks can best be noticed with use of network-based IDS.

Some possible downsides to network-based IDS include encrypted packet payloads and high-speed networks, both of which inhibit the effectiveness of packet interception and deter packet interpretation. Examples of network-based IDS include Shadow, Snort !, Dragon, NFR, RealSecure, and NetProwler. (IDS Group, 1999)


5. Host Based IDS:

Host-based systems were the first type of IDS to be developed and implemented . These systems collect and analyze data that originate on a computer that hosts a service, such as a Web server. Once this data is aggregated for a given computer, it can either be analyzed locally or sent to a separate/central analysis machine. One example of a host-based system is programs that operate on a system and receive application or operating system audit logs. These programs are highly effective for detecting insider abuses. Residing on the trusted network systems themselves, they are close to the network's authenticated users. If one of these users attempts unauthorized activity, host-based systems usually detect and collect the most pertnent information in the quickest possible manner. In addition to detecting unauthorized insider activity, host-based systems are also effective at detecting unauthorized file modification. (IDS Group, 1999)

On the down side, host-based systems can get unwieldy. With several thousand possible endpoints on a large network, collecting and aggregating separate specific computer information for each individual machine may prove inefficient and ineffective. In addition, if an intruder disables the data collection on any given computer, the IDS on that machine will be rendered useless because there is no backup. (IDS Group, 1999)

Possible host-based IDS implementations include Windows NT/2000 Security Event Logs, RDMS audit sources, Enterprise Management systems audit data (such as Tivoli), and UNIX Syslog in their raw forms or in their secure forms such as Solaris' BSM; host-based commercial products include RealSecure, ITA, Squire, and Entercept, to name a few. (IDS Group, 1999)


6. NIDS and HIDS Used in Combination:

The two types of intrusion detection systems differ significantly from each other, but complement one another well. The network architecture of host-based is agent-based, which means that a software agent resides on each of the hosts that will be governed by the system. In addition, more efficient host-based intrusion detection systems are capable of monitoring and collecting system audit trails in real time as well as on a scheduled basis, thus distributing both CPU utilization and network overhead and providing for a flexible means of security administration. (Anonymous, 2001)

In a proper IDS implementation, it would be advantageous to fully integrate the network intrusion detection system, such that it would filter alerts and notifications in an identical manner to the host-based portion of the system, controlled from the same central location. In doing so, this provides a convenient means of managing and reacting to misuse using both types of intrusion detection. That said, as an organization introduces an IDS into its network to augment its current information security strategy, the primary focus of the intrusion detection system should be host-based.

Although network intrusion detection has its merits and certainly must be incorporated into a proper IDS solution, it has historically been incapable of evolving to comply with the growing technology of data communications. Most NIDS perform miserably, if at all, on switched networks, fast networks of speeds over 100 Mbps, and encrypted networks.

Furthermore, somewhere in the range of 80 - 85 percent of security incidents originates from within an organization. Consequently, intrusion detection systems should rely predominantly on host-based components, but should always make use of NIDS to complete the defense. In short, a truly secure environment requires both a network and host-based intrusion detection implementation to provide for a robust system that is the basis for all of the monitoring, response, and detection of computer misuse. (Anonymous, 2001)


7. Conventional IDS:


7.1 Benefits:

The main benefit of IDS is its manageability.  An ID allows centralized, large-scale ease-of-use for many different networks.  For example, ISS's RealSecure 3.0 system allows administrators to easily access the status of many machines (and in turn the networks they monitor) by simply clicking.  Another benefit of IDS is its automation.  As in the example above, when an event is triggered different actions can be configured to run, such as email or paging various personnel. (Recourse Technologies, 2001)


7.2 Shortcomings

• An ID is by no means the ultimate solution, as it has many significant shortcomings.  Since IDS relies on exploit signatures, it is imperative for the system to have the most update collection of exploits available.  This is near impossible since new vulnerabilities are found on a daily basis.  As stated in Network Computing , “No IDS vendor has any sort of push technology in place to update its signature database.”  This failure to recognize the newest attacks must be corrected for IDS to truly become a standard. (Recourse Technologies, 2001)


• Since IDS must survey each packet on the wire, speed becomes an important issue.  On a 10Mbps Ethernet segment, IDS will work just fine.  But when placed on a network backbone running at 100Mbps all modern IDS stumbles to keep up with the traffic.  With new fiber being laid at higher speeds each day, IDS must constantly play catch-up. (Recourse Technologies, 2001)


• An ID also currently lacks the ability to detect network and user trends.  For example, there is a private military security system able to detect portscans as infrequent as 2 a day.  IDS are not at all intelligent in this respect, and often fail to detect even obvious network traffic patterns. (Recourse Technologies, 2001)


8. IDS Techniques:

Now that we have examined the two basic types of IDS and why they should be used together, we can investigate how they go about doing their job. There are four basic techniques used to detect intruders: anomaly detection, misuse detection (signature detection), target monitoring, and stealth probes.


• Anomaly Detection:

Designed to uncover abnormal patterns of behavior, the IDS establishes a baseline of normal usage patterns, and anything that widely deviates from it gets flagged as a possible intrusion. What is considered to be an anomaly can vary, but normally, any incident that occurs on frequency greater than or less than two standard deviations from the statistical norm raises an eyebrow. An example of this would be if a user logs on and off of a machine 20 times a day instead of the normal 1 or 2. Also , if a computer is used at 2:00 AM when normally no one outside of business hours should have access, this should raise some suspicions. At another level, anomaly detection can investigate user patterns, such as profiling the programs executed daily. If a user in the graphics department suddenly starts accessing accounting programs or compiling code, the system can properly alert its administrators. (Anonymous, 2001), (Recourse Technologies, 2001)

The benefit of this method is that it can detect the anomalies without having to understand the underlying cause behind the anomalies; however, legitimate use of the system can trigger anomalies leading to a very high number of false positives. (Anonymous, 2001), (Recourse Technologies, 2001)


• Misuse Detection (Signature Detection):

Commonly called signature detection, this method uses specifically known patterns of unauthorized behavior to predict and detect subsequent similar attempts. These specific patterns are called signatures. For host-based intrusion detection, one example of a signature is "three failed logins." For network intrusion detection, a signature can be as simple as a specific pattern that matches a portion of a network packet. For instance, packet content signatures or header content signatures can indicate unauthorized actions, such as improper FTP initiation. The occurrence of a signature might not signify an actual attempted unauthorized access (for example, it can be an honest mistake), but it is a good idea to take each alert seriously. Depending on the robustness and seriousness of a signature that is triggered , some alarm, response, or notification should be sent to the proper authorities. (Anonymous, 2001), (Recourse Technologies, 2001)

Nowadays, the majority of commercial IDS products on the market are based upon a system that examines the network traffic for specific patterns of attack. This means that for every exploit, the IDS vendor must code a signature specifically for that attack in order to detect it, and therefore the attack must be known . Almost all IDS systems are structured around a large signature database and attempt to compare every packet to every signature in the database. (Anonymous, 2001), (Recourse Technologies, 2001)

Unfortunately, there are some significant flaws with signature-based systems is the time it takes the IDS vendor to identify new attacks, create a signature, and release an update. Attacks like Code Red and Nimda cannot be identified by signature-based systems until the signature is added to the database, leaving a window of opportunity for attacks to penetrate the network unnoticed. Unfortunately, a new attack does the most damage during this window of opportunity. (Anonymous, 2001), (Recourse Technologies, 2001)


• Target Monitoring:

These systems do not actively search for anomalies or misuse, but instead look for the modification of specified files. This is more of a corrective control, designed to uncover an unauthorized action after it occurs in order to reverse it. One way to check for the covert editing of files is by computing a cryptographic hash beforehand and comparing this to new hashes of the file at regular intervals. This type of system is the easiest to implement, because it does not require constant monitoring by the administrator. Integrity checksum hashes can be computed at whatever intervals you wish, and on either all files or just the mission/system critical files. (Anonymous, 2001), (Recourse Technologies, 2001)


• Stealth Probes:

This technique attempts to detect any attackers that choose to carry out their mission over prolonged periods of time . Attackers, for example, will check for system vulnerabilities and open ports over a two-month period, and wait another two months to actually launch the attacks. Stealth probes collect a wide-variety of data throughout the system, checking for any methodical attacks over a long period of time . They take a wide-area sampling and attempt to discover any correlating attacks. In effect, this method combines anomaly detection and misuse detection in an attempt to uncover suspicious activity. (Anonymous, 2001), (Recourse Technologies, 2001)


9. The importance of intrusion detection?

Of the security incidents that occur on a network, the vast majority (up to 85 percent by many estimates) come from inside the network. These attacks may consist of otherwise authorized users who are disgruntled employees. The remainder comes from the outside, in the form of denial of service attacks or attempts to penetrate a network infrastructure. Intrusion detection systems remain the only proactive means of detecting and responding to threats that stem from both inside and outside a corporate network.

Intrusion detection systems are an integral and necessary element of a complete information security infrastructure performing as "the logical complement to network firewalls." [BAC99] Simply put, IDS tools allow for complete supervision of networks, regardless of the action being taken , such that information will always exist to determine the nature of the security incident and its source.


10. Evaluation Criteria for choosing IDS

There are three important points to consider when an organization chooses an Intrusion Detection System. The first point is the fact that different organization requires different IDS. As Greg Shipley mentioned in the chapter “The Defender's Toolkit”:  (Anonymous, 2001)


There is no “one size fits all” IDS solution on the market today, and
  I highly doubt there will be one anytime soon. The IDS product landscape is a diverse one.

Each organization should determine all the important parameters and requirements and then choose the most appropriate product to meet those requirements.

The second point is the rapid changes in the IDS technology. The main reason for these changes is that IDS technology is still in its infancy days and it's not yet mature. In as short as six months the issues to consider when choosing IDS could change. The most important   issue is to be conscious of the age of any information about IDS.


The third important point is to understand that choosing IDS is choosing two things:

A product and a partner who will be updating the product.   As mentioned before IDSs are very time sensitive. Therefore the usefulness of a product depends directly on product updates. Organizations should evaluate the vendor's track record in regard to product updates and evaluate the core components of a product before its bonus features. The core components of IDS to evaluate are listed below:


• Depth of coverage

The ability to detect a wide array of attacks is one of the most important components of IDS. The product should be capable of detecting more than a handful attacks. In NIDS, a set of attack signatures should be bundled in the solution. In HIDS the inspection of log files is not sufficient. The HIDS solution should also support all the platforms that the organization needs to monitor.


• Accuracy of coverage

The accuracy of a system is hard to determine without testing the system. A big problem with most NIDS solutions is false positives. The products, which are designed with reduction of false positives in mind, would be more accurate for any organization.


• Robust architecture

The multiple components of an intrusion detection solution should be designed to increase the effectiveness of the product. The engines and the IDS framework should be designed with strength in mind. The product should be able to resist both attacks and basic evasion techniques.


• Scalability

There are multiple factors that affect the scalability of IDS. The two most important factors are in the areas of high-bandwidth and data management. The bandwidth issue applies to NIDS devices. Many products are not able to monitor high-bandwidth environments. In the other hand, many products have problem to monitor, store and present large volume of alert data. Of course these issues are not relevant in environments where there are only a few ID devices to watch over a few connections.


• Management framework

The ability to present the data related to detected attacks is as crucial as the ability to detect attacks. These data should be presented clearly and efficiently. The system should provide the easy access to the required information. The usefulness of a system depends on ability of accessing attack and alert data.


• Timely updates

The need for timely IDS product updates is critical because new attacks continue to surface. All IDS models need to be timely updated.


• Customizability

Some intrusion detection products allow for a diverse range of customization, whereas others are fairly static and inflexible. Each organization should consider its needs of customization. It is probable that one organization does not need customization today but they might need that functionality in the future.


• Skill set requirements

Like any other component, intrusion detection devices need trained staff to operate the solution.


11. A Sample IDS solution: SNORT

There are a number of IDS solutions in the market. The most popular is SNORT, created by Marty Roesch. SNORT is an open source solution. It uses an NIDS model, and has a fairly extensive set of plug-ins and supporting applications.


12. Conclusion:

As security incidents become more numerous, IDS tools are becoming increasingly necessary. They round out the security arsenal, working in conjunction with other information security tools, such as firewalls, and allow for the complete supervision of all network activity. This information can, in turn, help to determine network misuse, its nature, and its source. Not surprisingly, the sale of IDS tools continue to climb and revenues should reach the $1 billion mark within the next two years. These intrusion detection tools use several techniques to help them determine what qualifies as an intrusion versus normal traffic. Whether a system uses anomaly detection, misuse detection, target monitoring, or stealth probes, they generally fall into one of two categories: network-based or host-based. Each category has strengths and weaknesses that should be measured against the requirements for each separate target environment. Ideally, the best IDS tools combine both approaches under one management console. That way, the user gets comprehensive coverage, making sure to guard against as many threats as possible. Whatever the choice, whether it is host-based, network-based, or a hybrid of the two, it is clear that using intrusion detection systems is an important and necessary tool in the security manager's arsenal.


13. Reference:


[1] Packetstorm.securify.com is a popular network security site.  The programs were TribeFloodNetwork and trinoo.

Copyright © 2002-2010 SpyHat. All Rights Reserved
 
SpyHat