OSSEC Intrustion Detection System

OSSEC is an open source host based intrusion detection system (IDS). An IDS is one of the most important tools available to a security administrator. As a host based IDS (or HIDS), OSSEC is uniquely advantaged to monitor activity from the server side. Although a network based IDS may be able to spot malicious traffic and identify attacks based on traffic, a HIDS can look directly at log files and system behavior to spot oddities such as successful brute force attacks or evidence of rootkit installation.

OSSEC runs on most operating systems, including Windows, Linux, FreeBSD, OpenBSD, Mac OS, and Solaris. OSSEC has multiple installation modes, but all are based on the idea that there is an OSSEC server and clients that report to the server. You can do a stand alone installation though, that includes both the client and server component in one installation. The server installation also includes a local client, so that your OSSEC server will monitor itself.

OSSEC has multiple capabilities that distinguish it. It can perform near real time log file analysis. The clients monitor log files for activity defined by rules that can be deployed on the server or on the client. Alerts are generated based on log file analysis which can then trigger e-mails to administrators or other activity.

OSSEC will also perform file integrity checking. It will compare hash values of files over time which can alert you to critical file system changes (which could indicate that a trojan was installed). OSSEC will also monitor critical Windows registry settings. OSSEC can perform root kit detection, based on behaviors, which is much more accurate than simply relying on signatures.

OSSEC's alerts are also time based. This means that OSSEC can trigger alerts based on behavior over time. One good example of this type of functionality is the alert that gets generated if a user logs into a system after repeated failures (which could indicate a successful brute force attack).

OSSEC can also use active response. Of course, as with any active response or Intrusion Prevention System (IPS) one must use caution. An active response could be used to create a denial of service situation. OSSEC uses scripts tied to rules that allow it to respond to specific alerts in specific ways. For instance, you can set up OSSEC so that if it notices a dictionary (brute force) attack it blocks the attacker. This can be extremely effective at preventing those ubiquitous SSH brute force attacks. OSSEC can even be configured so that multiple clients can respond using active response to an incident on just one client. For instance, if an attacker tries to brute force into SSH on one of your machines you can have all your machines block port 22 to that attacker. You can even set times on the responses, so as to limit the response to a certain duration.

OSSEC installations require a server and a client (although, as noted before, you can perform a stand alone installation). The client and the server communicate over an encrypted UDP channel that utilizes port 514 in version 1.4 and prior and port 1514 on version 1.5. You must be sure that your firewalls permit traffic between the client and the server on the appropriate port in order for OSSEC to function.

OSSEC generates a key (to use with synchronous encryption) unique to each client. This key is tied to the client IP address and prevents snooping on the conversation or the injection of bogus alerts or spoofing a client. This can cause some problems with clients that utilize DHCP, however. There are work arounds that allow you to cut keys for entire IP ranges that overcome this drawback, but you lose some of the integrity afforded through the encrypted channel by doing this.

OSSEC is a wonderful intrusion detection tool, but it is not by all means perfect. Default installations tend to be "noisy", generating a lot of alerts for events that range from anomalous to system misconfigurations to actual attacks. You must spend some time tuning your rule set and alerts in order to avoid producing too much traffic. OSSEC also suffers from problems where the server or agent might die silently. It is a good idea to schedule OSSEC to restart daily to prevent this problem. Solaris installations are also notoriously buggy. Don't be surprised if you run into problems on Solaris.

Despite these drawbacks, OSSEC has some wonderful features that recommend it. OSSEC, as a HID, allows organizations without access to their networking infrastructure to deploy security monitoring. OSSEC also has an active and responsive developer community. Sending questions to developers via IRC and e-mail almost always gets a quick response. OSSEC is truly cross platform. A server installation can monitor clients on numerous different operating systems. OSSEC also alerts administrators to a broad swath of threats, from web exploits to brute force attacks.

Additionally OSSEC can be configured to log to MySQL or PostgreSQL databases. This makes for easy automated analysis and trending over time. It also makes searching alerts and doing data mining possible. OSSEC also uses an XML based rule set that is both easy to learn and intuitive. Adjusting rules and creating new ones is fairly straightforward and simple. OSSEC can also integrate easily with other monitoring systems such as Nagios and BASE.

Installing OSSEC takes less than 15 minutes so you can be up and running in less time that it'll take you to get a cup of coffee. You can find further documentation about OSSEC at their website http://www.ossec.net and on their wiki at http://www.ossec.net/wiki.