Host intrusion detection with OSSEC

Keep your corporate network secure with open source OSSEC, an intrusion detection and prevention services tool that provides host agent and file integrity agent capabilities on Windows and Linux. In this tip, a security expert provides a walkthrough for installation and configuration.

No one wants an intruder in the corporate network, and one key way to fight against intruders is to be able to...

detect them.

More open source security tools:
Nmap and the open source debate 

Rsync and Amanda: Keeping your data safe with open source backup

Intrusion detection and prevention services (IDS/IPS) are broken down into two broad categories: network- and host-based services. Network-based IDS/IPS (Snort, for example) detects and potentially prevents network-borne attacks. Host-based IDS/IPS detects and potentially prevents threats at a host level. In this article, we'll discuss OSSEC, an open source IDS/IPS that works on the host level.

Host IDS/IPS is further broken down into host agents, which defend against network and root kit attacks, and file integrity agents like Tripwire that monitor and alert on file system changes.

OSSEC is an open source tool written by Daniel Cid that provides both host agent and file integrity agent capabilities. It works on both Linux and Windows, but I am going to focus on the Linux version.

OSSEC can detect rootkits, do log analysis (for a number of services like Bind, Apache and Squid and also for third-party logs from Cisco and Netscreen devices amongst others -- see here for a full list -- and perform integrity checking. It has an active response module that can respond to threats and attacks.

OSSEC can be deployed as a stand-alone agent or as part of a distributed network of agents with a central OSSEC server controlling their configuration and settings.

To install OSSEC, you first need to download it, like so:

# wget
# wget

Note that I also download the sig file for the package. I'll use this to verify the package. To do this, I also need to download and import the OSSEC GPG public key like so:

# wget
# gpg --import OSSEC-GPG-KEY.asc

Then I verify the downloaded file against the GPG key. # gpg --verify ossec-hids-0.9-1a.tar.gz.sig ossec-hids-0.9-1a.tar.gz

If the resulting output acknowledges a good signature, then the package file should be okay (obviously this assumes you trust the developer's security).

Unpack the package and change into the resulting directory. To install OSSEC, you need to run the script, like so:

# ./

The first thing you need to select is your language, the default being en for English. Enter the required language and press Enter to continue to the next screen. The next screen tells you about your installation environment, including the prerequisite of a C compiler. Press Enter to continue to the next screen.

The next screen prompts you for the type of installation:

1- What kind of installation do you want (server, agent, local or help)?

There are three types of installation: server, agent and local (the help option details each installation type). A server installation creates a central server that can monitor a number of remote hosts called agents.

To create an agent, the agent installation is run on the host and it is then connected to a host running the server installation. The last installation type, local, is used on a single stand-alone host and combines the features of server and agent on that host.

If you only have one host to monitor, you'd use the local installation method. We're going to use this local method. Type 'local' and press Enter to continue.

The next step is to specify the installation location. It defaults to /var/ossec. Specify the location or accept the default and press Enter to continue.

Next, we need to configure email notifications and specify the email address and SMTP server used to send the emails. OSSEC uses email notifications to alert you to events and anomalies. I recommend you turn it on and specify a suitable address to receive these alerts.

3.1- Do you want e-mail notification? (y/n) [y]: y
  - What's your e-mail address?
  - What's your SMTP server ip/host?

The following steps specify which components of the OSSEC are enabled. The components are integrity checking, rootkit detection engine and active response. Specify y or n to enable or disable them.

Integrity checking is file and directory integrity checking in the vein of Tripwire. Rootkit detection finds any rootkits (code designed to help an attacker maintain access to your host by concealing processes, files and logs) that might be installed on your host. The active response component allows OSSEC to respond to threats by blocking attacking hosts either by adding them to the hosts.deny file or adding blocking rules to a firewall. You can also specify a whitelist of hosts that should never be blocked.

A final component, log analysis, is enabled by default. Log analysis automatically analyses the contents of selected log files, /var/log/secure and /var/log/messages for example, and automatically alerts on any anomalies detected. You can also specify additional log files to be analyzed by adding localfile entries to the ossec.conf configuration file (located in the /var/ossec/etc/ directory by default).

After configuring your installation, the install script will compile and then copy the required files to the specified location. The last stage of the installation process will start the various OSSEC processes (each process roughly corresponds to an OSSEC component). You can then start and stop the OSSEC processes using the command:

# /var/ossec/bin/ossec-control start 


# /var/ossec/ bin/ossec-control ossec stop

OSSEC is now running and your host is being monitored for intrusions and anomalies.

But your work isn't quite done yet. Any intrusion detection agent installed out of the box is generally configured to detect very broadly. This is also known as being installed 'untuned'.

Intrusion agents require initial, and then regular, tuning to suit your environment. This tuning includes tuning the rules and signatures to reflect the combination of applications and services you have running on your hosts, specifying additional logging sources and adjusting the criticality of alerts to reflect those issues most important to your environment.

To do this you need to add or adjust the rules contained in the XML files in the /var/ossec/rules directory (you can read about the format of these files at

Additional information on the rules and other elements of OSSEC can be found in the manual. At the Web site, there is also an FAQ, a Wiki and mailing lists that can provide assistance in installing and configuring OSSEC. I recommend you read carefully through these resources in order to enable you to deploy OSSEC in the most effective and secure manner for your environment.

P.S. Recently another development in the OSSEC world is the ability to integrate with BASE aka the Basic Analysis and Security Engine.

BASE is a descendent of the ACID analysis tool used by many people to analyze Snort IDS output. If you also use Snort, this allows you to integrate your network and hosts intrusion detection output in one analysis engine.

Did you find this tip helpful? Email us and let us know what tips you'd like to see.

This was last published in October 2006

Dig Deeper on Linux security risks and threats

Start the conversation

Send me notifications when other members comment.

By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Please create a username to comment.