Event correlation is the art of taking data, usually from host or applications logs, and discerning useful information from that data. It can be used for a range of activities such as alerting on events like hardware or application failures, notification of security-related events like failed logins, triggering of actions-based matched events and even longterm trending of events and application behavior.
Many organizations consider event correlation a complicated and costly exercise with expensive tools and time-consuming configuration, but it doesn't necessarily have to be. In this two-part tip, I will look at a free open source tool called SEC (Simple Event Correlator) that is easy to learn and provides powerful correlation rules. In the first half of this tip, I will demonstrate how to install and configure SEC. In the second half, I'll introduce you to SEC correlation rules.
SEC was written by Risto Vaarandi. It is designed for small to medium enterprises; SEC falls into a niche between low-end event correlation tools (like shell scripts "greping" log files) and commercial high-end event correlation tools like Intellitactics or Cisco MARS. SEC is often integrated with tools like Snort, Nagios, HP OpenView and CiscoWorks.
Although it is not as powerful as the high-end tools, SEC should meet the event correlation requirements of most non-enterprise environments, and it could even be suitable for some organizations that don't need the powerful capabilities of the higher end tools.
Written in Perl, SEC is designed to read incoming data feeds (from regular files, named pipes, or standard input) and to correlate that data based on user-defined rules. These rules, divided into types, use regular expressions to match incoming data against a rule and perform some form of action (such as generating an alert).
This correlation can be a simple match; for example, every time a failed login is detected, an email alert could be generated. Or rules can specify more complicated types of multi-stage correlations that involve multiple items of data; for example, matching a log entry and then monitoring for additional log items of the same type, measuring them against a threshold and alerting if the threshold is exceeded.
The first step to getting up and running with SEC is, obviously, to download SEC. You can get the SEC package from the Download section of the SEC Web site; it uses Sourceforge to store its releases. Packages are available for the package management systems of common Linux distributions like Debian, Gentoo, Ubuntu and Fedora, as well as ports for various flavors of BSD. The current version of SEC at the time of writing was 2.4.
Download and unpack the package like so:
# wget http://optusnet.dl.sourceforge.net/sourceforge/simple-evcorr/sec-2.4.beta2.tar.gz # tar -zxf sec-2.4.beta2.tar.gz
The package contains the sec.pl script that is the correlation engine, start-up or init scripts, a man page, a C application for linking HP Openview with SEC and a tool to convert older SEC rules into version 2.0 form. Copy the sec.pl script and man page to appropriate locations for your distribution, for example:
# cp sec.pl /usr/local/sbin # cp sec.pl.man /usr/local/man/man8/sec.pl.8
Running SEC is simple. Execute the sec.pl script on the command line with a few simple options.
# /usr/local/sbin/sec.pl -conf=/usr/local/etc/rules.cfg -input=/var/log/messages -detach -pid=/var/run/sec.pid -log=/var/log/sec.log
The first option, -conf, allows you to specify a file containing your event correlation rules. You can specify multiple -conf options to include multiple rules files. I will briefly discuss SEC rules in part two of this tip. The -input option specifies the source or sources of events to be correlated; like the -conf option, it can be specified multiple times to include multiple sources of events:
# /usr/local/sbin/sec.pl -input=/var/log/messages -input=/var/log/secure -input=/var/log/named -input=/var/log/mysqld …
Input sources can include files, named pipes or standard input.
The next option, -detach, detaches the sec.pl process and runs it as a daemon. By default, SEC would normally run in the foreground. The -pid option specifies the location of a pid file for the SEC daemon.
The last option, -log, specifies a log file for the SEC process. You could also use the -syslog option to specify a syslog facility for SEC to log to. You can specify a number of other options also, which you can see in the SEC man page.
You can also use the startup scripts that come with the SEC package to start and stop the sec.pl script. Scripts are available for Red Hat Linux, BSD and Solaris.
Of course, running SEC is just the beginning; in order to make use of it, you need to define event correlation rules. I'll discuss the SEC rules in part two of this tip. In the meantime you can have a look at some sample rules at http://www.estpak.ee/~risto/sec/examples.html and http://www.bleedingthreats.com/sec/. You can also find more information about SEC in the FAQ at http://www.estpak.ee/~risto/sec/FAQ.html.
This was first published in November 2006