section_4.md

Osquery and Elasticsearch

Osquery is a cross-platform system monitoring tool that exposes information about a system through a SQL interface. Elasticsearch is a distributed schema-free search server based on Lucene. In this tutorial, you will learn first how to send osquery logs to syslog, and then send those logs to Elasticsearch.

##Osquery

Osquery reporting

Osquery runs in two modes - osqueryi, an interctive mode, and osqueryd, a daemon mode. Osquery's daemon mode allows for scheduled configurable queries to be run without user interaction. Conveniently, osqueryd natively supports sending the result of its scheduled queries to syslog in a JSON format.

To quickly setup osqueryd, once installed, copy its example configuration file osquery.example.conf (in my system it islocated in /usr/share/osquery/) to /etc/osquery/osquery.conf. Once copied, change the logger plugin in the osquery configuration file from file to syslog. To do this, change the line

"logger_plugin": "filesystem", 

to

"logger_plugin": "syslog", 

For reference, these logs are sent with a facility of 19, or "local3". On my Ubuntu 14.04 installation, Rsyslog was not logging these messages. To fix that, I have added the following line to the end of my rsyslog.conf file:

local3.* -/var/log/syslog

###The syslog-ng config file

To gather the logs from osquery, you need a source that collects system messages, and a filter that can identify logs from osqueryd.

An example of such a source is:

source s_local {
        system();
};

An example of such a filter is:

filter osqueryd {

program("^osqueryd.*");
};

##Elasticsearch

At the time of writing, there is not an official Elasticsearch destination in the syslog-ng repository, but it will be added soon. Until then, the source for a suitable destination can be found at https://github.com/juhaszviktor/ESDestination. Full documentation on the destination can be found there.

###Requirements

• Elasticsearch libraries
• SyslogNg.jar
• JDK min 1.7 for building
• JRE min 1.7 for usage

###Building You may need to change the directory of the relevant .jar files. If you do, you can specify their locations by:

javac -classpath "/usr/lib/syslog-ng/3.6/SyslogNg.jar:/usr/share/elasticsearch/lib/elasticsearch-1.4.4.jar:/usr/share/elasticsearch/lib/log4j-1.2.17.jar" -d bin src/org/syslog_ng/elasticsearch/ElasticSearchDestination.java

###Packaging

jar -cvf ESDestination.jar -C bin .

###Example Configuration

####Environment

• In this example the SyslogNg.jar file is under the directory /usr/lib/syslog-ng/3.6
• The elasticsearch libraries are under the /usr/share/elasticsearch/lib/ directory
• The built .jar file is under the /usr/local/ directory

####Configuration example The following configuration file example takes messages from osquery and sends them to Elasticsearch:

@version: 3.6

options {
  threaded(yes);
  use_rcptid(yes);
};

source s_local {
        system();
};

filter osqueryd {

program("^osqueryd.*");
};


destination d_es {
  java(
    class_path("/usr/local/ESDestination.jar:/usr/share/elasticsearch/lib/*.jar")
    class_name("org.syslog_ng.elasticsearch.ElasticSearchDestination")
    option("index", "syslog-ng_${YEAR}.${MONTH}.${DAY}")
    option("type", "test")
    option("cluster", "syslog-ng")
    option("flush_limit", "100")
  );
};

log {
  source(s_local);
  filter(osqueryd);
  destination(d_es);
  flags(flow-control);
};