Getting AuditD logs from a Linux Host

[krobertson71]

We are attempting to capture audit events from our Linux hosts. Particularly when someone tries to access a file or folder and is rejected.

These events are broken up over 3 records so getting them into NLS is not happening.

The log file in question is /var/log/audit/audit.log. Standard Linux log file.

Has anyone had issues with the log format? Is NLS setup already to handle this but we are doing something wrong on our end? All our syslog events are being forward to NLS via a central syslog server.

Here are the 3 entries that show my access of /var/log/audit/audit.log.

Code: Select all

type=SYSCALL msg=audit(1471971838.950:32791): arch=c000003e syscall=2 success=no exit=-13 a0=7ffff362bd96 a1=0 a2=7ffff362a3c0 a3=7ffff362bf11 items=1 ppid=7949 pid=8205 auid=12001 uid=12001 gid=12001 euid=12001 suid=12001 fsuid=12001 egid=12001 sgid=12001 fsgid=12001 tty=pts0 ses=5315 comm="cat" exe="/bin/cat" key="file_access"
type=CWD msg=audit(1471971838.950:32791):  cwd="/home/userxxx"
type=PATH msg=audit(1471971838.950:32791): item=0 name="/var/log/audit/audit.log"

[mcapra]

A simple grok filter did the trick for me:

Code: Select all

if [program] == 'audit_log' {
grok {
  pattern => [ "type=%{DATA:audit_type}\smsg=audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\):.*?( msg=\'(?<sub_msg>.*?)\')?$" ]
  named_captures_only => true
}
kv {
  exclude_keys => [ "msg", "type" ]
}
}

Using the following setup one-liner:

Code: Select all

sudo bash setup-linux.sh -s 192.168.67.3 -p 5544 -f "/var/log/audit/audit.log" -t audit_log

There are also likely permissions considerations when dealing with the audit log. Mine is 400 owned by root.root.

[krobertson71]

The problem is they are not getting parsed at all. Very hard to find them. The snippet I provided was from the audit.log file itself. Very difficult to find the events in NLS as they are not getting parsed. when I try to search UID=12001 it just returns matches to UID.

I have even tried message:"uid=12001"... still no dice.

[rkennedy]

Can you show us how you're having the logs input into NLS? Are they using a syslog input or raw tcp / udp?

[krobertson71]

According to our Linux head admin AuditD "audit.log" is not in a syslog format. It is forwarded to NLS via a rsyslog imfile plugin.


cat /etc/rsyslog.d/01_auditd.conf
# Input for auditd
$InputFileName /var/log/audit/audit.log
$InputFileTag auditd:
$InputFileStateFile state-auditd
$InputFileSeverity info
$InputFileFacility local5
$InputRunFileMonitor

[krobertson71]

He was sending it to the Syslog input on port 5544 so I assume it was getting parsed as syslog?

Like I mentioned before, he is using an imfile to put the file together. I am going to try to have him send it into the raw port.

I there another input I should set up?

[mcapra]

I am going to try to have him send it into the raw port.

That is what I would recommend. With raw tcp/udp inputs it's a bit easier to identify specific parsing issues.

[krobertson71]

Not sure if that is going to work.

An event generated by auditd is split into 3 records. That is why I am having the issue. Does your grok filter stitch these records together?

[mcapra]

If all you need is the uid, and all of the files follow the same conventions in terms of saying "this is the uid", you should be able to write a grok filter to capture that field.

Say you're taking in 3 files that are held locally (just as an example). You could define the inputs like so:

Code: Select all

    file {
            type => "AuditD"
            path => "/var/log/audit1.log"
    }
    file {
            type => "AuditD"
            path => "/var/log/audit2.log"
    }
    file {
            type => "AuditD"
            path => "/var/log/audit3.log"
    }

You could write a filter to pull out the uid from those files like so using the type as the common identifier:

Code: Select all

if [type] == 'AuditD' {
     grab my uid
}

The problem is that you will still have 3 separate entries for the 3 separate files; There isn't a very tidy way to take data from 3 separate inputs and say "make this one input".

If one system event is split into 3 different files, you're essentially dealing with 3 separate events as far as NLS is concerned. You'd be farther ahead writing a script to combine those 3 files, ship that file to logstash, then do your filtering. It's a limitation of logstash unfortunately.

You could probably do clever things with an elasticsearch output to merge several different events into a single event, but that's really a bit outside of what the ELK stack was designed for and is more trouble than it's worth (just my opinion).

[eloyd]

Logstash does have a multiline codec, but I've never stitched it in to NLS. If you're interested in the codec, you can read about it here: https://www.elastic.co/guide/en/logstas ... iline.html and it should be straightforward to implement.