rsyslog using regex

[WVUhealth]

Hello all.
I've been trying to use the brief examples in my rsyslog to throw logs into a file before it hits the end and sends to nagios logger..

Has anyone gotten rsyslog regex to work ? Would you mind sharing ?
Here is the full string in the messages that id like to throw into a diff file

Sep 14 08:55:01 Linux-System127 systemd: Starting Session 174675 of user nagios.
msg, regex, "Starting Session .* of user nagios." -/var/log/nagios-logs

Now i got my example from
http://www.rsyslog.com/doc/v8-stable/co ... lters.html
:msg, regex, "fatal .* error"
POSIX regular expression. It matches when the string contains the words “fatal” and “error” with anything in between (e.g. “fatal net error” and “fatal lib error” but not “fatal error” as two spaces are required by the regular expression!).

When i rehup rsyslog it gives me
rsyslogd-2207: error during parsing file /etc/rsyslog.conf, on or before line 52: errors occured in file '/etc/rsyslog.conf' around line 52 [try http://www.rsyslog.com/e/2207 ]

rsyslogd -v output
rsyslogd 7.4.7, compiled with:
FEATURE_REGEXP: Yes
FEATURE_LARGEFILE: No
GSSAPI Kerberos 5 support: Yes
FEATURE_DEBUG (debug build, slow code): No
32bit Atomic operations supported: Yes
64bit Atomic operations supported: Yes
Runtime Instrumentation (slow code): No
uuid support: Yes

[rkennedy]

Could you please post your entire rsyslog.conf file for us to look at?

[WVUhealth]

Here is the basic rsyslog.conf
Can you give see any issues with the regex i was trying to use that i have commented out

Code: Select all

# msg, regex, "Starting Session .* of user nagios." -/var/log/cron
#& stop

full rsyslog.conf file.

Code: Select all

# rsyslog configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

# The imjournal module bellow is now used as a message source instead of imuxsock.
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imjournal # provides access to the systemd journal
#$ModLoad imklog # reads kernel messages (the same are read from journald)
#$ModLoad immark  # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514

# Provides TCP syslog reception
#$ModLoad imtcp
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Where to place auxiliary files
$WorkDirectory /var/lib/rsyslog

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

# Turn off message reception via local log socket;
# local messages are retrieved through imjournal now.
$OmitLocalLogging on

# File to store the position in the journal
$IMJournalStateFile imjournal.state


#### RULES ####

:msg, contains, "nrpe" -/var/log/nrpe.log
& stop
##local firewall logs redirect
:msg, contains, "MAC=00:0d:3a:12:15:59" -/var/log/firewall.log
& stop
# msg, regex, "Starting Session .* of user nagios." -/var/log/cron
#& stop

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages

# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure

# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog


# Log cron stuff
cron.*                                                  /var/log/cron

# Everybody gets emergency messages
*.emerg                                                 :omusrmsg:*

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          /var/log/spooler

# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

[mcapra]

At a very basic level, you could use the contains operand to look for messages that contain "Starting Session" and "of user nagios." and forward the entire event to a log file.

I did this with messages similar to EXIT: nrpe status=0 pid=25830 duration=0(sec) like so:

Code: Select all

if $msg contains 'EXIT: nrpe status' then /var/log/nrpe_exit.log

Which produces the following log file (with the entire message):

Code: Select all

[root@localhost log]# tail /var/log/nrpe_exit.log
Sep 20 15:36:46 localhost xinetd[904]: EXIT: nrpe status=0 pid=627 duration=0(sec)
Sep 20 15:36:47 localhost xinetd[904]: EXIT: nrpe status=0 pid=631 duration=0(sec)
Sep 20 15:36:52 localhost xinetd[904]: EXIT: nrpe status=0 pid=639 duration=0(sec)
Sep 20 15:36:58 localhost xinetd[904]: EXIT: nrpe status=0 pid=653 duration=1(sec)
Sep 20 15:37:01 localhost xinetd[904]: EXIT: nrpe status=0 pid=642 duration=5(sec)
Sep 20 15:37:03 localhost xinetd[904]: EXIT: nrpe status=0 pid=1390 duration=0(sec)
Sep 20 15:37:04 localhost xinetd[904]: EXIT: nrpe status=0 pid=1397 duration=0(sec)
Sep 20 15:37:15 localhost xinetd[904]: EXIT: nrpe status=0 pid=1401 duration=0(sec)
Sep 20 15:37:19 localhost xinetd[904]: EXIT: nrpe status=0 pid=1405 duration=0(sec)
Sep 20 15:37:23 localhost xinetd[904]: EXIT: nrpe status=0 pid=1412 duration=0(sec)

You could also do your regex match like so:

Code: Select all

# Catch Nagios session start messages, redirect them to a file
:msg, regex, "Starting Session .* of user nagios."      /var/log/nagios_session.log

Which produces the following file with my test inputs:

Code: Select all

[root@localhost log]# tail /var/log/nagios_session.log
Sep 20 16:00:18 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 214675 of user nagios.
Sep 20 16:00:19 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 214675 of user nagios.
Sep 20 16:00:19 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 214675 of user nagios.
Sep 20 16:00:19 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 214675 of user nagios.
Sep 20 16:00:20 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 214675 of user nagios.
Sep 20 16:01:23 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 224675 of user nagios.
Sep 20 16:01:26 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 234675 of user nagios.
Sep 20 16:01:29 localhost root: Sep 14 08:55:01 Linux-System127 systemd: Starting Session 244675 of user nagios.

I think the -at the start of your output as well as a missing : from the beginning of the filter is what is tripping stuff up. The above both worked for me on rsyslog 7.4.7.

I would also make sure that the file you are trying to log to exists and that the rsyslogd daemon can write to it, or that the rsyslogd daemon has write permissions on the /var/log/ path.

[WVUhealth]

Thanks for posting my syntax error.. I just wanted to let you know that as you can seen i had the line commented out and i deleted the : by accident ..
But i did take out the hyphen - not getting the rsyslog error but still not matching ..!!!
PS Thank you for the posting about using msg contains but im really trying to see how to utilize regex ..

The GOOD NEWS!!

So i dropped your config right under my rule and kept mine commented out.. WTF yours worked.. Nothing diff other than the log destination..
Ok im not 100% happy about this but whatever for now..
One issue im still having.. The rule is still not stopping the log and rsyslog is still throwing that log to /var/log/messages..

#:msg, regex, "Starting Session .* of user nagios." /var/log/cron
#& stop
:msg, regex, "Starting Session .* of user nagios." /var/log/nagios_session.log
& stop

[WVUhealth]

added to have the thread notify me when reply is posted,.. Sorry it took so long to respond to your reply.

[mcapra]

See if discarding the message after adding it to the file helps. See line 3 with the ~ character:

Code: Select all

# Catch Nagios session start messages, redirect them to a file
:msg, regex, "Starting Session .* of user nagios."      /var/log/nagios_session.log
:msg, regex, "Starting Session .* of user nagios."      ~

[WVUhealth]

No still making it all the way to the catchall *.info;mail.none;authpriv.none;cron.none /var/log/messages
Ive even tried
:msg, regex, "Starting Session .* of user nagios." /var/log/nagios_session.log
& ~
:msg, regex, "Starting Session .* of user nagios." ~

I have stopped using the ~ as i get logs like
rsyslogd-2307: warning: ~ action is deprecated, consider using the 'stop' statement instead [try http://www.rsyslog.com/e/2307 ]

[WVUhealth]

I did use your other example and match on msg conatins right afterwards and it did stop processing the log.
:msg, regex, "Starting Session .* of user nagios." /var/log/nagios_session.log
& stop
# :msg, regex, "Starting Session .* of user nagios." ~
if $msg contains 'of user nagios.' then ~

Im good with this outcome and your support.. As this proved my regex foo is bad..
and just because i think it should work one way doesn't mean it dose.. Plus i get to use the UNIX way of thinking .. Use a tool as many times as you need it till you get what you want.. ie
if i cant make it work with one statement .. add another..

Thanks again for the help ..

[mcapra]

Sure thing

Is it alright if we lock this thread and mark the issue as resolved?