rsyslog stopped shipping to NLS

[dhend]

I have had rsyslog shipping to our log server for about a week now, and last night all logs suddenly stopped shipping. The only thing I changed was a filter for a new log. Here is the error for syslog:

Code: Select all

rsyslogd: version 4.6.2, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: WARNING: rsyslogd is running in compatibility mode. Automatically generated config directives may interfer with your rsyslog.conf settings. We suggest upgrading your config and adding -c4 as the first rsyslogd option.
rsyslogd: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: the last error occured in /etc/rsyslog.d/90-nagioslogserver_opt_dotcms_log_2.5_catalina.out.conf, line 13:"$InputFilePersistStateInterval 20000"
rsyslogd: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: the last error occured in /etc/rsyslog.d/90-nagioslogserver_opt_dotcms_log_2.5_dotcms.log.conf, line 14:"$InputFilePersistStateInterval 0"
rsyslogd: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: the last error occured in /etc/rsyslog.d/90-nagioslogserver_opt_dotcms_log_2.5_dotcms_access.2014-11-14.log.conf, line 13:"$InputFilePersistStateInterval 20000"
rsyslogd: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: the last error occured in /etc/rsyslog.d/90-nagioslogserver_opt_dotcms_log_2.5_dotcms_access.2014-11-17.log.conf, line 13:"$InputFilePersistStateInterval 20000"
rsyslogd: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: the last error occured in /etc/rsyslog.d/90-nagioslogserver_opt_dotcms_log_2.5_dotcms_access.2014-11-18.log.conf, line 13:"$InputFilePersistStateInterval 20000"
rsyslogd: invalid or yet-unknown config file command - have you forgotten to load a module? [try http://www.rsyslog.com/e/3003 ]
rsyslogd: the last error occured in /etc/rsyslog.d/90-nagioslogserver_opt_dotcms_log_2.5_dotcms_access.2014-11-19.log.conf, line 13:"$InputFilePersistStateInterval 20000"
rsyslogd: the last error occured in /etc/rsyslog.conf, line 35:"$IncludeConfig /etc/rsyslog.d/*.conf"
rsyslogd: CONFIG ERROR: could not interpret master config file '/etc/rsyslog.conf'. [try http://www.rsyslog.com/e/2124 ]
rsyslogd: Warning: backward compatibility layer added to following directive to rsyslog.conf: ModLoad immark

I went back and looked at the syslog in /var/log/syslog for the past few days and it has been printing the same error code.

rsyslog.conf:

Code: Select all

#rsyslog v3 config file

# if you experience problems, check
# http://www.rsyslog.com/troubleshoot for assistance

#### MODULES ####

$ModLoad imfile
$ModLoad imuxsock.so    # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so      # provides kernel logging support (previously done by rklogd)
#$ModLoad immark.so     # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp.so
#$UDPServerRun 514

#$ModLoad imgssapi      # provides GSSAPI syslog reception

# Provides TCP syslog reception
#$ModLoad imtcp.so
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on


# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf
#rsyslog v3 config file

# if you experience problems, check
# http://www.rsyslog.com/troubleshoot for assistance

#### MODULES ####

$ModLoad imfile
$ModLoad imuxsock.so    # provides support for local system logging (e.g. via logger command)
$ModLoad imklog.so      # provides kernel logging support (previously done by rklogd)
#$ModLoad immark.so     # provides --MARK-- message capability

# Provides UDP syslog reception
#$ModLoad imudp.so
#$UDPServerRun 514

#$ModLoad imgssapi      # provides GSSAPI syslog reception

# Provides TCP syslog reception
#$ModLoad imtcp.so
#$InputTCPServerRun 514


#### GLOBAL DIRECTIVES ####

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# File syncing capability is disabled by default. This feature is usually not required,
# not useful and an extreme performance hit
#$ActionFileEnableSync on

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf

#### RULES ####

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                 /dev/console

# Log cron stuff
cron.*                  /var/log/cron

# Everybody gets emergency messages
*.emerg                 *

# Save news errors of level crit and higher in a special file.
uucp,news.crit  /var/log/spooler

# Save boot messages also to boot.log
local7.info             /var/log/boot.log

#
# Standard Configuration
#
*.crit                                  /var/log/console
syslog.info                             /var/log/syslog
mail.info                               /var/log/maillog
auth.info;authpriv.info /var/log/authlog
lpr.info                                /var/log/lpd-errs
*.info;cron.none;mail.none;auth.none;authpriv.none;syslog.none;lpr.none /var/log/messages
# Fallback in case splunkd light forwarder fails
#*.notice               @loghost.unix.medcity.net

# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* @@IPADDRESS

#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/spppl/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

Any help is GREATLY appreciated.

[scottwilkerson]

It appears that the version of rsyslogd on your box does not have this option (from rsyslog imfile docs -> http://www.rsyslog.com/doc/imfile.html):

$InputFilePersistStateInterval 20000
Available in 4.7.3+, 5.6.2+

You can safely remove that line.

[dhend]

Awesome, thank you. I was unsure if I could just remove it. I did resolve my problem however. It was an issue with a filter I had to drop certain lines in a log.