log source with "rolling" filename

[zakhemj]

Hello

My question is regarding creating a logsource for logs with custom names that change over time. For example, new Apache httpd access and error log files are creating every hour and the file names contain the timestamp. I thought passing a wildcard in the logsource would handle the scenario, but logs cease to be collected once the hour has passed:

e.g.
Apache httpd accesslogs

creating a log source:

Code: Select all

sudo bash setup-linux.sh -s (NLS fqdn) -p 5544 -f /srv/sandboxes/sb10/logs/*access_log -t apache_access

Your system $PATH does not include /sbin and /usr/sbin. This could be the result of installing GNOME rather than creating a clean system.
Adding /sbin and /usr/sbin to $PATH.
Detected rsyslog 5.8.12
Detected rsyslog work directory /var/lib/rsyslog
Destination Log Server: (NLS fqdn):5544
Creating /etc/rsyslog.d/90-nagioslogserver_srv_sandboxes_sb10_logs_web21-sb10-20150316-1300-access_log.conf...
SELinux is disabled.
rsyslog configuration check passed.
Restarting rsyslog service with 'service'...
Shutting down system logger:                               [  OK  ]
Starting system logger:                                    [  OK  ]
Okay.
rsyslog is running with the new configuration.
Visit your Nagios Log Server dashboard to verify that logs are being received.

corresponding rsyslog.d conf file:

Code: Select all

cat 90-nagioslogserver_srv_sandboxes_sb10_logs_web21-sb10-20150316-1300-access_log.conf

$ModLoad imfile
$InputFilePollInterval 10
$PrivDropToGroup adm
$WorkDirectory /var/lib/rsyslog

# Input for apache_access
$InputFileName /srv/sandboxes/sb10/logs/web21-sb10-20150316-1300-access_log
$InputFileTag apache_access:
$InputFileStateFile nls-state-srv_sandboxes_sb10_logs_web21-sb10-20150316-1300-access_log # Must be unique for each file being polled
# Uncomment the folowing line to override the default severity for messages
# from this file.
#$InputFileSeverity info
$InputFilePersistStateInterval 20000
$InputRunFileMonitor

# Forward to Nagios Log Server and then discard, otherwise these messages
# will end up in the syslog file (/var/log/messages) unless there are other
# overriding rules.
if $programname == 'apache_access' then @@(NLS fqdn):5544
if $programname == 'apache_access' then ~

I have attempted editing the conf file to match all files that end with "access_log" but to no avail.

Any help would be appreciated. I would be happy to provide more information if necessary.

Thanks.

[jolson]

Hello,

After some testing, I am experiencing the same issues that you are. I found this post in the rsyslog documentation: http://www.rsyslog.com/doc/master/confi ... mfile.html

File [/path/to/file]

(Required Parameter) The file being monitored. So far, this must be an absolute name (no macros or templates). Note that wildcards are supported at the file name level (see “Wildcards” above for more details).

Note that wildcards are supported as of rsyslog version 8.5. The rsyslog version that our script uses is typically 5.8.10, which is why globbing is not working properly.

There are a few workarounds:
-Download and install rsyslog version 8.5+, which would enable wildcard usage.
-Download and configure syslog-ng, which will be able to parse files via wildcard and send log files properly.
-Create a symlink that points to the logfile which you want to monitor, and use a script to change the re-point the symlink to the new logs appropriately.

Code: Select all

ln -s /var/log/httpd/access_log-20150315 apache_access

Let me know if this helps. Thanks!

[zakhemj]

Hi jolson

Thanks for your reply. I am setting up the symlink option now. I was hoping to avoid this but didn't realize the limitation on rsyslog5 limitation with wildcards.

Take care.

[jolson]

Sounds good - let us know if this works for you. Thanks!