Page 1 of 2

duplicate message log when use NFS and rsyslog

Posted: Fri Jan 15, 2021 2:35 am
by sacom01
Hi all,
I have some aix servers which need push log messages to Nagios LS.
I use NFS, and rsyslog but the log messages will be duplicated.
Do we have any ways apart from rsyslog in this case?
Thank you.

Re: duplicate message log when use NFS and rsyslog

Posted: Fri Jan 15, 2021 5:37 pm
by cdienger
I'm not sure I fully understand the setup. The same logs are being sent twice - once via NFS somehow and the other via rsyslog? Can you please explain more? We may be able to configure something on NLS to drop logs, but ideally it would be best to not send duplicates in the first place.

Re: duplicate message log when use NFS and rsyslog

Posted: Sun Jan 17, 2021 11:00 pm
by sacom01
hi cdienger,
Thanks for your respone.
This is my setup :
1. There are many log files on Filesystem1 on server A, i use NFS to mount Filesystem1 to server B.
2. On server B, using scripts to COPY log files from mounted filesystem (Filesystem1) to local filesystem (Filesystem2) (name log_files_bk).
3. Using rsyslog to push log_files_bk to Nagios server --> Duplicate massages.

I dont know the COPY step is cause of duplicate or not and how to fix.

Re: duplicate message log when use NFS and rsyslog

Posted: Tue Jan 19, 2021 10:40 am
by cdienger
Is there configuration on server A also sends logs via rsyslog? Can you disable the COPY and see if logs are still sent from server A?

Re: duplicate message log when use NFS and rsyslog

Posted: Tue Jan 19, 2021 8:46 pm
by sacom01
hi cdienger,
There is no rsyslog config on server A. The log file on server A is realtime log, but we can not get log from this svr, so we use script to copy log row by row from server A to server B. After that use rsyslog to push log from server B to Nagios LS.
Filesystem 2 is local filesystem of ServerB.
thanks.

Re: duplicate message log when use NFS and rsyslog

Posted: Wed Jan 20, 2021 12:26 pm
by cdienger
Can you provide details or a copy of the scrip that copies logs from one machine to the other?

Can you provide the rsyslog configuration files for server B?

I want to check to see if the logs are getting duplicated on on server B somehow and rsyslog is picking up both.

Re: duplicate message log when use NFS and rsyslog

Posted: Wed Jan 20, 2021 10:39 pm
by sacom01
Mny thanks cdienger,
Here is the copy scripts :
cp -p /log/runtime.log /logbk/runtime.log
cp -p /log/database.log /logbk/database.log
cp -p /log/ejb.log /logbk/ejb.log
cp -p /log/locking.log /logbk/locking.log
cp -p /log/mdb.log /logbk/mdb.log

rsyslog config file :
$ModLoad imfile
$InputFileName /logbk/*.log
$InputFilePollInterval 10
$InputFileTag processlog
$InputFileStateFile /tmp/runtime
$InputFileSeverity info
$InputFileFacility local2
$InputRunFileMonitor
$InputUDPServerKeepalive on

template (name="TraditionalFormat" type="string" string="%msg%\n")
local2.* @192.168.xxx.xx:5050;TraditionalFormat

Re: duplicate message log when use NFS and rsyslog

Posted: Thu Jan 21, 2021 6:16 pm
by cdienger
Disable or remove the $InputFileStateFile line from rsyslog configuration. This can cause problems when using wildcards to monitor multiple logs. Rsyslog will automatically generate state files as needed with the InputFileStateFile option removed.

Re: duplicate message log when use NFS and rsyslog

Posted: Fri Jan 22, 2021 8:47 pm
by sacom01
hi cdienger,
i have tried remove the $InputFileStateFile line from rsyslog configuration and push message but it still getting duplicate issue :(

Re: duplicate message log when use NFS and rsyslog

Posted: Mon Jan 25, 2021 3:56 pm
by cdienger
Did you restart rsyslog after making the change?