Page 1 of 2
Lock File Moved after Config Failure
Posted: Wed May 29, 2019 12:09 pm
by blariv
Hi all,
I have been having this problem for a while now. I thought it was addressed in an earlier post of mine concerning my monitoring engine not starting. I have been digging around in the forum and found this post. It is exactly my issue. I tried following the fixes but when I made all the changes my engine would just hang. Attached is my system profile.
https://support.nagios.com/forum/viewto ... ck#p273193
Thanks,
Brian
Re: Lock File Moved after Config Failure
Posted: Wed May 29, 2019 3:39 pm
by npolovenko
Hello,
@blariv. Please run the following commands and show us the location of the nagios.lock file in each directory.
Code: Select all
cat /etc/rc.d/init.d/nagios | grep NagiosRunFile=
cat /usr/local/nagiosxi/scripts/nom_restore_nagioscore_checkpoint.sh | grep lockfile=
cat /usr/local/nagiosxi/scripts/nom_restore_nagioscore_checkpoint_specific.sh | grep lockfile=
cat /usr/local/nagios/etc/nagios.cfg | grep lock
Re: Lock File Moved after Config Failure
Posted: Wed May 29, 2019 4:07 pm
by blariv
I had changed these before to all be the "/var/run/nagios.lock" that are now commented out
[root@nagiosxi libexec]# cat /etc/rc.d/init.d/nagios | grep NagiosRunFile=
#NagiosRunFile=/var/run/nagios.lock
NagiosRunFile=${prefix}/var/nagios.lock
[root@nagiosxi libexec]# cat /usr/local/nagiosxi/scripts/nom_restore_nagioscore_checkpoint.sh | grep lockfile=
#lockfile="/var/run/nagios.lock"
lockfile="/usr/local/nagios/var/nagios.lock"
[root@nagiosxi libexec]# cat /usr/local/nagiosxi/scripts/nom_restore_nagioscore_checkpoint_specific.sh | grep lockfile=
#lockfile="/var/run/nagios.lock"
lockfile="/usr/local/nagios/var/nagios.lock"
[root@nagiosxi libexec]# cat /usr/local/nagios/etc/nagios.cfg | grep lock
lock_file=/usr/local/nagios/var/nagios.lock
#lock_file=/var/run/nagios.lock
Re: Lock File Moved after Config Failure
Posted: Wed May 29, 2019 4:58 pm
by npolovenko
@blariv, Is the nagios process starting with this lock file configuration? Or is it hanging? Is the lock file path getting reverted to the one you commented out?
Re: Lock File Moved after Config Failure
Posted: Thu May 30, 2019 6:35 am
by blariv
its starts fine with this config, however if we happen to have a failed "apply configuration" due to misconfigured hosts or services the lock file location in /usr/local/nagios/etc/nagios.cfg changes to "lock_file=/var/run/nagios.lock". the engine will no longer start until I change the lock file location back to "lockfile="/usr/local/nagios/var/nagios.lock", delete the old locks, reload the deamon and then start nagios.
I can repeat this at any time as well if you need logs.
Re: Lock File Moved after Config Failure
Posted: Thu May 30, 2019 10:57 am
by npolovenko
@blariv, This is strange. Are you on CentOS 6 or CentOS 7?
What's the location of the NagiosRunFile in /etc/init.d/nagios ?(if you have one on your server)
Please change all nagios.lock file locations to /var/run/nagios.lock.
Then run the following commands in order:
Code: Select all
service crond stop
service npcd stop
service nagios stop
service ndo2db stop
pkill -9 -u nagios
for i in $(ipcs -q | grep nagios |awk '{print $2}'); do ipcrm -q $i; done
rm -rf /usr/local/nagiosxi/var/dbmaint.lock
rm -rf /usr/local/nagiosxi/var/event_handler.lock
rm -rf /usr/local/nagiosxi/scripts/reconfigure_nagios.lock
service mysqld restart
service ndo2db start
service nagios start
service npcd start
service crond start
And if nagios process "hangs" again, please send me the /usr/local/nagios/var/nagios.log.
Also, when you mentioned that your engine just hangs, did you mean it's not starting, or Apply Configuration hangs?
Re: Lock File Moved after Config Failure
Posted: Thu May 30, 2019 11:07 am
by blariv
very strange indeed.
we are running RHEL7
init.d
NagiosRunFile=${prefix}/var/nagios.lock
the nagios service just hangs, not the apply config.
I will give this a shot and post results.
Re: Lock File Moved after Config Failure
Posted: Thu May 30, 2019 11:59 am
by npolovenko
Sounds good.
Re: Lock File Moved after Config Failure
Posted: Thu May 30, 2019 2:39 pm
by blariv
same problem
see attached log and screenshot
Re: Lock File Moved after Config Failure
Posted: Thu May 30, 2019 2:53 pm
by npolovenko
@blariv, Please keep the lock file set to /var/run/ for now. Just to be sure, it's currently set to /var/run in all locations that were mentioned in this thread?
What's the output of all the following commands?
ls -ld /var/run/
ls -l /var/run/
ps -ef | grep nagios.cfg
/usr/local/nagiosxi/scripts/manage_services.sh status nagios