Unable to start Database Backend

[ssax]

Please send the output of these commands:

Code: Select all

su - nagios
sudo /usr/local/nagiosxi/scripts/manage_services.sh status ndo2db

Thank you

[[email protected]]

Hi,


Output as follow

[root@cs1-prd-cmn-nag01 ~]# su - nagios


[nagios@cs1-prd-cmn-nag01 ~]$ sudo /usr/local/nagiosxi/scripts/manage_services.sh status ndo2db
● ndo2db.service - Nagios Data Out Daemon
Loaded: loaded (/usr/lib/systemd/system/ndo2db.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2017-02-01 02:46:36 MST; 18h ago
Docs: http://www.nagios.org/documentation
Process: 35137 ExecStopPost=/bin/rm -f /usr/local/nagios/var/ndo2db.pid (code=exited, status=0/SUCCESS)
Process: 35135 ExecStart=/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f (code=exited, status=1/FAILURE)
Main PID: 35135 (code=exited, status=1/FAILURE)

Feb 01 02:46:36 cs1-prd-cmn-nag01 systemd[1]: Started Nagios Data Out Daemon.
Feb 01 02:46:36 cs1-prd-cmn-nag01 systemd[1]: Starting Nagios Data Out Daemon...
Feb 01 02:46:36 cs1-prd-cmn-nag01 systemd[1]: ndo2db.service: main process exited, code=exited, status=1/FAILURE
Feb 01 02:46:36 cs1-prd-cmn-nag01 systemd[1]: Unit ndo2db.service entered failed state.
Feb 01 02:46:36 cs1-prd-cmn-nag01 systemd[1]: ndo2db.service failed.
[nagios@cs1-prd-cmn-nag01 ~]$

[ssax]

Please post the output of these commands (run as root or sudo):

Code: Select all

ps aux | grep ndo2db.cfg | grep -v grep | awk '{print $2}' | xargs kill -9
systemctl restart ndo2db.service
systemctl status ndo2db.service
journalctl -xe

Thank you

[SteveBeauchemin]

I apologize for jumping in here. I just wanted to share what I recently experienced with my own ndo2db setup. I wanted that damn red icon to turn green too.

On a crash and burn system I tried to learn what I could. I noticed that old versus new ndo2db uses a different lock file name. And that the new version has a new parameter. I got these all jumbled up while playing but did finally get it clear to myself. Be aware of your ndo2db version. Decide whether you will use lock or pid in the PIDFile name.

You should know the ndo2db version that is on your system.

Code: Select all

/usr/local/nagios/bin/ndo2db --version

I currently have the old one in production at NDO2DB 2.0.0

I found that the new version on ndo2db uses the -f parameter, and the old version does not. So you care what version is there.

Only if you need the older version to run, on Red Hat 7, you need to change how it gets invoked by removing the -f
Note: Always copy the systemctl file you will change from the default install location to the /etc location.

Code: Select all

cp /usr/lib/systemd/system/ndo2db.service /etc/systemd/system/ndo2db.service
vi /etc/systemd/system/ndo2db.service
#ExecStart=/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
ExecStart=/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg

then tell the OS that you made an update

Code: Select all

systemctl daemon-reload

Now you can run version 2.0.0

If you do have the new version, NDO2DB 2.1.2, then leave that -f in place.

In both cases you really need to check the .pid and .lock file names and how they are defined.

Make sure that the PID definition is consistent between the config files. I made them all become pid.
I used the default RH7 stuff and made the rest match it. The systemctl syntax is what I consider as default.
in /usr/lib/systemd/system/ndo2db.service
--> PIDFile=/usr/local/nagios/var/ndo2db.pid
--> ExecStopPost=/bin/rm -f /usr/local/nagios/var/ndo2db.pid

I changed these other files from lock to pid so they would match the new default.
in /etc/rc.d/init.d/ndo2db
--> Ndo2dbRunFile=$Ndo2dbVarDir/ndo2db.pid
in /usr/local/nagios/etc/ndo2db.cfg
--> lock_file=/usr/local/nagios/var/ndo2db.pid

Then stop all ndo2db processes,

Code: Select all

systemctl stop ndo2db

and kill off any extraneous ndo2db processes that may still be running.

Go to the lock/pid file location and remove any ndo2db.lock or ndo2db.pid files. If there are any.

Then see if it will run the way you expect it to.

Code: Select all

systemctl restart ndo2db
systemctl status ndo2db

Yesterday, I did a totally clean OS install of CentOS 7, and then installed Nagios XI 5.4.1, and I see that the lock versus pid names are not resolved for ndo2db yet.

I followed these steps and the setup is now good. I have no idea if this is what you have as a problem.
But you should at least verify the ndo2db version and make sure that all 3 configuration files reference either pid or lock. Try to get them consistent.

Steve B

[dwhitfield]

I apologize for jumping in here.

No problem at all, @SteveBeauchemin! You're always helpful!

[email protected], does that help you with your issue?

[[email protected]]

Hi SteveBeauchemin,

You are awesome, thanks for the resolution.

I just followed the steps which you provided and i executed few additional steps too.


Steps as follows:


1. Followed steps provided by you.

2. removed ndo2db.lock file in /usr/local/nagios/var/ and created ndo2db.pid file.

3. #systemctl stop ndo2db

4. #/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
Could not bind socket: Address already in use

5. #rm /usr/local/nagios/var/ndo.sock
rm: remove socket ‘/usr/local/nagios/var/ndo.sock’? y

6. #/usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f

7. #service ndo2db restart
Restarting ndo2db (via systemctl): [ OK ]

8. #service ndo2db status
ndo2db (pid 60344) is running...
[root@cs1-prd-cmn-nag01 /]# systemctl status ndo2db
● ndo2db.service - Nagios Data Out Daemon
Loaded: loaded (/usr/lib/systemd/system/ndo2db.service; enabled; vendor prese t: disabled)
Active: active (running) since Fri 2017-02-03 01:03:28 MST; 1min 22s ago
Docs: http://www.nagios.org/documentation
Process: 36342 ExecStopPost=/bin/rm -f /usr/local/nagios/var/ndo2db.pid (code= exited, status=0/SUCCESS)
Main PID: 60344 (ndo2db)
CGroup: /system.slice/ndo2db.service
├─60344 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2...
├─60985 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2...
└─60986 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2...

Feb 03 01:03:42 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming timedevents.
Feb 03 01:03:42 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming systemcommands.
Feb 03 01:03:42 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming servicechecks.
Feb 03 01:03:42 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming hostchecks.
Feb 03 01:03:42 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming eventhandlers.
Feb 03 01:04:43 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming timedevents.
Feb 03 01:04:43 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming systemcommands.
Feb 03 01:04:43 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming servicechecks.
Feb 03 01:04:43 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming hostchecks.
Feb 03 01:04:43 cs1-prd-cmn-nag01 ndo2db[60986]: Trimming eventhandlers.

Now no alerts on my Nagios XI screen.

Thank you for all who helped me to resolve my issue.


Thanks,

[rkennedy]

Thanks Steve!

Great to hear this one resolved. Going to lock it up!

Feel free to create a new thread should you have a question in the future.