Nagios service not comming up after apply changes ...

[vishfx]

Hi Nagios Team,

After doing apply changes in NagiosXI 5.5.7 , nagios service is not coming up.
Below is the tail from nagios.log:

Code: Select all

[1543295135] Nagios 4.2.4 starting... (PID=66482)
[1543295135] Local time is Mon Nov 26 23:05:35 CST 2018
[1543295135] LOG VERSION: 2.0
[1543295135] qh: Socket '/usr/local/nagios/var/rw/nagios.qh' successfully initialized
[1543295135] qh: core query handler registered
[1543295135] nerd: Channel hostchecks registered successfully
[1543295135] nerd: Channel servicechecks registered successfully
[1543295135] nerd: Channel opathchecks registered successfully
[1543295135] nerd: Fully initialized and ready to rock!
[1543295135] wproc: Successfully registered manager as @wproc with query handler
[1543295135] wproc: Registry request: name=Core Worker 66483;pid=66483
[1543295135] wproc: Registry request: name=Core Worker 66484;pid=66484
[1543295135] wproc: Registry request: name=Core Worker 66486;pid=66486
[1543295135] wproc: Registry request: name=Core Worker 66488;pid=66488
[1543295135] wproc: Registry request: name=Core Worker 66490;pid=66490
[1543295135] wproc: Registry request: name=Core Worker 66491;pid=66491
[1543295135] wproc: Registry request: name=Core Worker 66493;pid=66493
[1543295135] wproc: Registry request: name=Core Worker 66489;pid=66489
[1543295135] wproc: Registry request: name=Core Worker 66492;pid=66492
[1543295135] wproc: Registry request: name=Core Worker 66487;pid=66487
[1543295135] wproc: Registry request: name=Core Worker 66494;pid=66494
[1543295135] wproc: Registry request: name=Core Worker 66485;pid=66485
[1543295135] mod_gearman: initialized version 2.1.1 (libgearman 0.33)
[1543295135] Event broker module '/usr/lib64/mod_gearman2/mod_gearman2.o' initialized successfully.
[1543295135] ndomod: NDOMOD 2.1.2 (11-14-2016) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
[1543295135] ndomod: Successfully connected to data sink.  0 queued items to flush.
[1543295135] ndomod registered for process data
[1543295135] ndomod registered for log data'
[1543295135] ndomod registered for system command data'
[1543295135] ndomod registered for event handler data'
[1543295135] ndomod registered for notification data'
[1543295135] ndomod registered for comment data'
[1543295135] ndomod registered for downtime data'
[1543295135] ndomod registered for flapping data'
[1543295135] ndomod registered for program status data'
[1543295135] ndomod registered for host status data'
[1543295135] ndomod registered for service status data'
[1543295135] ndomod registered for adaptive program data'
[1543295135] ndomod registered for adaptive host data'
[1543295135] ndomod registered for adaptive service data'
[1543295135] ndomod registered for external command data'
[1543295135] ndomod registered for aggregated status data'
[1543295135] ndomod registered for retention data'
[1543295135] ndomod registered for contact data'
[1543295135] ndomod registered for contact notification data'
[1543295135] ndomod registered for acknowledgement data'
[1543295135] ndomod registered for state change data'
[1543295135] ndomod registered for contact status data'
[1543295135] ndomod registered for adaptive contact data'
[1543295135] Event broker module '/usr/local/nagios/bin/ndomod.o' initialized successfully.

Below are more screenshots:

nagios-core-not-startingup-Capture.PNG

nagios-core-not-startingup2-Capture.PNG

Kindly assist.

Regards,
Vish.

[vishfx]

I managed to bring it up using below steps:

1) Comment below line in nagios.cfg :
#broker_module=/usr/lib64/mod_gearman2/mod_gearman2.o config=/etc/mod_gearman2/module.conf eventhandler=no

2) sudo /usr/bin/systemctl start nagios

3) Un-Comment below line in nagios.cfg :
broker_module=/usr/lib64/mod_gearman2/mod_gearman2.o config=/etc/mod_gearman2/module.conf eventhandler=no

4) sudo /usr/bin/systemctl start nagios

After this step nagios process started running.

Would you know what is going on here ?

Regards,
Vish

[vishfx]

mod_gearman_issue.zip

This work around doesn't seem to be working anymore.

But when I disable mod_gearman2.o , nagios comes up.

Am running Nagios 5.5.5 and Downgraded Nagios Core to 4.2.4 ( https://support.nagios.com/kb/article.php?id=823 )

nagios-core-not-startingup3-Capture.PNG

Below are the rpm's installed using https://assets.nagios.com/downloads/nag ... ios_XI.pdf :

Code: Select all

gearmand.x86_64                1:0.33-2               installed
gearmand-debuginfo.x86_64      1:0.33-2               installed
gearmand-devel.x86_64          1:0.33-2               installed
gearmand-server.x86_64         1:0.33-2               installed
mod_gearman2.x86_64            2.1.1-1.el7.centos     installed

PFA strace output.

Kindly assist.

Regards,
Vish.

[cdienger]

The order that XI and gearman are stopped and started matters. Following the commands on page 6 of https://assets.nagios.com/downloads/nag ... ios_XI.pdf to first stop the services and then start them.

[vishfx]

Hi,

Tried following the order, still nagios does not come up.

In my case , Only gearmand is running on NagiosXI web node.
mod_gearman worker are running on 2 remote nodes.

Also, enable core dump in nagios.cfg
Nagios generated a core dump and crashed.
PFA core dump file.

This is critical for us, as at this point, Monitoring is down, kindly assist.

Regards,
Vish.

[cdienger]

Please open a a ticket at https://support.nagios.com/tickets/ and we can take a closer look.

[vishfx]

Below support ticket has been opened.
https://support.nagios.com/tickets/tickets.php?id=4033

[cdienger]

I've taken the ticket and responded.