XI System Component Status red XI 5.5.2

[sib]

Hi

After having installed XI 5.5.2 some of the XI System Compontent Status are red

sstatus.PNG

The services seem to be running I think. Also I reset the config permissions

Code: Select all

# /usr/local/nagiosxi/scripts/reset_config_perms.sh

--- reset_config_perms.sh ------------
> Setting CCM script permissions
> Setting script permissions
> Setting special component script permissions
> Setting configuration file/directory permissions
> Setting perfdata directory and RRD permissions
> Setting NOM checkpoint user:group permissions
> + Setting CCM configuration file user:group permissions
> + Setting Recurring Downtime file user:group permissions
> + Setting BPI configuration file user:group permissions
--------------------------------------

No SELinux issues.

[scottwilkerson]

Can you show the output of the following

Code: Select all

service crond status
change -l nagios
getenforce

[sib]

Code: Select all

# chage -l nagios
Last password change                                    : Aug 08, 2018
Password expires                                        : Oct 07, 2018
Password inactive                                       : never
Account expires                                         : never
Minimum number of days between password change          : 2
Maximum number of days between password change          : 60
Number of days of warning before password expires       : 7

Code: Select all

# service crond status
Redirecting to /bin/systemctl status crond.service
● crond.service - Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-08-10 15:23:11 CEST; 28min ago
 Main PID: 1032 (crond)
    Tasks: 1
   Memory: 3.6M
   CGroup: /system.slice/crond.service
           └─1032 /usr/sbin/crond -n

Aug 10 15:23:11 lbtss1000.ux.bisinfo.org systemd[1]: Started Command Scheduler.
Aug 10 15:23:11 lbtss1000.ux.bisinfo.org systemd[1]: Starting Command Scheduler...
Aug 10 15:23:11 lbtss1000.ux.bisinfo.org crond[1032]: (CRON) INFO (RANDOM_DELAY will be scaled with factor 60% if used.)
Aug 10 15:23:11 lbtss1000.ux.bisinfo.org crond[1032]: (CRON) INFO (running with inotify support)

Code: Select all

# getenforce
Disabled

[scottwilkerson]

Can you send the output of

Code: Select all

tail -50 /var/log/cron

And a profile.zip from Admin -> System Profile

thanks

[sib]

Hi

Some erros have been cleared.

sstatus2.PNG

This is the cron output

Code: Select all

Aug 13 09:27:01 lbtss1000 CROND[14199]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 09:27:01 lbtss1000 CROND[14198]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 09:27:01 lbtss1000 CROND[14201]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 09:27:01 lbtss1000 CROND[14197]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 09:27:01 lbtss1000 CROND[14196]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 09:27:01 lbtss1000 CROND[14206]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 09:27:01 lbtss1000 CROND[14208]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14663]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14665]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14666]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14670]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14671]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14674]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14668]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14675]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14677]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 09:28:01 lbtss1000 CROND[14678]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15126]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15131]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15129]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15130]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15138]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15135]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15136]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15137]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15140]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 09:29:02 lbtss1000 CROND[15141]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15608]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15610]: (root) CMD (LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lib/mrtg/mrtg.lock --confcache-file /var/lib/mrtg/mrtg.ok)
Aug 13 09:30:01 lbtss1000 CROND[15611]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15616]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15614]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15615]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15620]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php >> /usr/local/nagiosxi/var/dbmaint.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15619]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15618]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15612]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Aug 13 09:30:01 lbtss1000 CROND[15617]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15629]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Aug 13 09:30:01 lbtss1000 CROND[15628]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16092]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16094]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16102]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16103]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16097]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16098]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16101]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16099]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16104]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Aug 13 09:31:01 lbtss1000 CROND[16100]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)

I cannot get a build profile. Even after disabling SELinux

Code: Select all

PROFILE BUILD FAILED
Array
(
)
CODE: 1

[scottwilkerson]

Lets run the following to get the others up

Code: Select all

service ndo2db stop
service nagios stop
killall -9 nagios
rm -f /usr/local/nagios/var/ndo*ck
service ndo2db start
service npcd restart
service nagios start

[sib]

Code: Select all

# service ndo2db stop
Redirecting to /bin/systemctl stop ndo2db.service

[root][lbtss1000.ux.bisinfo.org][/home/sa002854][0]
# service nagios stop
Redirecting to /bin/systemctl stop nagios.service

[root][lbtss1000.ux.bisinfo.org][/home/sa002854][0]
# killall -9 nagios
nagios: no process found

[root][lbtss1000.ux.bisinfo.org][/home/sa002854][1]
# rm -f /usr/local/nagios/var/ndo*ck

[root][lbtss1000.ux.bisinfo.org][/home/sa002854][0]
# service ndo2db start
Redirecting to /bin/systemctl start ndo2db.service

[root][lbtss1000.ux.bisinfo.org][/home/sa002854][0]
# service npcd restart
NPCD Stopped.
NPCD started.

[root][lbtss1000.ux.bisinfo.org][/home/sa002854][0]
# service nagios start
Redirecting to /bin/systemctl start nagios.service

That is done. State is still the same. Everything seems to be working though.. Monitoring service is at least executing checks.

[scottwilkerson]

Can you send a profile.zip from Admin -> System Profile along with the /usr/local/nagiosxi/var/xi-sys.cfg from the server.

Thanks

[sib]

profile.zip does not work. Getting

Code: Select all

PROFILE BUILD FAILED
Array
(
)
CODE: 1

I do see the profile output though.

xi-sys.cfg.txt

[scottwilkerson]

Can you open a ticket here and reference this thread
https://support.nagios.com/tickets/

We may need to setup a remote session to take a look at the server.