Nagios XI - Monitoring Engine & Performance Grapher are RED on UI

[veeravamsi]

Hi Team,

We have recently installed ( Clean Install ) Nagios XI on Oracle 8 system. Installation went fine and able to get to GUI , however Noticed two of the System Components are in RED : Monitoring Engine & Performance Grapher.

MonitoringEngineError.PNG

Have tried repair_database script , stopping and restart all services , rebooting system etc. Still no luck.

Below are some of observations in few logs :

tail -20 event_handler.log
----------------------------------------------------------------
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
DELETED LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock'
EVENT HANDLER EXITING
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
DELETED LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock'
EVENT HANDLER EXITING
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
DELETED LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock'
EVENT HANDLER EXITING
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
DELETED LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock'
EVENT HANDLER EXITING
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
DELETED LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock'
EVENT HANDLER EXITING
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
DELETED LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock'
EVENT HANDLER EXITING
LOCKFILE '/usr/local/nagiosxi/var/event_handler.lock' CREATED
[root@dennagxip4 var]# tail -20 cleaner.log
KEEPING ALL ERROR CHECKPOINTS
DIR: /usr/local/nagiosxi/nom/checkpoints/nagiosxi
NUMFOUND: 3
KEEPING ALL SNAPSHOTS
----------------------------------
Running callbacks:
----------------------------------
DIR: /usr/local/nagiosxi/nom/checkpoints/nagioscore
NUMFOUND: 3
KEEPING ALL GOOD CHECKPOINTS
DIR: /usr/local/nagiosxi/nom/checkpoints/nagioscore/errors
ls: cannot access '/usr/local/nagiosxi/nom/checkpoints/nagioscore/errors/*.gz': No such file or directory
NUMFOUND: 0
KEEPING ALL ERROR CHECKPOINTS
DIR: /usr/local/nagiosxi/nom/checkpoints/nagiosxi
NUMFOUND: 3
KEEPING ALL SNAPSHOTS
----------------------------------
Running callbacks:
----------------------------------





tail -20 cmdsubsys.log
----------------------------------------------------------------
PROCESS COMMAND: CMD=11, DATA=[""]
CMDLINE=sudo /usr/local/nagiosxi/scripts/manage_services.sh start nagios

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.

sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
OUTPUT=
RETURNCODE=1
................................
PROCESSED 2 COMMANDS
..........................................................
PROCESSED 0 COMMANDS
............................................................
PROCESSED 0 COMMANDS
.........................




tail -100 reportengine.log
----------------------------------------------------------------
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
No such file or directory
Database Error: Could not connect to database
No such file or directory
Database Error: Could not connect to database
No such file or directory
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
No such file or directory
Database Error: Could not connect to database
No such file or directory
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
Connection refused
Database Error: Could not connect to database
Connection refused

[jsimon]

Hi @veeravamsi,

Can you post the output when you run this command?

Code: Select all

systemctl status mysqld

[veeravamsi]

service mysqld status
Redirecting to /bin/systemctl status mysqld.service
● mysqld.service - MySQL 8.0 database server
Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
Active: active (running) since Thu 2024-01-18 23:43:02 EST; 48s ago
Process: 185429 ExecStopPost=/usr/libexec/mysql-wait-stop (code=exited, status=0/SUCCESS)
Process: 185566 ExecStartPost=/usr/libexec/mysql-check-upgrade (code=exited, status=0/SUCCESS)
Process: 185481 ExecStartPre=/usr/libexec/mysql-prepare-db-dir mysqld.service (code=exited, status=0/SUCCESS)
Process: 185456 ExecStartPre=/usr/libexec/mysql-check-socket (code=exited, status=0/SUCCESS)
Main PID: 185518 (mysqld)
Status: "Server is operational"
Tasks: 64 (limit: 48548)
Memory: 444.3M
CGroup: /system.slice/mysqld.service
└─185518 /usr/libexec/mysqld --basedir=/usr

Jan 18 23:43:01 dennagxip4 systemd[1]: Starting MySQL 8.0 database server...
Jan 18 23:43:02 dennagxip4 systemd[1]: Started MySQL 8.0 database server.

[jmichaelson]

Can you check Admin->Monitoring Config->Config Snapshots to ensure that the configuration is applied. A bad configuration will prevent the services from starting. If it shows bad, restore the last known good one and apply it.

[veeravamsi]

Its clean install so only default monitors related to localhost are present and when i try apply config ( by disabling few of the monitors ) it fails :

1.PNG

When i click show Errors its empty and written config :

2.PNG

I have also tried Config File Management Options under Tools : Config Write , Delete , Verify options .. no errors there

3.PNG

[jsimon]

Can you run the following commands and compare the values for the passwords at the bottom of each (nagiosxipass, dbmaintpass, nagiosqlpass, ndoutilspass, mysqlpass)? We need to make sure that these match each other:

Code: Select all

cat /usr/local/nagiosxi/var/xi-sys.cfg

Code: Select all

cat /usr/local/nagiosxi/etc/xi-sys.cfg

(The second one may require a "sudo" prefix depending on your local permissions)

The next step I'd recommend after confirming that these match (they should, as this is a fresh install, but since things are a little odd we should confirm...) would be to try manually connecting to your database. Try running this command for your various databases with the local credentials you have for them -- use the passwords from the xi-sys.cfg files.

Code: Select all

mysql -u<username> -p<password> -D <database>

If you're able to connect manually, please provide the output of this command, and the output of /var/log/mysql, if present.

Code: Select all

"echo 'select user from mysql.user' | mysql -uroot p<password>"

[veeravamsi]

These are the entries i see didnt find any nagiosxipass, dbmaintpass, nagiosqlpass, ndoutilspass, mysqlpass entries , otherwise those two files match with entries same .


[root@~]# cat /usr/local/nagiosxi/var/xi-sys.cfg
apachegroup='apache'
apacheuser='apache'
chownbin='/bin/chown'
groupaddbin='/usr/sbin/groupadd'
groupdelbin='/usr/sbin/groupdel'
htpasswdbin='/usr/bin/htpasswd'
httpdconfdir='/etc/httpd/conf.d'
httpdconf='/etc/httpd/conf/httpd.conf'
httpdroot='/var/www/html'
httpd='httpd'
crond='crond'
mkdirbin='/bin/mkdir'
mrtgcfg='/etc/mrtg/mrtg.cfg'
mysqld='mysqld'
pgsqlrootpass='postgres'
pgsqlpass=''
nagioscmdgroup='nagcmd'
nagiosgroup='nagios'
nagiosuser='nagios'
nagioswebpwd=''
nagioswebuser='nagiosadmin'
phpconfd='/etc/php.d'
phpini='/etc/php.ini'
proddir='/usr/local/nagiosxi'
useraddbin='/usr/sbin/useradd'
usermodbin='/usr/sbin/usermod'
userdelbin='/usr/sbin/userdel'
mibsdir='/usr/share/snmp/mibs'
phpconfdcli='/etc/php.d'





mysqlpass=''


xiver='5.10.0'
distro='OracleServer'
version='8.9'
ver='8'
architecture='x86_64'
dist='el8'
arch='x86_64'
ntpd='chronyd'
php_extension_dir='/usr/lib64/php/modules'
make_j_flag='4'


And im able to connect to databases using root

3.PNG

Output of mysql.user :

4.PNG

[jsimon]

Alright, I see that you're on XI version 5.10 from that last reply, so I don't actually think we're on the right track, the password issue I was assuming this was is actually not an issue for your version. It seems like the issue might be your sudoers setup. I would recommend taking a look at the following support link and follow the "Resolving the Problem" steps to see if this resolves the issue.

https://support.nagios.com/kb/article.p ... ategory=44

Let us know how this turns out!

[veeravamsi]

Followed those steps to recreate the sudoers permissions.

No luck with those two Monitoring Components in RED. Tried rebooting the system too ( no luck ) , though had to recreate the folders because i dont see them in temp.d config.

1009 mkdir /var/run/httpd
1010 chown apache.apache /var/run/httpd
1011 systemctl restart httpd

1014 mkdir /var/run/mysqld
1015 chown mysql.mysql /var/run/mysqld
1016 systemctl restart mysqld

Otherwise still no luck.

[jsimon]

Sorry to hear you're still dealing with this! At this point I'd recommend opening a support ticket. Our dedicated support techs will be able to dig into this deeper and provide more personalized support for this issue.

https://answerhub.nagios.com/support/s/