Nagios XI Webconsole is blank

[lgaddam]

Hi,

Recently on secondary nagios /usr reached 100% and restroring primary backup was unsuccessfull.
So, we have extended space to /usr and made the threshold to 65%.

Now, we did primary backup restoring on Secondary. Nagios core console is working fine but Nagios XI webconsole is showing blank. No data or error is shown.

Below is the error showing in /var/log/httpd/error.log

[Mon Jan 06 23:04:43 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html.var denied
[Mon Jan 06 23:05:02 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.php denied
[Mon Jan 06 23:05:02 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html denied
[Mon Jan 06 23:05:02 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html.var denied
[Mon Jan 06 23:05:22 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.php denied
[Mon Jan 06 23:05:22 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html denied
[Mon Jan 06 23:05:22 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html.var denied


Below are the versions for reference:

[root@nagiosp02 ~]# cat /usr/local/nagiosxi/var/xiversion
###################################
# DO NOT DELETE THIS FILE!
# Nagios XI version information
###################################
full=5.4.3
major=5
minor=4.3
releasedate=2017-03-16
release=5403

[root@nagiosp02 ~]# php -v
PHP 5.1.6 (cli) (built: Oct 28 2014 09:19:42)
Copyright (c) 1997-2006 The PHP Group
Zend Engine v2.1.0, Copyright (c) 1998-2006 Zend Technologies

[root@nagiosp02 ~]# apachectl -v
Server version: Apache/2.2.3
Server built: Sep 27 2013 07:53:10

[root@nagiosp02 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.9 (Tikanga)

[Box293]

[root@nagiosp02 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.9 (Tikanga)

It appears that you are using RHEL 5.x which is no longer a supported version. This is going to cause issues with restores as certain modules may no longer operate correctly as part of the restore process.

You need to get your system migrated over to a version 6 or 7 system, the restore process supports this.

[lgaddam]

Sure we will check with management abt this & take action.
But till now its working fine.....suddenly webconsole not working.
Could you help me to make it temporarily work.

[lmiltchev]

Try troubleshooting your issue by following the steps, outlined in the KB article below:

https://support.nagios.com/kb/article/n ... ce-25.html

As @Box293 suggested, I would encourage you to upgrade your system to 6 or 7, preferably 7. RHEL 5.x is no longer supported.

[lgaddam]

All looks good from secondary....notifications are going to respective teams for critical alerts and everyhthing seemd ok , I have checked with Nagios core console, its same as primary restore was successful which happens daily from primary backup.

Only issue with Nagios xi webconsole….can u help me any issues with that permissions suddenly raised.....any php issue....something....

definetly we will go to rhel 6 or 7.....but temporarily we have to make it up the webconsole running fine

[lmiltchev]

Have you tried following all of the steps, outlined in our Nagios XI - Can’t Log Into The Web Interface article?

Restart apache:

Code: Select all

service httpd restart

then try logging in the web UI. When you see a blank page, check the apache error log:

Code: Select all

tail -100 /var/log/httpd/error_log

Do you see any errors that stand out?

[lgaddam]

Before I restart httpd or login webgui, im getting below errors for every minute gap...
---------------------------------------------------------------------------------------------------------------

[Mon Jan 13 07:38:43 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.php denied
[Mon Jan 13 07:38:43 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html denied
[Mon Jan 13 07:38:43 2020] [error] [client 127.0.0.1] (13)Permission denied: access to /nagiosxi/backend/index.html.var denied

After I restart, below is the log.
-----------------------------------------------------------

[Mon Jan 13 07:39:33 2020] [notice] caught SIGTERM, shutting down
[Mon Jan 13 07:39:33 2020] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Mon Jan 13 07:39:33 2020] [notice] SSL FIPS mode disabled
[Mon Jan 13 07:39:33 2020] [notice] Digest: generating secret for digest authentication ...
[Mon Jan 13 07:39:33 2020] [notice] Digest: done
[Mon Jan 13 07:39:33 2020] [notice] SSL FIPS mode disabled
[Mon Jan 13 07:39:33 2020] [notice] Apache/2.2.3 (Red Hat) configured -- resuming normal operations

After I login Webgui followed by httpd restart, below is the log.
--------------------------------------------------------------------------------

[Mon Jan 13 07:43:12 2020] [error] [client 192.168.146.81] PHP Warning: Unknown: failed to open stream: Permission denied in Unknown on line 0
[Mon Jan 13 07:43:12 2020] [error] [client 192.168.146.81] PHP Warning: Unknown: Failed opening '/usr/local/nagiosxi/html/index.php' for inclusion (include_path='.:/usr/share/pear:/usr/share/php') in Unknown on line 0
[Mon Jan 13 07:43:13 2020] [error] [client 192.168.146.81] File does not exist: /var/www/html/favicon.ico, referer: http://192.168.200.132/nagiosxi/

Still, the permission denied errors for every minute continues....


Please resolve this for temporary, im discussing with mangement to migrate Nagios to RHEL 6 or 7.

========================================================================================

Currently Nagios running on Physical servers with RHEL 5 on primary and seconday. Help me below points.

Which is best RHEL 6 or RHEL 7 to have Nagios running ?
Do i need to go with new physical servers or VM's ?
If i go with VM, for primary and secondary, do we face any issues as the current environment running on physical servers ?
Is it ok to migrate from physical severs to VMs, we dont face any issues right ?

[lmiltchev]

Have you tried following ALL of the steps, outlined in our Nagios XI - Can’t Log Into The Web Interface article?

When you run out of disk space, you almost always will have crashed tables in the database. What is the output of the command below:

Code: Select all

tail -50 /var/log/mysqld.log

If you have db with crashed tables, you would need to run a repair on it.

The permissions issues could be related to SELinux running in enforcing mode, or a wrong/expired password for the nagiosadmin user. You could also have wrong permissions on the nagios and nagiosxi directories and sub-directories. Have you done any "hardening" of the server or modified it from the default in any way?

You may have issues with postgres db if you are using one (if you are using an old XI or upgraded from an old XI).

There are many factors that can cause this issue. Blank pages are often associated with php errors.

Can you log into Nagios Core and view the web UI fine?

Code: Select all

http://ip address/nagios

Which is best RHEL 6 or RHEL 7 to have Nagios running ?

I would recommend using RHEL 7. This way you won't have to upgrade it any time soon again.

Do i need to go with new physical servers or VM's ?

There are pros and cons to each method. It depends on your environment. For very large environments, it is recommended that you go with a physical server. VMs have some limitations and they often use shared resources, which can degrade performance. On the other hand, using VMs is very convenient - you could create snapshots, and revert to them if case something wrong goes with your server.

If i go with VM, for primary and secondary, do we face any issues as the current environment running on physical servers ?
Is it ok to migrate from physical severs to VMs, we dont face any issues right ?

I still don't know what is causing your current issues in order to answer your question, but in general, you could use either one - a VM or a physical server. It is a matter of preference, and your specific environment.

In any case, your current Nagios XI instance is NOT supported as it runs on RHEL 5. We can try to help you with your current issue, but we cannot guarantee that we will be able to solve it. Please migrate your Nagios XI to a RHEL 7 server as soon as possible if you wish to keep receiving support from us. Thank you!

[lgaddam]

Yes definitely. We need to go for latest version of RHEL 7. Thanks for the advise.
Im in discussions with management, once get approvals, will raise a ticket for the migration.

Im successfully able to login Nagios core with nagiosadmin user, all notifications working fine. For reference i have attached the screenshot.
SElinux is disabled.

[root@nagiosp02 ~]# sestatus
SELinux status: disabled
[root@nagiosp02 ~]#

I have done db repair when issue raised, it didn't worked out to resolve. After that I haven't done it again.

Output for "tail -50 /var/log/mysqld.log", is given below for todays errors.

200113 20:15:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:15:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:20:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:20:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:25:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:25:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:30:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:30:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:35:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed
200113 20:35:02 [ERROR] /usr/libexec/mysqld: Table './nagios/nagios_logentries' is marked as crashed and last (automatic?) repair failed

Im not sure what these errors are...is there something wrong with DB ?

Do i need to again run "repair_databases.sh" script to resolve this issue ?

[lgaddam]

Sorry, missed attachment.