Delay in updation of Nagios Front end screen

This support forum board is for support questions relating to Nagios XI, our flagship commercial network monitoring solution.
jatindeepsharma
Posts: 70
Joined: Wed Apr 04, 2018 10:52 am

Delay in updation of Nagios Front end screen

Post by jatindeepsharma »

Front end of nagios making an delay while updation, first it is letting all queue to filled up to 256000 & then messages are getting reduced in a go(like storm). rather than updating after every minute it is taking delay up to 20 to 25 minutes for updation, Can you please suggest?
benjaminsmith
Posts: 5324
Joined: Wed Aug 22, 2018 4:39 pm
Location: saint paul

Re: Delay in updation of Nagios Front end screen

Post by benjaminsmith »

Hello @jatindeepsharma,

Ok, I believe I have an understanding of the issue - messages queue filling up and Nagios taking to long to update check results. How long has this been happening on the server?

Follow this article to increase the kernel message queue settings on your server, restart and let it run for a while.

NDOUtils - Message Queue Exceeded

Next, PM your system profile so we can review the logs? Thanks.

To send us your system profile.
Login to the Nagios XI GUI using a web browser
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and share this in a private message and then reply to this post to bring it up in the queue.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.

Be sure to check out our Knowledgebase for helpful articles and solutions!
jatindeepsharma
Posts: 70
Joined: Wed Apr 04, 2018 10:52 am

Re: Delay in updation of Nagios Front end screen

Post by jatindeepsharma »

Hello @Benjaminsmith,
this issue is happening from last 1.5 month. I have followed this document & increased kernel messages queue settings.
please check i have PM you profile.zip file.

Regards
Jatindeep
benjaminsmith
Posts: 5324
Joined: Wed Aug 22, 2018 4:39 pm
Location: saint paul

Re: Delay in updation of Nagios Front end screen

Post by benjaminsmith »

Hello Jatindeep,

Thanks for the profile. Even though many are passive, you're running a large number of service checks (Total Services: 149424 ). We usually recommend adding an additional XI server after 20K checks.

1. Is this a physical server or a VM?

2. I noticed the nagios database has crashed tables, run the following command to repair.

Code: Select all

service mysqld stop
/usr/local/nagiosxi/scripts/repairmysql.sh nagios
Once the script has completed, start the MySQL database server with the following command:

Code: Select all

service mysqld start
3. Post the output of the following commands:

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table

Code: Select all

echo "SELECT relname as Table, pg_size_pretty(pg_total_relation_size(relid)) As Size, pg_size_pretty(pg_total_relation_size(relid) - pg_relation_size(relid)) as ExternalSize FROM pg_catalog.pg_statio_user_tables ORDER BY pg_total_relation_size(relid) DESC;" | psql nagiosxi nagiosxi
4. Please take a look at the following tips for boosting system performance.
Maximizing Performance In Nagios X
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.

Be sure to check out our Knowledgebase for helpful articles and solutions!
jatindeepsharma
Posts: 70
Joined: Wed Apr 04, 2018 10:52 am

Re: Delay in updation of Nagios Front end screen

Post by jatindeepsharma »

Hello @Benjaminsmith,

Thanks for your reply, i am repairing my database right now & after repairing, I will get back to you with output of given commands.

Regards
Jatindeep
benjaminsmith
Posts: 5324
Joined: Wed Aug 22, 2018 4:39 pm
Location: saint paul

Re: Delay in updation of Nagios Front end screen

Post by benjaminsmith »

Hi Jatindeep,

Sounds good. We'll wait for your update.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.

Be sure to check out our Knowledgebase for helpful articles and solutions!
jatindeepsharma
Posts: 70
Joined: Wed Apr 04, 2018 10:52 am

Re: Delay in updation of Nagios Front end screen

Post by jatindeepsharma »

Hello Benjamin,

i have repaired SQL database tables but when i am putting query i am getting below errors. please suggest.

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table
ERROR 1045 (28000): Access denied for user 'root'@'127.0.0.1' (using password: YES)

echo "SELECT relname as Table, pg_size_pretty(pg_total_relation_size(relid)) As Size, pg_size_pretty(pg_total_relation_size(relid) - pg_relation_size(relid)) as ExternalSize FROM pg_catalog.pg_statio_user_tables ORDER BY pg_total_relation_size(relid) DESC;" | psql nagiosxi nagiosxi
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
benjaminsmith
Posts: 5324
Joined: Wed Aug 22, 2018 4:39 pm
Location: saint paul

Re: Delay in updation of Nagios Front end screen

Post by benjaminsmith »

Hello,

Those commands are to check the size of the database tables. Have you changed the default passwords?

With regards to the initial problem, has the performance improved after repairing the database and increasing the default kernel message queue settings?

What is the output of the following command?

Code: Select all

ipcs
Take a look at the guide below for tips on improving system performance.
Maximizing Performance In Nagios XI
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.

Be sure to check out our Knowledgebase for helpful articles and solutions!
jatindeepsharma
Posts: 70
Joined: Wed Apr 04, 2018 10:52 am

Re: Delay in updation of Nagios Front end screen

Post by jatindeepsharma »

Hello Benjamin,

Problem is still there after repairing tables.it is getting delay of about 35 minutes now. right now current time is 9:19 AM & it is showing data of 8:36 AM at front end. please suggest.
I will look into password of SQL database.

output of ipcs is given below.
[root@NagiosXI ~]# ipcs

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x0052e2c1 0 postgres 600 37879808 34
0x00000000 32769 gdm 600 393216 2 dest
0x00000000 65538 gdm 600 393216 2 dest
0x00000000 98307 gdm 600 393216 2 dest
0x00000000 131076 gdm 600 393216 2 dest

------ Semaphore Arrays --------
key semid owner perms nsems
0x00000000 0 root 600 1
0x00000000 65537 root 600 1
0x0052e2c1 131074 postgres 600 17
0x0052e2c2 163843 postgres 600 17
0x0052e2c3 196612 postgres 600 17
0x0052e2c4 229381 postgres 600 17
0x0052e2c5 262150 postgres 600 17
0x0052e2c6 294919 postgres 600 17
0x0052e2c7 327688 postgres 600 17
0x00000000 360457 apache 600 1
0x00000000 393226 apache 600 1
0x00000000 425995 apache 600 1

------ Message Queues --------
key msqid owner perms used-bytes messages
0x90000002 163840 nagios 600 261919744 255781
jatindeepsharma
Posts: 70
Joined: Wed Apr 04, 2018 10:52 am

Re: Delay in updation of Nagios Front end screen

Post by jatindeepsharma »

Hello Benjamin,

I am using VM machine, is that the number of hosts & passive checks that are causing this problem?
right now we have around 1700 hosts on this server & in coming months i will increase more hosts on it.(i.e. it will also increase passive checks on it).

Regards
Jatindeep
Locked