Nagios process at 100% CPU, system is crawling

An open discussion forum for obtaining help with Nagios Core. Nagios Core users of all experience levels are welcome here. Subforum have been created for the discussion of Nagios Core and Nagios Plugin development.

NOTE: The SourceForge.net mailing lists have been deprecated in favor of this forum in order to expedite support and provide additional features not available on the old mailing list.

Re: Nagios process at 100% CPU, system is crawling

Postby scottwilkerson » Mon Jul 30, 2018 3:28 pm

consulvation wrote:I had top running when I clicked on the Nagios Service Problems link on the left. It took 1m 15s to load the page.


Can you do this again and after a few seconds of waiting take a screen capture of the top window?
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 11144
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Nagios process at 100% CPU, system is crawling

Postby consulvation » Mon Jul 30, 2018 4:39 pm

Here you go...

2018-07-30_17-37-32.png
Top
consulvation
 
Posts: 16
Joined: Thu Jul 19, 2018 10:07 pm

Re: Nagios process at 100% CPU, system is crawling

Postby scottwilkerson » Tue Jul 31, 2018 8:09 am

Looks like the load has come down considerably, and the nagios process is not in the list of top processes.

The one item listed is a call the the statusjson.cgi which can be intensive if the call is for historical data
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 11144
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Nagios process at 100% CPU, system is crawling

Postby consulvation » Tue Jul 31, 2018 2:42 pm

Well, yes, the nagios process doesn't appear at all in the top anymore. I suppose making it a deamon helped that. But every call to a cgi starts by taking 100% CPU. It still takes over 1 minute to load any page on the site. It's still unusable. We also stopped getting email notifications. I guess I am wondering what else is wrong because if I stop the nagios process, there is a clear increase in performance even from the command line. Do you have any ideas? This did not happen in v3 by the way, this is strictly new to v4. Thanks for all your time on this, I do appreciate it, but this doesn't seem to be solved.
consulvation
 
Posts: 16
Joined: Thu Jul 19, 2018 10:07 pm

Re: Nagios process at 100% CPU, system is crawling

Postby consulvation » Tue Jul 31, 2018 3:28 pm

Also, I should add that it looks like Nagios is stuck in the past. Looking at the status, the last checks were from 4 days ago. Looking at the scheduling queue, it's from 4 days ago.

2018-07-31_16-22-23.png
2018-07-31_16-22-23.png (15.1 KiB) Viewed 109 times


2018-07-31_16-22-23.png
2018-07-31_16-22-23.png (15.1 KiB) Viewed 109 times
Attachments
2018-07-31_16-25-27.png
consulvation
 
Posts: 16
Joined: Thu Jul 19, 2018 10:07 pm

Re: Nagios process at 100% CPU, system is crawling

Postby consulvation » Tue Jul 31, 2018 4:02 pm

Further looking into this, I decided to roll back the changes and move it out of Daemon mode for a second and then I renamed the retention.dat and status.dat files to .old and restarted the nagios process. Those files had gotten to be 2.6GB each. In doing so, the web site is now flying at lightening speeds. I presume this will not last and eventually slow down as the files accumulate. Should they be getting that large?
consulvation
 
Posts: 16
Joined: Thu Jul 19, 2018 10:07 pm

Re: Nagios process at 100% CPU, system is crawling

Postby scottwilkerson » Wed Aug 01, 2018 9:26 am

If this is still running the maint version you may have fixed the problem

Their was a bug in 4.4.1 that duplicated comments etc, which could have been causing your issue

consulvation wrote:Also, I should add that it looks like Nagios is stuck in the past. Looking at the status, the last checks were from 4 days ago. Looking at the scheduling queue, it's from 4 days ago.


Is this resolved as well?
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 11144
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Nagios process at 100% CPU, system is crawling

Postby consulvation » Wed Aug 01, 2018 3:48 pm

It seems to be fine so far. The status.dat and retention.dat files seem to be holding in the 800-900K realm, I will continue to see if it increases significantly. I am still running the maint version, just not as a deamon.

Yes, once I renamed the super large status and retention.dat files, it became current and all host checks are being done, notifications are working again as well.

I presume I should be able to install the next stable release without any problems once it becomes available. Thanks again for your help.
consulvation
 
Posts: 16
Joined: Thu Jul 19, 2018 10:07 pm

Re: Nagios process at 100% CPU, system is crawling

Postby scottwilkerson » Wed Aug 01, 2018 4:00 pm

Great Feel free to open a new issue if this changes.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 11144
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

Re: Nagios process at 100% CPU, system is crawling

Postby scottwilkerson » Wed Aug 08, 2018 7:37 am

Unlocking per user request
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
User avatar
scottwilkerson
DevOps Engineer
 
Posts: 11144
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises

PreviousNext

Return to Nagios Core

Who is online

Users browsing this forum: Google [Bot] and 19 guests