Could not stat() check result file

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Could not stat() check result file

Post by abrist »

As long as the defunct clear out, it is WAI. Are you using a broker? (like mklivestatus)
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
paulds
Posts: 8
Joined: Thu Aug 01, 2013 7:30 am

Re: Could not stat() check result file

Post by paulds »

Hm, I don't think so. Our nagios.cfg contains "event_broker_options=-1", but we don't have any broker_module specified.
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Could not stat() check result file

Post by abrist »

What is the memory usage on this server? Is it swapping? You may want to see if the stat() errors are concurrent with nagios restarts. The init script / restart of the nagios process can (under the right circumstances) remove the lock file before all operations are complete, this may lead to these errors.
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
paulds
Posts: 8
Joined: Thu Aug 01, 2013 7:30 am

Re: Could not stat() check result file

Post by paulds »

The system is not swapping. I've looked back at our resource utilization logs, and at the time of all of the recent warnings, there was zero swap utilization, and roughly 3 - 3.5 GB of RAM (out of 8GB total) was in use. Also, FWIW, the system load has been pretty consistently around 1.00 (out of 8.00 on this 8-core system).

The warning messages do not coincide with nagios restarts.

All good theories, but alas. :)
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Could not stat() check result file

Post by abrist »

Is there any other odd errors/warnings in the logs near the time of each of these errors? I am running out of ideas as to the cause - any other details you can provide?
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
paulds
Posts: 8
Joined: Thu Aug 01, 2013 7:30 am

Re: Could not stat() check result file

Post by paulds »

Nothing else in either nagios.log or the system logs that shows up around the same times as these warnings.

I'm going to try using the audit subsystem to log all accesses of the checkresults directory, and then watch the system logs for another occurrence of this warning, and then see if I can correlate anything with that.
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Could not stat() check result file

Post by abrist »

That sounds great. I will leave the thread open. Respond back when more info has been collected.
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
Locked