Latencies increase drastically after 33 hourse of uptime

Support forum for Nagios Core, Nagios Plugins, NCPA, NRPE, NSCA, NDOUtils and more. Engage with the community of users including those using the open source solutions.
dnelson
Posts: 14
Joined: Mon Dec 23, 2013 11:07 am

Re: Latencies increase drastically after 33 hourse of uptime

Post by dnelson »

Hi abrist,

Thanks for the link. The link you provided appears to be for Nagios 4.x and differs slightly from Nagios 3.x. I'm including both here to avoid any confusion.

Nagios 3.x large installation tweaks: http://nagios.sourceforge.net/docs/3_0/ ... weaks.html

Nagios 4.x large installation tweaks: http://nagios.sourceforge.net/docs/nagi ... weaks.html

Regards,
David Nelson
abrist
Red Shirt
Posts: 8334
Joined: Thu Nov 15, 2012 1:20 pm

Re: Latencies increase drastically after 33 hourse of uptime

Post by abrist »

dnelson wrote: The link you provided appears to be for Nagios 4.x and differs slightly from Nagios 3.x.
True enough. Both versions do limit the environment variables, so make sure that your checks do not use them before you deploy the changes to production. (most plugins do not use them anymore, but there are some older, heavily customizable notification addods that do . . .)
Former Nagios employee
"It is turtles. All. The. Way. Down. . . .and maybe an elephant or two."
VI VI VI - The editor of the Beast!
Come to the Dark Side.
dnelson
Posts: 14
Joined: Mon Dec 23, 2013 11:07 am

Solution - Disabled embedded perl

Post by dnelson »

On the advice of an individual that was experiencing the same issues as I was, I disabled the embedded Perl interpreter (set 'enable_embedded_perl=0' in nagios.cfg). Restarted Nagios and waited. It's now been 5+ days since doing this and service and host latencies are stable in the 200 and 400 mS ranges, respectively; the 1 minute load average mean is 1.78 with the max being 2.48; kernel/system time is stable at around 12%.

In summary:
- Large installation tweaks enabled (use_large_installation_tweaks=1)
- Free child process memory commented (#free_child_process_memory=0)
- Child processes fork twice commented (#child_processes_fork_twice=1)
- Disabled the embedded perl interpreter (enable_embedded_perl=0)

Thank you for all your suggestions and ideas.

David Nelson
Locked