nagios dead but subsys locked

[vinothsethuram]

last 30 doesn't have any info related to downtime as this issue was happened 14 hours ago. So i retrieved last 1000 lines and took the lines whose timestamp matches the nagios downtime

[slansing]

Do you have a snippit of the nagios logs at this time as lmiltchev asked?

[vinothsethuram]

Do you have a snippit of the nagios logs at this time as lmiltchev asked?

last 30 doesn't have any info related to downtime. I retrieved last 1000 and took snippet which matches

Code: Select all

[1388367293] wproc: 'Core Worker 5127' seems to be choked. ret = 704000; bufsize = 826406: errno = 104 (Connection reset by peer)
[1388367293] wproc: iocache_read() from Core Worker 5127 returned -1: Connection reset by peer
[1388367293] wproc: Socket to worker Core Worker 5127 broken, removing
[1388367293] wproc: 'Core Worker 5129' seems to be choked. ret = -1; bufsize = 341: errno = 11 (Resource temporarily unavailable)
[1388367294] wproc: Job with id '11167' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11168' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11169' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11170' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11171' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11172' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11173' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11174' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11175' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11176' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11177' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11178' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11179' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11180' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11181' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11182' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11183' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11184' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11185' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11186' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11187' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11188' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11189' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11190' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11191' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11192' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11193' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11196' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11198' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11194' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11195' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11197' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11208' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11204' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11200' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11203' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11205' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11206' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11214' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11216' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11201' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11213' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11211' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11215' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11223' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11221' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11219' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11217' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11225' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11222' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11209' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11233' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11228' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11212' doesn't exist on Core Worker 5129.
[1388367294] wproc: Job with id '11224' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11210' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11232' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11234' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11230' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11229' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11239' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11237' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11238' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11227' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11246' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11244' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11242' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11251' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11226' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11249' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11255' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11252' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11254' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11248' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11256' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11259' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11260' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11264' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11220' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11257' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11263' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11243' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11231' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11269' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11268' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11273' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11245' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11270' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11199' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11271' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11272' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11247' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11277' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11265' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11275' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11283' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11280' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11284' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11281' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11279' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11278' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11287' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11288' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11292' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11291' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11258' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11293' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11300' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11296' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11282' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11294' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11302' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11295' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11301' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11305' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11303' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11306' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11235' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11308' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11297' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11299' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11307' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11312' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11285' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11289' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11304' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11218' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11318' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11241' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11316' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11315' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11317' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11310' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11240' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11309' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11324' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11325' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11321' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11322' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11261' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11323' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11202' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11313' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11329' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11286' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11250' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11331' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11253' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11262' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11266' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11267' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11276' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11311' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11326' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11332' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11335' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11336' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11338' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11343' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11328' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11330' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11340' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11334' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11341' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11342' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11346' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11333' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11349' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11339' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11356' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11357' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11348' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11354' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11353' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11358' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11350' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11359' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11207' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11274' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11290' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11298' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11351' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11344' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11320' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11314' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11355' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11319' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11352' doesn't exist on Core Worker 5129.
[1388367295] wproc: Job with id '11327' doesn't exist on Core Worker 5129.
[1388367302] wproc: Job with id '11345' doesn't exist on Core Worker 5129.
[1388367303] wproc: Job with id '11236' doesn't exist on Core Worker 5129.
[1388367303] wproc: Job with id '11337' doesn't exist on Core Worker 5129.

[slansing]

This is a known bug with the worker processes, and I "think" it may have been resolved in 4.0.2, what version are you running?

[vinothsethuram]

4.0.1.Is that this the reason for the nagios downtime ? Please brief about the cause so that I can explain the same to business. And also let me know how to update the nagios when another version is in live

[slansing]

Well, you can upgrade to version 4.0.2, which may resolve your issue. You can use this guide to upgrade to the new version, as well as get the new plugins:

http://assets.nagios.com/downloads/nagi ... Source.pdf

[vinothsethuram]

I hope PDF is for new installation. I wan to upgrade nagios from 4.0.1 to latest version.

[abrist]

The document has indeed been updated for 4.0.2. Best of luck!