Nagios Issue with nagios: wproc: 'Core Worker'

[haile711]

Good morning everyone,

Today I experienced an issue with Nagios Database couldn't start. My /var/log directory keep running out of space because of this error from nagios keep writing into the "messages" file and it's growing really really fast. Below are the example of the error.

OS: Red Hat Enterprise Linux Server release 6.6 (Santiago)
Nagios Core 4.1.1

Code: Select all

Mar  3 09:14:05 MA100DLVMON812 nagios: Unable to send check for host 'ma100mlvprt002' to worker (ret=-2)
Mar  3 09:14:05 MA100DLVMON812 nagios: wproc: 'Core Worker 20275' seems to be choked. ret = -1; bufsize = 117: errno = 11 (Resource temporarily unavailable)
Mar  3 09:14:05 MA100DLVMON812 nagios: Unable to run check for service 'Cpu-Usage' on host 'ma100mlvmsp122'
Mar  3 09:14:05 MA100DLVMON812 nagios: wproc: 'Core Worker 20276' seems to be choked. ret = -1; bufsize = 115: errno = 11 (Resource temporarily unavailable)
Mar  3 09:14:05 MA100DLVMON812 nagios: Unable to run check for service 'Uptime' on host 'macyds126'
Mar  3 09:14:06 MA100DLVMON812 nagios: wproc: 'Core Worker 20275' seems to be choked. ret = -1; bufsize = 117: errno = 11 (Resource temporarily unavailable)
Mar  3 09:14:06 MA100DLVMON812 nagios: Unable to run check for service 'Memory' on host 'MA100MLVNAV008'
Mar  3 09:14:06 MA100DLVMON812 nagios: wproc: 'Core Worker 20276' seems to be choked. ret = -1; bufsize = 117: errno = 11 (Resource temporarily unavailable)
Mar  3 09:14:06 MA100DLVMON812 nagios: Unable to run check for service 'Cpu-Usage' on host 'MA100MLVNAV006'
Mar  3 09:14:06 MA100DLVMON812 nagios: wproc: 'Core Worker 20275' seems to be choked. ret = -1; bufsize = 113: errno = 11 (Resource temporarily unavailable)

[ssax]

One thing I recommend is to set use_syslog=0 in the /usr/local/nagios/etc/nagios.cfg since it's technically duplicating them in there since it already logs it to /usr/local/nagios/var/nagios.log.

Do you see any crashed tables or anything in your

Code: Select all

/var/log/mysqld.log

?

Thank you

[haile711]

good morning ssax,

This is the only thing I see in the mysqld.log that is concerning.
ssax, I'm more concern with the error, instead on the out of space on the /var/log. I just want to know what that particular error means, and is there a way for us to fix it.
Thanks

Code: Select all

160303  6:10:07 [Warning] Disk is full writing './nagios/nagios_logentries.TMD' (Errcode: 28). Waiting for someone to free space... (Expe
ct up to 60 secs delay for server to continue after freeing disk space)
160303  6:10:07 [Warning] Retry in 60 secs. Message reprinted in 600 secs
160303  6:20:07 [Warning] Disk is full writing './nagios/nagios_logentries.TMD' (Errcode: 28). Waiting for someone to free space... (Expe
ct up to 60 secs delay for server to continue after freeing disk space)
160303  6:20:07 [Warning] Retry in 60 secs. Message reprinted in 600 secs

[ssax]

It means you're running out of space and it can't do what it's trying to do. What is the output of these commands:

Code: Select all

df -h
df -i

[haile711]

ssax,
It seems to go away now, after I made changes to the ulimits..Thanks for your help

[rkennedy]

Glad to hear! I'll go ahead and mark this thread as resolved.

Feel free to open a new one if you ever need assistance in the future!