return code of 24 is out of bounds

[ouahiba]

Hi,
I have more than 600 services that went down with this error "return code of 24 is out of bounds".
I can't find the cause.
Could you please help me?
Thanks

[rkennedy]

Can you post your /usr/local/nagios/var/nagios.log file, and also the output of df -i? Additionally, is anything in your /var/log/messages that looks related?

[ouahiba]

Hi,
Here the logs:
notification.log:1464518165;INFORMATION;5972;NOTIFICATION SERVICE MAIL - Receiver : - Message : ***** Notification de la supervision ***** - - Type: PROBLEM - - Service: disk_c - Equipement: QVIPWGTXA01 - Adresse IP: 172.31.1.164 - Catégorie: Serveur Windows - Etat: CRITICAL - Date/Heure: 29-05-2016 12:36:04 - - Consigne: http://**/w/Telephonie_fixe_IP/QVIPWGTXA01#disk_c - Informations détaillées : (Return code of 24 is out of bounds)
notification.log:1464518166;INFORMATION;6015;NOTIFICATION SERVICE MAIL - Receiver : - Message : ***** Notification de la supervision ***** - - Type: PROBLEM - - Service: disk_e_data - Equipement: QVIPWGTXA01 - Adresse IP: 172.31.1.164 - Catégorie: Serveur Windows - Etat: CRITICAL - Date/Heure: 29-05-2016 12:36:05 - - Consigne: http://**/w/Telephonie_fixe_IP/QVIPWGTXA01#disk_e_data - Informations détaillées : (Return code of 24 is out of bounds)
notification.log:1464519882;INFORMATION;22472;NOTIFICATION SERVICE MAIL - Receiver : - Message : ***** Notification de la supervision ***** - - Type: ACKNOWLEDGEMENT - - Service: ram - Equipement: QVIPWMCPI01 - Adresse IP: 172.31.1.81 - Catégorie: Serveur Windows - Etat: CRITICAL - Date/Heure: 29-05-2016 13:04:42 - - Acquitté par: api- - Consigne: http://**/w/IT_SUPERVISION/QVIPWMCPI01#ram - Informations détaillées : (Return code of 24 is out of bounds)
notification.log:1464519883;INFORMATION;22559;NOTIFICATION SERVICE MAIL - Receiver : - Message : ***** Notification de la supervision ***** - - Type: ACKNOWLEDGEMENT - - Service: cpu - Equipement: QVIPWMCPI01 - Adresse IP: 172.31.1.81 - Catégorie: Serveur Windows - Etat: CRITICAL - Date/Heure: 29-05-2016 13:04:43 - - Acquitté par: api - - Consigne: http://**/w/IT_SUPERVISION/QVIPWMCPI01#cpu - Informations détaillées : (Return code of 24 is out of bounds)
notification.log:1464524236;INFORMATION;19598;NOTIFICATION SERVICE MAIL - Receiver : - Message : ***** Notification de la supervision ***** - - Type: ACKNOWLEDGEMENT - - Service: disk_c - Equipement: QVIPWMCPI01 - Adresse IP: 172.31.1.81 - Catégorie: Serveur Windows - Etat: CRITICAL - Date/Heure: 29-05-2016 14:17:15 - - Acquitté par: api - - Consigne: http://**/w/IT_SUPERVISION/QVIPWMCPI01#disk_c - Informations détaillées : (Return code of 24 is out of bounds)

syslog.log
[1464786198] SERVICE EVENT HANDLER: LSDuMy6dZtK9fbYRgjoA2E0xe;rqu8cwMbiDBQXEhY7Cfe4ygxv;OK;SOFT;2;write_syslog_service!NAGIOS-CLIENT!
%CISTATE%~%HOSTALIAS%~%SERVICEDESC%~%OUTPUT%~%INSTRUCTIONS%~{\\"id\\":%CIID%,\\"type\\":%CITYPE%}!1!1
[1464786340] SERVICE EVENT HANDLER: DNMpJWR561AncjmLtSszxeCVh;KJLj0i6DZsbd9QvuMGe3BxRgc;OK;SOFT;4;write_syslog_service!NAGIOS-SERVAIR!
%CISTATE%~%HOSTALIAS%~%SERVICEDESC%~%OUTPUT%~%INSTRUCTIONS%~{\\"id\\":%CIID%,\\"type\\":%CITYPE%}!1!1
[1464786374] HOST EVENT HANDLER: LB08Qj2675HrXtakGhZSYNvzg;UP;SOFT;2;write_syslog_host!LABEL_VIADEIS!%CISTATE%~%HOSTALIAS%~~%OUTPUT%~%
INSTRUCTIONS%~{\\"id\\":%CIID%,\\"type\\":%CITYPE%}!1

[tgriep]

Have you tried to restart the Nagios server to see if that works for you?
Can you login to the server, run the following and post the output?

Code: Select all

df -h
df -i
chage -l nagios

Can you run one of the checks that is failing from a shell and post the output so we can view it?

[ouahiba]

here the results:

root@QVIPLVDSI01:~ $ df -h
Sys. de fichiers Taille Utilisé Dispo Uti% Monté sur
/dev/mapper/VSB--vg-root 7,2G 5,5G 1,3G 82% /
udev 488M 4,0K 488M 1% /dev
tmpfs 100M 264K 100M 1% /run
none 5,0M 0 5,0M 0% /run/lock
none 498M 5,6M 492M 2% /run/shm
/dev/sda1 228M 28M 189M 13% /boot
root@QVIPLVDSI01:~ $ df -i
Sys. de fichiers Inœuds IUtil. ILibre IUti% Monté sur
/dev/mapper/VSB--vg-root 474208 118476 355732 25% /
udev 124909 446 124463 1% /dev
tmpfs 127335 340 126995 1% /run
none 127335 9 127326 1% /run/lock
none 127335 2 127333 1% /run/shm
/dev/sda1 124496 231 124265 1% /boot
root@QVIPLVDSI01:~ $ chage -l nagios
Last password change : janv. 23, 2014
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : 0
Maximum number of days between password change : 99999
Number of days of warning before password expires : 7

Thanks

[lmiltchev]

Some systems return 24 if there are too many files open... What's your environment? Run the following commands and show the output:

Code: Select all

uname -a
cat /etc/*release
lsb_release -a

Can you show us the actual command that you are running from the command line along with the output of it?

Does restarting nagios eliminate the issue?

[ouahiba]

here the results:
1.Linux QVIPLVDSI01 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 16:19:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

2.DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.4 LTS"
NAME="Ubuntu"
VERSION="12.04.4 LTS, Precise Pangolin"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu precise (12.04.4 LTS)"
VERSION_ID="12.04"

3.No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.4 LTS
Release: 12.04
Codename: precise

[lmiltchev]

Can you show us the actual command ("failing" check) that you are running from the command line along with the output of it?

Does restarting nagios eliminate the issue?

[ouahiba]

this issue appears only twice a year but we would like to know why.
do you have any ideas?

[tgriep]

There isn't enough information to give a valid reason for this. We would need to see the nagios.log file from that day to see if there are any errors / details that could help debug this issue.