/n & \n showing up in service descriptions

[sreinhardt]

Based on your output there, I can see that every single \ is being replaced with \\. This is definitely not a known core or gearman issue, and is likely something to do with your command reaping check results. Could you post your nagios.cfg and commands.cfg files please. We would like to verify what may have changed in there.

[rajasegar]

Based on your output there, I can see that every single \ is being replaced with \\. This is definitely not a known core or gearman issue, and is likely something to do with your command reaping check results. Could you post your nagios.cfg and commands.cfg files please. We would like to verify what may have changed in there.

See attached.

config_files.zip

[sreinhardt]

I think we found your issue! Looks like you still have the sed replace lines that was a manual modification for gearman on r1.0 or r1.1, simply reverting this should resolve the issue.

Services FROM:

Code: Select all

define command {
       command_name                  		process-service-perfdata-file-bulk
       command_line                  		sed -i 's/\\n//g' /usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}

TO:
define command {
       command_name                  		process-service-perfdata-file-bulk
       command_line                  		/usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}

Hosts FROM:

Code: Select all

define command {
       command_name                  		process-host-perfdata-file-bulk
       command_line                  		sed -i 's/\\n//g' /usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}

TO:
define command {
       command_name                  		process-host-perfdata-file-bulk
       command_line                  		/usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}

Give nagios a restart (service nagios restart) and some time for check results to come in, see if that clears up the issues for you.

[rajasegar]

I think we found your issue! Looks like you still have the sed replace lines that was a manual modification for gearman on r1.0 or r1.1, simply reverting this should resolve the issue.

Services FROM:

Code: Select all

define command {
       command_name                  		process-service-perfdata-file-bulk
       command_line                  		sed -i 's/\\n//g' /usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}

TO:
define command {
       command_name                  		process-service-perfdata-file-bulk
       command_line                  		/usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}

Hosts FROM:

Code: Select all

define command {
       command_name                  		process-host-perfdata-file-bulk
       command_line                  		sed -i 's/\\n//g' /usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}

TO:
define command {
       command_name                  		process-host-perfdata-file-bulk
       command_line                  		/usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}

Give nagios a restart (service nagios restart) and some time for check results to come in, see if that clears up the issues for you.

This was the instructions from your team, I double confirmed the steps to implement Gearman with just last week.
Anyway I will check this out

[rajasegar]

After making the changes I am getting all kind of errors. The \\, \n are still there.
The graphs stopped updating.

Code: Select all

[1404347273] Warning: Attempting to execute the command "/usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/1404347273.perfdata.service" resulted in a return code of 126.  Make sure the script or binary you are trying to execute actually exists...
[1404347273] Warning: Attempting to execute the command "/usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/1404347273.perfdata.host" resulted in a return code of 126.  Make sure the script or binary you are trying to execute actually exists...
[1404347287] Warning: Attempting to execute the command "/usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/1404347287.perfdata.host" resulted in a return code of 126.  Make sure the script or binary you are trying to execute actually exists...
[1404347287] Warning: Attempting to execute the command "/usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/1404347287.perfdata.service" resulted in a return code of 126.  Make sure the script or binary you are trying to execute actually exists...
[1404347293] SERVICE ALERT: BHQPDMESXIU06;VMH CPU Usage;WARNING;SOFT;2;ESX3 WARNING - cpu usage=74.94 %
[1404347303] Warning: Attempting to execute the command "/usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/1404347303.perfdata.host" resulted in a return code of 126.  Make sure the script or binary you are trying to execute actually exists...
[1404347303] Warning: Attempting to execute the command "/usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/1404347303.perfdata.service" resulted in a return code of 126.  Make sure the script or binary you are trying to execute actually exists...

Code: Select all

define command {
       command_name                  		process-host-perfdata-file-bulk
       command_line                  		/usr/local/nagios/var/host-perfdata && /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}	

define command {
       command_name                  		process-service-perfdata-file-bulk
       command_line                  		/usr/local/nagios/var/service-perfdata && /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service
}	

I reverted back to the original commands with sed and the error messages went off.
The graphs started updating again.
Please double check your instructions.

[sreinhardt]

You are right, this was the original quick fix for gearman with r1.0, which has since been resolved. I missed a portion after the sed, the output saying what is wrong probably should have lead you to see this, but the changes you need to make are:

Code: Select all

define command {
       command_name                        process-host-perfdata-file-bulk
       command_line                        /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.host
}   

define command {
       command_name                        process-service-perfdata-file-bulk
       command_line                        /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/xidpe/$TIMET$.perfdata.service

Just and additional note, you commands must have been altered somewhat from defaults, probably nothing to worry about, but the default commands look like:

Code: Select all

define command {
       command_name                             process-service-perfdata-file-pnp-bulk
       command_line                             /bin/mv /usr/local/nagios/var/service-perfdata /usr/local/nagios/var/spool/perfdata/service-perfdata.$TIMET$
}
define command {
       command_name                             process-host-perfdata-file-pnp-bulk
       command_line                             /bin/mv /usr/local/nagios/var/host-perfdata /usr/local/nagios/var/spool/perfdata/host-perfdata.$TIMET$
}

[rajasegar]

Made the changes in Dev.

Graphs stopped updating.

04-07-2014 08-15-17 AM.png

The \n are still there.

04-07-2014 08-15-55 AM.png

Performance Data still has \n

04-07-2014 08-14-21 AM.png

Some graphs went missing, CPU is there but storage all gone.

No error messages in nagios.log

Code: Select all

/var/log/mod_gearman/mod_gearman_worker.log
[2014-07-04 07:39:50][7063][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:4730 -> libgearman/connection.cc:498

Code: Select all

/var/log/mod_gearman/mod_gearman_neb.log
[2014-07-04 08:29:24][18578][ERROR] sending job to gearmand failed: flush(GEARMAN_COULD_NOT_CONNECT) localhost:4730 -> libgearman/connection.cc:498

Reverted back. Restarted nagios , gearman services, npcd. Graphs started updating.
However the storage graphs still not showing up

No error messages after restart in the following files.
/usr/local/nagios/var/nagios.log
/var/log/mod_gearman/mod_gearman_neb.log
/var/log/mod_gearman/mod_gearman_worker.log

[abrist]

We need to see these problems in action. Could you open a ticket at [email protected] ?

[rajasegar]

We need to see these problems in action. Could you open a ticket at [email protected] ?

Done.

[sreinhardt]

Locking, ticket opened.