Failed api json format service checks

[bmiadmins]

Hello,

First, some details of our install.

XI :

Latest Available Version: 2014R2.7
Installed Version: 2014R2.7
Last Update Check: 2015-05-20 09:20:01


CORE :

Nagios Core 4.0.8
Copyright (c) 2009-present Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 08-12-2014
License: GPL

Foreman Service Check Script Link :

https://github.com/infoxchange/opstools ... foreman.pl


The issue I am seeing is that I can run the command from the cli and all works fine. Meaning, I get content from the foreman api with no errors. However, in order to get this to work I was forced to build a new version of perl which is not in the path of nagios. I set the perl in the script as a work around. Additionally, the command listing nagios calls the correct version of perl with absolute path set. So, what I end getting out of the logs is in the apache error logs, not the nagios.log file. So, I was thinking that this may be caused by how nagios is trying to read in the content. Can some one please give me some ideas where to check?

Thanks,

I was needing to get your help on figuring out

[tgriep]

Could you show us how you ran the command from the command line and how the check is defined in Nagios XI?
Can you provide the error you are receiving?

[bmiadmins]

Hello,

CLI :

Code: Select all

/usr/lib64/nagios/plugins/check_foreman.pl -H foreman -k -v -l nagios -a PASSWORD -o 60

The output is what I would expect :

Code: Select all

Connecting to foreman:80
--------------- GET http://foreman:80/api/dashboard?format=json
Authorization: Basic HASH
Host: foreman
User-Agent: libwww-perl/6.13
---------------
HTTP/1.1 200 OK
Cache-Control: must-revalidate, private, max-age=0
Connection: close
Date: Wed, 20 May 2015 18:09:05 GMT
ETag: "STRING"
Server: Apache/2.2.15 (Red Hat)
Content-Type: application/json; charset=utf-8
Apipie-Checksum: STRING
Client-Date: Wed, 20 May 2015 18:09:05 GMT
Client-Peer: 10.50.1.146:80
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
Foreman_api_version: 1
Foreman_version: 1.8.0
Set-Cookie: _session_id=COOKIE; path=/; HttpOnly
Set-Cookie: request_method=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
Status: 200 OK
X-Powered-By: Phusion Passenger 4.0.18
X-Rack-Cache: miss
X-Request-Id: STRING
X-Runtime: 0.814660
X-UA-Compatible: IE=Edge,chrome=1
Content has 354 bytes
{"total_hosts":EDITED}
changed
  http://foreman:80/api/hosts?format=json&per_page=10000&search=last_report+>+"25+minutes+ago"+and+(status.applied+>+0+or+status.restarted+>+0)+and+(status.failed+%3D+0)
--------------- GET http://foreman:80/api/hosts?format=json&per_page=10000&search=last_report+>+"25+minutes+ago"+and+(status.applied+>+0+or+status.restarted+>+0)+and+(status.failed+%3D+0)
Authorization: Basic HASH
Host: foreman
User-Agent: libwww-perl/6.13
---------------
HTTP/1.1 200 OK
Cache-Control: must-revalidate, private, max-age=0
Connection: close
Date: Wed, 20 May 2015 18:09:05 GMT
ETag: "STRING"
Server: Apache/2.2.15 (Red Hat)
Content-Type: application/json; charset=utf-8
Apipie-Checksum: STRING
Client-Date: Wed, 20 May 2015 18:09:06 GMT
Client-Peer: 10.50.1.146:80
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
Foreman_api_version: 1
Foreman_version: 1.8.0
Set-Cookie: _session_id=COOKIE; path=/; HttpOnly
Set-Cookie: request_method=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
Status: 200 OK
X-Powered-By: Phusion Passenger 4.0.18
X-Rack-Cache: miss
X-Request-Id: STRING
X-Runtime: 0.155361
X-UA-Compatible: IE=Edge,chrome=1
[{"host":{"name":"EDIT","id":240,"hostgroup_id":13,"operatingsystem_id":8}},{"host":{"name":"n1xdmgintd02.dev.bmi.org","id":345,"hostgroup_id":18,"operatingsystem_id":4}},{"host":{"name":"EDIT","id":40,"hostgroup_id":24,"operatingsystem_id":7}},{"host":{"name":"EDIT","id":49,"hostgroup_id":33,"operatingsystem_id":7}},{"host":{"name":"n1xsfgt01.bmi.org","id":20,"hostgroup_id":9,"operatingsystem_id":4}},{"host":{"name":"n2xspep01.bmi.org","id":251,"hostgroup_id":null,"operatingsystem_id":4}},{"host":{"name":"EDIT","id":252,"hostgroup_id":null,"operatingsystem_id":4}},{"host":{"name":"naxcuedbt01.rapidcue.com","id":234,"hostgroup_id":13,"operatingsystem_id":7}},{"host":{"name":"naxratp01.bmi.org","id":185,"hostgroup_id":13,"operatingsystem_id":8}}]
out_of_sync
  http://foreman:80/api/hosts?format=json&per_page=10000&search=%28+last_report+<+"60+minutes+ago"+or+not+has+last_report+%29+and+status.enabled+%3D+true

**EDITED**

So, the above shows I am getting content back and its being cleaned up byt the script somewhat.

COMMAND DEFINITION :

Code: Select all

define command {
       command_name                             check_foreman
       command_line                             /usr/local/perls/5_20/bin/perl /usr/lib64/nagios/plugins/check_foreman.pl $HOSTNAME$ $ARG1$
}

SERVICE :

Code: Select all

define service {
	host_name			n2xforemanp01.bmi.org
	service_description		check_foreman
	use				generic-service
	display_name			check_foreman
	check_command			check_foreman! -l nagios -a _Prq-PrRWxJZbv7PyD6v -G unmanaged -o 60!!!!!!!
	process_perf_data		1
	retain_status_information	1
	retain_nonstatus_information	1
	register			1
	}

The only error I am seeing at this time is the following :

Code: Select all

malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "<!DOCTYPE HTML PUBLI...") at /usr/lib64/nagios/plugins/check_foreman.pl line 270.
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "<!DOCTYPE HTML PUBLI...") at /usr/lib64/nagios/plugins/check_foreman.pl line 270.
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "<!DOCTYPE HTML PUBLI...") at /usr/lib64/nagios/plugins/check_foreman.pl line 270.
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "<!DOCTYPE HTML PUBLI...") at /usr/lib64/nagios/plugins/check_foreman.pl line 270.

And from the nagios.log :

Code: Select all

[1432145817] Warning: Return code of 255 for check of service 'check_foreman' on host 'foreman.bmi.org' was out of bounds.

The above error is only showing up when nagios calls it. I am a bit lost.

[tmcdonald]

I think you set up your command incorrectly:

Code: Select all

Command definition:

/usr/local/perls/5_20/bin/perl /usr/lib64/nagios/plugins/check_foreman.pl $HOSTNAME$ $ARG1$


With ARG1 substituted in:

/usr/local/perls/5_20/bin/perl /usr/lib64/nagios/plugins/check_foreman.pl $HOSTNAME$ -l nagios -a _Prq-PrRWxJZbv7PyD6v -G unmanaged -o 60


Intended CLI command:

/usr/lib64/nagios/plugins/check_foreman.pl -H foreman -k -v -l nagios -a PASSWORD -o 60

You can see you are missing the "/usr/local/perls/5_20/bin/perl" prefix when run from the CLI, and that different arguments are being passed to the perl script than what you demonstrated from the CLI.

And bear in mind, the $HOSTNAME$ macro will grab host_name, which appears to be "n2xforemanp01.bmi.org" from your service definition, not "foreman" as in your CLI example.

[Box293]

To add to tmcdonald's post, you should define the command as follows:

Code: Select all

define command {
       command_name                             check_foreman
       command_line                             /usr/lib64/nagios/plugins/check_foreman.pl -H $HOSTNAME$ $ARG1$
}

In your service definition I don't know why "-G unmanaged" is required as it is not in your test command.

[bmiadmins]

Hello,

Sorry it took me so long to reply. This issue is now fixed.

I ended up using the below :

/usr/local/perls/5_20/bin/perl $USER1$/check_foreman.pl -H $ARG4$ $ARG1$ $ARG2$ $ARG3$.