Nagios Support Forum

Posted: **Tue Sep 01, 2015 8:24 pm**

I'm using a perl/LWP query from a bash shell that should return all of the status detail of the services with STATUS=CRITICAL, and fairly consistently (most of the time, but not always - maybe due to the current number of outstanding CRITICAL issues to process), the JSON response output is truncated which results in bad data that can't be parsed due to imbalanced JSON syntax.

Environment Info:
CentOS release 6.3 (Final)
glibc: Version 2.12, 1.149.el6_6.9
Nagios Core 4.1.1 (compiled from source)
Nagios Plugins 2.1.1 (compiled from source)
Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips SVN/1.6.11
At the time of the query, there were roughly 17 services currently with critical status

Query (which is truncated):
perl -MLWP::Simple -e "getprint 'https://{user}:{pwd}@hostname/nagios/cg ... s=critical'"
{
"format_version": 0,
"result": {
"query_time": 1441155089000,
"cgi": "statusjson.cgi",
"user": "{user}",
"query": "servicelist",
"query_status": "beta",
"program_start": 1441150670000,
"last_data_update": 1441155082000,
"type_code": 0,
"type_text": "Success",
"message": ""
},
"data": {
"selectors": {
"service_status": 16
},
"servicelist": {
"xxxxxxxxxxxxx": {
"{xxxxxxxxxxxxx}": {
"host_name": "xxxxxxxxxxxxx",
"description": "{xxxxxxxxxxxxx}",
"plugin_output": "Log backup status. File does not exist: {xxxxxxxxxxxxx}. Aborting.",
"long_plugin_output": "",
"perf_data": "",
"max_attemps": 1,
"current_attempt": 1,
"status": 16,
"last_update": 1441155081000,
"has_been_checked": true,
"should_be_scheduled": false,
"last_check": 1441152347000,
"check_options": 0,
"check_type": 1,
"checks_enabled": false,
"last_state_change": 1441152347000,
"last_hard_state_change": 1441152347000,
"last_hard_state": 2,
"last_time_ok": 1441152347000,
"last_time_warning": 0,
"last_time_unknown": 0,
"last_time_critical": 0,
"state_type": 1,
"last_notification": 1441152347000,
"next_notification": 1441155947000,
"no_more_notifications": false,
"notifications_enabled": true,
"problem_has_been_acknowledged": false,
"acknowledgement_type": 0,
"current_notification_number": 1,
"accept_passive_checks": true,
"event_handler_enabled": true,
"flap_detection_enabled": false,
"is_flapping": false,
"percent_state_change": 6.25,
"latency": 0.63,
"execution_time": 0.00,
"scheduled_downtime_depth": 0,
"process_performance_data": false,
"obsess": true
}
},
"{xxxxxxxxxxxxx}": {
"{xxxxxxxxxxxxx}": {
"host_name": "{xxxxxxxxxxxxx}",
"description": "{xxxxxxxxxxxxx}",

From Apache /etc/httpd/logs/ssl_error_log at the exact time of query failure:
…
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] *** glibc detected *** /usr/local/nagios/sbin/statusjson.cgi: realloc(): invalid next size: 0x0000000002d0fa00 ***
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] ======= Backtrace: =========
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /lib64/libc.so.6[0x3f81e75e66]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /lib64/libc.so.6[0x3f81e7b937]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /lib64/libc.so.6(realloc+0xe5)[0x3f81e7baf5]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x42fbb2]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x43166d]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x431ad8]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x431ad8]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x431ad8]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x431ad8]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x431ad8]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x40918f]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /lib64/libc.so.6(__libc_start_main+0xfd)[0x3f81e1ed5d]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] /usr/local/nagios/sbin/statusjson.cgi[0x401b49]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] ======= Memory map: ========
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 00400000-00445000 r-xp 00000000 fc:01 661238 /usr/local/nagios/sbin/statusjson.cgi
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 00645000-0064b000 rw-p 00045000 fc:01 661238 /usr/local/nagios/sbin/statusjson.cgi
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 0064b000-00651000 rw-p 00000000 00:00 0
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 023b9000-02d1a000 rw-p 00000000 00:00 0 [heap]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f81a00000-3f81a20000 r-xp 00000000 fc:01 20881 /lib64/ld-2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f81c1f000-3f81c20000 r--p 0001f000 fc:01 20881 /lib64/ld-2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f81c20000-3f81c21000 rw-p 00020000 fc:01 20881 /lib64/ld-2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f81c21000-3f81c22000 rw-p 00000000 00:00 0
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f81e00000-3f81f8a000 r-xp 00000000 fc:01 27128 /lib64/libc-2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f81f8a000-3f8218a000 ---p 0018a000 fc:01 27128 /lib64/libc-2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f8218a000-3f8218e000 r--p 0018a000 fc:01 27128 /lib64/libc-2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f8218e000-3f8218f000 rw-p 0018e000 fc:01 27128 /lib64/libc
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] -2.12.so
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f8218f000-3f82194000 rw-p 00000000 00:00 0
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f85200000-3f85216000 r-xp 00000000 fc:01 32163 /lib64/libgcc_s-4.4.7-20120601.so.1
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f85216000-3f85415000 ---p 00016000 fc:01 32163 /lib64/libgcc_s-4.4.7-20120601.so.1
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 3f85415000-3f85416000 rw-p 00015000 fc:01 32163 /lib64/libgcc_s-4.4.7-20120601.so.1
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7f1aecef5000-7f1aecefc000 r--s 00000000 fc:01 35745 /usr/lib64/gconv/gconv-modules.cache
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7f1aecefc000-7f1aecf1d000 rw-p 00000000 00:00 0
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7f1aecf1d000-7f1af2dae000 r--p 00000000 fc:01 35861 /usr/lib/locale/locale-archive
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7f1af2dae000-7f1af2db1000 rw-p 00000000 00:00 0
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7f1af2db7000-7f1af2dba000 rw-p 00000000 00:00 0
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7fff75ec1000-7fff75ed6000 rw-p 00000000 00:00 0 [stack]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] 7fff75fa4000-7fff75fa5000 r-xp 00000000 00:00 0 [vdso]
[Wed Sep 02 00:51:30 2015] [error] [client aa.bbb.ccc.ddd] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

I suspect I will be opening a Bug report, but wanted to post here first in the event anybody else has seen this, and if they know of a fix.

Appreciate any help, thanks!
Michael

Posted: **Wed Sep 02, 2015 1:40 pm**

I was unable to replicate this, the only difference is that I'm using CentOS 6.6, I updated glibc to match, and enabled mod_dav_svn, I even enabled SSL for the test.

Environment Info:
CentOS release 6.6 (Final)
glibc-2.12-1.149.el6_6.9.x86_64
Core 4.1.1
Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips SVN/1.6.11
I setup 17 critical services and received the full output and no errors in the ssl_error_log

Query:

Code: Select all

perl -MLWP::Simple -e "getprint 'https://myuser:mypass@localhost/nagios/cgi-bin/statusjson.cgi?query=servicelist&details=true&servicestatus=critical'"
{
  "format_version": 0,
  "result": {
    "query_time": 1441218852000,
    "cgi": "statusjson.cgi",
    "user": "nagiosadmin",
    "query": "servicelist",
    "query_status": "released",
    "program_start": 1441217585000,
    "last_data_update": 1441218844000,
    "type_code": 0,
    "type_text": "Success",
    "message": ""
  },
  "data": {
    "selectors": {
      "service_status": 16
    },
    "servicelist": {
      "localhost": {
        "Check Dummy 1": {
          "host_name": "localhost",
          "description": "Check Dummy 1",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218677000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217597000,
          "last_hard_state_change": 1441217777000,
          "last_hard_state": 2,
          "last_time_ok": 1441217597000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218677000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441218977000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 10": {
          "host_name": "localhost",
          "description": "Check Dummy 10",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218689000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217609000,
          "last_hard_state_change": 1441217789000,
          "last_hard_state": 2,
          "last_time_ok": 1441217609000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218689000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441218989000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 11": {
          "host_name": "localhost",
          "description": "Check Dummy 11",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218701000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217621000,
          "last_hard_state_change": 1441217801000,
          "last_hard_state": 2,
          "last_time_ok": 1441217621000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218701000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219001000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 12": {
          "host_name": "localhost",
          "description": "Check Dummy 12",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218713000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217633000,
          "last_hard_state_change": 1441217813000,
          "last_hard_state": 2,
          "last_time_ok": 1441217633000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218713000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219013000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 13": {
          "host_name": "localhost",
          "description": "Check Dummy 13",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218725000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217645000,
          "last_hard_state_change": 1441217825000,
          "last_hard_state": 2,
          "last_time_ok": 1441217645000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218725000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219025000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 14": {
          "host_name": "localhost",
          "description": "Check Dummy 14",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218737000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217657000,
          "last_hard_state_change": 1441217837000,
          "last_hard_state": 2,
          "last_time_ok": 1441217657000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218737000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219037000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 15": {
          "host_name": "localhost",
          "description": "Check Dummy 15",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218749000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217669000,
          "last_hard_state_change": 1441217849000,
          "last_hard_state": 2,
          "last_time_ok": 1441217669000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218749000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219049000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 16": {
          "host_name": "localhost",
          "description": "Check Dummy 16",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218761000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217681000,
          "last_hard_state_change": 1441217861000,
          "last_hard_state": 2,
          "last_time_ok": 1441217681000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218761000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219061000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 17": {
          "host_name": "localhost",
          "description": "Check Dummy 17",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218773000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217693000,
          "last_hard_state_change": 1441217873000,
          "last_hard_state": 2,
          "last_time_ok": 1441217693000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218773000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219073000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 2": {
          "host_name": "localhost",
          "description": "Check Dummy 2",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218785000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217705000,
          "last_hard_state_change": 1441217885000,
          "last_hard_state": 2,
          "last_time_ok": 1441217705000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218785000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219085000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 3": {
          "host_name": "localhost",
          "description": "Check Dummy 3",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218797000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217717000,
          "last_hard_state_change": 1441217897000,
          "last_hard_state": 2,
          "last_time_ok": 1441217717000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218797000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219097000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 4": {
          "host_name": "localhost",
          "description": "Check Dummy 4",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218809000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217729000,
          "last_hard_state_change": 1441217909000,
          "last_hard_state": 2,
          "last_time_ok": 1441217729000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218809000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219109000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 5": {
          "host_name": "localhost",
          "description": "Check Dummy 5",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218821000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217741000,
          "last_hard_state_change": 1441217921000,
          "last_hard_state": 2,
          "last_time_ok": 1441217741000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218821000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219121000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 6": {
          "host_name": "localhost",
          "description": "Check Dummy 6",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218833000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217753000,
          "last_hard_state_change": 1441217933000,
          "last_hard_state": 2,
          "last_time_ok": 1441217753000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218833000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441219133000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.86,
          "latency": 0.00,
          "execution_time": 0.01,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 7": {
          "host_name": "localhost",
          "description": "Check Dummy 7",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218545000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217765000,
          "last_hard_state_change": 1441217945000,
          "last_hard_state": 2,
          "last_time_ok": 1441217765000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218545000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441218845000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.99,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 8": {
          "host_name": "localhost",
          "description": "Check Dummy 8",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218557000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217777000,
          "last_hard_state_change": 1441217957000,
          "last_hard_state": 2,
          "last_time_ok": 1441217777000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218557000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441218857000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.99,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        },
        "Check Dummy 9": {
          "host_name": "localhost",
          "description": "Check Dummy 9",
          "plugin_output": "CRITICAL",
          "long_plugin_output": "",
          "perf_data": "",
          "max_attempts": 4,
          "current_attempt": 4,
          "status": 16,
          "last_update": 1441218844000,
          "has_been_checked": true,
          "should_be_scheduled": true,
          "last_check": 1441218569000,
          "check_options": 0,
          "check_type": 0,
          "checks_enabled": true,
          "last_state_change": 1441217789000,
          "last_hard_state_change": 1441217969000,
          "last_hard_state": 2,
          "last_time_ok": 1441217789000,
          "last_time_warning": 0,
          "last_time_unknown": 0,
          "last_time_critical": 1441218569000,
          "state_type": 1,
          "last_notification": 0,
          "next_notification": 0,
          "next_check": 1441218869000,
          "no_more_notifications": false,
          "notifications_enabled": false,
          "problem_has_been_acknowledged": false,
          "acknowledgement_type": 0,
          "current_notification_number": 0,
          "accept_passive_checks": true,
          "event_handler_enabled": true,
          "flap_detection_enabled": true,
          "is_flapping": false,
          "percent_state_change": 5.99,
          "latency": 0.00,
          "execution_time": 0.00,
          "scheduled_downtime_depth": 0,
          "process_performance_data": true,
          "obsess": true
        }
      }
    }
  }
}

Posted: **Wed Sep 02, 2015 2:09 pm**

Bug Tracker issue "0000635: statsjson crashes" (against Nagios Core 4.0.8) at URL: http://tracker.nagios.org/view.php?id=635 is an exact match to this problem.

Opened "GitHub Issue 77" at URL https://github.com/NagiosEnterprises/na ... /issues/77 to report this issue against Nagios Core 4.1.1.

Posted: **Wed Sep 02, 2015 4:32 pm**

mobrien03 wrote:Bug Tracker issue "0000635: statsjson crashes" (against Nagios Core 4.0.8) at URL: http://tracker.nagios.org/view.php?id=635 is an exact match to this problem.

Opened "GitHub Issue 77" at URL https://github.com/NagiosEnterprises/na ... /issues/77 to report this issue against Nagios Core 4.1.1.

Thank you for the bug report.

Posted: **Thu Sep 03, 2015 9:08 am**

I've had it running on a cron job since yesterday every minute and I'm unable to reproduce it, I had 17 criticals at all times, I'll leave it running and check again tomorrow.

Posted: **Thu Sep 03, 2015 6:21 pm**

What is the last object in the output? It might be some status text in a service that perhaps is being confused as JSON notation and causing weirdness.

Posted: **Fri Sep 04, 2015 10:09 am**

mobrien03 wrote:I'm using a perl/LWP query from a bash shell that should return all of the status detail of the services with STATUS=CRITICAL, and fairly consistently (most of the time, but not always - maybe due to the current number of outstanding CRITICAL issues to process), the JSON response output is truncated which results in bad data that can't be parsed due to imbalanced JSON syntax.

Environment Info:
CentOS release 6.3 (Final)
glibc: Version 2.12, 1.149.el6_6.9
Nagios Core 4.1.1 (compiled from source)
Nagios Plugins 2.1.1 (compiled from source)
Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips SVN/1.6.11
At the time of the query, there were roughly 17 services currently with critical status

I suspect I will be opening a Bug report, but wanted to post here first in the event anybody else has seen this, and if they know of a fix.

This usually happens when different parts (program vs library, etc) are compiled/linked against different versions of glibc. It may also happen if everything was compiled against one version of glibc, then the glibc on your system gets updated.

Try going to the nagios core directory and doing:

Code: Select all

make clean
make all
make install

and see if it's still a problem.

Posted: **Wed Nov 18, 2015 10:31 am**

This problem appears to be fixed in branch https://github.com/NagiosEnterprises/na ... 4.1.2-Pre1

Please download the source and run:

Code: Select all

./configure
make cgis
make install-cgis

and see if the problem is fixed.

Nagios Support Forum

statusjson.cgi failing truncate response output

statusjson.cgi failing truncate response output

Re: statusjson.cgi failing/truncating response output

Re: statusjson.cgi failing truncate response output

Re: statusjson.cgi failing truncate response output

Re: statusjson.cgi failing truncate response output

Re: statusjson.cgi failing truncate response output

Re: statusjson.cgi failing truncate response output

Re: statusjson.cgi failing truncate response output