livestatus VERY slow

[fpaladinw]

Hi,
we are using from years nagioscore + livestatus with success on our big linux clusters, and a problem arose just in the last cluster.

This is our last installation:

Nagios Core 4.4.3
Livestatus 1.5.0p11

We have to to this simple query:

[root@CLUSTER1 ~]# cat query_services
GET services
Columns: host_name description last_hard_state host_groups scheduled_downtime_depth host_scheduled_downtime_depth acknowledged host_acknowledged active_checks_enabled plugin_output host_downtimes downtimes state state_type current_attempt max_check_attempts notes_expanded contact_groups check_command

It works, but it take too long, 100x the usual time:

[root@CLUSTER1 ~]# time cat query_services | /usr/local/bin/unixcat /install/nagios/var/rw/live | wc -l
13902

real 0m11.751s
user 0m0.001s
sys 0m0.008s


If I remove the field "notes_expanded" in the query we obtain the usual timing, around 0.1 seconds:

[root@CLUSTER1 ~]# cat query_services-without-notes_expanded
GET services
Columns: host_name description last_hard_state host_groups scheduled_downtime_depth host_scheduled_downtime_depth acknowledged host_acknowledged active_checks_enabled plugin_output host_downtimes downtimes state state_type current_attempt max_check_attempts contact_groups check_command


[root@CLUSTER1 ~]# time cat query_services-without-notes_expanded | /usr/local/bin/unixcat /install/nagios/var/rw/live | wc -l
13902

real 0m0.130s
user 0m0.001s
sys 0m0.009s


Any tip? Is a nagios or livestatus problem?

Thanks,
Federico

PS: in other installations, for example

Nagios Core 4.3.2
livestatus-1.2.6p16

all works in reasonable time: the presence of notes_expanded triple the query time, not make it 100x!!


[root@CLUSTER2 ~]# time cat query_services | /usr/local/bin/unixcat /install/nagios/var/rw/live | wc -l
51553

real 0m0.469s
user 0m0.003s
sys 0m0.026s

[root@master02(A3) ~]# time cat query_services-without-notes_expanded | /usr/local/bin/unixcat /install/nagios/var/rw/live | wc -l
51553

real 0m0.176s
user 0m0.004s
sys 0m0.013s

[scottwilkerson]

Any tip? Is a nagios or livestatus problem?

I hate to pass the buck, but this has to be a livestatus issue as notes_expanded isn't any part of the Nagios Core project, so it must happen in something that livestatus does.

[fpaladinw]

Any tip? Is a nagios or livestatus problem?

I hate to pass the buck, but this has to be a livestatus issue as notes_expanded isn't any part of the Nagios Core project, so it must happen in something that livestatus does.

OK, that's a piece of information anyway, thanks.

So, if you are suggesting me to contact livestatus distributors, that's seems to me a not-so-easy task because livestatus is referred here

https://checkmk.com/cms_livestatus.html

but obviously we do not have checkmk product so this contact

https://checkmk.com/support.html

is not for us.

If you have some suggestion about how to address this request please tell us, thank you,
Federico

[scottwilkerson]

Unfortunately that is going to be your best bet as we are not the developers of livestatus and do not have knowledge of it's code base or reported bugs.