Page 2 of 2
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Fri Aug 12, 2016 7:39 am
by gsl_ops_practice
That is correct, the output I am getting in the NLS GUI and via the command you supplied are what I am after. I also have a feeling it's a bug somewhere, I hope you would be able to duplicate this behavior in-house.
Can you please advise what are the next steps to get this resolve? This monitoring functionality is now a production requirement for our customer-facing application.
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Fri Aug 12, 2016 12:43 pm
by mcapra
We were able to replicate this behavior in house. Unfortunately, despite my best efforts, I was not able to find an immediate solution for this particular query that didn't involve revising it. You might try altering the query to say "response AND 200" instead of "response = 200" and see if that produces the same results by querying the Elasticsearch API directly:
Code: Select all
curl -XGET 127.0.0.1:9200/_search?pretty -d '{"query":{"filtered":{"query":{"bool":{"should":[{"query_string":{"query":"*"}}]}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1470754104641,"to":1470754404641}}},{"fquery":{"query":{"query_string":{"query":"*apache_access*"}},"_cache":true}},{"fquery":{"query":{"query_string":{"query":"response AND 200"}},"_cache":true}}],"must_not":[{"fquery":{"query":{"query_string":{"query":"*staging*"}},"_cache":true}},{"fquery":{"query":{"query_string":{"query":"*10.30*"}},"_cache":true}}]}}}}}'
I filed an internal bug report for this issue (ID 9294).
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Fri Aug 19, 2016 3:18 pm
by gsl_ops_practice
Thank you for raising an internal bug for this.
Using Response AND 200 unfortunately doesn't work. "Response" is a field that is generated by the apache global filter and the equal sign is the only way to filter out for the response codes that are required for monitoring.
Can you please advise the approximate timeframe for getting this bug fixed?
Thanks,
Alex
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Mon Aug 22, 2016 10:53 am
by tmcdonald
Unfortunately bug fixes are in the hands of the developers, and they have their own system for prioritizing fixes. Fortunately, whenever a release is made they make sure to put in the bug or feature request ID in the changelog:
https://assets.nagios.com/downloads/nag ... NGES-5.TXT
So if you keep an eye out for 9294 you should see it, possibly in the next release.
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Mon Aug 22, 2016 11:09 am
by mcapra
We don't typically offer timeframes on bugfixes. If it was a simple hot-patch it would be one thing, but this particular issue is pretty nitty-gritty into the elasticsearch parsing scheme.
However, I think we're getting to the bottom of this now. If you want to check the value of a specific field in elasticsearch, the syntax is different than that of a generic query. If you wanted to check the response field via the elasticsearch (or NLS) API, you would be better off using the "term" identifier rather than the "query_string" identifier. For example, this will return all entries with the response field equal to 200:
Code: Select all
{
"query":{
"constant_score":{
"filter":{
"term":{
"response":200
}
}
}
}
}
Or from the CLI:
Code: Select all
curl -XGET 127.0.0.1:9200/_search?pretty -d '{"query":{"constant_score":{"filter":{"term":{"response":200}}}}}'
So, replacing the old response = 200 logic, give this a try and see if it produces the results you're looking for. Be mindful of the timestamp in this one since it may be invalid now:
Code: Select all
curl -XGET 127.0.0.1:9200/_search?pretty -d '{"query":{"filtered":{"query":{"bool":{"should":[{"query_string":{"query":"*"}}]}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1470754104641,"to":1470754404641}}},{"fquery":{"query":{"query_string":{"query":"*apache_access*"}},"_cache":true}},{"query":{"constant_score":{"filter":{"term":{"response":200}}}}}],"must_not":[{"fquery":{"query":{"query_string":{"query":"*staging*"}},"_cache":true}},{"fquery":{"query":{"query_string":{"query":"*10.30*"}},"_cache":true}}]}}}}}'
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Tue Aug 23, 2016 2:23 pm
by gsl_ops_practice
Hello,
The query you posted does now produce a valid reply, please see sample output below. I tried to adapt my NagiosXI query to match the syntax with
Code: Select all
{"query_string":{"query":"response":200"}}
but that did not produce a valid result in the search. Any other suggestions on how to modify the nagiosxi check?
Code: Select all
"_index" : "logstash-2016.08.09",
"_type" : "apache_access",
"_id" : "NNNNNNNNNNNNNNNNNNNNNNNN",
"_score" : 1.0,
"_source":{"message":"111.111.111.111 - - [09/Aug/2016:14:49:50 +0000] \"POST /webservice/webservice HTTP/1.1\" 200 2737 \"-\" \"Java/1.5.0\" inbytes=1939 outbytes=3051\n","@version":"1","@timestamp":"2016-08-09T14:49:50.000Z","type":"apache_access","host":"111.111.111.111","priority":133,"timestamp":["Aug 9 14:49:53","09/Aug/2016:14:49:50 +0000"],"logsource":"webserver","program":"apache_access","severity":5,"facility":16,"facility_label":"local0","severity_label":"Notice","clientip":"111.111.111.111","ident":"-","auth":"-","verb":"POST","request":"/webservice/webservice","httpversion":"1.1","response":200,"bytes":2737,"referrer":"\"-\"","agent":"\"Java/1.5.0\"","inbytes":1939,"outbytes":3051}
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Tue Aug 23, 2016 2:37 pm
by mcapra
Again, using the
query_string identifier isn't going to work here. That logic should be replaced with the
term identifier I provided in my last post.
Looking at your first post, the drop-in replacement should look something like this:
Code: Select all
check_xi_service_nagioslogserver!--url='http://1.1.1.1/nagioslogserver/' --apikey='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' --minutes='2' --warn='500' --crit='1000' --query='{"query":{"filtered":{"query":{"bool":{"should":[{"query_string":{"query":"*"}}]}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1470754104641,"to":1470754404641}}},{"fquery":{"query":{"query_string":{"query":"*apache_access*"}},"_cache":true}},{"query":{"constant_score":{"filter":{"term":{"response":200}}}}}],"must_not":[{"fquery":{"query":{"query_string":{"query":"*staging*"}},"_cache":true}},{"fquery":{"query":{"query_string":{"query":"*10.30*"}},"_cache":true}}]}}}}}'
Again being mindful of the timestamps in use here.
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Tue Aug 23, 2016 3:13 pm
by gsl_ops_practice
Fantastic, this now works. Now for the most important part - how would you structure the query to show transactions where Response NOT = 200?
I tried below but it was throwing the same "Could not get data from NagiosLog server"
Code: Select all
check_xi_service_nagioslogserver!--url='http://1.1.1.1/nagioslogserver/' --apikey='NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN' --minutes='2' --warn='500' --crit='1000' --query='{"query":{"filtered":{"query":{"bool":{"should":[{"query_string":{"query":"*"}}]}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1470754104641,"to":1470754404641}}},{"fquery":{"query":{"query_string":{"query":"*apache_access*"}},"_cache":true}},"must_not":[{"query":{"constant_score":{"filter":{"term":{"response":200}}}}}],"must_not":[{"fquery":{"query":{"query_string":{"query":"*staging*"}},"_cache":true}},{"fquery":{"query":{"query_string":{"query":"*10.30*"}},"_cache":true}}]}}}}}'
We have a more complex query in NagiosLog where we say "Response NOT = 200", "Response NOT = 302", "Response NOT = 304", trying to figure out how we can string that together if at the moment we can't depend on setting up a NagiosLog check with the wizard.
Re: NagiosXI -> NagiosLog specific type of query broken
Posted: Tue Aug 23, 2016 3:25 pm
by mcapra
You would basically apply that same
term logic, except instead of putting it in the
must block you would put it in the
must_not block and alter it for the proper response codes. If you wanted to pull all entries where the response code is not 200, 302, or 304, the query might look like this:
Code: Select all
{"query":{"filtered":{"query":{"bool":{"should":[{"query_string":{"query":"*"}}]}},"filter":{"bool":{"must":[{"range":{"@timestamp":{"from":1470754104641,"to":1470754404641}}},{"fquery":{"query":{"query_string":{"query":"*apache_access*"}},"_cache":true}}],"must_not":[{"fquery":{"query":{"query_string":{"query":"*staging*"}},"_cache":true}},{"fquery":{"query":{"query_string":{"query":"*10.30*"}},"_cache":true}},{"query":{"constant_score":{"filter":{"term":{"response":200}}}}},{"query":{"constant_score":{"filter":{"term":{"response":302}}}}},{"query":{"constant_score":{"filter":{"term":{"response":304}}}}}]}}}}}