Export Nagios Log Server data to CSV

[Sampath.Basireddy]

Is there any way to export Nagios Log Server data to CSV?

I know there is an option "Export as CSV" for exporting, but if I want to export data for longer period of times or hundreds of thousands of logs, what are my options?

I tried "Export as CSV" to export upto 5000 logs, but anything more than that doesn't seem to be even loading completely.

Any help is greatly appreciated.

[cdienger]

Check out https://exchange.nagios.org/directory/U ... rt/details written by our good friend @mcapra.

[Sampath.Basireddy]

Thank You @ cdienger.

I tried the java tool, but when I use it with "-query" it fails with below error message.

Code: Select all

com.google.gson.stream.MalformedJsonException: Expected name at line 1 column 2 path $.
Malformed JSON detected in query. Please check your JSON.

Please excuse me if it is a dumb question - Am assuming query in the sense, the "raw" or "services" part from json file that is exported from queries in NLS?

Here is how I ran it:

Code: Select all

java -jar nlsexport.jar -host=localhost -date_start=2019.02.26 -date_end=2019.02.26 -output_path=/home/sampath/ -output_format=csv -query='{\"query\":{\"list\":{\"0\":{\"query\":\"host:\\\"192.168.0.1\\\"\",\"id\":0,\"color\":\"#03a645\",\"alias\":\"\",\"pin\":false,\"type\":\"lucene\",\"enable\":true}},\"ids\":[0]},\"filter\":{\"list\":{\"0\":{\"from\":\"now-1d\",\"to\":\"now\",\"field\":\"@timestamp\",\"type\":\"time\",\"active\":true,\"id\":0,\"mandate\":\"must\"},\"1\":{\"type\":\"field\",\"field\":\"program\",\"query\":\"\\\"ohs-access_log\\\"\",\"mandate\":\"must\",\"active\":true,\"alias\":\"\",\"id\":1}},\"ids\":[0,1]}}'

[cdienger]

The exported version contains some escape characters that need to be removed to make it valid json. Try:

Code: Select all

java -jar nlsexport.jar -host=localhost -date_start=2019.02.26 -date_end=2019.02.26 -output_path=/home/sampath/ -output_format=csv -query='{"query":{"list":{"0":{"query":"host:192.168.0.1","id":0,"color":"#03a645","alias":"","pin":false,"type":"lucene","enable":true}},"ids":[0]},"filter":{"list":{"0":{"from":"now-1d","to":"now","field":"@timestamp","type":"time","active":true,"id":0,"mandate":"must"},"1":{"type":"field","field":"program","query":"ohs-access_log","mandate":"must","active":true,"alias":"","id":1}},"ids":[0,1]}}'

https://jsonlint.com/ can be used to validate the json.

[Sampath.Basireddy]

No luck, it is throwing error:

Code: Select all

[root@brnagiosls1 sampath]# java -jar nlsexport.jar -host=localhost -date_start=2019.03.04 -date_end=2019.03.04 -output_path=/home/sampath/ -output_format=csv -query='{"query":{"list":{"0":{"query":"host:192.168.0.1","id":0,"color":"#03a645","alias":"","pin":false,"type":"lucene","enable":true}},"ids":[0]},"filter":{"list":{"0":{"from":"now-1d","to":"now","field":"@timestamp","type":"time","active":true,"id":0,"mandate":"must"},"1":{"type":"field","field":"program","query":"ohs-access_log","mandate":"must","active":true,"alias":"","id":1}},"ids":[0,1]}}'


Creating directory /home/sampath/nls-export-1551840370649/
[1551840370937]  Writing data from [logstash-2019.03.04/syslog_json].
Exception in thread "main" java.lang.NullPointerException
	at ElasticSearch.startIndexScroll(ElasticSearch.java:252)
	at Main.doit(Main.java:123)
	at Main.<init>(Main.java:37)
	at run.doMain(run.java:97)
	at run.main(run.java:50)
[root@brnagiosls1 sampath]#

[cdienger]

It looks like the 'services' query will result in a 400 bad request message from the elasticsearch service. Use the raw query instead making sure to remove the backslashes first.