Page 1 of 1
Export Nagios Log Server data to CSV
Posted: Thu Feb 28, 2019 6:23 pm
by Sampath.Basireddy
Is there any way to export Nagios Log Server data to CSV?
I know there is an option "Export as CSV" for exporting, but if I want to export data for longer period of times or hundreds of thousands of logs, what are my options?
I tried "Export as CSV" to export upto 5000 logs, but anything more than that doesn't seem to be even loading completely.
Any help is greatly appreciated.
Re: Export Nagios Log Server data to CSV
Posted: Fri Mar 01, 2019 1:28 pm
by cdienger
Re: Export Nagios Log Server data to CSV
Posted: Fri Mar 01, 2019 5:18 pm
by Sampath.Basireddy
Thank You
@ cdienger.
I tried the java tool, but when I use it with "-query" it fails with below error message.
Code: Select all
com.google.gson.stream.MalformedJsonException: Expected name at line 1 column 2 path $.
Malformed JSON detected in query. Please check your JSON.
Please excuse me if it is a dumb question - Am assuming query in the sense, the "raw" or "services" part from json file that is exported from queries in NLS?
Here is how I ran it:
Code: Select all
java -jar nlsexport.jar -host=localhost -date_start=2019.02.26 -date_end=2019.02.26 -output_path=/home/sampath/ -output_format=csv -query='{\"query\":{\"list\":{\"0\":{\"query\":\"host:\\\"192.168.0.1\\\"\",\"id\":0,\"color\":\"#03a645\",\"alias\":\"\",\"pin\":false,\"type\":\"lucene\",\"enable\":true}},\"ids\":[0]},\"filter\":{\"list\":{\"0\":{\"from\":\"now-1d\",\"to\":\"now\",\"field\":\"@timestamp\",\"type\":\"time\",\"active\":true,\"id\":0,\"mandate\":\"must\"},\"1\":{\"type\":\"field\",\"field\":\"program\",\"query\":\"\\\"ohs-access_log\\\"\",\"mandate\":\"must\",\"active\":true,\"alias\":\"\",\"id\":1}},\"ids\":[0,1]}}'
Re: Export Nagios Log Server data to CSV
Posted: Mon Mar 04, 2019 10:30 am
by cdienger
The exported version contains some escape characters that need to be removed to make it valid json. Try:
Code: Select all
java -jar nlsexport.jar -host=localhost -date_start=2019.02.26 -date_end=2019.02.26 -output_path=/home/sampath/ -output_format=csv -query='{"query":{"list":{"0":{"query":"host:192.168.0.1","id":0,"color":"#03a645","alias":"","pin":false,"type":"lucene","enable":true}},"ids":[0]},"filter":{"list":{"0":{"from":"now-1d","to":"now","field":"@timestamp","type":"time","active":true,"id":0,"mandate":"must"},"1":{"type":"field","field":"program","query":"ohs-access_log","mandate":"must","active":true,"alias":"","id":1}},"ids":[0,1]}}'
can be used to validate the json.
Re: Export Nagios Log Server data to CSV
Posted: Tue Mar 05, 2019 9:49 pm
by Sampath.Basireddy
No luck, it is throwing error:
Code: Select all
[root@brnagiosls1 sampath]# java -jar nlsexport.jar -host=localhost -date_start=2019.03.04 -date_end=2019.03.04 -output_path=/home/sampath/ -output_format=csv -query='{"query":{"list":{"0":{"query":"host:192.168.0.1","id":0,"color":"#03a645","alias":"","pin":false,"type":"lucene","enable":true}},"ids":[0]},"filter":{"list":{"0":{"from":"now-1d","to":"now","field":"@timestamp","type":"time","active":true,"id":0,"mandate":"must"},"1":{"type":"field","field":"program","query":"ohs-access_log","mandate":"must","active":true,"alias":"","id":1}},"ids":[0,1]}}'
Creating directory /home/sampath/nls-export-1551840370649/
[1551840370937] Writing data from [logstash-2019.03.04/syslog_json].
Exception in thread "main" java.lang.NullPointerException
at ElasticSearch.startIndexScroll(ElasticSearch.java:252)
at Main.doit(Main.java:123)
at Main.<init>(Main.java:37)
at run.doMain(run.java:97)
at run.main(run.java:50)
[root@brnagiosls1 sampath]#
Re: Export Nagios Log Server data to CSV
Posted: Wed Mar 06, 2019 11:16 am
by cdienger
It looks like the 'services' query will result in a 400 bad request message from the elasticsearch service. Use the raw query instead making sure to remove the backslashes first.