Page 1 of 5

How to export query results to a text file?

Posted: Tue Aug 25, 2015 7:50 pm
by onthax
Hi All

New to Nagios but have run a query to get the logs for an event and can see them on screen.

The vendor would like a text output of the logs at the relevant time.

How can i export the results of the query to txt?

Re: How to export query results to a text file?

Posted: Wed Aug 26, 2015 10:52 am
by jolson
This is one of our most requested features (log reporting), however the functionality is not built-in yet. You best bet is to use an API call to elasticsearch, though the results will be messy. You can get the appropriate query using the following button:
2015-08-26 10_49_05-Dashboard • Nagios Log Server - Firefox Developer Edition.png
Enter the resulting query into the command line of Nagios Log Server, and text results will be returned. Let me know if that's close to what you're looking for. If not, know that we are aware of the desire for reporting functionality. I have added your forum post to the feature request. :)

Re: How to export query results to a text file?

Posted: Thu Sep 03, 2015 11:46 pm
by onthax
jolson wrote:This is one of our most requested features (log reporting), however the functionality is not built-in yet. You best bet is to use an API call to elasticsearch, though the results will be messy. You can get the appropriate query using the following button:
2015-08-26 10_49_05-Dashboard • Nagios Log Server - Firefox Developer Edition.png
Enter the resulting query into the command line of Nagios Log Server, and text results will be returned. Let me know if that's close to what you're looking for. If not, know that we are aware of the desire for reporting functionality. I have added your forum post to the feature request. :)

Hi Jolson,

Thanks for the quick response,

unfortunately i don't see the button you are specifying.
Capture.PNG

Re: How to export query results to a text file?

Posted: Fri Sep 04, 2015 9:04 am
by jolson
It should be in the top right-hand corner of your panels.
2015-09-04 09_03_51-Cortana.png

Re: How to export query results to a text file?

Posted: Tue Sep 08, 2015 10:41 pm
by onthax
Hi Jolson,

As per the previous screenshot and this one attached, i don't have an i icon.
Capture.PNG

Re: How to export query results to a text file?

Posted: Wed Sep 09, 2015 11:02 am
by jolson
Interesting. Are you performing your search using the top menu bar? There was a recent bug that was resolved where if you search using the upper bar, the 'inspect' icon would fail to appear. Try using the following search bar instead:
2015-09-09 11_01_03-Action center.png

Re: How to export query results to a text file?

Posted: Wed Sep 09, 2015 11:05 pm
by onthax
Thanks Jolson, that did the trick

Only issue i'm getting now is that when i'm exporting the logs, the output is only 100k in size where as there should be 26,000 records

Syntax of query, hostname and ip addresses removed.

Code: Select all

curl -XGET 'http://nagiosservername/nagioslogserver/index.php/api/backend/logstash-2015.08.13/_search?pretty&token=c1adf9710ddf8fe68dd293ace08cacf104720a50' -d '{
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "should": [
            {
              "query_string": {
                "query": "*"
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "fquery": {
                "query": {
                  "query_string": {
                    "query": "host:(\"x.x.x.x\")"
                  }
                },
                "_cache": true
              }
            },
            {
              "range": {
                "@timestamp": {
                  "from": 1439485200000,
                  "to": 1439488800000
                }
              }
            }
          ]
        }
      }
    }
  },
  "highlight": {
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647,
    "pre_tags": [
      "@start-highlight@"
    ],
    "post_tags": [
      "@end-highlight@"
    ]
  },
  "size": 250,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "ignore_unmapped": true
      }
    },
    {
      "@timestamp": {
        "order": "desc",
        "ignore_unmapped": true
      }
    }
  ]
}' > /root/output.txt

Re: How to export query results to a text file?

Posted: Thu Sep 10, 2015 12:27 pm
by jolson
You might try expanding the time period - does the output (/root/output.txt) contain the data that you're looking for?

Re: How to export query results to a text file?

Posted: Thu Sep 10, 2015 9:17 pm
by Box293
I had a bit of a dig into this and I have somewhat of a solution.

First, I clicked the i icon on the "ALL EVENTS" panel to get my query.

Next, when the query is executed at the command line, I found that the number of results returned was always 10. Regardless of the value that was specificed here: "size": 250,

So I removed "size": 250, from the query and at the beginning of the query before the -d I added &size=20 and I get 20 results.

Code: Select all

curl -XGET 'http://lsproduction01/nagioslogserver/index.php/api/backend/logstash-2015.09.11,logstash-2015.09.10/_search?pretty&token=c8d0c7a3a064a065339f5f9e21dd7a63e540890d&size=20' -d '{ "query": { "filtered": { "query": { "bool": { "should": [ { "query_string": { "query": "message:DHCP*" } } ] } }, } } }'
So then I couldn't work out how to get all results easily. I found if I set size=0 then I got this output:

Code: Select all

{
    "took": 7,
    "timed_out": false,
    "_shards": {
        "total": 10,
        "successful": 10,
        "failed": 0
    },
    "hits": {
        "total": 801835,
        "max_score": 0,
        "hits": [
            
        ]
    }
Which tells me I have 801835 results. So then I tried:

Code: Select all

curl -XGET 'http://lsproduction01/nagioslogserver/index.php/api/backend/logstash-2015.09.11,logstash-2015.09.10/_search?pretty&token=c8d0c7a3a064a065339f5f9e21dd7a63e540890d&size=801835' -d '{ "query": { "filtered": { "query": { "bool": { "should": [ { "query_string": { "query": "message:DHCP*" } } ] } }, } } }'
Which did nothing. So I reduced the number to 8000 and it worked, spewing a lot of output on the screen.

I tried adding a -connect-timeout but that did not help. Not sure where to go from here.

The final bit is to output it to a file with -o /tmp/curl_results.txt

Code: Select all

curl -XGET 'http://lsproduction01/nagioslogserver/index.php/api/backend/logstash-2015.09.11,logstash-2015.09.10/_search?pretty&token=c8d0c7a3a064a065339f5f9e21dd7a63e540890d&size=8000' -d '{ "query": { "filtered": { "query": { "bool": { "should": [ { "query_string": { "query": "message:DHCP*" } } ] } }, } } }' -o /tmp/curl_results.txt
So all you need to do is work out how to get a large number of results to output.

Let us know how this works for you.

Re: How to export query results to a text file?

Posted: Thu Mar 24, 2016 4:12 pm
by robjohn
Just wondering if the dev team has created a reporting/exporting solution as of yet or can an estimate of when it could be available?