How to export query results to a text file?

[onthax]

Hi All

New to Nagios but have run a query to get the logs for an event and can see them on screen.

The vendor would like a text output of the logs at the relevant time.

How can i export the results of the query to txt?

[jolson]

This is one of our most requested features (log reporting), however the functionality is not built-in yet. You best bet is to use an API call to elasticsearch, though the results will be messy. You can get the appropriate query using the following button:

2015-08-26 10_49_05-Dashboard • Nagios Log Server - Firefox Developer Edition.png

Enter the resulting query into the command line of Nagios Log Server, and text results will be returned. Let me know if that's close to what you're looking for. If not, know that we are aware of the desire for reporting functionality. I have added your forum post to the feature request.

[onthax]

This is one of our most requested features (log reporting), however the functionality is not built-in yet. You best bet is to use an API call to elasticsearch, though the results will be messy. You can get the appropriate query using the following button:

2015-08-26 10_49_05-Dashboard • Nagios Log Server - Firefox Developer Edition.png

Enter the resulting query into the command line of Nagios Log Server, and text results will be returned. Let me know if that's close to what you're looking for. If not, know that we are aware of the desire for reporting functionality. I have added your forum post to the feature request.

Hi Jolson,

Thanks for the quick response,

unfortunately i don't see the button you are specifying.

Capture.PNG

[jolson]

It should be in the top right-hand corner of your panels.

2015-09-04 09_03_51-Cortana.png

[onthax]

Hi Jolson,

As per the previous screenshot and this one attached, i don't have an i icon.

Capture.PNG

[jolson]

Interesting. Are you performing your search using the top menu bar? There was a recent bug that was resolved where if you search using the upper bar, the 'inspect' icon would fail to appear. Try using the following search bar instead:

2015-09-09 11_01_03-Action center.png

[onthax]

Thanks Jolson, that did the trick

Only issue i'm getting now is that when i'm exporting the logs, the output is only 100k in size where as there should be 26,000 records

Syntax of query, hostname and ip addresses removed.

Code: Select all

curl -XGET 'http://nagiosservername/nagioslogserver/index.php/api/backend/logstash-2015.08.13/_search?pretty&token=c1adf9710ddf8fe68dd293ace08cacf104720a50' -d '{
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "should": [
            {
              "query_string": {
                "query": "*"
              }
            }
          ]
        }
      },
      "filter": {
        "bool": {
          "must": [
            {
              "fquery": {
                "query": {
                  "query_string": {
                    "query": "host:(\"x.x.x.x\")"
                  }
                },
                "_cache": true
              }
            },
            {
              "range": {
                "@timestamp": {
                  "from": 1439485200000,
                  "to": 1439488800000
                }
              }
            }
          ]
        }
      }
    }
  },
  "highlight": {
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647,
    "pre_tags": [
      "@start-highlight@"
    ],
    "post_tags": [
      "@end-highlight@"
    ]
  },
  "size": 250,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "ignore_unmapped": true
      }
    },
    {
      "@timestamp": {
        "order": "desc",
        "ignore_unmapped": true
      }
    }
  ]
}' > /root/output.txt

[jolson]

You might try expanding the time period - does the output (/root/output.txt) contain the data that you're looking for?

[Box293]

I had a bit of a dig into this and I have somewhat of a solution.

First, I clicked the i icon on the "ALL EVENTS" panel to get my query.

Next, when the query is executed at the command line, I found that the number of results returned was always 10. Regardless of the value that was specificed here: "size": 250,

So I removed "size": 250, from the query and at the beginning of the query before the -d I added &size=20 and I get 20 results.

Code: Select all

curl -XGET 'http://lsproduction01/nagioslogserver/index.php/api/backend/logstash-2015.09.11,logstash-2015.09.10/_search?pretty&token=c8d0c7a3a064a065339f5f9e21dd7a63e540890d&size=20' -d '{ "query": { "filtered": { "query": { "bool": { "should": [ { "query_string": { "query": "message:DHCP*" } } ] } }, } } }'

So then I couldn't work out how to get all results easily. I found if I set size=0 then I got this output:

Code: Select all

{
    "took": 7,
    "timed_out": false,
    "_shards": {
        "total": 10,
        "successful": 10,
        "failed": 0
    },
    "hits": {
        "total": 801835,
        "max_score": 0,
        "hits": [
            
        ]
    }

Which tells me I have 801835 results. So then I tried:

Code: Select all

curl -XGET 'http://lsproduction01/nagioslogserver/index.php/api/backend/logstash-2015.09.11,logstash-2015.09.10/_search?pretty&token=c8d0c7a3a064a065339f5f9e21dd7a63e540890d&size=801835' -d '{ "query": { "filtered": { "query": { "bool": { "should": [ { "query_string": { "query": "message:DHCP*" } } ] } }, } } }'

Which did nothing. So I reduced the number to 8000 and it worked, spewing a lot of output on the screen.

I tried adding a -connect-timeout but that did not help. Not sure where to go from here.

The final bit is to output it to a file with -o /tmp/curl_results.txt

Code: Select all

curl -XGET 'http://lsproduction01/nagioslogserver/index.php/api/backend/logstash-2015.09.11,logstash-2015.09.10/_search?pretty&token=c8d0c7a3a064a065339f5f9e21dd7a63e540890d&size=8000' -d '{ "query": { "filtered": { "query": { "bool": { "should": [ { "query_string": { "query": "message:DHCP*" } } ] } }, } } }' -o /tmp/curl_results.txt

So all you need to do is work out how to get a large number of results to output.

Let us know how this works for you.

[robjohn]

Just wondering if the dev team has created a reporting/exporting solution as of yet or can an estimate of when it could be available?