Problem searching for logs in a time span > 200 days

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
li_alm
Posts: 19
Joined: Thu Oct 13, 2016 4:44 am

Problem searching for logs in a time span > 200 days

Post by li_alm »

Hello,

Selecting a time interval larger than 200 days (using the timepicker) does not produce any results in the dashboard.

E.g.: [03 March 2017 - today (19 September 2017)] produces output, while [02 march 2017 - today] does not
E.g.: [13 January 2017 - 01 August 2017] produces output, while [12 january 2017 - 01 august 2017] does not

Is this a known issue?
Is there a solution?

Thank you.
Regards,
Liviu
dwasswa

Re: Problem searching for logs in a time span > 200 days

Post by dwasswa »

Hi @li_alm,

There different factors that could be causing this issue.

It depends on how your data is being distributed...

How many nodes?
How many instances?

Also most importantly this could be something to do with Disk (more disk means better performance /efficiency) because time span of > 200 days worth of logs is a lot of data even for read.

You also have to consider the CPU load.

Therefore this goes back to Disk,CPU load,how much data you have,how much you is stored a day and how its distributed...
li_alm
Posts: 19
Joined: Thu Oct 13, 2016 4:44 am

Re: Problem searching for logs in a time span > 200 days

Post by li_alm »

Hello,

I have 1 nagios running on a virtual machine (1 node, 1 instance), but it is not relevant. I do not think resources are the problem.
The search just seems to be ignored - it's not like it takes time to process.

Everything works fine when the time span is <= 199 days.

Is 200 days a magic number somewhere in the elasticsearch/logstash/kibana settings?

Thanks.
Regards,
Liviu
User avatar
hsmith
Agent Smith
Posts: 3539
Joined: Thu Jul 30, 2015 11:09 am
Location: 127.0.0.1
Contact:

Re: Problem searching for logs in a time span > 200 days

Post by hsmith »

How long are you keeping indices open for on the settings page?
Former Nagios Employee.
me.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Problem searching for logs in a time span > 200 days

Post by scottwilkerson »

hsmith wrote:How long are you keeping indices open for on the settings page?
This was what I was thinking as well, it's possible you are reaching back much further than you have indexes open
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
li_alm
Posts: 19
Joined: Thu Oct 13, 2016 4:44 am

Re: Problem searching for logs in a time span > 200 days

Post by li_alm »

Could you, please, give me more details on where shoud I look?
(unfortunately, I do not understand what do you actually mean by "settings page")

Thank you.
Liviu
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Problem searching for logs in a time span > 200 days

Post by scottwilkerson »

In Administration -> Backup & Maintenance there are settings to close/delete the indexes after xx number of days. If the indexes are closed/deleted they cannot be queried.

You can also look at Administration -> Index Status to see what your oldest index is.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
li_alm
Posts: 19
Joined: Thu Oct 13, 2016 4:44 am

Re: Problem searching for logs in a time span > 200 days

Post by li_alm »

Backup & Maintainance
Close indexes older than: 0 days
Delete indexes older than: 0 days
Oldest index: 28.03.2017

I should be able to select the time interval 01.01.2017 - today and get all the logs (of course, the output will begin with 28.03.2017), but I am unable to do that.
I can only go back 199 days.

Thank you.
Liviu
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Problem searching for logs in a time span > 200 days

Post by scottwilkerson »

Hmm, I dug into this a bit more, and I am seeing the same result as you are.

I am going to file a bug report to have the developers take a look at this.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
Locked