How long should it take for a new sources to appear?

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:
Contact Box293

How long should it take for a new sources to appear?

Post by Box293 »

I've configured a host to send syslogs to Log Server 2015R1.2b.

I can see the traffic hitting the Log Server by watching a TCP Dump.

On the Home page, BEFORE I configured the remote host it said "Receiving logs from 5 hosts." AFTER I configured the remote host it said "Receiving logs from 6 hosts."

I go to the Dashboard to see the new logs coming in, query is:

Code: Select all

host:"10.25.6.1"
Nothing shows on the screen.

Under Administration > System Status:
I tried restarting Logstash Collector -> no change
I tried restarting Elasticsearch Database -> no change
logstash.log
elasticsearch.log

Code: Select all

cat /etc/localtime
AEST-10AEDT,M10.1.0,M4.1.0/3

Code: Select all

cat /etc/php.ini | grep date.timezone
; http://www.php.net/manual/en/datetime.configuration.php#ini.date.timezone
date.timezone = Australia/Melbourne
I had this problem the other day and it seemed to work fine when I came back later.
You do not have the required permissions to view the files attached to this post.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Top
itsupport_sgp
Posts: 19
Joined: Fri Aug 15, 2014 12:06 am

Re: How long should it take for a new sources to appear?

Post by itsupport_sgp »

Hi,

I encounter this before as well.
For ours, it is due to the datetime, you might want to double check on it.

If it works for you in the past, it should not be the NLS firewall too.

Cheers.
Top
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:
Contact Box293

Re: How long should it take for a new sources to appear?

Post by Box293 »

Where did you check the date and time?

I checked on the sending server and discovered it's timezone was not correctly set. I just fixed that and rebooted the sending server however nothing is show in the Dashboard (tcpdump shows incoming data).
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Top
itsupport_sgp
Posts: 19
Joined: Fri Aug 15, 2014 12:06 am

Re: How long should it take for a new sources to appear?

Post by itsupport_sgp »

Hi,

Try to type 'datetime' in the NLS CLI.
It will shows you the time.

Regards,
Top
slansing
Posts: 7698
Joined: Mon Apr 23, 2012 4:28 pm
Location: Travelling through time and space...

Re: How long should it take for a new sources to appear?

Post by slansing »

You can also check for differences between the system time and php time by opening up php.ini. The main commands I run to check time for both XI and NLS are:

Code: Select all

date

hwclock

cat /etc/php.ini | grep date.time
Top
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:
Contact Box293

Re: How long should it take for a new sources to appear?

Post by Box293 »

These all seem correct

Code: Select all

[root@lsproduction ~]# date
Fri Jan 23 07:42:52 AEDT 2015
[root@lsproduction ~]# hwclock
Fri 23 Jan 2015 07:42:56 AM AEDT  -0.489382 seconds
[root@lsproduction ~]# cat /etc/php.ini | grep date.time
; http://www.php.net/manual/en/datetime.configuration.php#ini.date.timezone
date.timezone = Australia/Melbourne
[root@lsproduction ~]#
tcpdump still shows logs hitting server but dashboard result returns 0

Code: Select all

tcpdump src host 10.25.6.1 and tcp dst port 5544 and dst host 10.25.5.80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
07:45:00.245023 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 4069869443:4069869605, ack 2323282946, win 115, options [nop,nop,TS val 12552869 ecr 139734191], length 162
07:45:00.247060 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 162:435, ack 1, win 115, options [nop,nop,TS val 12552869 ecr 139744208], length 273
07:45:00.251540 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 435:596, ack 1, win 115, options [nop,nop,TS val 12552870 ecr 139744211], length 161
07:45:00.252993 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 596:865, ack 1, win 115, options [nop,nop,TS val 12552871 ecr 139744215], length 269
07:45:00.258910 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 865:1038, ack 1, win 115, options [nop,nop,TS val 12552872 ecr 139744216], length 173
07:45:00.259327 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 1038:1281, ack 1, win 115, options [nop,nop,TS val 12552872 ecr 139744222], length 243
07:45:00.267497 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 1281:1461, ack 1, win 115, options [nop,nop,TS val 12552874 ecr 139744222], length 180
07:45:00.271406 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 1461:2062, ack 1, win 115, options [nop,nop,TS val 12552874 ecr 139744230], length 601
07:45:00.271602 IP vCenter.box293.local.35235 > lsproduction.box293.local.5544: Flags [P.], seq 2062:2170, ack 1, win 115, options [nop,nop,TS val 12552875 ecr 139744234], length 108
^C
9 packets captured
173 packets received by filter
78 packets dropped by kernel
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Top
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:
Contact scottwilkerson

Re: How long should it take for a new sources to appear?

Post by scottwilkerson »

Something to look at, if you go to Administration -> Index Status

Do you have any indexes that are coming in with old dates?

I ask because the logs are placed in the index based on the time in the logs and not the current time.

So you have have them in there, but might need to look at a day/month/year/decade ago to see them
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Top
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:
Contact Box293

Re: How long should it take for a new sources to appear?

Post by Box293 »

I do have some that date back to 2014.01.01

How can I determine when an index was last updated?

If I do a Dashboard search for the past 2 years with the query host:"10.25.6.1" I do not see anything dating back that far. There are some entries for this host that came in yesterday but no entries today. However the events over time only seem to reflect a 24 hour period, not a 2 year period.
Screenshot.png
ls -al /usr/local/nagioslogserver/elasticsearch/data/ebb8a63c-023a-421f-9bd5-7ec355a6ce8f/nodes/0/indices/
total 332
drwxrwxr-x 83 nagios nagios 4096 Jan 22 19:00 .
drwxrwxr-x 4 nagios nagios 4096 Nov 24 12:46 ..
drwxrwxr-x 8 nagios nagios 4096 Nov 24 12:46 kibana-int
drwxrwxr-x 8 nagios nagios 4096 Jan 1 00:00 logstash-2014.01.01
drwxrwxr-x 8 nagios nagios 4096 Jan 1 19:00 logstash-2014.01.02
drwxrwxr-x 8 nagios nagios 4096 Jan 2 19:00 logstash-2014.01.03
drwxrwxr-x 8 nagios nagios 4096 Jan 3 19:00 logstash-2014.01.04
drwxrwxr-x 8 nagios nagios 4096 Jan 4 19:00 logstash-2014.01.05
drwxr-xr-x 8 nagios users 4096 Jan 5 19:00 logstash-2014.01.06
drwxr-xr-x 8 nagios users 4096 Jan 6 19:00 logstash-2014.01.07
drwxr-xr-x 8 nagios users 4096 Jan 7 19:00 logstash-2014.01.08
drwxr-xr-x 8 nagios users 4096 Jan 8 19:00 logstash-2014.01.09
drwxr-xr-x 8 nagios users 4096 Jan 9 19:00 logstash-2014.01.10
drwxr-xr-x 8 nagios users 4096 Jan 10 19:00 logstash-2014.01.11
drwxr-xr-x 8 nagios users 4096 Jan 11 19:00 logstash-2014.01.12
drwxr-xr-x 8 nagios users 4096 Jan 12 19:00 logstash-2014.01.13
drwxr-xr-x 8 nagios users 4096 Jan 13 19:00 logstash-2014.01.14
drwxr-xr-x 8 nagios users 4096 Jan 14 19:00 logstash-2014.01.15
drwxr-xr-x 8 nagios users 4096 Jan 15 19:00 logstash-2014.01.16
drwxrwxr-x 8 nagios nagios 4096 Nov 24 12:47 logstash-2014.11.17
drwxrwxr-x 8 nagios nagios 4096 Nov 24 12:47 logstash-2014.11.24
drwxrwxr-x 8 nagios nagios 4096 Nov 24 19:00 logstash-2014.11.25
drwxrwxr-x 8 nagios nagios 4096 Nov 25 19:00 logstash-2014.11.26
drwxrwxr-x 8 nagios nagios 4096 Nov 26 19:00 logstash-2014.11.27
drwxrwxr-x 8 nagios nagios 4096 Nov 27 19:00 logstash-2014.11.28
drwxrwxr-x 8 nagios nagios 4096 Nov 28 19:00 logstash-2014.11.29
drwxrwxr-x 8 nagios nagios 4096 Nov 29 19:00 logstash-2014.11.30
drwxrwxr-x 8 nagios nagios 4096 Nov 30 19:00 logstash-2014.12.01
drwxrwxr-x 8 nagios nagios 4096 Dec 1 19:00 logstash-2014.12.02
drwxrwxr-x 8 nagios nagios 4096 Dec 2 19:00 logstash-2014.12.03
drwxrwxr-x 8 nagios nagios 4096 Dec 3 19:00 logstash-2014.12.04
drwxrwxr-x 8 nagios nagios 4096 Dec 4 19:00 logstash-2014.12.05
drwxrwxr-x 8 nagios nagios 4096 Dec 5 19:00 logstash-2014.12.06
drwxrwxr-x 8 nagios nagios 4096 Dec 6 19:00 logstash-2014.12.07
drwxrwxr-x 8 nagios nagios 4096 Dec 7 19:00 logstash-2014.12.08
drwxrwxr-x 8 nagios nagios 4096 Dec 8 19:00 logstash-2014.12.09
drwxrwxr-x 8 nagios nagios 4096 Dec 9 19:00 logstash-2014.12.10
drwxrwxr-x 8 nagios nagios 4096 Dec 10 19:00 logstash-2014.12.11
drwxrwxr-x 8 nagios nagios 4096 Dec 11 19:00 logstash-2014.12.12
drwxrwxr-x 8 nagios nagios 4096 Dec 12 19:00 logstash-2014.12.13
drwxrwxr-x 8 nagios nagios 4096 Dec 13 19:00 logstash-2014.12.14
drwxrwxr-x 8 nagios nagios 4096 Dec 14 19:00 logstash-2014.12.15
drwxrwxr-x 8 nagios nagios 4096 Dec 15 19:00 logstash-2014.12.16
drwxrwxr-x 8 nagios nagios 4096 Dec 16 19:00 logstash-2014.12.17
drwxrwxr-x 8 nagios nagios 4096 Dec 17 19:00 logstash-2014.12.18
drwxrwxr-x 8 nagios nagios 4096 Dec 18 19:00 logstash-2014.12.19
drwxrwxr-x 8 nagios nagios 4096 Dec 19 19:00 logstash-2014.12.20
drwxrwxr-x 8 nagios nagios 4096 Dec 20 19:00 logstash-2014.12.21
drwxrwxr-x 8 nagios nagios 4096 Dec 22 08:51 logstash-2014.12.22
drwxrwxr-x 8 nagios nagios 4096 Dec 22 19:00 logstash-2014.12.23
drwxrwxr-x 8 nagios nagios 4096 Dec 23 19:00 logstash-2014.12.24
drwxrwxr-x 8 nagios nagios 4096 Dec 24 19:00 logstash-2014.12.25
drwxrwxr-x 8 nagios nagios 4096 Dec 25 19:00 logstash-2014.12.26
drwxrwxr-x 8 nagios nagios 4096 Dec 26 19:00 logstash-2014.12.27
drwxrwxr-x 8 nagios nagios 4096 Dec 27 19:00 logstash-2014.12.28
drwxrwxr-x 8 nagios nagios 4096 Dec 28 19:00 logstash-2014.12.29
drwxrwxr-x 8 nagios nagios 4096 Dec 29 19:00 logstash-2014.12.30
drwxrwxr-x 8 nagios nagios 4096 Dec 30 19:00 logstash-2014.12.31
drwxrwxr-x 8 nagios nagios 4096 Dec 31 19:00 logstash-2015.01.01
drwxrwxr-x 8 nagios nagios 4096 Jan 2 11:00 logstash-2015.01.02
drwxrwxr-x 8 nagios nagios 4096 Jan 3 11:00 logstash-2015.01.03
drwxrwxr-x 8 nagios nagios 4096 Jan 4 11:00 logstash-2015.01.04
drwxrwxr-x 8 nagios nagios 4096 Jan 5 11:00 logstash-2015.01.05
drwxr-xr-x 8 nagios users 4096 Jan 6 11:00 logstash-2015.01.06
drwxr-xr-x 8 nagios users 4096 Jan 7 11:00 logstash-2015.01.07
drwxr-xr-x 8 nagios users 4096 Jan 8 11:00 logstash-2015.01.08
drwxr-xr-x 8 nagios users 4096 Jan 9 11:00 logstash-2015.01.09
drwxr-xr-x 8 nagios users 4096 Jan 10 11:00 logstash-2015.01.10
drwxr-xr-x 8 nagios users 4096 Jan 11 11:00 logstash-2015.01.11
drwxr-xr-x 8 nagios users 4096 Jan 12 11:00 logstash-2015.01.12
drwxr-xr-x 8 nagios users 4096 Jan 13 11:00 logstash-2015.01.13
drwxr-xr-x 8 nagios users 4096 Jan 14 11:00 logstash-2015.01.14
drwxr-xr-x 8 nagios users 4096 Jan 15 11:00 logstash-2015.01.15
drwxr-xr-x 8 nagios users 4096 Jan 16 11:00 logstash-2015.01.16
drwxr-xr-x 8 nagios users 4096 Jan 16 19:00 logstash-2015.01.17
drwxr-xr-x 8 nagios users 4096 Jan 17 19:00 logstash-2015.01.18
drwxr-xr-x 8 nagios users 4096 Jan 18 19:00 logstash-2015.01.19
drwxr-xr-x 8 nagios users 4096 Jan 19 19:00 logstash-2015.01.20
drwxr-xr-x 8 nagios users 4096 Jan 20 19:00 logstash-2015.01.21
drwxr-xr-x 8 nagios users 4096 Jan 21 19:00 logstash-2015.01.22
drwxr-xr-x 8 nagios users 4096 Jan 22 19:00 logstash-2015.01.23
drwxrwxr-x 4 nagios nagios 4096 Nov 24 12:46 nagioslogserver
drwxrwxr-x 8 nagios nagios 4096 Nov 24 12:46 nagioslogserver_log
You do not have the required permissions to view the files attached to this post.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Top
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:
Contact scottwilkerson

Re: How long should it take for a new sources to appear?

Post by scottwilkerson »

Can you post the following

Code: Select all

cat /etc/sysconfig/clock
Lets make sure it has

Code: Select all

ZONE="Australia/Melbourne"
then restart logstash

Code: Select all

service logstash restart
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
Top
User avatar
Box293
Too Basu
Posts: 5126
Joined: Sun Feb 07, 2010 10:55 pm
Location: Deniliquin, Australia
Contact:
Contact Box293

Re: How long should it take for a new sources to appear?

Post by Box293 »

That seems the be a problem:

Code: Select all

cat /etc/sysconfig/clock
ZONE="US/Eastern"
UTC=False
Now it is:

Code: Select all

cat /etc/sysconfig/clock
ZONE="Australia/Melbourne"
UTC=False
I restarted logstash and BANG new entries are coming on.

Awesome, great stuff, case closed

Certainly needs to be a procedure for setting the timezone for Log Server.
As of May 25th, 2018, all communications with Nagios Enterprises and its employees are covered under our new Privacy Policy.
Top
Locked

Return to “Nagios Log Server”