Logs are sent to LS, but don't show up for hours

[GldRush98]

So forgive me if this is something simple, but I'm not super familiar with Log Server and am struggling to understand what is happening with this system.
The LS server is not loaded down, it only has 8 systems sending logs to it right now and sits idle most of the time.
We can see the log leave the router on time, but they don't show up in LS until something like 4 and a half hours later. You can see the lag in the difference between the timestamp and the timestamp in the message. (screenshot attached)

[scottwilkerson]

Do both of these machines (sending server and Log Server) have the correct timezones on the server? And is the time correct on each?

[GldRush98]

Yes, timezones are the same on both. That was my first thought as well, but wasn't it.

[cdienger]

Is this happening with just the one router or is this happening with all 8 devices? Is there more than just the default inputs and filters configured? Please provide a copy of the config found under Administration > Global > Global Configuration > View > All Files Combined.

[GldRush98]

It is happening with just this one device. When other devices send data it shows up right away in LS.

[cdienger]

Can you provide more information on the router? Model, version, etc... It sounds like there could be additional settings that may need to be set for it to use the proper time for its syslogs. I'd also like to see a tcpdump take on the NLS server:

Code: Select all

yum -y install tcpdump
tcpdump -s 0 -i any host w.x.y.z and port 5544

where w.x.y.z is the IP the logs are coming from and 5544 is the default syslog port(change this accordingly if needed). Let it run for a couple minutes then use CTRL+C to stop it. Feel free to PM it to me as it may contain sensitive info.

[GldRush98]

PM sent. Hope it helps, but doesn't look like much to me.

[cdienger]

Well, it was only one packet but it shows us that the timestamp on the packet is 16:54 and the syslog message logged with 12:20. Judging by the time this came in, I would say the 12:20 time is the more correct time. What does running date from the NLS command line return ?

[GldRush98]

Bingo. That was it. So why did this only effect one device and not the others?

[cdienger]

Not entirely sure. Do the other devices include timestamp information in their messages? My thought is that it may have been a problem for the other devices as well but this one was more obvious.