Some Hosts stops sending to Nagios Log Server.

[TimePlan]

Hi,

we have a rather small Nagios Log Server setup with 1 instance/server (v2.1.6) and 33 windows host sending windows event logs with nxlog (v2.9.1716).

we have all 33 hosts sending logs find until, all of sudden a single or a few hosts stop sending their logs.
the only thing that seemed to have worked so far is rebooting the Nagios log server, then it works for a couple of days and then some other hosts stops sending.

I have taken over this system from a former collegue so i am rather new at Nagios Log Server, not sure what i am looking for.

The nxlog datalog show this error, but that is also in on the hosts that work:

Code: Select all

2020-05-18 10:45:29 WARNING stopping nxlog service
2020-05-18 10:45:29 WARNING nxlog-ce received a termination request signal, exiting...
2020-05-18 10:45:31 INFO connecting to 172.17.9.58:3515
2020-05-18 10:45:31 INFO nxlog-ce-2.9.1716 started
2020-05-18 10:45:32 WARNING Due to a limitation in the Windows EventLog subsystem, a query cannot contain more than 256 sources.
2020-05-18 10:45:32 WARNING The following sources are omitted to avoid exceeding the limit in the generated query:  Microsoft-Windows-SMBWitnessClient/Informational Microsoft-Windows-StateRepository/Operational Microsoft-Windows-StateRepository/Restricted Microsoft-Windows-Storage-ClassPnP/Operational Microsoft-Windows-Storage-Storport/Operational Microsoft-Windows-Storage-Tiering/Admin Microsoft-Windows-StorageManagement/Operational Microsoft-Windows-StorageSpaces-Driver/Diagnostic Microsoft-Windows-StorageSpaces-Driver/Operational Microsoft-Windows-StorageSpaces-ManagementAgent/WHC Microsoft-Windows-StorageSpaces-SpaceManager/Diagnostic Microsoft-Windows-StorageSpaces-SpaceManager/Operational Microsoft-Windows-Store/Operational Microsoft-Windows-SystemSettingsThreshold/Operational Microsoft-Windows-TaskScheduler/Maintenance Microsoft-Windows-TaskScheduler/Operational Microsoft-Windows-TCPIP/Operational Microsoft-Windows-TerminalServices-ClientUSBDevices/Admin Microsoft-Windows-TerminalServices-ClientUSBD

One thing i did notice is a netstat -ano | grep "IP of non working host" shows two or more established connections were as those that work only have a single connection.

[jdunitz]

Hello,

Are you running an older version of log server, perhaps?
You might also want to have a look at your firewall setup, if any.

See this previous thread for some ideas:
https://support.nagios.com/forum/viewto ... 38&t=51455

Thanks!
--Jeffrey

[TimePlan]

Thank you for your response.

we are running the newst version of the log server (v2.1.6)
i tried updating the nxlog-ce version on one host on the 18th May, the log server was rebooted so it started working again.

today another host stopped sending so installed the lastest version of nxlog-ce and that aswell, when i was done and checked the log server for sending host, the server from 18th May has stopped sending at 11.00 (Danish time) until that point the server was listed as sending. Also the server that stopped sending today was not sending either.

To me this indicates the nxlog-ce update did not resolve the issue.

To my awareness we dont have firewall rules based on time (they are either on or off) so while it could be it i think the problem lies elsewhere.
is there somewhere in nagios log server og nxlog-ce i can see if its a firewall network problem?

in the nxlog > data > nxlog.conf i am seeing alot of "Error could not connect to socket" but not after a restart of the nxlog-ce service.

[jdunitz]

The easiest way to check for firewall issues is:

nmap -Pn -p3515 X.X.X.X

If what you get back says anything about it being "filtered", something is in the way, blocking or filtering your connection.

(you can download nmap for Windows).

--Jeffrey

[TimePlan]

Thank you jdunitz,

Since my last post we have restarted the log server every night, and all hosts has been sending non stop.

I will stop the reboot and wait for the first host to stop and then check with nmap.

[benjaminsmith]

Hi @TimePlan,

I will stop the reboot and wait for the first host to stop and then check with nmap.

Sounds good. We'll wait for your update.