ls -la /var/lib/rsyslog/ [17-02-08 9:28:35]
total 764564
drwx------. 2 root root 24576 Feb 8 09:28 ./
drwxr-xr-x. 33 root root 4096 Jan 18 09:25 ../
-rw------- 1 root root 126 Feb 8 09:28 imjournal.state
-rw-------. 1 root root 1049470 Dec 4 07:11 nlsFwdRule0.00000097
-rw-------. 1 root root 1049562 Dec 4 09:07 nlsFwdRule0.00000098
-rw-------. 1 root root 1049003 Dec 4 11:04 nlsFwdRule0.00000099
-rw-------. 1 root root 1048808 Dec 4 14:59 nlsFwdRule0.00000100
-rw-------. 1 root root 1048780 Dec 4 16:57 nlsFwdRule0.00000101
-rw-------. 1 root root 1049464 Dec 4 18:53 nlsFwdRule0.00000102
-rw-------. 1 root root 1049050 Dec 4 22:46 nlsFwdRule0.00000103
-rw-------. 1 root root 1049738 Dec 5 00:40 nlsFwdRule0.00000104
-rw-------. 1 root root 1049273 Dec 5 04:32 nlsFwdRule0.00000105
-rw-------. 1 root root 1048604 Dec 5 06:28 nlsFwdRule0.00000106
-rw-------. 1 root root 1049209 Dec 5 08:24 nlsFwdRule0.00000107
-rw-------. 1 root root 1049078 Dec 5 12:18 nlsFwdRule0.00000108
-rw-------. 1 root root 1049586 Dec 5 14:14 nlsFwdRule0.00000109
-rw-------. 1 root root 1048759 Dec 5 16:10 nlsFwdRule0.00000110
.....
Seems like there are a lot of nlsFwdRule0 files in /var/lib/rsyslog. SELinux is disabled on this server. What could be casuing this?
This server had SELinux in the past, but it was disabled. I tried changing the port to our standard Linux syslog port and restarted rsyslog. Can I just remove all these files?
Those are spool files rsyslog uses on the back-end. From their docs:
Please note that actual spool files are only created if the remote server is down and there is no more space in the in-memory queue.
The implication being that the NLS cluster this machine is shipping to was/is unreachable. Once rsyslog runs out of memory to store events in, it writes the raw data to disk. If there aren't a bunch of files being generated frequently, there's likely just a very large backlog that rsyslog is churning through. If you notice those files being generated consistently, something may be going wrong within rsyslog. Though I would first verify this machine is able to communicate with Nagios Log Server over the designated port.
Aaah yes indeed this server had two nic's and a DNS server confgured which had no record for our nls servers. Added the nls servers to hosts and seems solved. Plese close this thread. Tx!