Page 1 of 1

/var/spool/snmptt - many files

Posted: Wed Apr 01, 2015 10:53 am
by gravitas
We are running Nagiox 2014R2.6. The lvm_root filed up. it ran out of space, but also inodes. df -i showed 0 inodes available even after I deleted some files.

Eventually, I moved some files and extended lvm_root.


The problem seems to stem from the 1.6 million files in /var/spool/snmptt. It has a size of 7GB, but why so many files. There has to be something wrong. It just keeps spitting out files,,,

-rw-r--r-- 1 root root 359 Apr 1 15:48 #snmptt-trap-1427903338760530
-rw-r--r-- 1 root root 357 Apr 1 15:48 #snmptt-trap-1427903339067666
-rw-r--r-- 1 root root 362 Apr 1 15:48 #snmptt-trap-1427903339216766
-rw-r--r-- 1 root root 362 Apr 1 15:48 #snmptt-trap-1427903339302945
-rw-r--r-- 1 root root 357 Apr 1 15:48 #snmptt-trap-1427903339403718
-rw-r--r-- 1 root root 361 Apr 1 15:48 #snmptt-trap-1427903339565997
-rw-r--r-- 1 root root 362 Apr 1 15:49 #snmptt-trap-1427903346306110
-rw-r--r-- 1 root root 362 Apr 1 15:49 #snmptt-trap-1427903346393499
-rw-r--r-- 1 root root 359 Apr 1 15:49 #snmptt-trap-1427903346775887
-rw-r--r-- 1 root root 240 Apr 1 15:49 #snmptt-trap-1427903347585478
-rw-r--r-- 1 root root 240 Apr 1 15:49 #snmptt-trap-1427903347681689
-rw-r--r-- 1 root root 240 Apr 1 15:49 #snmptt-trap-1427903347784686
-rw-r--r-- 1 root root 240 Apr 1 15:49 #snmptt-trap-1427903347887742
-rw-r--r-- 1 root root 240 Apr 1 15:49 #snmptt-trap-1427903347981256
-rw-r--r-- 1 root root 285 Apr 1 15:49 #snmptt-trap-1427903348088280
-rw-r--r-- 1 root root 285 Apr 1 15:49 #snmptt-trap-1427903348202722
-rw-r--r-- 1 root root 285 Apr 1 15:49 #snmptt-trap-1427903348295305
-rw-r--r-- 1 root root 285 Apr 1 15:49 #snmptt-trap-1427903348392888
-rw-r--r-- 1 root root 285 Apr 1 15:49 #snmptt-trap-1427903348474349
-rw-r--r-- 1 root root 360 Apr 1 15:49 #snmptt-trap-1427903348852597
-rw-r--r-- 1 root root 359 Apr 1 15:49 #snmptt-trap-1427903349356114
-rw-r--r-- 1 root root 359 Apr 1 15:49 #snmptt-trap-1427903359250641
-rw-r--r-- 1 root root 362 Apr 1 15:49 #snmptt-trap-1427903366170041
-rw-r--r-- 1 root root 359 Apr 1 15:49 #snmptt-trap-1427903369116686
-rw-r--r-- 1 root root 359 Apr 1 15:49 #snmptt-trap-1427903369439603
[root@nagios snmptt]#


Does anybody have a suggestion?

Re: /var/spool/snmptt - many files

Posted: Wed Apr 01, 2015 11:05 am
by tmcdonald
The first thing is those trap files will need to be removed. With that many in the directory most processes can't stat() them so they do not get processed. That also means that a regular rm will not work, you need to use this:

Code: Select all

find /var/spool/snmptt -type f -delete
This command can take a while to complete. Then make sure that snmptt is running:

Code: Select all

service snmptt status
service snmptt restart
Please show the output of those commands.

Re: /var/spool/snmptt - many files

Posted: Wed Apr 01, 2015 11:08 am
by jolson
It looks like you are receiving traps that are not being processed.

The daemon snmptrapd writes the files to the spool directory. (/var/spool/snmptt)

If you have no need for these files, you could stop the snmptrapd service and delete them.

Code: Select all

service snmptrapd stop

Code: Select all

find /var/spool/snmptt/. -name snmptt-trap-* -delete
Keep in mind that this will delete all of the trap data in that directory - the command may take a while to run.

There may be a problem with the snmptt daemon, since your traps are not being processed fast enough.
The snmptt daemon picks up the files from the spool directory and processes them.

Is The snmptt Service Running?

Code: Select all

service snmptt status
You could also check on the following log files:

Code: Select all

tail /var/log/snmptt/snmpttsystem.log
tail /var/log/snmptt/snmptt.log
tail /var/log/snmptt/snmpttunknown.log
Thanks - let me know if the above helps!

Re: /var/spool/snmptt - many files

Posted: Wed Apr 01, 2015 5:20 pm
by gravitas
Yes, it was hard to delete then all. I just removed the directory and recreated it with proper permission.

Then, I started the snmptt service again. After watching the directory "watch -d -n 1 ls -l". It is processing them now. thx