Nagios core using lot of memory

[lafargeuser]

My nagios core server (Centos) shows that I am using a lot of memory, even though nothing is running. Why?
Due to I am getting warning alerts.

free -m
total used free shared buffers cached
Mem: 3034 2866 168 0 337 1941
-/+ buffers/cache: 587 2447
Swap: 5023 0 5023

[mguthrie]

Can you post the output from the following:

Code: Select all

ps aux --sort -rss

Also, what version of Nagios Core are you running? Make sure you've got embedded perl disabled as well, it's been known to leak memory with some perl plugins:


In nagios.cfg

Code: Select all

 enable_embedded_perl=0
 use_embedded_perl_implicitly=0

[lafargeuser]

output of # ps aux --sort -rss (Also disabled embedded perl )

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 3471 1.4 3.0 126972 95156 ? Ssl Jan28 444:06 /usr/bin/python
root 22123 4.9 1.7 148028 53228 ? Sl Feb15 192:15 splunkd -p 8089
root 22225 0.0 1.1 203844 34800 ? Sl Feb15 0:18 python -O /opt/
root 22291 0.0 0.4 37604 15216 ? Sl Feb15 0:50 [splunkd pid=22
root 22282 0.0 0.4 37480 15148 ? Sl Feb15 1:42 [splunkd pid=22
root 22283 0.0 0.4 37480 15148 ? Sl Feb15 1:43 [splunkd pid=22
root 22284 0.0 0.4 37480 15148 ? Sl Feb15 1:43 [splunkd pid=22
root 20125 0.0 0.3 24188 12408 ? Ss Feb09 0:01 /usr/sbin/httpd
root 4214 0.0 0.3 26112 10668 ? SN Jan28 0:03 /usr/bin/python
root 3247 0.0 0.3 12116 10488 ? Ss Jan28 0:00 /usr/sbin/resto
root 3738 0.0 0.3 17672 10028 ? Ss Jan28 0:00 cupsd
apache 30751 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30752 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30753 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30754 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30755 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30756 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30757 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
apache 30758 0.0 0.2 24384 7196 ? S Feb17 0:00 /usr/sbin/httpd
root 3704 0.0 0.1 13468 4644 ? S Jan28 0:00 python ./hpssd.
68 3602 0.0 0.1 6324 4296 ? Ss Jan28 1:01 hald
root 2711 0.0 0.1 38012 3824 ? Sl Jan28 6:29 /usr/sbin/vmtoo
root 23747 0.1 0.0 9920 2896 ? Ss 10:46 0:00 sshd: root@pts/
root 3919 0.0 0.0 60216 2736 ? Sl Jan28 0:00 libvirtd --daem
root 2972 0.0 0.0 2320 2316 ? S<Ls Jan28 0:00 iscsid
root 22124 0.0 0.0 25688 2020 ? Ss Feb15 1:30 [splunkd pid=22
root 22285 0.0 0.0 26308 1928 ? Ss Feb15 0:00 [splunkd pid=22
root 22287 0.0 0.0 26308 1928 ? Ss Feb15 0:00 [splunkd pid=22
root 22289 0.0 0.0 26308 1928 ? Ss Feb15 0:00 [splunkd pid=22
root 22295 0.0 0.0 26308 1928 ? Ss Feb15 0:00 [splunkd pid=22
root 3780 0.0 0.0 9300 1916 ? Ss Jan28 0:01 sendmail: accep
root 716 0.0 0.0 3092 1640 ? S<s Jan28 0:00 /sbin/udevd -d
smmsp 3789 0.0 0.0 8148 1516 ? Ss Jan28 0:00 sendmail: Queue
xfs 3875 0.0 0.0 3840 1480 ? Ss Jan28 0:00 xfs -droppriv -
root 23756 0.0 0.0 4672 1480 pts/0 Ss 10:46 0:00 -bash
avahi 3967 0.0 0.0 2728 1404 ? Ss Jan28 0:17 avahi-daemon: r
root 3677 0.0 0.0 30336 1388 ? Ssl Jan28 0:17 automount
root 3568 0.0 0.0 12736 1264 ? Ssl Jan28 0:00 pcscd
root 3833 0.0 0.0 5292 1200 ? Ss Jan28 0:01 crond
dbus 3454 0.0 0.0 13120 1176 ? Ssl Jan28 0:03 dbus-daemon --s
root 3724 0.0 0.0 7068 1068 ? Ss Jan28 0:15 /usr/sbin/sshd
root 4216 0.0 0.0 2564 1052 ? SN Jan28 0:00 /usr/libexec/ga
root 3603 0.0 0.0 3164 988 ? S Jan28 0:00 hald-runner
root 23974 0.0 0.0 4368 936 pts/0 R+ 10:49 0:00 ps aux --sort -
root 3757 0.0 0.0 2728 904 ? Ss Jan28 0:31 xinetd -stayali
nagios 23964 0.0 0.0 12800 904 ? Ssl 10:48 0:00 /usr/local/nagi
68 3611 0.0 0.0 2020 812 ? S Jan28 0:00 hald-addon-acpi
68 3616 0.0 0.0 2020 800 ? S Jan28 0:00 hald-addon-keyb
root 3222 0.0 0.0 13544 792 ? S<sl Jan28 0:16 auditd
root 3485 0.0 0.0 2172 764 ? Ss Jan28 0:00 /usr/sbin/hcid
root 3395 0.0 0.0 1868 744 ? Ss Jan28 0:00 rpc.statd
root 3699 0.0 0.0 5156 744 ? Ss Jan28 0:00 ./hpiod
root 3224 0.0 0.0 13100 724 ? S<sl Jan28 0:08 /sbin/audispd
root 3429 0.0 0.0 5820 648 ? Ss Jan28 0:00 rpc.idmapd
root 3624 0.0 0.0 1976 636 ? S Jan28 15:17 hald-addon-stor
root 1 0.0 0.0 2072 632 ? Ss Jan28 0:30 init [3]
rpc 3349 0.0 0.0 1816 608 ? Ss Jan28 0:00 portmap
root 2965 0.0 0.0 32588 576 ? Ssl Jan28 0:00 brcm_iscsiuio
root 3261 0.0 0.0 1728 572 ? Ss Jan28 0:11 syslogd -m 0
root 3583 0.0 0.0 1676 532 ? Ss Jan28 0:00 /usr/sbin/acpid
root 3491 0.0 0.0 1748 512 ? Ss Jan28 0:00 /usr/sbin/sdpd
root 2991 0.0 0.0 2172 508 ? Ss Jan28 0:09 mcstransd
root 3648 0.0 0.0 1916 464 ? Ss Jan28 0:00 /usr/bin/hidd -
root 4152 0.0 0.0 1664 444 tty1 Ss+ Jan28 0:00 /sbin/mingetty
root 4148 0.0 0.0 3516 440 ? S Jan28 0:00 /usr/sbin/smart
root 3902 0.0 0.0 2268 436 ? Ss Jan28 0:00 /usr/sbin/atd
root 4153 0.0 0.0 1664 428 tty2 Ss+ Jan28 0:00 /sbin/mingetty
root 4155 0.0 0.0 1664 428 tty4 Ss+ Jan28 0:00 /sbin/mingetty
root 4156 0.0 0.0 1664 428 tty5 Ss+ Jan28 0:00 /sbin/mingetty
nobody 4090 0.0 0.0 1832 424 ? S Jan28 0:25 /usr/sbin/dnsma
root 4158 0.0 0.0 1664 424 tty6 Ss+ Jan28 0:00 /sbin/mingetty
root 4154 0.0 0.0 1664 420 tty3 Ss+ Jan28 0:00 /sbin/mingetty
root 2971 0.0 0.0 1868 416 ? Ss Jan28 0:00 iscsid
root 3264 0.0 0.0 1680 404 ? Ss Jan28 0:00 klogd -x
root 3316 0.0 0.0 2472 368 ? Ss Jan28 1:32 irqbalance
root 3804 0.0 0.0 1908 368 ? Ss Jan28 0:00 gpm -m /dev/inp
avahi 3968 0.0 0.0 2600 316 ? Ss Jan28 0:00 avahi-daemon: c
root 2 0.0 0.0 0 0 ? S< Jan28 0:06 [migration/0]
root 3 0.0 0.0 0 0 ? SN Jan28 0:00 [ksoftirqd/0]
root 4 0.0 0.0 0 0 ? S< Jan28 0:00 [watchdog/0]
root 5 0.0 0.0 0 0 ? S< Jan28 0:06 [migration/1]
root 6 0.0 0.0 0 0 ? SN Jan28 0:00 [ksoftirqd/1]
root 7 0.0 0.0 0 0 ? S< Jan28 0:00 [watchdog/1]
root 8 0.0 0.0 0 0 ? S< Jan28 0:07 [migration/2]
root 9 0.0 0.0 0 0 ? SN Jan28 0:01 [ksoftirqd/2]
root 10 0.0 0.0 0 0 ? S< Jan28 0:00 [watchdog/2]
root 11 0.0 0.0 0 0 ? S< Jan28 0:10 [migration/3]
root 12 0.0 0.0 0 0 ? SN Jan28 0:00 [ksoftirqd/3]
root 13 0.0 0.0 0 0 ? S< Jan28 0:00 [watchdog/3]
root 14 0.0 0.0 0 0 ? S< Jan28 15:05 [events/0]
root 15 0.0 0.0 0 0 ? S< Jan28 0:00 [events/1]
root 16 0.0 0.0 0 0 ? S< Jan28 0:00 [events/2]
root 17 0.0 0.0 0 0 ? S< Jan28 0:00 [events/3]
root 18 0.0 0.0 0 0 ? S< Jan28 0:00 [khelper]
root 19 0.0 0.0 0 0 ? S< Jan28 0:00 [kthread]
root 25 0.0 0.0 0 0 ? S< Jan28 0:00 [kblockd/0]
root 26 0.0 0.0 0 0 ? S< Jan28 0:00 [kblockd/1]
root 27 0.0 0.0 0 0 ? S< Jan28 0:04 [kblockd/2]
root 28 0.0 0.0 0 0 ? S< Jan28 0:00 [kblockd/3]
root 29 0.0 0.0 0 0 ? S< Jan28 0:00 [kacpid]
root 192 0.0 0.0 0 0 ? S< Jan28 0:00 [cqueue/0]
root 193 0.0 0.0 0 0 ? S< Jan28 0:00 [cqueue/1]
root 194 0.0 0.0 0 0 ? S< Jan28 0:00 [cqueue/2]
root 195 0.0 0.0 0 0 ? S< Jan28 0:00 [cqueue/3]
root 198 0.0 0.0 0 0 ? S< Jan28 0:00 [khubd]
root 200 0.0 0.0 0 0 ? S< Jan28 0:00 [kseriod]
root 277 0.0 0.0 0 0 ? S Jan28 0:00 [khungtaskd]
root 278 0.0 0.0 0 0 ? S Jan28 3:23 [pdflush]
root 279 0.0 0.0 0 0 ? S Jan28 0:29 [pdflush]
root 280 0.0 0.0 0 0 ? S< Jan28 0:16 [kswapd0]
root 281 0.0 0.0 0 0 ? S< Jan28 0:00 [aio/0]
root 282 0.0 0.0 0 0 ? S< Jan28 0:00 [aio/1]
root 283 0.0 0.0 0 0 ? S< Jan28 0:00 [aio/2]
root 284 0.0 0.0 0 0 ? S< Jan28 0:00 [aio/3]
root 502 0.0 0.0 0 0 ? S< Jan28 0:00 [kpsmoused]
root 570 0.0 0.0 0 0 ? S< Jan28 0:00 [mpt_poll_0]
root 571 0.0 0.0 0 0 ? S< Jan28 0:00 [mpt/0]
root 572 0.0 0.0 0 0 ? S< Jan28 0:00 [scsi_eh_0]
root 578 0.0 0.0 0 0 ? S< Jan28 0:00 [ata/0]
root 579 0.0 0.0 0 0 ? S< Jan28 0:00 [ata/1]
root 580 0.0 0.0 0 0 ? S< Jan28 0:00 [ata/2]
root 581 0.0 0.0 0 0 ? S< Jan28 0:00 [ata/3]
root 582 0.0 0.0 0 0 ? S< Jan28 0:00 [ata_aux]
root 593 0.0 0.0 0 0 ? S< Jan28 0:00 [kstriped]
root 614 0.0 0.0 0 0 ? S< Jan28 0:00 [ksnapd]
root 652 0.0 0.0 0 0 ? S< Jan28 10:59 [kjournald]
root 683 0.0 0.0 0 0 ? S< Jan28 0:01 [kauditd]
root 2048 0.0 0.0 0 0 ? S< Jan28 0:00 [kmpathd/0]
root 2049 0.0 0.0 0 0 ? S< Jan28 0:00 [kmpathd/1]
root 2050 0.0 0.0 0 0 ? S< Jan28 0:00 [kmpathd/2]
root 2051 0.0 0.0 0 0 ? S< Jan28 0:00 [kmpathd/3]
root 2052 0.0 0.0 0 0 ? S< Jan28 0:00 [kmpath_handle]
root 2116 0.0 0.0 0 0 ? S< Jan28 0:00 [kjournald]
root 2561 0.0 0.0 0 0 ? S< Jan28 0:00 [vmmemctl]
root 2818 0.0 0.0 0 0 ? S< Jan28 0:00 [iscsi_eh]
root 2904 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_addr]
root 2920 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_mcast]
root 2921 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_inform]
root 2922 0.0 0.0 0 0 ? S< Jan28 0:00 [local_sa]
root 2928 0.0 0.0 0 0 ? S< Jan28 0:00 [iw_cm_wq]
root 2934 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_cm/0]
root 2935 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_cm/1]
root 2936 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_cm/2]
root 2937 0.0 0.0 0 0 ? S< Jan28 0:00 [ib_cm/3]
root 2943 0.0 0.0 0 0 ? S< Jan28 0:00 [rdma_cm]
root 3382 0.0 0.0 0 0 ? S< Jan28 0:00 [rpciod/0]
root 3383 0.0 0.0 0 0 ? S< Jan28 0:00 [rpciod/1]
root 3384 0.0 0.0 0 0 ? S< Jan28 0:00 [rpciod/2]
root 3385 0.0 0.0 0 0 ? S< Jan28 0:00 [rpciod/3]
root 3521 0.0 0.0 0 0 ? S< Jan28 0:00 [krfcommd]

[scottwilkerson]

Actually looking at this

Code: Select all

free -m
total used free shared buffers cached
Mem: 3034 2866 168 0 337 1941
-/+ buffers/cache: 587 2447
Swap: 5023 0 5023

while 2866 is marked "used" you have 1941 that us cached, so the actual use is really 925. Additionally, it looks like biggest memory users are some python scripts you are running and splunkd....