NagiosXI ram disk full

[scottwilkerson]

This looks like almost everything is correct, it should take 5-15 minutes for performance data to start filling in on your graphs.

I did find an old ndo2db process running that should be killed or it could cause problems.

Code: Select all

kill -9 6448

[thanigaivel.a]

As per your request, had killed the ndo2db process and restarted the service. It's been more than 1 hour and still performance graph is not working (attached the error screeshot).

Also, found that the "Submit passive check result" option was not working.


[root@usa0300lv6348 checkresults]# kill -9 6448
-bash: kill: (6448) - No such process
[root@usa0300lv6348 checkresults]# ps -ef |grep ndo2db
root 12182 16160 0 09:12 pts/0 00:00:00 grep --color=auto ndo2db
[root@usa0300lv6348 checkresults]#


[root@usa0300lv6348 checkresults]# service ndo2db stop
Stopping ndo2db (via systemctl): [ OK ]
[root@usa0300lv6348 checkresults]# ps -ef |grep ndo2db
root 13888 16160 0 09:13 pts/0 00:00:00 grep --color=auto ndo2db
[root@usa0300lv6348 checkresults]# rm -f /usr/local/nagios/var/ndo2db.lock /usr/local/nagios/var/ndo.sock
[root@usa0300lv6348 checkresults]# rm -f /usr/local/nagios/var/ndo2db.lock /usr/local/nagios/var/ndo.sock
[root@usa0300lv6348 checkresults]# service ndo2db start
Starting ndo2db (via systemctl): [ OK ]
[root@usa0300lv6348 checkresults]# ps -ef |grep ndo2db
nagios 14290 1 0 09:13 ? 00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
nagios 14320 14290 0 09:13 ? 00:00:00 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
nagios 14321 14320 38 09:13 ? 00:00:01 /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg -f
root 14478 16160 0 09:13 pts/0 00:00:00 grep --color=auto ndo2db
[root@usa0300lv6348 checkresults]#

[scottwilkerson]

Did you verify other hosts, or just the one in this search?

Does this host have recent results like you would expect displayed on the host detail page?

Do the perfdata files exists again?

Code: Select all

ls -l /var/nagiosramdisk/*perfdata

How about the xidpe spool

Code: Select all

ls -l /var/nagiosramdisk/spool/xidpe/
ls -l /var/nagiosramdisk/spool/perfdata/

What is the error that is leading you to believe this

Also, found that the "Submit passive check result" option was not working.

[thanigaivel.a]

It looks again ramdisk is filling to 100%, even after the work around you gave and looks its not moving to the spool path.

[root@usa0300lv6348 ~]# ls -l /var/nagiosramdisk/*perfdata
-rw-r--r-- 1 nagios nagios 200114176 Jan 31 10:41 /var/nagiosramdisk/host-perfdata
-rw-r--r-- 1 nagios nagios 632893440 Jan 31 10:41 /var/nagiosramdisk/service-perfdata
[root@usa0300lv6348 ~]# ls -l /var/nagiosramdisk/spool/xidpe/
total 0
[root@usa0300lv6348 ~]# ls -l /var/nagiosramdisk/spool/perfdata/
total 0
[root@usa0300lv6348 ~]# df -hl
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-lv_root 45G 26G 20G 58% /
devtmpfs 4.8G 0 4.8G 0% /dev
tmpfs 4.9G 0 4.9G 0% /dev/shm
tmpfs 4.9G 537M 4.3G 11% /run
tmpfs 4.9G 0 4.9G 0% /sys/fs/cgroup
/dev/sda1 976M 203M 707M 23% /boot
/dev/mapper/vg00-lv_var 30G 18G 13G 60% /var
/dev/mapper/vg00-lv_audit 1014M 222M 793M 22% /var/log/audit
/dev/mapper/vg00-lv_openv 10G 70M 10G 1% /usr/openv
/dev/mapper/vg00-lv_opt 50G 177M 50G 1% /opt
/dev/mapper/vg00-lv_export 4.0G 2.8G 1.3G 69% /export
/dev/mapper/vg00-lv_tmp 10G 45M 10G 1% /tmp
/dev/mapper/vg00-lv_nagiosxi 50G 7.4G 43G 15% /usr/local
tmpfs 984M 0 984M 0% /run/user/1000
tmpfs 800M 800M 0 100% /var/nagiosramdisk
tmpfs 984M 0 984M 0% /run/user/420873
tmpfs 984M 0 984M 0% /run/user/1001
[root@usa0300lv6348 ~]#

[thanigaivel.a]

May I know your team's working hours, because I felt getting delayed response, due I want to fix this issue asap.

I work in US-EST shifts. morning 9am-6pm

[tgriep]

When you changed the commands in the XI Core Config Manager > Commands menu, there was a space added in the command line which is causing the performance data to not get processes and also filling up the ramdisk.
Go in to the Core Config Manager > Commands menu and edit this command

Code: Select all

process-host-perfdata-file-bulk

Change the command line from

Code: Select all

/bin/mv /var/nagiosramdisk/host-perfdata /var/nagiosramdisk/spool/xidpe/ $TIMET$.perfdata.host

to

Code: Select all

/bin/mv /var/nagiosramdisk/host-perfdata /var/nagiosramdisk/spool/xidpe/$TIMET$.perfdata.host

Save it and edit this command

Code: Select all

process-service-perfdata-file-bulk

Change the command line from

Code: Select all

/bin/mv /var/nagiosramdisk/service-perfdata /var/nagiosramdisk/spool/xidpe/ $TIMET$.perfdata.service

to

Code: Select all

/bin/mv /var/nagiosramdisk/service-perfdata /var/nagiosramdisk/spool/xidpe/$TIMET$.perfdata.service

Save it and Apply the Config.
Delete the following files from the server

Code: Select all

/var/nagiosramdisk/host-perfdata
/var/nagiosramdisk/service-perfdata

Give the system 15 to 20 minutes to gather data and see if the Graphs start to populate.

Our normal business hours (Central Time) are listed below:
Monday – Thursday: 9:00 am – 5:00 pm
Friday: 9:00 am – 2:00 pm

[thanigaivel.a]

thanks, Now the performance graph is working.

[dwhitfield]

It sounds like this issue has been resolved. Is it okay if we lock this thread? Thanks for choosing the Nagios forums!