NagiosXI ram disk full

[thanigaivel.a]

Hi Team,

As per the recommendation, to increase nagiosXI performance have enabled ram disk in our environment. Though, I gave (800MB) more than the value where script recommends during installation, but now it got filled 100%.

So please guide us to resolve the issue.

Code: Select all

[root@usa0300lv6348 libexec]# cat  /etc/sysconfig/nagios
USE_RAMDISK=1
RAMDISK_DIR=/var/nagiosramdisk
RAMDISK_SIZE=800
if [ "`mount |grep "${RAMDISK_DIR} type tmpfs"`"X == "X" ]; then
   mount -t tmpfs -o size=${RAMDISK_SIZE}m tmpfs ${RAMDISK_DIR}
fi
mkdir -p -m 775 ${RAMDISK_DIR} ${RAMDISK_DIR}/tmp ${RAMDISK_DIR}/spool ${RAMDISK_DIR}/spool/checkresults ${RAMDISK_DIR}/spool/xidpe ${RAMDISK_DIR}/spool/perfdata
chown -R nagios:nagios ${RAMDISK_DIR}
[root@usa0300lv6348 libexec]#
[root@usa0300lv6348 libexec]# df -h /var/nagiosramdisk
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           800M  800M     0 100% /var/nagiosramdisk
[root@usa0300lv6348 libexec]#

Regards,
A.Thanigaivel

[tgriep]

When you setup the ramdisk, did you run Automatic RAM Disk Installation script from the guide below or did you manually edit the configuration files?
https://assets.nagios.com/downloads/nag ... giosXI.pdf

One thing to try is to restart the processes that use and process the files in the ramdisk. To do that, run the following as root.
service npcd restart
service nagios restart

It may take a while to clean up the directory but if it still fails, run the following commands and post the /tmp/info.txt file.
df -h >/tmp/info.txt
df -i >>/tmp/info.txt
ls -lR /var/nagiosramdisk >>/tmp/info.txt

We also may need to see a System Profile to check the settings, can you post it of PM it to me?
To get your system profile. Login to the Nagios XI GUI using a web browser.
Click the "Admin" > "System Profile" Menu
Click the "Download Profile" button
Save the profile.zip file and upload it to the post.

[thanigaivel.a]

Hi team,

I had configured ram disk using Automatic RAM Disk Installation script and also tried the below commands, but disk space not reducing. had provided the requested informations.

service npcd restart
service nagios restart

[root@usa0300lv6348 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-lv_root 45G 26G 20G 58% /
devtmpfs 4.8G 0 4.8G 0% /dev
tmpfs 4.9G 0 4.9G 0% /dev/shm
tmpfs 4.9G 505M 4.4G 11% /run
tmpfs 4.9G 0 4.9G 0% /sys/fs/cgroup
/dev/sda1 976M 203M 707M 23% /boot
/dev/mapper/vg00-lv_var 30G 17G 14G 56% /var
/dev/mapper/vg00-lv_audit 1014M 222M 793M 22% /var/log/audit
/dev/mapper/vg00-lv_openv 10G 70M 10G 1% /usr/openv
/dev/mapper/vg00-lv_opt 50G 177M 50G 1% /opt
/dev/mapper/vg00-lv_export 4.0G 2.8G 1.3G 69% /export
/dev/mapper/vg00-lv_tmp 10G 33M 10G 1% /tmp
/dev/mapper/vg00-lv_nagiosxi 50G 7.3G 43G 15% /usr/local
tmpfs 984M 0 984M 0% /run/user/1000
tmpfs 800M 800M 0 100% /var/nagiosramdisk
tmpfs 984M 0 984M 0% /run/user/420873
vnx-01:/fs101/home/athanigai 3.1T 3.1T 56G 99% /home/athanigai
tmpfs 984M 0 984M 0% /run/user/1001
ns80:/fs002/home/arunprar 688G 560G 129G 82% /home/arunprar
ns80:/fs062/home/asahu 772G 636G 137G 83% /home/asahu
ns80:/fs054/home/aschroet 1.7T 1.5T 242G 86% /home/aschroet
ns80:/fs054/home/asenthil 1.7T 1.5T 242G 86% /home/asenthil
[root@usa0300lv6348 ~]#
[root@usa0300lv6348 ~]# df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/mapper/vg00-lv_root 23592960 64944 23528016 1% /
devtmpfs 1256630 437 1256193 1% /dev
tmpfs 1259363 1 1259362 1% /dev/shm
tmpfs 1259363 714 1258649 1% /run
tmpfs 1259363 16 1259347 1% /sys/fs/cgroup
/dev/sda1 65536 350 65186 1% /boot
/dev/mapper/vg00-lv_var 15728640 11793 15716847 1% /var
/dev/mapper/vg00-lv_audit 524288 96 524192 1% /var/log/audit
/dev/mapper/vg00-lv_openv 5242880 4 5242876 1% /usr/openv
/dev/mapper/vg00-lv_opt 26214400 9170 26205230 1% /opt
/dev/mapper/vg00-lv_export 2097152 23380 2073772 2% /export
/dev/mapper/vg00-lv_tmp 5242880 43 5242837 1% /tmp
/dev/mapper/vg00-lv_nagiosxi 26214400 19564 26194836 1% /usr/local
tmpfs 1259363 1 1259362 1% /run/user/1000
tmpfs 1259363 10 1259353 1% /var/nagiosramdisk
tmpfs 1259363 1 1259362 1% /run/user/420873
vnx-01:/fs101/home/athanigai 257949694 94593233 163356461 37% /home/athanigai
tmpfs 1259363 1 1259362 1% /run/user/1001
ns80:/fs002/home/arunprar 87953854 17396306 70557548 20% /home/arunprar
ns80:/fs062/home/asahu 98699134 22994731 75704403 24% /home/asahu
ns80:/fs054/home/aschroet 216968062 54116460 162851602 25% /home/aschroet
ns80:/fs054/home/asenthil 216968062 54116460 162851602 25% /home/asenthil
[root@usa0300lv6348 ~]#

[root@usa0300lv6348 ~]# ls -lR /var/nagiosramdisk
/var/nagiosramdisk:
total 819200
drwxr-xr-x 2 nagios nagios 40 Dec 20 13:44 checkresults
-rw-r--r-- 1 nagios nagios 296361984 Jan 23 15:32 host-perfdata
-rw-r--r-- 1 nagios nagios 5791744 Jan 23 15:29 objects.cache
-rw-r--r-- 1 nagios nagios 536707072 Jan 23 15:32 service-perfdata
drwxrwxr-x 5 nagios nagios 100 Dec 20 13:44 spool
drwxrwxr-x 2 nagios nagios 40 Jan 23 15:29 tmp

/var/nagiosramdisk/checkresults:
total 0

/var/nagiosramdisk/spool:
total 0
drwxrwxr-x 2 nagios nagios 40 Jan 23 15:29 checkresults
drwxrwxr-x 2 nagios nagios 40 Dec 20 13:44 perfdata
drwxrwxr-x 2 nagios nagios 40 Dec 20 13:44 xidpe

/var/nagiosramdisk/spool/checkresults:
total 0

/var/nagiosramdisk/spool/perfdata:
total 0

/var/nagiosramdisk/spool/xidpe:
total 0

/var/nagiosramdisk/tmp:
total 0
[root@usa0300lv6348 ~]#

regards,
thanigaivel.a

[tgriep]

I am thinking one of the commands in the nagios.cfg file of the commands.cfg file did not get updated.
Can you run the following as root and post the output?

Code: Select all

grep nagiosramdisk /usr/local/nagios/etc/*

[thanigaivel.a]

hi team,

FYI.

Code: Select all

[root@usa0300lv6348 ~]#  grep nagiosramdisk /usr/local/nagios/etc/*
grep: /usr/local/nagios/etc/cfgprep: Is a directory
grep: /usr/local/nagios/etc/hosts: Is a directory
grep: /usr/local/nagios/etc/import: Is a directory
grep: /usr/local/nagios/etc/mkdir: Is a directory
/usr/local/nagios/etc/nagios.cfg:service_perfdata_file=/var/nagiosramdisk/service-perfdata
/usr/local/nagios/etc/nagios.cfg:host_perfdata_file=/var/nagiosramdisk/host-perfdata
/usr/local/nagios/etc/nagios.cfg:check_result_path=/var/nagiosramdisk/spool/checkresults
/usr/local/nagios/etc/nagios.cfg:object_cache_file=/var/nagiosramdisk/objects.cache
/usr/local/nagios/etc/nagios.cfg:status_file=/var/nagiosramdisk/status.dat
/usr/local/nagios/etc/nagios.cfg:temp_path=/var/nagiosramdisk/tmp
grep: /usr/local/nagios/etc/pnp: Is a directory
grep: /usr/local/nagios/etc/services: Is a directory
grep: /usr/local/nagios/etc/static: Is a directory

[thanigaivel.a]

attached the profile

[tgriep]

Thanks for the profile. It looks like the commands that move the performance data have not been changed.
In the PDF file at the link below, go to page 8 and edit the commands. After that, the system should process the data which will keep the ramdisk from filling up.
https://assets.nagios.com/downloads/nag ... giosXI.pdf

[thanigaivel.a]

Had followed the steps provided and restarted the service. so far disk space for ramdisk has not yet reduced.

Also, we are facing some other issues with performance graph is not working and acknowledge & add comments section is not working even after giving comments over it.

thanks in advance

regards,
A.Thanigaivel

[dwhitfield]

As long as the ramdisk is full, performance data is not likely to work. Try running the following:

Code: Select all

chage -I -1 -m 0 -M 99999 -E -1 nagios

Also, can you send a new profile so we can verify the ramdisk is now set up properly?

[thanigaivel.a]

Had done the changes as per the instructions provided on the pdf document. and also zipped the (/var/nagiosramdisk/host* & /var/nagiosramdisk/service*) files and moved to the /var/nagiosramdisk/spool/perfdata, and restarted the services. now the disk space also freed.

However, still performance graphs is not working. kindly help me out.

Attached the profile.

[root@usa0300lv6348 spool]# df -hl
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg00-lv_root 45G 26G 20G 58% /
devtmpfs 4.8G 0 4.8G 0% /dev
tmpfs 4.9G 0 4.9G 0% /dev/shm
tmpfs 4.9G 505M 4.4G 11% /run
tmpfs 4.9G 0 4.9G 0% /sys/fs/cgroup
/dev/sda1 976M 203M 707M 23% /boot
/dev/mapper/vg00-lv_var 30G 17G 14G 57% /var
/dev/mapper/vg00-lv_audit 1014M 222M 793M 22% /var/log/audit
/dev/mapper/vg00-lv_openv 10G 70M 10G 1% /usr/openv
/dev/mapper/vg00-lv_opt 50G 177M 50G 1% /opt
/dev/mapper/vg00-lv_export 4.0G 2.8G 1.3G 69% /export
/dev/mapper/vg00-lv_tmp 10G 45M 10G 1% /tmp
/dev/mapper/vg00-lv_nagiosxi 50G 7.4G 43G 15% /usr/local
tmpfs 984M 0 984M 0% /run/user/1000
tmpfs 800M 20M 781M 3% /var/nagiosramdisk
tmpfs 984M 0 984M 0% /run/user/420873
tmpfs 984M 0 984M 0% /run/user/1001