Page 1 of 2
Using a RAMdisk
Posted: Mon Apr 13, 2015 7:33 am
by WillemDH
Hello,
Today I started with configuring a RAMdisk on my XI production server. I followed all steps in
http://assets.nagios.com/downloads/nagi ... giosXI.pdf except the one to move the perfdata spool folder. I'm not yet sure however if the RAMdisk helps to improve performance.
Please check the attached graphs. As you can see avg cpu wait was 1.6%. After moving to a RAMdisk (from 13:00), cpu wait is avg 5.3 %. I hope the situation will calm down a bit, but I would have expected a small improvement after 60 minutes at least. Did the same procedure on my QA, but the load on that server is really too low to notice any difference.
Thoughts?
Grtz
Willem
Re: Using a RAMdisk
Posted: Mon Apr 13, 2015 8:58 am
by tmcdonald
Using a ramdisk is more for reducing disk read/write times than for reducing CPU usage.
Re: Using a RAMdisk
Posted: Mon Apr 13, 2015 9:46 am
by WillemDH
iowait is time that the processor/processors are waiting (i.e. is in an idle state and does nothing), during which there in fact was outstanding disk I/O requests.
http://veithen.github.io/2013/11/18/iowait-linux.html
So wouldn't making use of a ramdisk decrease cpu wait? I'll see if I can get some disk io graphs tomorrow.
Re: Using a RAMdisk
Posted: Mon Apr 13, 2015 9:54 am
by abrist
I think the most important bit from the at article:
For a given CPU, the I/O wait time is the time during which that CPU was idle (i.e. didn’t execute any tasks) and there was at least one outstanding disk I/O operation requested by a task scheduled on that CPU (at the time it generated that I/O request).
So I think both of you are right:
tmcdonald wrote:Using a ramdisk is more for reducing disk read/write times than for reducing CPU usage.
True, which will in turn reduce CPU wait:
WillemDH wrote:So wouldn't making use of a ramdisk decrease cpu wait?
Exactly. It is important to remember that CPU wait is a form of CPU idle, and IO Wait is a form of CPU wait. You are technically *not* reducing CPU usage by using a ramdisk, you are instead reducing the time that CPUs idle when there are still tasks to complete.
Re: Using a RAMdisk
Posted: Wed Apr 22, 2015 3:53 am
by WillemDH
Ok, I completely followed the RAMdisk procedure and it seems my graphs stopped working after changing the commands in the end of the procedure and applying configuration. I tried restarting the NPCD service, but my graphs are still empty starting from the moment I applied configuration.
I took a snapshot and will revert to the snapshot, but what could be the reason my graphs stopped working? What else can I do to troubleshoot this issue?
It seems some files are being created in the perfdata folder on the ramdisk, but I'm not sure if this looks like it should be:
Code: Select all
ls -la /var/nagiosramdisk/spool/perfdata/
total 1300
drwxrwxr-x 2 nagios nagios 220 Apr 22 10:47 .
drwxrwxr-x 5 nagios nagios 100 Apr 22 10:17 ..
-rw-r--r-- 1 nagios nagios 172335 Apr 22 10:46 1429692361.perfdata.service-PID-53690
-rw-r--r-- 1 nagios nagios 306145 Apr 22 10:46 1429692377.perfdata.service-PID-53692
-rw-r--r-- 1 nagios nagios 188663 Apr 22 10:46 1429692391.perfdata.service
-rw-r--r-- 1 nagios nagios 2336 Apr 22 10:46 1429692406.perfdata.host
-rw-r--r-- 1 nagios nagios 197109 Apr 22 10:46 1429692406.perfdata.service
-rw-r--r-- 1 nagios nagios 1154 Apr 22 10:46 1429692421.perfdata.host
-rw-r--r-- 1 nagios nagios 277195 Apr 22 10:47 1429692421.perfdata.service
-rw-r--r-- 1 nagios nagios 931 Apr 22 10:47 1429692437.perfdata.host
-rw-r--r-- 1 nagios nagios 162994 Apr 22 10:47 1429692437.perfdata.service
Code: Select all
ls -la /var/nagiosramdisk/spool/xidpe/
total 360
drwxrwxr-x 2 nagios nagios 80 Apr 22 10:48 .
drwxrwxr-x 5 nagios nagios 100 Apr 22 10:17 ..
-rw-r--r-- 1 nagios nagios 18584 Apr 22 10:48 1429692511.perfdata.host
-rw-r--r-- 1 nagios nagios 348118 Apr 22 10:48 1429692511.perfdata.service
Code: Select all
ls -la /var/nagiosramdisk/
total 29220
drwxrwxrwt 4 nagios nagios 160 Apr 22 10:51 .
drwxr-xr-x. 23 root root 4096 Apr 13 13:04 ..
-rw-r--r-- 1 nagios nagios 0 Apr 22 10:51 host-perfdata
-rw-r--r-- 1 nagios nagios 12201097 Apr 22 10:29 objects.cache
-rw-r--r-- 1 nagios nagios 16010 Apr 22 10:51 service-perfdata
drwxrwxr-x 5 nagios nagios 100 Apr 22 10:17 spool
-rw-r--r-- 1 nagios nagios 17695236 Apr 22 10:51 status.dat
drwxrwxr-x 2 nagios nagios 40 Apr 22 10:29 tmp
Please advise how to get this working. It's only the part "Moving Performance Data to the RAM Disk" that is not working.
Grtz
Willem
Re: Using a RAMdisk
Posted: Wed Apr 22, 2015 4:17 am
by WillemDH
Ah, right before I was considering restoring a snapshot, I decided to give the server one last reboot (plus apply config, as my hosts are still greyed out untill nagios service is restarted) it seems the graphs started working again. So this is a good thing I guess. I do not see an immediate performance improvement however, but it's probably a bit early to tell. it might be a good idea to suggest a reboot in the RAMdisk documentation? i'll add some graphs to this thread later.
Some questions: Is it possible to rollback the RAMdisk setup? I noted all original configurations, so if I reconfigured everything like it was, it should just start working again without RAMdisk?
Re: Using a RAMdisk
Posted: Wed Apr 22, 2015 8:05 am
by WillemDH
Ok, another update on this. It seems our Nagios XI server is very sensitive to storage migrations of other hosts, even when they are running on different datastores and different ESXi server. I will try to make a post about cpuwait on my blog and explain the process I've gone through this afternoon. In short, it seems a colleague of me was working from home and updating the frmware of some ESXi server, migrating all the vm guests. Even when they were running on another datastore and another ESXi server, the impact on the Nagios XI server was noticed resulting in triple cpu wait and cpu load.
I hope the following days will show if using a RAMdisk did improve something or not.
Grtz
Willem
Re: Using a RAMdisk
Posted: Wed Apr 22, 2015 10:23 am
by lmiltchev
I hope the following days will show if using a RAMdisk did improve something or not.
Keep us posted, Willem. It would be nice to have some feedback/stats on the performance improvements.
Re: Using a RAMdisk
Posted: Wed Apr 22, 2015 12:02 pm
by BanditBBS
Willem,
Not sure if you use nagiosmobile or not, but Nagios never added this to their ramdisk instructions yet that I can see:
Modify the status file and object cache file variables in /usr/local/nagiosmobile/include.inc.php to match the /var/nagiosramdisk location.
Re: Using a RAMdisk
Posted: Wed Apr 22, 2015 2:39 pm
by WillemDH
Hey Bandit,
I'm not (yet) using Nagios Mobile. Thanks for the tip though! Could someone from Nagios support answer this question:
Some questions: Is it possible to rollback the RAMdisk setup? I noted all original configurations, so if I reconfigured everything like it was, it should just start working again without RAMdisk?
Grtz
Willem