Using a RAMdisk

[WillemDH]

Hello,

Today I started with configuring a RAMdisk on my XI production server. I followed all steps in http://assets.nagios.com/downloads/nagi ... giosXI.pdf except the one to move the perfdata spool folder. I'm not yet sure however if the RAMdisk helps to improve performance.

Please check the attached graphs. As you can see avg cpu wait was 1.6%. After moving to a RAMdisk (from 13:00), cpu wait is avg 5.3 %. I hope the situation will calm down a bit, but I would have expected a small improvement after 60 minutes at least. Did the same procedure on my QA, but the load on that server is really too low to notice any difference.

Thoughts?

Grtz

Willem

[tmcdonald]

Using a ramdisk is more for reducing disk read/write times than for reducing CPU usage.

[WillemDH]

iowait is time that the processor/processors are waiting (i.e. is in an idle state and does nothing), during which there in fact was outstanding disk I/O requests.

http://veithen.github.io/2013/11/18/iowait-linux.html

So wouldn't making use of a ramdisk decrease cpu wait? I'll see if I can get some disk io graphs tomorrow.

[abrist]

I think the most important bit from the at article:

For a given CPU, the I/O wait time is the time during which that CPU was idle (i.e. didn’t execute any tasks) and there was at least one outstanding disk I/O operation requested by a task scheduled on that CPU (at the time it generated that I/O request).

So I think both of you are right:

Using a ramdisk is more for reducing disk read/write times than for reducing CPU usage.

True, which will in turn reduce CPU wait:

So wouldn't making use of a ramdisk decrease cpu wait?

Exactly. It is important to remember that CPU wait is a form of CPU idle, and IO Wait is a form of CPU wait. You are technically *not* reducing CPU usage by using a ramdisk, you are instead reducing the time that CPUs idle when there are still tasks to complete.

[WillemDH]

Ok, I completely followed the RAMdisk procedure and it seems my graphs stopped working after changing the commands in the end of the procedure and applying configuration. I tried restarting the NPCD service, but my graphs are still empty starting from the moment I applied configuration.

I took a snapshot and will revert to the snapshot, but what could be the reason my graphs stopped working? What else can I do to troubleshoot this issue?

It seems some files are being created in the perfdata folder on the ramdisk, but I'm not sure if this looks like it should be:

Code: Select all

ls -la /var/nagiosramdisk/spool/perfdata/
total 1300
drwxrwxr-x 2 nagios nagios    220 Apr 22 10:47 .
drwxrwxr-x 5 nagios nagios    100 Apr 22 10:17 ..
-rw-r--r-- 1 nagios nagios 172335 Apr 22 10:46 1429692361.perfdata.service-PID-53690
-rw-r--r-- 1 nagios nagios 306145 Apr 22 10:46 1429692377.perfdata.service-PID-53692
-rw-r--r-- 1 nagios nagios 188663 Apr 22 10:46 1429692391.perfdata.service
-rw-r--r-- 1 nagios nagios   2336 Apr 22 10:46 1429692406.perfdata.host
-rw-r--r-- 1 nagios nagios 197109 Apr 22 10:46 1429692406.perfdata.service
-rw-r--r-- 1 nagios nagios   1154 Apr 22 10:46 1429692421.perfdata.host
-rw-r--r-- 1 nagios nagios 277195 Apr 22 10:47 1429692421.perfdata.service
-rw-r--r-- 1 nagios nagios    931 Apr 22 10:47 1429692437.perfdata.host
-rw-r--r-- 1 nagios nagios 162994 Apr 22 10:47 1429692437.perfdata.service

Code: Select all

ls -la /var/nagiosramdisk/spool/xidpe/
total 360
drwxrwxr-x 2 nagios nagios     80 Apr 22 10:48 .
drwxrwxr-x 5 nagios nagios    100 Apr 22 10:17 ..
-rw-r--r-- 1 nagios nagios  18584 Apr 22 10:48 1429692511.perfdata.host
-rw-r--r-- 1 nagios nagios 348118 Apr 22 10:48 1429692511.perfdata.service

Code: Select all

 ls -la /var/nagiosramdisk/
total 29220
drwxrwxrwt   4 nagios nagios      160 Apr 22 10:51 .
drwxr-xr-x. 23 root   root       4096 Apr 13 13:04 ..
-rw-r--r--   1 nagios nagios        0 Apr 22 10:51 host-perfdata
-rw-r--r--   1 nagios nagios 12201097 Apr 22 10:29 objects.cache
-rw-r--r--   1 nagios nagios    16010 Apr 22 10:51 service-perfdata
drwxrwxr-x   5 nagios nagios      100 Apr 22 10:17 spool
-rw-r--r--   1 nagios nagios 17695236 Apr 22 10:51 status.dat
drwxrwxr-x   2 nagios nagios       40 Apr 22 10:29 tmp

Please advise how to get this working. It's only the part "Moving Performance Data to the RAM Disk" that is not working.

Grtz

Willem

[WillemDH]

Ah, right before I was considering restoring a snapshot, I decided to give the server one last reboot (plus apply config, as my hosts are still greyed out untill nagios service is restarted) it seems the graphs started working again. So this is a good thing I guess. I do not see an immediate performance improvement however, but it's probably a bit early to tell. it might be a good idea to suggest a reboot in the RAMdisk documentation? i'll add some graphs to this thread later.

Some questions: Is it possible to rollback the RAMdisk setup? I noted all original configurations, so if I reconfigured everything like it was, it should just start working again without RAMdisk?

[WillemDH]

Ok, another update on this. It seems our Nagios XI server is very sensitive to storage migrations of other hosts, even when they are running on different datastores and different ESXi server. I will try to make a post about cpuwait on my blog and explain the process I've gone through this afternoon. In short, it seems a colleague of me was working from home and updating the frmware of some ESXi server, migrating all the vm guests. Even when they were running on another datastore and another ESXi server, the impact on the Nagios XI server was noticed resulting in triple cpu wait and cpu load.
I hope the following days will show if using a RAMdisk did improve something or not.

Grtz

Willem

[lmiltchev]

I hope the following days will show if using a RAMdisk did improve something or not.

Keep us posted, Willem. It would be nice to have some feedback/stats on the performance improvements.

[BanditBBS]

Willem,

Not sure if you use nagiosmobile or not, but Nagios never added this to their ramdisk instructions yet that I can see:

Modify the status file and object cache file variables in /usr/local/nagiosmobile/include.inc.php to match the /var/nagiosramdisk location.

[WillemDH]

Hey Bandit,

I'm not (yet) using Nagios Mobile. Thanks for the tip though! Could someone from Nagios support answer this question:

Some questions: Is it possible to rollback the RAMdisk setup? I noted all original configurations, so if I reconfigured everything like it was, it should just start working again without RAMdisk?

Grtz

Willem