EC2 100% CPU usage for Nagios XI

[mengerink]

I have deployed an instance per the following instructions:

http://assets.nagios.com/downloads/nagi ... -Cloud.pdf

I have configured Nagios to monitor my mysql instance as well as a website. When I click on the monitoring tab of the instance running NagiosXI, it shows that the CPU has been at 100% nearly the entire time the service has been up.

Has anyone else suffered this? Does anyone know what to do to fix this so that the CPU isn't getting chewed up so much by Nagios.

[tmcdonald]

Can you run the "top" command on the CLI and post the results?

[mengerink]

Here goes. This is showing a 2.6% usage, but when I go to the AWS monitoring tab, it is still saying 100% which you can see in the attached image.

Code: Select all

top - 12:32:29 up 3 days, 22:25,  1 user,  load average: 0.72, 0.75, 0.72
Tasks: 131 total,   1 running, 130 sleeping,   0 stopped,   0 zombie
Cpu(s):  2.6%us,  0.9%sy,  0.0%ni, 96.2%id,  0.3%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:    604940k total,   432352k used,   172588k free,    10120k buffers
Swap:        0k total,        0k used,        0k free,    60880k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
16723 nagios    20   0  182m  13m 3324 S  5.0  2.2   0:00.15 check_mysql_hea
 2164 nagios    20   0 50412 1096  340 S  0.3  0.2   0:42.70 ndo2db
16561 nagios    20   0  217m  22m 7788 S  0.3  3.7   0:00.21 php
16663 ec2-user  20   0 15028 1328 1000 R  0.3  0.2   0:00.05 top
    1 root      20   0 19224  468  180 S  0.0  0.1   0:00.36 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
    4 root      20   0     0    0    0 S  0.0  0.0   0:00.24 ksoftirqd/0
    5 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 migration/0
    6 root      RT   0     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    7 root      20   0     0    0    0 S  0.0  0.0   0:00.17 events/0
    8 root      20   0     0    0    0 S  0.0  0.0   0:00.00 cpuset
    9 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khelper
   10 root      20   0     0    0    0 S  0.0  0.0   0:00.00 netns
   11 root      20   0     0    0    0 S  0.0  0.0   0:00.00 async/mgr
   12 root      20   0     0    0    0 S  0.0  0.0   0:00.00 pm
   13 root      20   0     0    0    0 S  0.0  0.0   0:00.00 xenwatch
   14 root      20   0     0    0    0 S  0.0  0.0   0:00.00 xenbus
   15 root      20   0     0    0    0 S  0.0  0.0   0:00.00 sync_supers
   16 root      20   0     0    0    0 S  0.0  0.0   0:00.00 bdi-default
   17 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kintegrityd/0
   18 root      20   0     0    0    0 S  0.0  0.0   0:03.88 kblockd/0
   19 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata/0
   20 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ata_aux
   21 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ksuspend_usbd  
   22 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khubd
   23 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kseriod
   24 root      20   0     0    0    0 S  0.0  0.0   0:00.00 md/0
   25 root      20   0     0    0    0 S  0.0  0.0   0:00.00 md_misc/0
   26 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khungtaskd
   27 root      20   0     0    0    0 S  0.0  0.0   0:08.57 kswapd0
   28 root      25   5     0    0    0 S  0.0  0.0   0:00.00 ksmd
   29 root      20   0     0    0    0 S  0.0  0.0   0:00.00 aio/0
   30 root      20   0     0    0    0 S  0.0  0.0   0:00.00 crypto/0
   35 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthrotld/0
   37 root      20   0     0    0    0 S  0.0  0.0   0:00.00 khvcd
   38 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kpsmoused
   39 root      20   0     0    0    0 S  0.0  0.0   0:00.00 usbhid_resumer
  147 root      20   0     0    0    0 S  0.0  0.0   0:17.48 jbd2/xvde1-8
  148 root      20   0     0    0    0 S  0.0  0.0   0:00.00 ext4-dio-unwrit
  213 root      16  -4 10780  516   52 S  0.0  0.1   0:00.03 udevd
  404 postgres  20   0  212m 5732 3532 S  0.0  0.9   0:01.32 postmaster  
  412 root      18  -2 10776  516   52 S  0.0  0.1   0:00.00 udevd
  415 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kstriped                

========================

[sreinhardt]

Your top looks pretty darn normal, but I agree the ec2 screen is a bit odd. If you wouldn't mind, let's go ahead and run the nagiostats and nagios xi server wizards against localhost. This should allow us to collect some additional system info on a continual basis and see if the ec2 metrics match up. Just a note, you don't need to set notifications on any of this, it's purely for statistics unless you want to be notified for these.

[mengerink]

Ok, I've run for a day.

Here's the current info:

Last Update: Tue Jul 15 2014 10:38:38 GMT-0700 (PDT)
Nagios BPI v2.31
Nagios Enterprises, LLC
BPI Documentation
Manually Edit Config

Essential group members are denoted with: **
Ok Local Services URL Group health is 100.00% with 0 problem(s) Example BPI Group Edit Delete
Ok localhost Current Load
OK - load average: 0.67, 0.61, 0.57
Ok localhost Current Users USERS OK - 0 users currently logged in
Ok localhost HTTP HTTP OK: HTTP/1.1 200 OK - 3193 bytes in 0.002 second response time
Ok** localhost PING PING OK - Packet loss = 0%, RTA = 0.07 ms

[Box293]

When I click on the monitoring tab of the instance running NagiosXI, it shows that the CPU has been at 100% nearly the entire time the service has been up

Are you able to view a graph that shows actual CPU usage as MHz instead of a percentage?

Just trying to look at things from a different angle.

[mengerink]

No, I do not seem to be able to change the units on the graph.

[tmcdonald]

I believe these two articles are of interest:

http://www.axibase.com/cloud/2010/07/22 ... tolen-cpu/
http://stackoverflow.com/questions/1170 ... -going-off

Essentially, top might be lying because the VM "steals" CPU time in order to run other things. The guest doesn't notice but the host does.