ncpa agent reporting wrong info

[Raghupradeep]

Hi,

I have installed ncpa-1.7.2 as monitoring agent on my linux server. After installation I came to notice ncpa is reporting wrong info for cpu usage, N/W in and out traffic and small variation in the amount of disk space used. The variation for CPU and N/W is huge. My server is never over 30% but ncpa is reporting it is always 100% utilized. Similar the case with N/W interface in and out traffic. Its reporting 510 Mbps for my server. This is a test server and the traffic to this is almost nil.

I am new to Nagios and NCPA, can you please help me on where to look for this issue and how to fix it.

Thanks & Regards
Raghupradeep

[sreinhardt]

Could you provide the following disk usage metrics from that server and an example of the ncpa output:

Code: Select all

df -h
df

Could you also provide examples of the cup and memory checks you are currently running against it? What OS is ncpa installed on currently?

[Raghupradeep]

The OS is RHEL 6 64bit.

Output of df -h
=============

Code: Select all

Filesystem                    Size  Used Avail Use% Mounted on
/dev/mapper/vg00-root_lvol    7.9G  3.8G  3.8G  51% /
tmpfs                        1003M     0 1003M   0% /dev/shm
/dev/sda1                     485M   88M  373M  19% /boot
/dev/mapper/vg00-home_lvol    2.0G  573M  1.4G  30% /home
/dev/mapper/vg00-opt_lv       4.0G  222M  3.6G   6% /opt
/dev/mapper/vg00-tmp_lvol     2.0G  290M  1.6G  16% /tmp
/dev/mapper/vg00-var_lv       2.0G  1.1G  847M  56% /var
/dev/mapper/vg00-dump_lv      2.0G   67M  1.9G   4% /var/crash
/dev/mapper/appvg-lvol01      542G  3.6G  511G   1% /mysql
/dev/mapper/vg00-yumcache_lv  6.0G  995M  4.7G  18% /var/cache/yum

Output of df
==========

Code: Select all

Filesystem                   1K-blocks    Used Available Use% Mounted on
/dev/mapper/vg00-root_lvol     8256952 3957336   3880188  51% /
tmpfs                          1026332       0   1026332   0% /dev/shm
/dev/sda1                       495844   89100    381144  19% /boot
/dev/mapper/vg00-home_lvol     2064208  586352   1373000  30% /home
/dev/mapper/vg00-opt_lv        4128448  227080   3691656   6% /opt
/dev/mapper/vg00-tmp_lvol      2064208  296940   1662412  16% /tmp
/dev/mapper/vg00-var_lv        2064208 1092584    866768  56% /var
/dev/mapper/vg00-dump_lv       2064208   68608   1890744   4% /var/crash
/dev/mapper/appvg-lvol01     567667808 3715856 535122512   1% /mysql
/dev/mapper/vg00-yumcache_lv   6192704 1018344   4859788  18% /var/cache/yum

The check for cpu is

Code: Select all

check_xi_ncpa_agent!-t 'token' -P 5693 -M cpu/percent -w 75 -c 90

The CPU is never utilized above 30 %. But NCPA says CPU is always 100 %. Below is the ncpa api snippet for cpu... got from url

Code: Select all

https://ipaddress:5693/api/cpu?token=nagios

{
  "value": {
    "cpu": {
      "count": [
        [
          1
        ], 
        "c"
      ], 
      "idle": [
        [
          689354.25
        ], 
        "ms"
      ], 
      "percent": [], 
      "system": [
        [
          2600095.93
        ], 
        "ms"
      ], 
      "user": [
        [
          712618.69
        ], 
        "ms"
      ]
    }
  }
}

https://ipaddress:5693/api/cpu/percent?token=nagios
{
  "value": {
    "percent": [
      [
        100.0
      ], 
      "%"
    ]
  }
}

The memory part seams to work fine. Its the interface throughput causing the problem, its reporting that the interface is receiving anf transmitting more than 100 Gbps

The check is

Code: Select all

check_xi_ncpa_agent!-t 'token' -P 5693 -M 'interface/eth0/bytes_recv' -d -u M -w 10 -c 100

Its reporting

Below is the interface api snippet...

Code: Select all

{
  "value": {
    "interface": {
      "lo": {
        "packets_sent": [
          8062595, 
          "c"
        ], 
        "dropin": [
          0, 
          "c"
        ], 
        "bytes_recv": [
          722836613, 
          "b"
        ], 
        "packets_recv": [
          8062595, 
          "c"
        ], 
        "errin": [
          0, 
          "c"
        ], 
        "dropout": [
          0, 
          "c"
        ], 
        "bytes_sent": [
          722836613, 
          "b"
        ], 
        "errout": [
          0, 
          "c"
        ]
      }, 
      "eth0": {
        "packets_sent": [
          19688953, 
          "c"
        ], 
        "dropin": [
          0, 
          "c"
        ], 
        "bytes_recv": [
          9359073753, 
          "b"
        ], 
        "packets_recv": [
          56761483, 
          "c"
        ], 
        "errin": [
          0, 
          "c"
        ], 
        "dropout": [
          0, 
          "c"
        ], 
        "bytes_sent": [
          95932164579, 
          "b"
        ], 
        "errout": [
          0, 
          "c"
        ]
      }
    }
  }
}

I am trying ncpa and its the basic thing that come as default monitoring embedded with ncpa (not an external plugin). I am not sure if I am missing some configuration.

[lmiltchev]

Let's try to deal with the cpu issue first. Run the following commands and show us the output:

On the client:

Code: Select all

top | head

On the nagios box:

Code: Select all

./check_ncpa.py -H <client ip> -t 'token' -P 5693 -M cpu/percent -w 75  -c 90

Also, post the "ncpa_listener.log" in code wraps.

[Raghupradeep]

The cpu issue is fixed. Once I bounced the server.

But the interface issue still remains. As per the ncpa api interface monitoring (https://IPADDRESS:5693/api/interface/eth0/) below is the output that I get ...

{
"value": {
"eth0": {
"packets_sent": [
572045,
"c"
],
"dropin": [
0,
"c"
],
"bytes_recv": [
158421773,
"b"
],
"packets_recv": [
1409126,
"c"
],
"errin": [
0,
"c"
],
"dropout": [
0,
"c"
],
"bytes_sent": [
479135352,
"b"
],
"errout": [
0,
"c"
]
}
}
}

Seams ncpa is checking the total bytes_sent and bytes_recv values which are incremental values as per my understanding. So how can we put a monitor on these counters ? I checked the server with other n/w utilities like nload.

I tried nload -m and this is the output that I got ...

Device eth0 [IPADDRESS] (1/2):
================================================================================================================================================
Incoming: Outgoing:
Curr: 12.72 kBit/s Curr: 22.62 kBit/s
Avg: 11.66 kBit/s Avg: 20.81 kBit/s
Min: 3.81 kBit/s Min: 3.78 kBit/s
Max: 42.84 kBit/s Max: 49.99 kBit/s
Ttl: 151.61 MByte Ttl: 457.31 MByte

Device lo [127.0.0.1] (2/2):
================================================================================================================================================
Incoming: Outgoing:
Curr: 3.69 kBit/s Curr: 3.69 kBit/s
Avg: 2.32 kBit/s Avg: 2.32 kBit/s
Min: 0.00 Bit/s Min: 0.00 Bit/s
Max: 20.71 kBit/s Max: 20.71 kBit/s
Ttl: 23.25 MByte Ttl: 23.25 MByte

So, seams ncpa interface monitoring is taking the total in and out bytes value (an incremental value) rather than using the Current rate. Is there any way to ask ncpa to check on current rate rather than total ? In the API I cant see and option to do so ?

[lgroschen]

I'm in the process of replicating this on my system, but looks like you might be right. If that's the case I'll open up an issue on the ncpa github page so the devs can add it to the next version.

EDIT: Added a bug report to the ncpa page. This may or may not be a bug, but you'll get your answer from here when the dev responds:
https://github.com/NagiosEnterprises/ncpa/issues/104

[lgroschen]

https://github.com/NagiosEnterprises/ncpa/issues/104

The lead developer of ncpa gave an answer to this question in response to my bug report and it does indeed exist already. Please read the above. Locking this post.


PM or post a new topic if you have further questions