Page 2 of 2

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 9:38 am
by mike-d-howell
Hello Vinh,

Trying:

Code: Select all

curl -k -v 'https://IP.OF.MY.SERVER:5693/api/?token=Yes-I-put-in-the-real-token'
RESULTS:

Code: Select all

# curl -k -v 'https://drupshop-db-replica-01.osufpp.org:5693/api/?token=fakestring'
*   Trying 172.16.1.22:5693...
* TCP_NODELAY set
* Connected to drupshop-db-replica-01.osufpp.org (172.16.1.22) port 5693 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=US; ST=Minnesota; L=St. Paul; O=Nagios Enterprises, LLC; OU=Development; CN=drupshop-db
*  start date: Feb 19 18:48:08 2020 GMT
*  expire date: Feb 16 18:48:08 2030 GMT
*  issuer: C=US; ST=Minnesota; L=St. Paul; O=Nagios Enterprises, LLC; OU=Development; CN=drupshop-db
*  SSL certificate verify result: self signed certificate (18), continuing anyway.
> GET /api/?token=fakestring HTTP/1.1
> Host: drupshop-db-replica-01.osufpp.org:5693
> User-Agent: curl/7.66.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 15652
< Access-Control-Allow-Origin: *
< X-Frame-Options: SAMEORIGIN
< Content-Security-Policy: frame-ancestors 'self'
< Date: Wed, 04 Aug 2021 14:17:27 GMT
<
{
    "root": {
        "services": [],
        "processes": [],
        "disk": {
            "mount": {},
            "logical": {
                "|opt|PostgreSQL|10": {
                    "used_percent": [
                        15.8,
                        "%"
                    ],
                    "used": [
                        8.8,
                        "GiB"
                    ],
                    "inodes_used": [
                        10995,
                        "inodes"
                    ],
                    "free": [
                        46.98,
                        "GiB"
                    ],
                    "device_name": [
                        "/dev/mapper/data-vol1"
                    ],
                    "inodes_free": [
                        3921165,
                        "inodes"
                    ],
                    "inodes": [
                        3932160,
                        "inodes"
                    ],
                    "fstype": "ext4",
                    "total": [
                        58.79,
                        "GiB"
                    ],
                    "opts": "rw,relatime",
                    "inodes_used_percent": [
                        1.0,
                        "%"
                    ]
                },
                "|sys|fs|cgroup|unified": {
                    "used_percent": [
                        0.0,
                        "%"
                    ],
                    "used": [
                        0.0,
                        "GiB"
                    ],
                    "inodes_used": [
                        0,
                        "inodes"
                    ],
                    "free": [
                        0.0,
                        "GiB"
                    ],
                    "device_name": [
                        "cgroup2"
                    ],
                    "inodes_free": [
                        0,
                        "inodes"
                    ],
                    "inodes": [
                        0,
                        "inodes"
                    ],
                    "fstype": "cgroup2",
                    "total": [
                        0.0,
                        "GiB"
                    ],
                    "opts": "rw,nosuid,nodev,noexec,relatime,nsdelegate",
                    "inodes_used_percent": [
                        0,
                        "%"
                    ]
                },
                "|": {
                    "used_percent": [
                        15.0,
                        "%"
                    ],
                    "used": [
                        2.5,
                        "GiB"
                    ],
                    "inodes_used": [
                        49535,
                        "inodes"
                    ],
                    "free": [
                        14.16,
                        "GiB"
                    ],
                    "device_name": [
                        "/dev/mapper/system-root"
                    ],
                    "inodes_free": [
                        1130113,
                        "inodes"
                    ],
                    "inodes": [
                        1179648,
                        "inodes"
                    ],
                    "fstype": "ext4",
                    "total": [
                        17.58,
                        "GiB"
                    ],
                    "opts": "rw,relatime",
                    "inodes_used_percent": [
                        5.0,
                        "%"
                    ]
                },
                "|sys|fs|bpf": {
                    "used_percent": [
                        0.0,
                        "%"
                    ],
                    "used": [
                        0.0,
                        "GiB"
                    ],
                    "inodes_used": [
                        0,
                        "inodes"
                    ],
                    "free": [
                        0.0,
                        "GiB"
                    ],
                    "device_name": [
                        ""
                    ],
                    "inodes_free": [
                        0,
                        "inodes"
                    ],
                    "inodes": [
                        0,
                        "inodes"
                    ],
                    "fstype": "bpf",
                    "total": [
                        0.0,
                        "GiB"
                    ],
                    "opts": "rw,nosuid,nodev,noexec,relatime,mode=700",
                    "inodes_used_percent": [
                        0,
                        "%"
                    ]
                }
            },
            "physical": {
                "sr0": {
                    "read_time": [
                        8,
                        "ms"
                    ],
                    "read_bytes": [
                        0.0,
                        "GiB"
                    ],
                    "write_count": [
                        0,
                        "c"
                    ],
                    "write_time": [
                        0,
                        "ms"
                    ],
                    "write_bytes": [
                        0.0,
                        "GiB"
                    ],
                    "read_count": [
                        9,
                        "c"
                    ]
                },
                "sda2": {
                    "read_time": [
                        20521,
                        "ms"
                    ],
                    "read_bytes": [
                        0.26,
                        "GiB"
                    ],
                    "write_count": [
                        653077,
                        "c"
                    ],
                    "write_time": [
                        194603,
                        "ms"
                    ],
                    "write_bytes": [
                        4.79,
                        "GiB"
                    ],
                    "read_count": [
                        7804,
                        "c"
                    ]
                },
                "sda": {
                    "read_time": [
                        20620,
                        "ms"
                    ],
                    "read_bytes": [
                        0.27,
                        "GiB"
                    ],
                    "write_count": [
                        653077,
                        "c"
                    ],
                    "write_time": [
                        194603,
                        "ms"
                    ],
                    "write_bytes": [
                        4.79,
                        "GiB"
                    ],
                    "read_count": [
                        8348,
                        "c"
                    ]
                },
                "sdb": {
                    "read_time": [
                        1205,
                        "ms"
                    ],
                    "read_bytes": [
                        0.03,
                        "GiB"
                    ],
                    "write_count": [
                        783092,
                        "c"
                    ],
                    "write_time": [
                        176178,
                        "ms"
                    ],
                    "write_bytes": [
                        7.56,
                        "GiB"
                    ],
                    "read_count": [
                        383,
                        "c"
                    ]
                },
                "sda1": {
                    "read_time": [
                        21,
                        "ms"
                    ],
                    "read_bytes": [
                        0.01,
                        "GiB"
                    ],
                    "write_count": [
                        0,
                        "c"
                    ],
                    "write_time": [
                        0,
                        "ms"
                    ],
                    "write_bytes": [
                        0.0,
                        "GiB"
                    ],
                    "read_count": [
                        146,
                        "c"
                    ]
                },
                "dm-2": {
                    "read_time": [
                        3296,
                        "ms"
                    ],
                    "read_bytes": [
                        0.02,
                        "GiB"
                    ],
                    "write_count": [
                        1983480,
                        "c"
                    ],
                    "write_time": [
                        391200,
                        "ms"
                    ],
                    "write_bytes": [
                        7.56,
                        "GiB"
                    ],
                    "read_count": [
                        482,
                        "c"
                    ]
                },
                "dm-0": {
                    "read_time": [
                        8,
                        "ms"
                    ],
                    "read_bytes": [
                        0.0,
                        "GiB"
                    ],
                    "write_count": [
                        0,
                        "c"
                    ],
                    "write_time": [
                        0,
                        "ms"
                    ],
                    "write_bytes": [
                        0.0,
                        "GiB"
                    ],
                    "read_count": [
                        98,
                        "c"
                    ]
                },
                "dm-1": {
                    "read_time": [
                        36184,
                        "ms"
                    ],
                    "read_bytes": [
                        0.22,
                        "GiB"
                    ],
                    "write_count": [
                        1196030,
                        "c"
                    ],
                    "write_time": [
                        131216,
                        "ms"
                    ],
                    "write_bytes": [
                        4.79,
                        "GiB"
                    ],
                    "read_count": [
                        10334,
                        "c"
                    ]
                }
            }
        },
        "user": {
            "count": [
                2,
                "users"
            ],
            "list": [
                [
                    "myname",
                    "myname"
                ],
                "users"
            ]
        },
        "memory": {
            "swap": {
                "used": [
                    0.0,
                    "GiB"
                ],
                "swapped_out": [
                    0.0,
                    "GiB"
                ],
                "swapped_in": [
                    0.0,
                    "GiB"
                ],
                "total": [
                    2.0,
                    "GiB"
                ],
                "percent": [
                    0.0,
                    "%"
                ],
                "free": [
                    2.0,
                    "GiB"
                ]
            },
            "virtual": {
                "available": [
                    1.57,
                    "GiB"
                ],
                "total": [
                    1.93,
                    "GiB"
                ],
                "percent": [
                    18.6,
                    "%"
                ],
                "free": [
                    0.94,
                    "GiB"
                ],
                "used": [
                    0.17,
                    "GiB"
                ]
            }
        },
        "interface": {
            "lo": {
                "packets_sent": [
                    71393,
                    "packets"
                ],
                "dropin": [
                    0,
                    "packets"
                ],
                "bytes_recv": [
                    0.01,
                    "GiB"
                ],
                "packets_recv": [
                    71393,
                    "packets"
                ],
                "errin": [
                    0,
                    "errors"
                ],
                "dropout": [
                    0,
                    "packets"
                ],
                "bytes_sent": [
                    0.01,
                    "GiB"
                ],
                "errout": [
                    0,
                    "errors"
                ]
            },
            "eth0": {
                "packets_sent": [
                    3974388,
                    "packets"
                ],
                "dropin": [
                    353,
                    "packets"
                ],
                "bytes_recv": [
                    0.67,
                    "GiB"
                ],
                "packets_recv": [
                    6459235,
                    "packets"
                ],
                "errin": [
                    0,
                    "errors"
                ],
                "dropout": [
                    0,
                    "packets"
                ],
                "bytes_sent": [
                    0.33,
                    "GiB"
                ],
                "errout": [
                    0,
                    "errors"
                ]
            }
        },
        "plugins": [],
        "system": {
            "node": "drupshop-db-rep",
            "machine": "x86_64",
            "uptime": [
                1071536.0,
                "s"
            ],
            "version": "#1 SMP Thu Jul 15 11:28:57 UTC 2021 (0b62bdb)",
            "time": 1628086647.792325,
            "release": "5.3.18-59.16-default",
            "timezone": [
                "CST",
                "CDT"
            ],
            "agent_version": "2.3.1",
            "system": "Linux",
            "processor": "x86_64"
        },
        "cpu": {
            "count": [
                [
                    2
                ],
                "cores"
            ],
            "idle": [
                [
                    1065426.67,
                    1067242.29
                ],
                "ms"
            ],
            "percent": [],
            "system": [
                [
                    802.28,
                    833.42
                ],
                "ms"
            ],
            "user": [
                [
                    2644.82,
                    2070.5
                ],
                "ms"
            ]
        }
    }
* Connection #0 to host drupshop-db-replica-01.osufpp.org left intact
I notice the Services value is empty.

I cannot downgrade to 2.2.0. Please note earlier I said when I upgraded the server from SP2 to SP3 (running NCPA 2.2.0) I got an error--NCPA stopped returning data at all. That triggered me to try upgrading the NCPA agent to 2.3.1 in the first place.

This is going to be a serious problem for our datacenter. We purchased and NagiosXI and rely on the NCPA agent to monitor dozens of services. As we upgrade our fleet to SP3, we will be unable to monitor those services.

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 10:31 am
by vtrac
Hi Mike,
Great!! ..... That last "curl" command works!! ... :-)
This command works:

Code: Select all

curl -k -v 'https://IP.OF.MY.SERVER:5693/api/?token=Yes-I-put-in-the-real-token'
Now, can you please try same command with "services" added?

Code: Select all

curl -k -v 'https://IP.OF.MY.SERVER:5693/api/services/?token=Yes-I-put-in-the-real-token'
Please update result?

Also, what user are you using to run the above command?

Can you please try both "root" and "nagios"?

Let me explain about the "curl" command we used:
If we use just "/api/", the it will list all CPU, disk and memory and many more but not running "services".
If we want to see running services, we must add the "services" to the command like "/api/services" for it to just list only services.


Best Regards,
Vinh

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 11:09 am
by vtrac
Hi Mike,
A senior member of my team has determined that NCPA v2.3.1 did not work with SLES 15 SP3 when checking for "services". Everything else works but "services"

He has suggested the following plugin as a work around:
https://exchange.nagios.org/directory/P ... ce/details

Could also use the check_init_service plugin but would need utils.sh as well to make it work, I tested the one above and works

Here's the test:
[nagios@xid ~]$ /usr/local/nagios/libexec/check_ncpa.py -H 192.168.232.238 -t nagiosprivate -p 5693 -M plugins/check_service -a '-s cron'
OK: Service cron is running! | 'status'=0;1;2;;
I'm very sorry for any inconvenience issue this has caused.


Best Regards,
Vinh

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 1:44 pm
by mike-d-howell
Hi Vinh,

Sorry to be a noob, but I'm having trouble with the ncpa_listener command your team member suggested. I used the link to install the check_service plugin to my NagiosXI installation.

Then I changed the service check in Nagios to the string listed, changing the values for -H and -t as necessary. I left all other values as-is.

Code: Select all

/usr/local/nagios/libexec/check_ncpa.py -H drupshop-db-replica-01.osufpp.org -t my-token -p 5693 -M plugins/check_service -a '-s cron'
OK: Service cron is running! | 'status'=0;1;2;;
After updating and forcing a refresh to the service monitor, I get the following:

Code: Select all

Critical
Usage: check_ncpa.py [options]

Options:
-h, --help show this help message and exit
-H HOSTNAME, --hostname=HOSTNAME
The hostname to be connected to.
-M METRIC, --metric=METRIC
The metric to check, this is defined on client system.
This would also be the plugin name in the plugins
directory. Do not attach arguments to it, use the -a
directive for that. DO NOT INCLUDE the api/
instruction.
-P PORT, --port=PORT Port to use to connect to the client.
-w WARNING, --warning=WARNING
Warning value to be passed for the check.
-c CRITICAL, --critical=CRITICAL
Critical value to be passed for the check.
-u UNITS, --units=UNITS
The unit prefix (k, Ki, M, Mi, G, Gi, T, Ti) for b and
B unit types which calculates the value returned.
-n UNIT, --unit=UNIT Overrides the unit with whatever unit you define. Does
not perform calculations. This changes the unit of
measurement only.
-a ARGUMENTS, --arguments=ARGUMENTS
Arguments for the plugin to be run. Not necessary
unless you're running a custom plugin. Given in the
same as you would call from the command line. Example:
-a '-w 10 -c 20 -f /usr/local'
-t TOKEN, --token=TOKEN
The token for connecting.
-T TIMEOUT, --timeout=TIMEOUT
Enforced timeout, will terminate plugins after this
amount of seconds. [60]
-d, --delta Signals that this check is a delta check and a local
state will kept.
-l, --list List all values under a given node. Do not perform a
check.
-v, --verbose Print more verbose error messages.
-D, --debug Print LOTS of error messages. Used mostly for
debugging.
-V, --version Print version number of plugin.
-q QUERYARGS, --queryargs=QUERYARGS
Extra query arguments to pass in the NCPA URL.
-s, --secure Require successful certificate verification. Does not
work on Python < 2.7.9.
-p, --performance Print performance data even when there is none. Will
print data matching the return code of this script
Am I not doing this right?

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 2:22 pm
by vtrac
Hi Mike,
You download the "check_service.tar.gz" file from:
https://exchange.nagios.org/directory/P ... ce/details

Bring this "check_service.tar.gz" file over to your NCPA remote agent under "/tmp"

Code: Select all

cd /tmp
tar -xzf check_service.tar.gz
cp check_service /usr/local/ncpa/plugins/
chmod 755 /usr/local/ncpa/plugins/check_service
Now, on your Nagios XI command prompt, run the below command to check for a service call "cron":

Code: Select all

/usr/local/nagios/libexec/check_ncpa.py -H drupshop-db-replica-01.osufpp.org -t my-token -p 5693 -M plugins/check_service -a '-s cron'
Basically, you use "check_ncpa.py" to run a module under the "plugins" folder, .... "plugins/check_service"
The "-M" is to run the modules under the "plugins" folder on the NCPA remote agent.


Best Regards,
Vinh

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 3:01 pm
by mike-d-howell
Hi Vinh,

I believe I got it to work. I had to go thru the Configuration Wizard to add a new NCPA check for the server. I checked the test for plugins, picked "check_service" in the drop down menu and put in "postgresql-10" as the value.

After I updated and forced a refresh on the service, it now appears to be finding the postgresql service. On the Configure Service screen, the "Monitor the Service with this command" shows:

Code: Select all

check_xi_ncpa!-t 'my-token' -P 5693  -M 'plugins/check_service' -q 'args=-s postgresql-10'
This seems like a useful work around--I will keep it handy as we go forward and I'll do a little more testing on my other SP3 boxen. Still, I hope the developers can correct the original bug on the NCPA listener service. It'd be much easier to update a single package than revamp the tests for all the boxes I manage. Thanks!

Re: No services for ncpa_listener 2.2.1 thru 2.3.1 on SLES15

Posted: Wed Aug 04, 2021 4:04 pm
by vtrac
Great!! ... very happy that you got the service check to work (as a work around) .... :-)

Again, I'm very sorry for all the troubles this has caused.

Locking thread!! ... :-)


Vinh