Errors on ceph plugin

[harinarne]

from the cli of the remote working fine but from web interface showing errors
cli output::
->./check_ceph_health --id nagios --keyring /etc/ceph/ceph.client.nagios.keyring
HEALTH OK

[[email protected]]:288:/usr/local/nagios/libexec
{Thu Feb 28 18:25:52 MST 2019}
->./check_ceph_df -i nagios -k /etc/ceph/ceph.client.nagios.keyring -W 29.12 -C 30.22 -d
RAW usage 0.06% | Usage=0.06%;29.12;30.22;;

[[email protected]]:289:/usr/local/nagios/libexec
{Thu Feb 28 18:25:52 MST 2019}
->./check_ceph_mgr
MGR OK: active: node03, standbys: node02, node06, node07, node05, node08, node04, node01

from the web interface::
NRPE: Unable to read output

[[email protected] ~]$ /usr/local/nagios/libexec/check_nrpe -H 10.189.131.52 -t 30 -c check_ceph_mgr -a '-i nagios -k /etc/ceph/ceph.client.nagios.keyring'
MGR ERROR: 2019-02-28 18:45:02.122964 7f639d090700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.nagios.keyring: (13) Permission denied

2019-02-28 18:45:02.122981 7f639d090700 0 librados: client.nagios initialization error (13) Permission denied
[errno 13] error connecting to the cluster
checking from NagiosXI server below is the ouuput:
/usr/local/nagios/libexec/check_nrpe -H 10.189.131.37 -t 30 -c check_ceph_health -a '-i nagios -k /etc/ceph/ceph.client.nagios.keyring'
NRPE: Unable to read output

kindly suggest us

[cdienger]

The user permissions the npre agent runs with don't appear to have access to /etc/ceph/ceph.client.nagios.keyring. What do the permissions of /etc/ceph/ceph.client.nagios.keyring look like? Try giving ownership of this file to the nagios user or set permissions on the file so the nagios user can access it.

[harinarne]

below is keyring permissions
->ls -la
total 28
drwxr-xr-x. 2 ceph ceph 104 Feb 28 19:54 .
drwxr-xr-x. 98 root root 8192 Mar 1 10:48 ..
-rw-------. 1 ceph ceph 63 Feb 26 14:32 ceph.client.admin.keyring
-rw-------. 1 ceph ceph 64 Feb 28 12:43 ceph.client.nagios.keyring
-rw-r--r--. 1 ceph ceph 1178 Feb 28 19:54 ceph.conf
-rw-r--r--. 1 root root 92 Jan 30 09:48 rbdmap

[cdienger]

You can try adding nagios to the ceph group:

usermod -G ceph nagios

and give the ceph group read permissions:

chmod 640 /etc/ceph/ceph.client.nagios.keyring

[harinarne]

we change the permissions and looks like below.


->ls -al
total 28
drwxr-xr-x. 2 ceph ceph 104 Feb 28 19:54 .
drwxr-xr-x. 98 root root 8192 Mar 1 10:48 ..
-rw-------. 1 ceph ceph 63 Feb 26 14:32 ceph.client.admin.keyring
-rw-------. 1 nagios nagios 64 Feb 28 12:43 ceph.client.nagios.keyring
-rw-r--r--. 1 ceph ceph 1178 Feb 28 19:54 ceph.conf
-rw-r--r--. 1 root root 92 Jan 30 09:48 rbdmap
and now output is getting socket timeout
->/usr/local/nagios/libexec/check_nrpe -H 10.189.131.52 -t 30 -c check_ceph_health -a '-i nagios -k /etc/ceph/ceph.client.nagios.keyring'
CHECK_NRPE STATE CRITICAL: Socket timeout after 30 seconds.

[cdienger]

Do you see any messages logged to /var/log/messages when this fails? Do other checks(check_users or something else defined in nrpe.cfg or common.cfg) work through check_nrpe or do they all fail with the same message? Do you get permission error message again if you switch back the ownership?

[harinarne]

The other commands are working fine with check_nrpe other than this below is the logs observed from the host machine
after chaning permissons
>ls -al
total 28
drwxr-xr-x. 2 ceph ceph 104 Feb 28 19:54 .
drwxr-xr-x. 98 root root 8192 Mar 1 10:48 ..
-rw-------. 1 ceph ceph 63 Feb 26 14:32 ceph.client.admin.keyring
-rw-------. 1 nagios nagios 64 Feb 28 12:43 ceph.client.nagios.keyring
-rw-r--r--. 1 ceph ceph 1178 Feb 28 19:54 ceph.conf
-rw-r--r--. 1 root root 92 Jan 30 09:48 rbdmap

logs::
Mar 1 15:24:14 node01 nrpe[278710]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:24:14 node01 nrpe[278711]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:24:21 node01 nrpe[278719]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:24:21 node01 nrpe[278720]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:24:34 node01 nrpe[278732]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:24:34 node01 nrpe[278733]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:24:51 node01 nrpe[278780]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:24:51 node01 nrpe[278781]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:25:00 node01 nrpe[278790]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:25:00 node01 nrpe[278791]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:25:01 node01 nrpe[278795]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:25:01 node01 nrpe[278796]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:25:16 node01 nrpe[278828]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:25:16 node01 nrpe[278829]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:25:41 node01 nrpe[278851]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:25:41 node01 nrpe[278852]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:25:42 node01 nrpe[278855]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:25:42 node01 nrpe[278856]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:25:49 node01 nrpe[278864]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:25:49 node01 nrpe[278865]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:26:03 node01 nrpe[278918]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:26:03 node01 nrpe[278919]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:26:10 node01 nrpe[278926]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:26:10 node01 nrpe[278927]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:26:11 node01 nrpe[278930]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:26:11 node01 nrpe[278931]: WARNING: my_system() seteuid(0): Operation not permitted
Mar 1 15:26:13 node01 nrpe[278934]: INFO: SSL/TLS initialized. All network traffic will be encrypted.
Mar 1 15:26:13 node01 nrpe[278935]: WARNING: my_system() seteuid(0): Operation not permitted

[cdienger]

As a test, change the ownership back to ceph:

chown ceph:ceph ceph.client.nagios.keyring

and set the permissions to read for everyone:

chmod 644 ceph.client.nagios.keyring

[harinarne]

you can close this

[cdienger]

Thanks for the follow up!