Issue with check_log over nrpe for Linux remote host

[Hikerguy86]

I have seen this topic posted about before, but I believe the permissions are correct in this case.

Problem: recieving "Log check error: Log file /var/log/secure is not readable!" over nrpe when executing check_log binary (named diff_log in nrpe.cfg on remote host for the command and pointing to /usr/lib64/nagios/plugins/check_log)

Code: Select all

[root@nagiosserver nrpe-3.0.1]# /usr/lib64/nagios/plugins/check_nrpe -H <ip> -c diff_log -a /var/log/secure /home/nagios/authFailures authentication failures
Log check error: Log file /var/log/secure is not readable!

however, this is not the case when the plugin is executed locally:

Code: Select all

[nagios@remotehost~]$ /usr/lib64/nagios/plugins/check_log -F /var/log/secure -O /home/nagios/authFailures -q "authentication failure"
Log check ok - 0 pattern matches found

Note, the nagios user on the cli locally on the remote host that nagios is to monitor was used to execute this check.

permissions on check_log:

Code: Select all

ls -l /usr/lib64/nagios/plugin/check_log
-rwxr-xr-x. 1 nagios nagios   5989 Sep 11  2015 check_log

mrpe.cfg snippet:

Code: Select all

log_facility=daemon
pid_file=/var/run/nrpe/nrpe.pid
server_port=5666
nrpe_user=nagios
nrpe_group=nagios
...

file acl settings:

Code: Select all

# file: var/log/secure
# owner: root
# group: root
user::rw-
user:nagios:r--
group::---
group:monitoring:r--
group:nagios:r--
mask::r--
other::---

Anyone have any ideas?

I have also restarted nrpe service a couple times, just for good measure. OS is RHEL 7.2

[avandemore]

To start with, I'd like to compare apples to apples. Why are there so many differences between syntax on local vs remote?

Please show the command definition for diff_log in the remote /usr/local/nagios/etc/nrpe.cfg

[Hikerguy86]

difference is from calling nrpe and paramaterizing the input. You have to call -F -O and -q, so I don't want to put it in each service def

snippet of nrpe.cfg:

Code: Select all

command[diff_log]=/usr/lib64/nagios/plugins/check_log -F $ARG1$ -O $ARG2$ -q $ARG3$

[tgriep]

I found this post that looks like could be a possible solution to the issue you are having.
https://support.nagios.com/forum/viewto ... 29&p=60435
What they did was add sudo to the command on the remote system but look at it for details.

Also, you may need to add double quotes around your query string in the nrpe command as well.

Code: Select all

/usr/lib64/nagios/plugins/check_nrpe -H <ip> -c diff_log -a /var/log/secure /home/nagios/authFailures "authentication failures"

[Hikerguy86]

Hi

yes I did see that thread however, in that case the nagios user was not able to locally run the check_log command without getting the same error. In my original post you can see, my shell is explicitly logged in as nagios the user and it is able to execute the command and return a result.

Code: Select all

[[b]nagios[/b]@remotehost~]$ /usr/lib64/nagios/plugins/check_log -F /var/log/secure -O /home/nagios/authFailures -q "authentication failure"
Log check ok - 0 pattern matches found

[Hikerguy86]

tried this anyway just to see and it failes with "NRPE: Unable to read output"

nrpe.cfg snippet:

Code: Select all

command[diff_log]=sudo /usr/lib64/nagios/plugins/check_log -F $ARG1$ -O $ARG2$ -q $ARG3$

nagios is now in sudoers file

Code: Select all

Defaults:nagios !requiretty
nagios ALL=NOPASSWD: /usr/lib64/nagios/plugins/check_log

from nagios server:

Code: Select all

[root@nagiosserver~]# /usr/lib64/nagios/plugins/check_nrpe -H remotehost_IP -c diff_log -a /var/log/secure /home/nagios/authFailure authentication
NRPE: Unable to read output

executing check_log on the target host locally:

Code: Select all

[nagios@remotehost~]$ /usr/lib64/nagios/plugins/check_log -F /var/log/secure -O /home/nagios/secure -q "authentication failure;"
Log check ok - 0 pattern matches found

Edit:

I put 2>&1 at the end of the command line in the nrpe.cfg file so that errors would be spit out onto stdout. error is:

Code: Select all

[root@nagiosserver~]# /usr/lib64/nagios/plugins/check_nrpe -H remotehost_IP -c diff_log -a /var/log/secure /home/nagios/authFailure authentication
sudo: sorry, you must have a tty to run sudo

when I comment out Defaults requiretty in /etc/sudoers, I get the following error:

Code: Select all

sudo: no tty present and no askpass program specified 

the nagios user has no password.

[tgriep]

That error is usually caused by the "Defaults requiretty" setting in the in the /etc/sudoers file.
Remove it and that should make the plugin work with the NRPE Agent.

[Hikerguy86]

beat you to it , but there is another error

Code: Select all

sudo: no tty present and no askpass program specified 

EDIT:

updated command in nrpe.cfg to:

Code: Select all

command[diff_log]=sudo -n /usr/lib64/nagios/plugins/check_log -F $ARG1$ -O $ARG2$ -q $ARG3$ 2>&1

new error:

Code: Select all

sudo: a password is required 

in /var/log/secure I see:

Code: Select all

Nov 10 09:31:44 remotehost sudo: pam_unix(sudo:auth): auth could not identify password for [nrpe]
Nov 10 09:31:44 remotehost sudo: pam_succeed_if(sudo:auth): requirement "uid >= 1000" not met by user "nrpe"
...
Nov 10 09:36:13 remotehostsudo sudo:    nrpe : user NOT in sudoers ; TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/lib64/nagios/plugins/check_log -F /var/log/secure -O /home/nagios/authFail -q authentication

but it is clearly there

Code: Select all

Defaults:nagios !requiretty
nagios ALL=NOPASSWD: /usr/lib64/nagios/plugins/check_log
#nagios ALL=(ALL) NOPASSWD: ALL


[nagios@remotehost~]$ sudo systemctl status nrpe
● nrpe.service - NRPE
   Loaded: loaded (/usr/lib/systemd/system/nrpe.service; enabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-11-10 09:36:04 EST; 8min ago
  Process: 22757 ExecStart=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d $NRPE_SSL_OPT (code=exited, status=0/SUCCESS)
 Main PID: 22758 (nrpe)
   CGroup: /system.slice/nrpe.service
           └─22758 /usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -d

added the nrpe user to /etc/sudoers and it seems like it is reading the file now, BUT every time the command is run, it is logged to /var/log/secure so it just detects itself! Not exactly the point

Code: Select all

(1) < Nov 10 10:09:05 remotehost sudo: nrpe : TTY=unknown : PWD=/ : USER=root : COMMAND=/usr/lib64/nagios/plugins/check_log -F /var/log/secure -O /home/nagios/authFailures -q authentication failure 

I don't understand why there is a user called nrpe when the "nrpe_user" in the cfg file is set to nagios

Code: Select all

grep "nrpe_user=" /etc/nagios/nrpe.cfg
nrpe_user=nagios

[tgriep]

That is strange, it looks like you are running NRPE as a daemon and that should run it as the user in the nrpe.cfg file.
Can you check and see what user it is running as by running the following on that server as root?

Code: Select all

ps -ef |grep nrpe

[tgriep]

I think I found where the nrpe user is setup. Take a look at the nrpe.service file and you will see that the nrpe user is defined in it.