Page 1 of 2
Permission denied errors in my audit logs
Posted: Wed Sep 16, 2020 8:59 am
by dhatha03
I am getting hundreds of permission denied messages in my Nagios server audit logs. For example,
node=<nagios server> msg=audit(09/16/2020 10:13:45.634:2983937) exit=-13(Permission denied) auid=nagios uid=nagios gid=nagios comm=snmpget exe=/usr/bin/snmpget
node=<nagios server> msg=audit(09/16/2020 10:13:48.181:2983940) exit=-13(Permission denied) auid=nagios uid=nagios gid=nagios comm=snmpwalk exe=/usr/bin/snmpwalk
node=<nagios server> msg=audit(09/16/2020 10:13:43.828:2983935) exit=-13(Permission denied) auid=nagios uid=nagios gid=nagcmd comm=php exe=/usr/bin/php
File permissions are as follows:
-rwxr-xr-x 1 root root 13920 Mar 22 2017 /usr/bin/snmpget
-rwxr-xr-x 1 root root 16824 Mar 22 2017 /usr/bin/snmpwalk
-rwxr-xr-x 1 root root 3273840 Mar 22 2017 /usr/bin/php
All SNMP checks within NagiosXI are executing and completing as expected, but our log files are filling up quickly with the messages above.
Re: Permission denied errors in my audit logs
Posted: Thu Sep 17, 2020 1:34 pm
by benjaminsmith
Hi,
The file permissions look right and are the same as my test server, do you have SELinux enabled?
Also, have you made any changes to the sudoers file, please post the output to the following command.
Regards,
Benjamin
Re: Permission denied errors in my audit logs
Posted: Tue Sep 22, 2020 9:17 am
by dhatha03
getenforce = Disabled
I cannot include the full sudoers file due to company restrictions, but the Nagios part is below.
User_Alias NAGIOSXI=nagios
User_Alias NAGIOSXIWEB=apache
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios status
NAGIOSXI ALL = NOPASSWD:/etc/init.d/nagios checkconfig
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd start
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd stop
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd restart
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd reload
NAGIOSXI ALL = NOPASSWD:/etc/init.d/npcd status
NAGIOSXI ALL = NOPASSWD:/usr/bin/php /usr/local/nagiosxi/scripts/components/autodiscover_new.php *
NAGIOSXI ALL = NOPASSWD:/usr/bin/php /usr/local/nagiosxi/scripts/send_to_nls.php *
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/components/getprofile.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/upgrade_to_latest.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/change_timezone.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_services.sh *
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/reset_config_perms.sh
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_ssl_config.sh *
NAGIOSXI ALL = NOPASSWD:/usr/local/nagiosxi/scripts/backup_xi.sh *
NAGIOSXIWEB ALL = NOPASSWD:/etc/init.d/snmptt restart
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/messages
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/httpd/error_log
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/tail -100 /var/log/mysqld.log
NAGIOSXIWEB ALL = NOPASSWD:/usr/bin/php /usr/local/nagiosxi/scripts/components/autodiscover_new.php *
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/components/getprofile.sh
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/repair_databases.sh
NAGIOSXIWEB ALL = NOPASSWD:/usr/local/nagiosxi/scripts/manage_services.sh *
Re: Permission denied errors in my audit logs
Posted: Wed Sep 23, 2020 10:28 am
by lmiltchev
The entries in the sudoers look fine.
Have you modified your system from the "default", which could've caused the issue, e.g. hardening the server, restricting cron to root user only, LDAP sudoers, TCP wrappers, etc.?
Please run the following commands and show the output in code wraps:
Code: Select all
grep requiretty /etc/sudoers
chage nagios -l
chage apache -l
grep nag /etc/group
grep nag /etc/passwd
tail -50 /var/log/cron
Also, PM me your latest profile (
Admin > System Config > System Profile > Download Profile).
Re: Permission denied errors in my audit logs
Posted: Wed Sep 23, 2020 1:59 pm
by dhatha03
profile.zip sent to you.
Code: Select all
[root@tvm-nagios01 ~]# grep requiretty /etc/sudoers
#Defaults requiretty
# changed in order to be able to use sudo without a tty. See requiretty above.
Code: Select all
[root@tvm-nagios01 ~]# chage nagios -l
Last password change : Sep 27, 2018
Password expires : Dec 05, 2020
Password inactive : Jan 09, 2021
Account expires : never
Minimum number of days between password change : 1
Maximum number of days between password change : 800
Number of days of warning before password expires : 7
Code: Select all
[root@tvm-nagios01 ~]# chage apache -l
Last password change : Sep 27, 2018
Password expires : never
Password inactive : never
Account expires : never
Minimum number of days between password change : -1
Maximum number of days between password change : -1
Number of days of warning before password expires : -1
Code: Select all
[root@tvm-nagios01 ~]# grep nag /etc/group
nagcmd:x:500:nagios,apache
nagios:x:501:nagios,apache,snmptt
Code: Select all
[root@tvm-nagios01 ~]# grep nag /etc/passwd
nagios:x:500:500:Nagios Owner Account:/home/nagios:/bin/bash
Code: Select all
[root@tvm-nagios01 ~]# grep nag /etc/passwd
nagios:x:500:500:Nagios Owner Account:/home/nagios:/bin/bash
[root@tvm-nagios01 ~]# tail -50 /var/log/cron
Sep 23 18:49:01 tvm-nagios01 CROND[15343]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Sep 23 18:49:01 tvm-nagios01 CROND[15344]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Sep 23 18:50:01 tvm-nagios01 CROND[15871]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Sep 23 18:50:01 tvm-nagios01 CROND[15873]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Sep 23 18:50:01 tvm-nagios01 CROND[15874]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Sep 23 18:50:02 tvm-nagios01 CROND[15939]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15940]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15942]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/dbmaint.php >> /usr/local/nagiosxi/var/dbmaint.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15938]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Sep 23 18:50:02 tvm-nagios01 CROND[15943]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15944]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15941]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15945]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15947]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15946]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15949]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Sep 23 18:50:02 tvm-nagios01 CROND[15948]: (root) CMD (LANG=C LC_ALL=C /usr/bin/mrtg /etc/mrtg/mrtg.cfg --lock-file /var/lib/mrtg/mrtg.lock --confcache-file /var/lib/mrtg/mrtg.ok)
Sep 23 18:51:01 tvm-nagios01 CROND[16378]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16381]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16380]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16382]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16379]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16384]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16386]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16383]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16387]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16385]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Sep 23 18:51:01 tvm-nagios01 CROND[16390]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16782]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16783]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16784]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16785]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16786]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16788]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16787]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16790]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16791]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16792]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Sep 23 18:52:01 tvm-nagios01 CROND[16796]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17181]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/deadpool.php >> /usr/local/nagiosxi/var/deadpool.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17183]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/perfdataproc.php >> /usr/local/nagiosxi/var/perfdataproc.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17184]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/event_handler.php >> /usr/local/nagiosxi/var/event_handler.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17186]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cmdsubsys.php >> /usr/local/nagiosxi/var/cmdsubsys.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17187]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/sysstat.php >> /usr/local/nagiosxi/var/sysstat.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17188]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/reportengine.php >> /usr/local/nagiosxi/var/reportengine.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17189]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/nom.php >> /usr/local/nagiosxi/var/nom.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17192]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/feedproc.php >> /usr/local/nagiosxi/var/feedproc.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17195]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/eventman.php >> /usr/local/nagiosxi/var/eventman.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17193]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/cleaner.php >> /usr/local/nagiosxi/var/cleaner.log 2>&1)
Sep 23 18:53:01 tvm-nagios01 CROND[17196]: (nagios) CMD (/usr/bin/php -q /usr/local/nagiosxi/cron/snmptt_service_results.php >> /usr/local/nagiosxi/var/snmptt_service_results.log 2>&1)
Re: Permission denied errors in my audit logs
Posted: Wed Sep 23, 2020 3:28 pm
by lmiltchev
Thank you for the profile! I found these errors in the apache error log:
sh: /usr/local/nagios/var/rw/nagios.cmd: Permission denied
sh: /usr/local/nagios/var/rw/nagios.cmd: Permission denied
sh: /usr/local/nagios/var/rw/nagios.cmd: Permission denied
sh: /usr/local/nagios/var/rw/nagios.cmd: Permission denied
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
sudo: no tty present and no askpass program specified
sudo: no tty present and no askpass program specified
So, let's check some permissions:
Code: Select all
ls -lad /usr/local/nagios /usr/local/nagios/var/ /usr/local/nagios/var/rw/
ls -la /usr/local/nagios/var/rw/
Also, make sure you have the following line in sudoers:
Can you run the command below and show the output?
Have you modified your system from the "default", which could've caused the issue, e.g. hardening the server, restricting cron to root user only, LDAP sudoers, TCP wrappers, etc.?
Re: Permission denied errors in my audit logs
Posted: Thu Sep 24, 2020 6:55 am
by dhatha03
cron is not restricted, selinux is disabled, server does connect to LDAP for local user authentication. I think I might try to rebuild the server and see if that helps. I will backup my existing configuration and then once rebuilt import it and see what happens.
Re: Permission denied errors in my audit logs
Posted: Thu Sep 24, 2020 9:10 am
by lmiltchev
Let us know how it goes. We will keep the topic open for the time being in case you have more questions.
Re: Permission denied errors in my audit logs
Posted: Thu Oct 01, 2020 9:34 am
by BackNBlack
Hi Guys,
We're going to hold off on rebuilding just yet. Thanks for the help, let's see where this new information leads us.
An update, the snmpget and snmpwalk entries have stopped but the php entries are constant and still persist.
I have the answers to your last questions:
Sudoers:
Code: Select all
#Defaults requiretty
Defaults !visiblepw
Directory permissions
Code: Select all
ls -lad /usr/local/nagios /usr/local/nagios/var/ /usr/local/nagios/var/rw/
drwxr-xr-x 8 root root 4096 Sep 27 2018 /usr/local/nagios
drwxrwxr-x 6 nagios nagios 4096 Oct 1 14:16 /usr/local/nagios/var/
drwsr-xr-x 2 nagios nagios 4096 Oct 1 12:04 /usr/local/nagios/var/rw/
PHP version
Code: Select all
su nagios -c "/usr/bin/php -v"
PHP 5.3.3 (cli) (built: Mar 22 2017 12:27:09)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies
Re: Permission denied errors in my audit logs
Posted: Thu Oct 01, 2020 10:13 am
by lmiltchev
Your nagios directory is owned by root...
drwxr-xr-x 8 root root 4096 Sep 27 2018 /usr/local/nagios
Change the ownership to nagios.
Code: Select all
chown nagios.nagios /usr/local/nagios
The permissions of the "/usr/local/nagios/var/rw/" directory are also wrong.
drwsr-xr-x 2 nagios nagios 4096 Oct 1 12:04 /usr/local/nagios/var/rw/
They have to be:
drwxrwsr-x nagios nagios
Fix the permissions by running:
Code: Select all
chmod u-s,g+ws /usr/local/nagios/var/rw
Also, add the following line to sudoers:
One last thing - you didn't show us the output of the command below as asked: