Problem monitoring MYSQL socket (Zimbra)

[thomsany]

Hey guys,

I have done many installations of the NRPE tool to monitor remote servers.
I have been monitoring always Zimbra 8.6 versions which have the MYSQL socket pointing at: /opt/zimbra/db/mysql.sock.

The option on the nrpe.cfg works just fine and the server returns the mysql status at the moment.

In a new server I installed the new version of Zimbra 8.7 and now for some reason the socket monitoring is not working.
It has a different path pointing now to: /opt/zimbra/data/tmp/mysql/mysql.sock.

I changed the path in the configuration file for that particular server pointing to the right location and I get this message.

Can't connect to local MySQL server through socket '/opt/zimbra/data/tmp/mysql/mysql.sock' (13)

All permissions are set the same on both servers and on one it works, on the other one it doesn't.
No special configuration was done to get it to work on either servers since Zimbra installs everything.

Any suggestions on what might be happening?

Thanks!
Teo

[rkennedy]

Can you show us the service definition from Nagios, the command definition it's running, and also your nrpe.cfg (from the Zimbra machine) so that we can see how your commands are referenced?

This should help us to see what's logically going on all around.

Also, what are the permissions on the sock file? ls -la /opt/zimbra/data/tmp/mysql/mysql.sock

[thomsany]

Hi

Service definition on Nagios host:

Code: Select all

define service {
       use                     generic-service
       hostgroup_name          zimbra
       service_description     MySQL
       check_command           check_nrpe!check_zimbra_mysql
       #contact_groups                  admins                  ; Notifications get sent out to everyone in the 'admins' group
       #notification_options            w,u,c,r                 ; Send notifications about warning, unknown, critical, and recovery events
       #notification_interval           60                      ; Re-notify about service problems every hour
       #notification_period             24x7                    ; Notifications can be sent out at any time
}

Mysql.sock permissions (Zimbra computer):

Code: Select all

[root@mail libexec]# ls -la /opt/zimbra/data/tmp/mysql/mysql.sock
srwxrwxrwx. 1 zimbra zimbra 0 Nov 16 12:38 /opt/zimbra/data/tmp/mysql/mysql.sock

NRPE.cfg configuration:

command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10
command[check_load]=/usr/local/nagios/libexec/check_load -w 15,10,5 -c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk -w 10% -c 5% -p /opt
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 150 -c 200
command[check_dns]=/usr/local/nagios/libexec/check_dns -H google.com -w 5 -c 8
command[check_zimbra_mailq]=/usr/local/nagios/libexec/check_mailq -w 100 -c 250 -M postfix
command[check_zimbra_mysql]=/usr/local/nagios/libexec/check_mysql -s /opt/zimbra/data/tmp/mysql/mysql.sock
command[check_cert]=/usr/local/nagios/libexec/check_http -S -H localhost -C 30

I even tried adding the user and password for the check_mysql and the error persists.

Thanks a lot in advance.
Teo

[rkennedy]

Could you try a su nagios - on the client machine? (or su to whichever user NRPE is running under in your environment)

Then, try to execute it again - it could be a parent folder permissions. What happens if you try directly from the CLI? I haven't tested this plugin much, but I had to specify credentials to get it to work properly over here.

Code: Select all

[root@bob libexec]# ./check_mysql -s /var/lib/mysql/mysql.sock
Access denied for user 'root'@'localhost' (using password: NO)

[root@bob libexec]# ./check_mysql -s /var/lib/mysql/mysql.sock -u root -p nagiosxi
Uptime: 609606  Threads: 75  Questions: 12482602  Slow queries: 0  Opens: 28011  Flush tables: 1  Open tables: 57  Queries per second avg: 20.476|Connections=392864c;;; Open_files=112;;; Open_tables=57;;; Qcache_free_memory=0;;; Qcache_hits=0c;;; Qcache_inserts=0c;;; Qcache_lowmem_prunes=0c;;; Qcache_not_cached=0c;;; Qcache_queries_in_cache=0;;; Queries=12482602c;;; Questions=12482602c;;; Table_locks_waited=6008c;;; Threads_connected=75;;; Threads_running=1;;; Uptime=609606c;;;

Just wondering if we'll be able to get more of an output going this route.

[thomsany]

Hmmm intersting, I am seeing the user nagios doesn't have the permission to run the command. I get the error mentioned before:

./check_mysql -s /opt/zimbra/data/tmp/mysql/mysql.sock -u root -p passwordhidden

If I do it with the root user it works.
How can I fix it?
I already set the owner for the whole nagios directory to user nagios using the following command:

chown nagios.nagios /usr/local/nagios
chown -R nagios.nagios /usr/local/nagios/libexec

The NRPE user that I am using is nagios. All other checks work, except this one.
Also I am using xinetd service.

[tgriep]

Can you run the following commands on the remote server and post the output?

Code: Select all

ls -l /usr/local/nagios/libexec
cat /etc/xinetd.d/nrpe

Thanks

[avandemore]

You'll want to execute the remote command with the appropriate user, eg zimbra.

You can have your command definition do a sudo to the correct account. zimbra is a special user so this is probably a good approach to any Zimbra metrics you want to monitor.

[thomsany]

Can you run the following commands on the remote server and post the output?

Code: Select all

ls -l /usr/local/nagios/libexec
cat /etc/xinetd.d/nrpe

Thanks

Here is the output:

Code: Select all

[root@mail ~]# ls -l /usr/local/nagios/libexec
total 7396
-rwxr-xr-x. 1 nagios nagios 208847 Nov 16 12:17 check_apt
-rwxr-xr-x. 1 nagios nagios   2346 Nov 16 12:17 check_breeze
-rwxr-xr-x. 1 nagios nagios 212932 Nov 16 12:17 check_by_ssh
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_clamd -> check_tcp
-rwxr-xr-x. 1 nagios nagios 161352 Nov 16 12:17 check_cluster
-r-xr-xr-x. 1 nagios nagios 210629 Nov 16 12:17 check_dhcp
-rwxr-xr-x. 1 nagios nagios 221169 Nov 16 12:17 check_dig
-rwxr-xr-x. 1 nagios nagios 225583 Nov 16 12:17 check_disk
-rwxr-xr-x. 1 nagios nagios   9385 Nov 16 12:17 check_disk_smb
-rwxr-xr-x. 1 nagios nagios 228587 Nov 16 12:17 check_dns
-rwxr-xr-x. 1 nagios nagios 125376 Nov 16 12:17 check_dummy
-rwxr-xr-x. 1 nagios nagios   3604 Nov 16 12:17 check_file_age
-rwxr-xr-x. 1 nagios nagios   6412 Nov 16 12:17 check_flexlm
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_ftp -> check_tcp
-rwxr-xr-x. 1 nagios nagios 368032 Nov 16 12:17 check_http
-r-xr-xr-x. 1 nagios nagios 225355 Nov 16 12:17 check_icmp
-rwxr-xr-x. 1 nagios nagios 168222 Nov 16 12:17 check_ide_smart
-rwxr-xr-x. 1 nagios nagios  15275 Nov 16 12:17 check_ifoperstatus
-rwxr-xr-x. 1 nagios nagios  13423 Nov 16 12:17 check_ifstatus
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_imap -> check_tcp
-rwxr-xr-x. 1 nagios nagios   6984 Nov 16 12:17 check_ircd
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_jabber -> check_tcp
-rwxr-xr-x. 1 nagios nagios 185975 Nov 16 12:17 check_load
-rwxr-xr-x. 1 nagios nagios   6595 Nov 16 12:17 check_log
-rwxr-xr-x. 1 nagios nagios  22756 Nov 16 12:17 check_mailq
-rwxr-xr-x. 1 nagios nagios 167396 Nov 16 12:17 check_mrtg
-rwxr-xr-x. 1 nagios nagios 168753 Nov 16 12:17 check_mrtgtraf
-rwxr-xr-x. 1 nagios nagios 203901 Nov 16 12:17 check_mysql
-rwxr-xr-x. 1 nagios nagios 190339 Nov 16 12:17 check_mysql_query
-rwxr-xr-x. 1 nagios nagios 185219 Nov 16 12:17 check_nagios
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_nntp -> check_tcp
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_nntps -> check_tcp
-rwxrwxr-x. 1 nagios nagios  81518 Nov 16 09:26 check_nrpe
-rwxr-xr-x. 1 nagios nagios 218313 Nov 16 12:17 check_nt
-rwxr-xr-x. 1 nagios nagios 217137 Nov 16 12:17 check_ntp
-rwxr-xr-x. 1 nagios nagios 206630 Nov 16 12:17 check_ntp_peer
-rwxr-xr-x. 1 nagios nagios 201803 Nov 16 12:17 check_ntp_time
-rwxr-xr-x. 1 nagios nagios 248762 Nov 16 12:17 check_nwstat
-rwxr-xr-x. 1 nagios nagios   8926 Nov 16 12:17 check_oracle
-rwxr-xr-x. 1 nagios nagios 191516 Nov 16 12:17 check_overcr
-rwxr-xr-x. 1 nagios nagios 221697 Nov 16 12:17 check_ping
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_pop -> check_tcp
-rwxr-xr-x. 1 nagios nagios 218391 Nov 16 12:17 check_procs
-rwxr-xr-x. 1 nagios nagios 184122 Nov 16 12:17 check_real
-rwxr-xr-x. 1 nagios nagios   9679 Nov 16 12:17 check_rpc
-rwxr-xr-x. 1 nagios nagios   1465 Nov 16 12:17 check_sensors
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_simap -> check_tcp
-rwxr-xr-x. 1 nagios nagios 282286 Nov 16 12:17 check_smtp
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_spop -> check_tcp
-rwxr-xr-x. 1 nagios nagios 183621 Nov 16 12:17 check_ssh
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_ssmtp -> check_tcp
-rwxr-xr-x. 1 nagios nagios 164972 Nov 16 12:17 check_swap
-rwxr-xr-x. 1 nagios nagios 266273 Nov 16 12:17 check_tcp
-rwxr-xr-x. 1 nagios nagios 185424 Nov 16 12:17 check_time
lrwxrwxrwx. 1 nagios nagios      9 Nov 16 12:17 check_udp -> check_tcp
-rwxr-xr-x. 1 nagios nagios 197932 Nov 16 12:17 check_ups
-rwxr-xr-x. 1 nagios nagios 161770 Nov 16 12:17 check_uptime
-rwxr-xr-x. 1 nagios nagios 156364 Nov 16 12:17 check_users
-rwxr-xr-x. 1 nagios nagios   3032 Nov 16 12:17 check_wave
-rwxr-xr-x. 1 nagios nagios   2121 Jul 13  2008 check_zmstatus.pl
-rwxr-xr-x. 1 nagios nagios 151873 Nov 16 12:17 negate
-rwxr-xr-x. 1 nagios nagios 149573 Nov 16 12:17 urlize
-rwxr-xr-x. 1 nagios nagios   1929 Nov 16 12:20 utils.pm
-rwxr-xr-x. 1 nagios nagios   2791 Nov 16 12:17 utils.sh

Xinetd NRPE:

Code: Select all

[root@mail ~]# cat /etc/xinetd.d/nrpe
# default: on
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
       	flags           = REUSE
        socket_type     = stream    
	port		= 5666    
       	wait            = no
        user            = nagios
	group		= nagios
       	server          = /usr/local/nagios/bin/nrpe
        server_args     = -c /usr/local/nagios/etc/nrpe.cfg --inetd
       	log_on_failure  += USERID
        disable         = no
	only_from       = 127.0.0.1 54.197.229.243
}

[thomsany]

You'll want to execute the remote command with the appropriate user, eg zimbra.

You can have your command definition do a sudo to the correct account. zimbra is a special user so this is probably a good approach to any Zimbra metrics you want to monitor.

How can I run the command as the "zimbra" user?
Where do I declare the user I am going to use?
Keep in mind on other Zimbra 8.6 versions I didn't have this issue. They work just fine with the nagios user. With this particular Zimbra version I have this issue.
Thanks a lot in advance for all the help!!

Teo

[avandemore]

Hmmm intersting, I am seeing the user nagios doesn't have the permission to run the command. I get the error mentioned before:

Please show the full output for the command.