Readonly File system checking
Posted: Mon Apr 30, 2018 1:46 pm
I was looking at the checks for checking readonly file systems in nagios exchange and I didn't find them to useful for an issue that was occuring in our environment with some blades disconnecting from the san. While that issue is being review by the vendors I am being tasked with creating a nagios check to tell us if any of the filesystems become readonly. Apparently i've received conflicting reviews where some folks have told me that proc mounts doesn't say RO and some other folks tell me that it is..... so I can't trust reading rw or ro stat from the os level . So i've decided to come up with my own.
So in our nagios test environment I created 3 file systems and mounted 2 as RO while a third was RW
When I execute it locally as nagios it fails displaying all filesystems as being Critical but when I execute it with sudo it works fine.
-sh-4.1$ /usr/local/nagios/libexec/check_readonly.sh
Critical: ReadOnly File Systems / /boot /home /home/mount1 /usr/mount2 /var/mount3
But when I run it with sudo it works fine.
-sh-4.1$ sudo /usr/local/nagios/libexec/check_readonly.sh
OK: All File Systems Writable
This checks reviews /proc/mounts for any mounted filesystem and attempts to write a file to them.
If it fails to write then it becomes critical
Works fine in my test environment with the nagios account having a lot more permissions than normal but when it comes to our test production environment I am getting
nrpe.cfg has
command[check_readonly]=sudo /usr/local/nagios/libexec/check_readonly.sh
sudoers file has same command on both servers
nagios ALL= NOPASSWD:/usr/local/nagios/libexec/check_readonly.sh
The comnmand itself on the testprod server from test nagios server comes out:
/usr/local/nagios/libexec/check_nrpe -H testprodserver -c check_readonly
NRPE: Unable to read output
The comnmand itself on the nagios production server to the test nagios server comes out:
/usr/local/nagios/libexec/check_nrpe -H testnagioserver -c check_readonly
OK: All File Systems Writable
when ro filesystems are mounted I get
Critical: ReadOnly File Systems /home/mount1 /usr/mount2
I know i've had these issues in the past implementing plugins from Nagios exchange but I can't recall what the fix was. I am assuming there is still a permissions issue for the testprodserver? The plugin has the same permissions on both servers.
So in our nagios test environment I created 3 file systems and mounted 2 as RO while a third was RW
When I execute it locally as nagios it fails displaying all filesystems as being Critical but when I execute it with sudo it works fine.
-sh-4.1$ /usr/local/nagios/libexec/check_readonly.sh
Critical: ReadOnly File Systems / /boot /home /home/mount1 /usr/mount2 /var/mount3
But when I run it with sudo it works fine.
-sh-4.1$ sudo /usr/local/nagios/libexec/check_readonly.sh
OK: All File Systems Writable
This checks reviews /proc/mounts for any mounted filesystem and attempts to write a file to them.
If it fails to write then it becomes critical
Works fine in my test environment with the nagios account having a lot more permissions than normal but when it comes to our test production environment I am getting
nrpe.cfg has
command[check_readonly]=sudo /usr/local/nagios/libexec/check_readonly.sh
sudoers file has same command on both servers
nagios ALL= NOPASSWD:/usr/local/nagios/libexec/check_readonly.sh
The comnmand itself on the testprod server from test nagios server comes out:
/usr/local/nagios/libexec/check_nrpe -H testprodserver -c check_readonly
NRPE: Unable to read output
The comnmand itself on the nagios production server to the test nagios server comes out:
/usr/local/nagios/libexec/check_nrpe -H testnagioserver -c check_readonly
OK: All File Systems Writable
when ro filesystems are mounted I get
Critical: ReadOnly File Systems /home/mount1 /usr/mount2
I know i've had these issues in the past implementing plugins from Nagios exchange but I can't recall what the fix was. I am assuming there is still a permissions issue for the testprodserver? The plugin has the same permissions on both servers.