"Session timeout" message while monitoring database paramete

[RIDS_I2MP]

Hi Team,

We are monitoring few database parameters on an AIX box. Successfully able to monitor some parameters as mentioned below, but receiving "session timeout" error message while monitoring 'rman_backup-problems'.

successfully monitored
- TNSping
- process usage
- session usage
- sga-data-buffer-hit-ratio
- sga-library-cache-hit-ratio

COMMAND: /usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib ORACLE_HOME=/usr/lib/oracle/12.1/client64

/usr/local/nagios/libexec/check_oracle_health --connect '\(DESCRIPTION=\(ADDRESS=\(PROTOCOL=TCP\)\(HOST=<IP>\)\(PORT=1521\)\)\(CONNECT_DATA=

\(SID=<DBname>\)\)\)' --username '<username>' --password '<password>' --mode rman-backup-problems --warning 3 --critical 5
OUTPUT: CRITICAL - connection could not be established within 60 seconds

Note: The leading backslash are not mentioned in the command, but when we click on "Test check command" the command appears in above format.

Kindly suggest.

[avandemore]

When run from the command line, please paste the full command and output like so:

Code: Select all

# su - nagios
$ <your_full_command>
<the_resulting_output>

[RIDS_I2MP]

Below is the output when command executed from Nagios Xi server:

[nagios@hostname libexec]$ ./check_oracle_health --connect '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=<remoteIP>)(PORT=1521))(CONNECT_DATA=(SID=<DBName>)))' --username '<username>' --password '<passwd>' --mode rman-backup-problems --warning 3 --critical 5
CRITICAL - cannot connect to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=<IP>)(PORT=1521))(CONNECT_DATA=(SID=<DBName>))). install_driver(Oracle) failed: Can't load '/usr/local/lib64/perl5/auto/DBD/Oracle/Oracle.so' for module DBD::Oracle: libocci.so.12.1: cannot open shared object file: No such file or directory at /usr/lib64/perl5/DynaLoader.pm line 200.
at (eval 19) line 3
Compilation failed in require at (eval 19) line 3.
Perhaps a required shared library or dll isn't installed where expected
at ./check_oracle_health line 4755

[SteveBeauchemin]

Couple suggestions... I monitor more than 100 oracle instances. This is what I do...

the --connect syntax I use is

Code: Select all

--connect [oracle IP]:[oracle port]/[oracle instance name]

for example

Code: Select all

--connect 111.222.333.444:1512/ora1

The other thing is the LD_LIBRARY_PATH and ORACLE_HOME environment variables. Either put them on the command line or make sure they are part of the system environment. For example, I have a file in /etc/profile.d named oracle.sh that looks like this and provides every user login those environment settings.

Code: Select all

cat /etc/profile.d/oracle.sh
# Oracle Required Variables for environment
export ORACLE_HOME=/usr/lib/oracle/12.1/client64
if [ -z "$LD_LIBRARY_PATH" ]; then
  export LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib
else
  export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/oracle/12.1/client64/lib
fi

It makes it easier to run the oracle tests from the command line. So you can either run this

Code: Select all

cd /usr/local/nagios/libexec
/usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib \
ORACLE_HOME=/usr/lib/oracle/12.1/client64 \
./check_oracle_health --connect 111.222.333.444:1512/ora1 \
--username [ORACLE_USERNAME] \
 --password [ORACLE_PASSWORD] \
 --mode rman-backup-problems \
 --warning 3 \
 --critical 5 

or this, which ever is easier for you

Code: Select all

cd /usr/local/nagios/libexec
./check_oracle_health --connect 111.222.333.444:1512/ora1 \
--username [ORACLE_USERNAME] \
 --password [ORACLE_PASSWORD] \
 --mode rman-backup-problems \
 --warning 3 \
 --critical 5 

This information is for running from the command line. In the Nagios GUI, the command definition probably has the env defined properly.
Fix your connect string and I think you will succeed.
Good luck.
Steve B

[tgriep]

Thanks @SteveBeauchemin for the help.
I agree, argument for the connect option doesn't seen right
One thing, the test check command button, sometimes adds too much escaping to the command and it may not function so do worry if it doesn't work for some commands.
The best thing is to look at the Service Details after the check has run and see if the output is correct.

If the Status Details for that service is bad, please run the following and post the output.

Code: Select all

grep check_oracle_health /usr/local/nagios/etc/commands.cfg
grep  rman-backup-problems /usr/local/nagios/etc/services/*
su nagios
/usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client/lib ORACLE_HOME=/usr/lib/oracle/12.1/client /usr/local/nagios/libexec/check_oracle_health -V

[RIDS_I2MP]

Hi Team,

Please find the below output:

[nagios@nagiosserver libexec]$ grep check_oracle_health /usr/local/nagios/etc/commands.cfg
command_line /usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib

ORACLE_HOME=/usr/lib/oracle/12.1/client64 $USER1$/check_oracle_health $ARG1$
command_line /usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib

ORACLE_HOME=/usr/lib/oracle/12.1/client64 $USER1$/check_oracle_health $ARG1$
command_line /usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib

ORACLE_HOME=/usr/lib/oracle/12.1/client64 $USER1$/check_oracle_health $ARG1$

[nagios@nagiosserver libexec]$ grep rman-backup-problems /usr/local/nagios/etc/services/*
/usr/local/nagios/etc/services/remotehostname1.cfg: service_description <SID/DBname> rman-backup-problems
/usr/local/nagios/etc/services/remotehostname1.cfg: check_command check_xi_oracleserverspace!--connect '(DESCRIPTION=(ADDRESS=

(PROTOCOL=TCP)(HOST=<IP>)(PORT=1521))(CONNECT_DATA=(SID=<DBname/SID>)))' --username '<DBusername>' --password '<DBpasswd>' --mode rman-backup-

problems --warning 3 --critical 5!!!!!!!
/usr/local/nagios/etc/services/remotehostname2.cfg: service_description <SID/DBname> rman-backup-problems
/usr/local/nagios/etc/services/remotehostname2.cfg: check_command check_xi_oracleserverspace!--connect '(DESCRIPTION=(ADDRESS=

(PROTOCOL=TCP)(HOST=<IP>)(PORT=1521))(CONNECT_DATA=(SID=<DBname/SID>)))' --username '<DBusername>' --password '<DBpasswd>' --mode rman-backup-

problems --warning 3 --critical 5!!!!!!!

[nagios@nagiosserver libexec]$ /usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client/lib ORACLE_HOME=/usr/lib/oracle/12.1/client

/usr/local/nagios/libexec/check_oracle_health -V
check_oracle_health (1.7.3)
This nagios plugin comes with ABSOLUTELY NO WARRANTY. You may redistribute
copies of this plugin under the terms of the GNU General Public License.


As suggested by you I tried running below command from command line but it is taking very long for the response:

cd /usr/local/nagios/libexec

/usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib \
ORACLE_HOME=/usr/lib/oracle/12.1/client64 \
./check_oracle_health --connect '(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=<IP>)(PORT=1521))(CONNECT_DATA=(SID=<SID/DBname>)))' --username
'<DBusername>' --password '<DBpasswd>' --mode rman-backup-problems --warning 3 --critical 5

from Nagios GUI the response is:

"(Service check timed out after 120.01 seconds)"

[tgriep]

The --connect option in your commands is not configured correctly and that is why the checks are failing.
This is the format the check should be using for the --connect string

Code: Select all

--connect <Hostname or IP address>:<port>/<SID>

The <Hostname or IP address> will be the Oracle server's name or IP.
The <Port> will be the port that is opened on the server for remote connections.
The <SID> is the Site Identifier. SID is a unique name for an Oracle database instance.

Here is an example of how the connect option should be formatted in your checks.

Code: Select all

--connect 192.168.1.1:1521/SID

It looks like the plugin is installed and functional as when you an the plugin on the command line, it returned the version.

If you update the --connect option in your service checks with the valid information, they should work for you.
You will probably have to contact your Oracle Admin for the information as well as setup the permissions in the database that you are using to connect to the database.

The Simplest way to monitor the Oracle Database it to use the Oracle Wizards in the XI GUI by going to the Config > Configuration Wizards menu and use the Oracle Wizards.

[RIDS_I2MP]

Hi Team,

Below is the output of the command suggested by your team, but didn't succeed:

[nagios@nagiosxi libexec]$ /usr/bin/env LD_LIBRARY_PATH=/usr/lib/oracle/12.1/client64/lib ORACLE_HOME=/usr/lib/oracle/12.1/client64 ./check_oracle_health --connect DBserverIp:1521/DBname --username 'username' --password 'passwd' --mode rman-backup-problems --warning 3 --critical 5
CRITICAL - cannot connect to DBserverIp:1521/DBname. ORA-12514: TNS:listener does not currently know of service requested in connect descriptor (DBD ERROR: OCIServerAttach)

Note: I am facing this issue only when I am trying to monitor "rman-backup-problem", whereas other database parmeters are monitored successfully on the same server and database.

Kindly help, this issue is lasting from long time

[tgriep]

If all of the other checks are working except the rman-backup-problems mode, then someone will have to login to the Oracle server and verify that the Recovery Manager is setup on the server and functioning correctly.
Take a look at this link for Troubleshooting RMAN operations on your server.
https://docs.oracle.com/cd/B28359_01/ba ... m#BRADV012

If the Recovery Manager is not going to be configured on the server, then you can delete the check in Nagios XI.

[SteveBeauchemin]

A TNS problem is a TNS problem.

Review your syntax. Try the exact same command with a different mode, such as connection-time.

If other commands work against the same DB, then delete this one that is not working, copy an existing service that is working properly, change the mode, and see if that helps.

The error message is telling you that your connect string is not okay. Compare 2 services side by side and see if you can spot the problem.

Until you get past the TNS issue you cannot even ask about RMAN.

That is what I would do.

Steve B