CGI Error created by "enable_environment_macros=1"

[uragusan]

We have install Nagios 4.08 on Debian 8U1 with Apache 2.4.
For our notifications we use nagios_mail.php Script by Otto Berger that required to enable “enable_environment_macros“ in Nagios.cfg.
After activation we get the CGI error in Nagios-error.log.

Code: Select all

[Thu Jul 09 11:39:06.910982 2015] [cgi:error] [pid 3549] [client 10.0.10.197:61954] Script timed out before returning headers: cmd.cgi, referer: https://de-h-nag01.domain.de/nagios/cgi-bin/cmd.cgi?cmd_typ=7&host=de-h-nas02&service=DSM+System+Status&force_check

[Thu Jul 09 11:44:06.926015 2015] [cgi:warn] [pid 3549] [client 10.0.10.197:61954] AH01220: Timeout waiting for output from CGI script /usr/local/nagios/sbin/cmd.cgi, referer: https://de-h-nag01.domain.de/nagios/cgi-bin/cmd.cgi?cmd_typ=7&host=de-h-nas02&service=DSM+System+Status&force_check

Have anyone idea with what it may be related ?

[abrist]

The environment may be sooooo big that the cgi is timing out waiting for it to return. You may need to increase the apache TimeOut directive value:
http://httpd.apache.org/docs/2.0/mod/core.html#timeout
This may cause issues with long waits with the core frontend though.

[uragusan]

Hi abrist, thanks for the answer.
I have change timeout to 600 sec for testing. After forcing the Service check and wait followed, nothing happens, the check was not started and I have become same CGI error in the Logs.
On top command we do not see high CPU or RAM usage.
Nagios 3.2.3 in Productive Environment have not this big Problem.

[jdalrymple]

Maybe apparmor or selinux is giving you some troubles?

I've had no issue when enabling envs on any of my many 4.x systems that I've setup, although I use strictly CentOS, and disabling selinux is the first thing I do.

[uragusan]

Hello jdalrymple,
Apparmor so as selinux do not run on our Debian 8 Nagios Server.

[abrist]

First turn on external command logging if it is disabled:

Code: Select all

sed -i 's/log_external_commands=.*/log_external_commands=1/g' /usr/local/nagios/etc/nagios.cfg

Verify:

Code: Select all

grep log_external /usr/local/nagios/etc/nagios.cfg

Restart Nagios:

Code: Select all

service nagios restart

Then try submitting the query directly to the cgi (instead of through apache):

Code: Select all

REMOTE_USER=nagiosadmin REQUEST_METHOD=GET QUERY_STRING="cmd_typ=7&host=de-h-nas02&service=DSM+System+Status&force_check" /usr/local/nagios/sbin/cmd.cgi > /tmp/cgi_test.txt

Code: Select all

echo "------------------" >> /tmp/cgi_test.txt

Now, try finishing the the command and submitting the forced check:

Code: Select all

REMOTE_USER=nagiosadmin REQUEST_METHOD=GET QUERY_STRING="cmd_typ=7&cmd_mod=2&host=de-h-nas02&service=DSM+System+Status&force_check=on&start_time=$(date '+%m-%d-%Y %H:%M:%S')&btnSubmit=Commit" /usr/local/nagios/sbin/cmd.cgi >> /tmp/cgi_test.txt

Code: Select all

echo "------------------" >> /tmp/cgi_test.txt

And then get a tail of the nagios.log looking specifically for submitted external commands:

Code: Select all

grep -i EXTERNAL /usr/local/nagios/var/nagios.log | tail -10 >> /tmp/cgi_test.txt

Post the contents of /tmp/cgi_test.txt here in code wraps.

[uragusan]

Now, try finishing the the command and submitting the forced check:

Code: Select all

REMOTE_USER=nagiosadmin REQUEST_METHOD=GET QUERY_STRING="cmd_typ=7&cmd_mod=2&host=de-h-nas02&service=DSM+System+Status&force_check=on&start_time=$(date '+%m-%d-%Y %H:%M:%S')&btnSubmit=Commit" /usr/local/nagios/sbin/cmd.cgi >> /tmp/cgi_test.txt

This Command do not work with enabled environment macros, I get Log only if I set "enable_environment_macros=0".
Interesting too, nagios so as nagios.debug Logs are not more filed, after activated environment.

[jdalrymple]

I can't reproduce your scenario. How did this Nagios instance get installed?

Code: Select all

root@localhost:~# cat /etc/debian_version
8.1

[uragusan]

I have take this tutorial.
http://www.2daygeek.com/nagios-4-0-8-in ... linuxmint/

With all this packages that we need for our checks.

Code: Select all

apt-get install wget build-essential apache2 php5-gd libgd2-xpm libgd2-xpm-dev libapache2-mod-php5 libperl-dev libssl-dev apache2-utils snmp libsnmp-dev php5-curl libconfig-general-perl php5-snmp curl libnet-dns-perl libdbd-sybase-perl libdbi-perl autoconf python libdatetime-perl make gcc libxml-simple-perl libio-socket-ssl-perl libnet-snmp-perl libsnmp-perl libsnmp-session-perl libconfig-general-perl libsnmp-dev libnagios-plugin-perl krb5-user libapache2-mod-auth-kerb cifs-utils python-pywbem

[jdalrymple]

I'm not sure what broke for you, but if you don't have a lot of investment in the system already, I'd be happy to share my command history that got my system running. As a matter of fact I'll do it anyway.

I removed incorrect or unimportant lines for clarity's sake:

Code: Select all

    2  apt-get install build-essential php5-gd libgd2-xpm-dev libapache2-mod-php5 libperl-dev libssl-dev apache2-utils apache2
    3  useradd nagios && passwd nagios
    4  groupadd nagcmd
    5  usermod -a -G nagcmd nagios
    6  usermod -a -G nagcmd www-data
    7  wget http://sourceforge.net/projects/nagios/files/nagios-4.x/nagios-4.0.8/nagios-4.0.8.tar.gz
    8  wget http://nagios-plugins.org/download/nagios-plugins-2.0.3.tar.gz
    9  tar -xzvf nagios-4.0.8.tar.gz
   10  cd nagios-4.0.8/
   11  ./configure --with-command-group=nagcmd
   12  make all
   13  make install
   14  make install-init
   15  make install-commandmode
   16  make install-config
   17  make install-webconf
   19  cd sample-config/
   21  cp httpd.conf /etc/apache2/sites-enabled/nagios.conf
   22  cd ..
   23  cp contrib/eventhandlers /usr/local/nagios/libexec/
   24  cp -R contrib/eventhandlers /usr/local/nagios/libexec/
   25  chown -R nagios:nagios /usr/local/nagios/libexec/
   26  a2enmod rewrite
   27  a2enmod cgi
   28  service apache2 restart
   29  htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
   30  /etc/init.d/nagios restart
   35  cd /usr/local/nagios/var/
   39  systemctl enable nagios.service
   40  /etc/init.d/nagios start
   41  /etc/init.d/nagios status
   42  cd
   44  tar -xzvf nagios-plugins-2.0.3.tar.gz
   45  ./configure --with-nagios-user=nagios --with-nagios-group=nagios
   46  cd nagios-plugins-2.0.3/
   47  ./configure --with-nagios-user=nagios --with-nagios-group=nagios
   48  make
   49  make install