Page 1 of 2

CGI Error created by "enable_environment_macros=1"

Posted: Thu Jul 09, 2015 4:57 am
by uragusan
We have install Nagios 4.08 on Debian 8U1 with Apache 2.4.
For our notifications we use nagios_mail.php Script by Otto Berger that required to enable “enable_environment_macros“ in Nagios.cfg.
After activation we get the CGI error in Nagios-error.log.

Code: Select all

[Thu Jul 09 11:39:06.910982 2015] [cgi:error] [pid 3549] [client 10.0.10.197:61954] Script timed out before returning headers: cmd.cgi, referer: https://de-h-nag01.domain.de/nagios/cgi-bin/cmd.cgi?cmd_typ=7&host=de-h-nas02&service=DSM+System+Status&force_check

[Thu Jul 09 11:44:06.926015 2015] [cgi:warn] [pid 3549] [client 10.0.10.197:61954] AH01220: Timeout waiting for output from CGI script /usr/local/nagios/sbin/cmd.cgi, referer: https://de-h-nag01.domain.de/nagios/cgi-bin/cmd.cgi?cmd_typ=7&host=de-h-nas02&service=DSM+System+Status&force_check
Have anyone idea with what it may be related ?

Re: CGI Error created by "enable_environment_macros=1"

Posted: Thu Jul 09, 2015 10:47 am
by abrist
The environment may be sooooo big that the cgi is timing out waiting for it to return. You may need to increase the apache TimeOut directive value:
http://httpd.apache.org/docs/2.0/mod/core.html#timeout
This may cause issues with long waits with the core frontend though.

Re: CGI Error created by "enable_environment_macros=1"

Posted: Fri Jul 10, 2015 4:05 am
by uragusan
Hi abrist, thanks for the answer.
I have change timeout to 600 sec for testing. After forcing the Service check and wait followed, nothing happens, the check was not started and I have become same CGI error in the Logs.
On top command we do not see high CPU or RAM usage.
Nagios 3.2.3 in Productive Environment have not this big Problem.

Re: CGI Error created by "enable_environment_macros=1"

Posted: Fri Jul 10, 2015 9:27 am
by jdalrymple
Maybe apparmor or selinux is giving you some troubles?

I've had no issue when enabling envs on any of my many 4.x systems that I've setup, although I use strictly CentOS, and disabling selinux is the first thing I do.

Re: CGI Error created by "enable_environment_macros=1"

Posted: Fri Jul 10, 2015 10:57 am
by uragusan
Hello jdalrymple,
Apparmor so as selinux do not run on our Debian 8 Nagios Server.

Re: CGI Error created by "enable_environment_macros=1"

Posted: Fri Jul 10, 2015 11:47 am
by abrist
First turn on external command logging if it is disabled:

Code: Select all

sed -i 's/log_external_commands=.*/log_external_commands=1/g' /usr/local/nagios/etc/nagios.cfg
Verify:

Code: Select all

grep log_external /usr/local/nagios/etc/nagios.cfg
Restart Nagios:

Code: Select all

service nagios restart
Then try submitting the query directly to the cgi (instead of through apache):

Code: Select all

REMOTE_USER=nagiosadmin REQUEST_METHOD=GET QUERY_STRING="cmd_typ=7&host=de-h-nas02&service=DSM+System+Status&force_check" /usr/local/nagios/sbin/cmd.cgi > /tmp/cgi_test.txt

Code: Select all

echo "------------------" >> /tmp/cgi_test.txt
Now, try finishing the the command and submitting the forced check:

Code: Select all

REMOTE_USER=nagiosadmin REQUEST_METHOD=GET QUERY_STRING="cmd_typ=7&cmd_mod=2&host=de-h-nas02&service=DSM+System+Status&force_check=on&start_time=$(date '+%m-%d-%Y %H:%M:%S')&btnSubmit=Commit" /usr/local/nagios/sbin/cmd.cgi >> /tmp/cgi_test.txt

Code: Select all

echo "------------------" >> /tmp/cgi_test.txt
And then get a tail of the nagios.log looking specifically for submitted external commands:

Code: Select all

grep -i EXTERNAL /usr/local/nagios/var/nagios.log | tail -10 >> /tmp/cgi_test.txt
Post the contents of /tmp/cgi_test.txt here in code wraps.

Re: CGI Error created by "enable_environment_macros=1"

Posted: Mon Jul 13, 2015 1:34 am
by uragusan
abrist wrote: Now, try finishing the the command and submitting the forced check:

Code: Select all

REMOTE_USER=nagiosadmin REQUEST_METHOD=GET QUERY_STRING="cmd_typ=7&cmd_mod=2&host=de-h-nas02&service=DSM+System+Status&force_check=on&start_time=$(date '+%m-%d-%Y %H:%M:%S')&btnSubmit=Commit" /usr/local/nagios/sbin/cmd.cgi >> /tmp/cgi_test.txt
This Command do not work with enabled environment macros, I get Log only if I set "enable_environment_macros=0".
Interesting too, nagios so as nagios.debug Logs are not more filed, after activated environment.

Re: CGI Error created by "enable_environment_macros=1"

Posted: Mon Jul 13, 2015 4:39 pm
by jdalrymple
I can't reproduce your scenario. How did this Nagios instance get installed?

Code: Select all

root@localhost:~# cat /etc/debian_version
8.1

Re: CGI Error created by "enable_environment_macros=1"

Posted: Tue Jul 14, 2015 1:19 am
by uragusan
I have take this tutorial.
http://www.2daygeek.com/nagios-4-0-8-in ... linuxmint/

With all this packages that we need for our checks.

Code: Select all

apt-get install wget build-essential apache2 php5-gd libgd2-xpm libgd2-xpm-dev libapache2-mod-php5 libperl-dev libssl-dev apache2-utils snmp libsnmp-dev php5-curl libconfig-general-perl php5-snmp curl libnet-dns-perl libdbd-sybase-perl libdbi-perl autoconf python libdatetime-perl make gcc libxml-simple-perl libio-socket-ssl-perl libnet-snmp-perl libsnmp-perl libsnmp-session-perl libconfig-general-perl libsnmp-dev libnagios-plugin-perl krb5-user libapache2-mod-auth-kerb cifs-utils python-pywbem

Re: CGI Error created by "enable_environment_macros=1"

Posted: Tue Jul 14, 2015 10:10 am
by jdalrymple
I'm not sure what broke for you, but if you don't have a lot of investment in the system already, I'd be happy to share my command history that got my system running. As a matter of fact I'll do it anyway.

I removed incorrect or unimportant lines for clarity's sake:

Code: Select all

    2  apt-get install build-essential php5-gd libgd2-xpm-dev libapache2-mod-php5 libperl-dev libssl-dev apache2-utils apache2
    3  useradd nagios && passwd nagios
    4  groupadd nagcmd
    5  usermod -a -G nagcmd nagios
    6  usermod -a -G nagcmd www-data
    7  wget http://sourceforge.net/projects/nagios/files/nagios-4.x/nagios-4.0.8/nagios-4.0.8.tar.gz
    8  wget http://nagios-plugins.org/download/nagios-plugins-2.0.3.tar.gz
    9  tar -xzvf nagios-4.0.8.tar.gz
   10  cd nagios-4.0.8/
   11  ./configure --with-command-group=nagcmd
   12  make all
   13  make install
   14  make install-init
   15  make install-commandmode
   16  make install-config
   17  make install-webconf
   19  cd sample-config/
   21  cp httpd.conf /etc/apache2/sites-enabled/nagios.conf
   22  cd ..
   23  cp contrib/eventhandlers /usr/local/nagios/libexec/
   24  cp -R contrib/eventhandlers /usr/local/nagios/libexec/
   25  chown -R nagios:nagios /usr/local/nagios/libexec/
   26  a2enmod rewrite
   27  a2enmod cgi
   28  service apache2 restart
   29  htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
   30  /etc/init.d/nagios restart
   35  cd /usr/local/nagios/var/
   39  systemctl enable nagios.service
   40  /etc/init.d/nagios start
   41  /etc/init.d/nagios status
   42  cd
   44  tar -xzvf nagios-plugins-2.0.3.tar.gz
   45  ./configure --with-nagios-user=nagios --with-nagios-group=nagios
   46  cd nagios-plugins-2.0.3/
   47  ./configure --with-nagios-user=nagios --with-nagios-group=nagios
   48  make
   49  make install