Unable to remove server

[danniiffxi]

I closed my previous thread too early.

I have followed the following document and everything has worked up until 'Delete Removed Instance' section
https://assets.nagios.com/downloads/nag ... luster.pdf

When i click the bin to delete


This happens, and it wont delete


Any suggestions?

[cdienger]

What version of NLS is this? What OS?

I'd like you to navigate to the page where the old node is seen and then run this on the command line:

Code: Select all

tail -f /var/log/httpd/*

and click the delete icon. Please provide anything that is generated in terminal when you take this action.

I'd also like to see a tcpdump taken while you do the same thing. This can be done with:

Code: Select all

yum -y install tcpdump
tcpdump -s 0 -i any -w output.pcap

Then click delete and use CTRL+C to stop it. You can PM me the log files as well as the output.pcap created by the tcpdump.

Finally, provide me with a copy of the nagioslogserver index and I'll be able to provide you commands to remove the entry from the command line. This can be exported to a file by running:

Code: Select all

curl -XPOST http://localhost:9200/nagioslogserver/_export?path=/tmp/nagioslogserver.tar.gz

The file it creates and that we'd like to see is /tmp/nagioslogserver.tar.gz.

[danniiffxi]

Hi cdienger

NL 2.1.5
Centos 6.9

I have also tried clearing internet cache, and using different browsers, same error.

Here is a copy of all logs in /var/log/httpd/ while executing the command to remove the instance

The output and other tar.gz I will PM to you now.

Everything below have been sanitised, real server names and IP changed

Thank you

Code: Select all

[root@naglp01 httpd]# tail -f /var/log/httpd/error_log
[Wed Apr 15 07:44:45 2020] [notice] caught SIGTERM, shutting down
[Wed Apr 15 07:45:40 2020] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Apr 15 07:45:40 2020] [notice] Digest: generating secret for digest authentication ...
[Wed Apr 15 07:45:40 2020] [notice] Digest: done
[Wed Apr 15 07:45:40 2020] [notice] Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips configured -- resuming normal operations
[Thu Apr 16 16:10:02 2020] [notice] caught SIGTERM, shutting down
[Thu Apr 16 16:10:55 2020] [notice] suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Thu Apr 16 16:10:55 2020] [notice] Digest: generating secret for digest authentication ...
[Thu Apr 16 16:10:55 2020] [notice] Digest: done
[Thu Apr 16 16:10:55 2020] [notice] Apache/2.2.15 (Unix) DAV/2 PHP/5.3.3 mod_ssl/2.2.15 OpenSSL/1.0.1e-fips configured -- resuming normal operations

ssl_request_log

Code: Select all

[root@naglp01 httpd]# tail -f /var/log/httpd/ssl_request_log
[17/Apr/2020:08:02:41 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/ HTTP/1.1" 30684
[17/Apr/2020:08:02:43 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/admin HTTP/1.1" 13626
[17/Apr/2020:08:02:46 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/media/css/elastichq.css?ver=2.1.5 HTTP/1.1" 9647
[17/Apr/2020:08:02:46 +0100] 132.162.177.239  TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "POST /nagioslogserver/api/system/status HTTP/1.1" 515
[17/Apr/2020:08:03:06 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/admin/cluster HTTP/1.1" 41554
[17/Apr/2020:08:03:07 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "POST /nagioslogserver/api/system/status HTTP/1.1" 515
[17/Apr/2020:08:03:09 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/admin/instance_status HTTP/1.1" 20954
[17/Apr/2020:08:03:10 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/media/icons/exclamation.png HTTP/1.1" 701
[17/Apr/2020:08:03:10 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "POST /nagioslogserver/api/system/status HTTP/1.1" 515
[17/Apr/2020:08:03:13 +0100] 132.162.177.239 TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 "GET /nagioslogserver/configure/remove_instance/aab1eafb-e8a4-4db7-a4d2-212dc4e48e12/YWRtaW4vaW5zdGFuY2Vfc3RhdHVz HTTP/1.1" -

Code: Select all

[root@naglp01 httpd]# tail -f /var/log/httpd/ssl_access_log
132.162.177.239 - - [17/Apr/2020:08:02:41 +0100] "GET /nagioslogserver/ HTTP/1.1" 200 30684
132.162.177.239 - - [17/Apr/2020:08:02:43 +0100] "GET /nagioslogserver/admin HTTP/1.1" 200 13626
132.162.177.239 - - [17/Apr/2020:08:02:46 +0100] "GET /nagioslogserver/media/css/elastichq.css?ver=2.1.5 HTTP/1.1" 200 9647
132.162.177.239 - - [17/Apr/2020:08:02:46 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 515
132.162.177.239 - - [17/Apr/2020:08:03:06 +0100] "GET /nagioslogserver/admin/cluster HTTP/1.1" 200 41554
132.162.177.239 - - [17/Apr/2020:08:03:07 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 515
132.162.177.239 - - [17/Apr/2020:08:03:09 +0100] "GET /nagioslogserver/admin/instance_status HTTP/1.1" 200 20954
132.162.177.239 - - [17/Apr/2020:08:03:10 +0100] "GET /nagioslogserver/media/icons/exclamation.png HTTP/1.1" 200 701
132.162.177.239 - - [17/Apr/2020:08:03:10 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 515
132.162.177.239 - - [17/Apr/2020:08:03:13 +0100] "GET /nagioslogserver/configure/remove_instance/aab1eafb-e8a4-4db7-a4d2-212dc4e48e12/YWRtaW4vaW5zdGFuY2Vfc3RhdHVz HTTP/1.1" 500 -

ssl_error_log

Code: Select all

[root@naglp01 httpd]# tail -f /var/log/httpd/ssl_error_log
[Tue Apr 14 11:17:55 2020] [warn] RSA server certificate CommonName (CN) `nagios-log' does NOT match server name!?
[Tue Apr 14 11:17:55 2020] [warn] RSA server certificate CommonName (CN) `nagios-log' does NOT match server name!?
[Wed Apr 15 07:45:40 2020] [warn] RSA server certificate CommonName (CN) `nagios-log' does NOT match server name!?
[Wed Apr 15 07:45:40 2020] [warn] RSA server certificate CommonName (CN) `nagios-log' does NOT match server name!?
[Thu Apr 16 16:10:55 2020] [warn] RSA server certificate CommonName (CN) `nagios-log' does NOT match server name!?
[Thu Apr 16 16:10:55 2020] [warn] RSA server certificate CommonName (CN) `nagios-log' does NOT match server name!?
[Fri Apr 17 08:02:13 2020] [error] [client 132.162.177.239 ] PHP Fatal error:  Call to undefined method Configure::config_item() in /nagios/var/www/html/nagioslogserver/application/controllers/Configure.php on line 0, referer: https://naglp01/nagioslogserver/login?redirect=configure/remove_instance/aab1eafb-e8a4-4db7-a4d2-212dc4e48e12/YWRtaW4vaW5zdGFuY2Vfc3RhdHVz
[Fri Apr 17 08:02:17 2020] [error] [client 132.162.177.239 ] PHP Fatal error:  Call to undefined method Configure::config_item() in /nagios/var/www/html/nagioslogserver/application/controllers/Configure.php on line 0
[Fri Apr 17 08:02:18 2020] [error] [client 132.162.177.239 ] PHP Fatal error:  Call to undefined method Configure::config_item() in /nagios/var/www/html/nagioslogserver/application/controllers/Configure.php on line 0
[Fri Apr 17 08:03:13 2020] [error] [client 132.162.177.239 ] PHP Fatal error:  Call to undefined method Configure::config_item() in /nagios/var/www/html/nagioslogserver/application/controllers/Configure.php on line 0, referer: https://naglp01/nagioslogserver/admin/instance_status

access_log

Code: Select all

[root@naglp01 ~]# tail -f /var/log/httpd/access_log
132.162.177.239 - - [16/Apr/2020:16:52:31 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 87 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
132.162.177.239 - - [16/Apr/2020:16:52:31 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 82 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
132.162.177.239 - - [16/Apr/2020:16:52:53 +0100] "POST /nagioslogserver/api/system/stop HTTP/1.1" 200 51 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
132.162.177.239 - - [16/Apr/2020:16:52:55 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 106 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
132.162.177.239 - - [16/Apr/2020:16:52:55 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 87 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
132.162.177.239 - - [16/Apr/2020:16:53:04 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 87 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
132.162.177.239 - - [16/Apr/2020:16:53:04 +0100] "POST /nagioslogserver/api/system/status HTTP/1.1" 200 106 "http://lxnaglp01/nagioslogserver/admin/system" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
172.31.32.140 - - [16/Apr/2020:18:07:12 +0100] "GET / HTTP/1.1" 302 285 "-" "Mozilla / 5.0(Windows NT 10.0; &) Gecko / 20100101 Firefox / 62.0"
172.31.32.140 - - [16/Apr/2020:18:19:13 +0100] "GET / HTTP/1.1" 302 285 "-" "Mozilla / 5.0(Windows NT 10.0; &) Gecko / 20100101 Firefox / 62.0"
132.162.177.239 - - [17/Apr/2020:08:01:44 +0100] "HEAD /nagioslogserver/configure/remove_instance/aab1eafb-e8a4-4db7-a4d2-212dc4e48e12/YWRtaW4vaW5zdGFuY2Vfc3RhdHVz HTTP/1.1" 303 - "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"

[cdienger]

Thanks for the data. I'll look into the cause but in the meantime I've sent you a command to remove the node.