Page 1 of 2
check_log is not collecting queried string
Posted: Tue May 11, 2021 11:58 am
by kn00567386
Hello,
I am using check_log to query string "AMQP server on 127.0.0.1:5671 is unreachable". But it is giving output other that queried string. What should be wrong. Please help. I am suspecting it is taking each word from string and showing output. How to fix this.
Below is command used for it.
--->
/usr/local/nagios/libexec/check_nrpe -H <IP> -t 30 -c check_log -a '-F /var/log/neutron/server.log -O /usr/lib64/nagios/plugins/500_log.txt -q 'AMQP server on 127.0.0.1:5671 is unreachable''
(4) < 2021-05-11 15:49:55.325 12342 DEBUG amqp [req-34a805d4-1b5f-4210-be2a-157cd8148a64 - - - - -] Start from server, version: 0.9, properties: {'information': 'Licensed under the MPL. See
http://www.rabbitmq.com/', 'product': 'RabbitMQ', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'exchange_exchange_bindings': True, 'connection.blocked': True, 'authentication_failure_close': True, 'direct_reply_to': True, 'basic.nack': True, 'per_consumer_qos': True, 'consumer_priorities': True, 'consumer_cancel_notify': True, 'publisher_confirms': True}, 'cluster_name': 'rabbit@ifra04-pvc01', 'platform': 'Erlang/OTP', 'version': '3.6.6'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: [u'en_US'] _on_start /usr/lib/python2.7/site-packages/amqp/connection.py:369|match=4;;;0
Re: check_log is not collecting queried string
Posted: Wed May 12, 2021 11:54 am
by ssax
The output is likely getting truncated so you're not seeing all 4 entries or the entirety of the messages.
What version of the remote NRPE agent are you running?
Code: Select all
/usr/local/nagios/libexec/check_nrpe -H X.X.X.X
Re: check_log is not collecting queried string
Posted: Fri May 14, 2021 8:36 am
by kn00567386
Please refer agent version below.
/usr/local/nagios/libexec/check_nrpe -H 192.168.4.136
NRPE v4.0.3
My worried point is why its not reporting only for queried string "AMQP server on 127.0.0.1:5671 is unreachable".
I dont see this string is occur and displayed in output. is it matching each word/letter for query string.
Re: check_log is not collecting queried string
Posted: Mon May 17, 2021 7:10 am
by kn00567386
ssax wrote:The output is likely getting truncated so you're not seeing all 4 entries or the entirety of the messages.
What version of the remote NRPE agent are you running?
Code: Select all
/usr/local/nagios/libexec/check_nrpe -H X.X.X.X
Please refer agent version below.
/usr/local/nagios/libexec/check_nrpe -H 192.168.4.136
NRPE v4.0.3
My worried point is why its not reporting only for queried string "AMQP server on 127.0.0.1:5671 is unreachable".
I dont see this string is occur and displayed in output. is it matching each word/letter for query string.
Re: check_log is not collecting queried string
Posted: Mon May 17, 2021 10:27 am
by benjaminsmith
Hi,
I'm helping Sean out with this topic as he is out today.
Can you run the plugin locally (on the remote host) as the nagios user. If it's working locally, then this likely an issue with passing arguments.
Please note, the plugin only scans for new entries, it may be necessary to manually add the "AMQP server on 127.0.0.1:5671 is unreachable" to the end of the log for testing purposes.
Regards,
Benjamin
Re: check_log is not collecting queried string
Posted: Wed May 19, 2021 7:03 am
by kn00567386
Strange thing is we don't see query string neither in source log file or old log file. Still we do see alerts for it.
[root@- ~]# grep “AMQP server on 127.0.0.1:5671 is unreachable” /var/log/neutron/server.log
[root@- ~]# grep “AMQP server on 127.0.0.1:5671 is unreachable” /usr/lib64/nagios/plugins/500_log.txt
Re: check_log is not collecting queried string
Posted: Wed May 19, 2021 5:07 pm
by benjaminsmith
Hi kn00567386,
That's very odd. When you run this locally, without the string in the file, are you getting valid output? Can you upload the nrpe.cfg file from the remote system, I'd like to see the command definition.
Thanks,
Benjamin
Re: check_log is not collecting queried string
Posted: Thu May 20, 2021 10:05 am
by kn00567386
find the attached nrpe.cfg file.I tried to run it frequently on locally and i dont see it unwated error.
[root@-~]# for i in 1 2 3 4 5 6 7 8 9 10
> do
> /usr/lib64/nagios/plugins/check_log -F /var/log/neutron/server.log -O /usr/lib64/nagios/plugins/500_log.txt -q ‘AMQP server on^C27.0.0.1:5671 is unreachable’
> done
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Log check ok - 0 pattern matches found|match=0;;;0
Re: check_log is not collecting queried string
Posted: Thu May 20, 2021 4:32 pm
by benjaminsmith
Hi
@kn00567386,
A couple of items we noticed. One is that when running the plugin locally you are logged in as root, try logging in as the nagios user account,
su - nagios, it's possible that this user cannot write an output file /usr/lib64/nagios/plugins/500_log.txt. Also, try using another file just to clear this out and start fresh.
Next, when running the check command from the XI server, enclose the query in double quotes and the whole argument in single quotes. For example:
Code: Select all
-a '-F /var/log/neutron/server.log -O /usr/lib64/nagios/plugins/500_log.txt -q "AMQP server on 127.0.0.1:5671 is unreachable"'
Let us know if that resolves the error.
Thanks,
Benjamin
Re: check_log is not collecting queried string
Posted: Thu May 20, 2021 4:32 pm
by ssax
Please run these commands as root on the system and send the full output:
Code: Select all
touch /tmp/testing.txt
/usr/local/nagios/libexec/check_log -F /tmp/testing.txt -O /tmp/testing.txt.log -q 'AMQP server on 127.0.0.1:5671 is unreachable'
echo 'test' >> /tmp/testing.txt
/usr/local/nagios/libexec/check_log -F /tmp/testing.txt -O /tmp/testing.txt.log -q 'AMQP server on 127.0.0.1:5671 is unreachable'
echo 'AMQP server on 127.0.0.1:5671 is unreachable' >> /tmp/testing.txt
/usr/local/nagios/libexec/check_log -F /tmp/testing.txt -O /tmp/testing.txt.log -q 'AMQP server on 127.0.0.1:5671 is unreachable'
It should look like this:
Code: Select all
[root@c77 ~]# touch /tmp/testing.txt
[root@c77 ~]# /usr/local/nagios/libexec/check_log -F /tmp/testing.txt -O /tmp/testing.txt.log -q 'AMQP server on 127.0.0.1:5671 is unreachable'
Log check data initialized...
[root@c77 ~]# echo 'test' >> /tmp/testing.txt
[root@c77 ~]# /usr/local/nagios/libexec/check_log -F /tmp/testing.txt -O /tmp/testing.txt.log -q 'AMQP server on 127.0.0.1:5671 is unreachable'
Log check ok - 0 pattern matches found|match=0;;;0
[root@c77 ~]# echo 'AMQP server on 127.0.0.1:5671 is unreachable' >> /tmp/testing.txt
[root@c77 ~]# /usr/local/nagios/libexec/check_log -F /tmp/testing.txt -O /tmp/testing.txt.log -q 'AMQP server on 127.0.0.1:5671 is unreachable'
(1) < AMQP server on 127.0.0.1:5671 is unreachable|match=1;;;0
What I was saying before is that this is not the entirely of that log messages:
Code: Select all
2021-05-11 15:49:55.325 12342 DEBUG amqp [req-34a805d4-1b5f-4210-be2a-157cd8148a64 - - - - -] Start from server, version: 0.9, properties: {'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', 'product': 'RabbitMQ', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'exchange_exchange_bindings': True, 'connection.blocked': True, 'authentication_failure_close': True, 'direct_reply_to': True, 'basic.nack': True, 'per_consumer_qos': True, 'consumer_priorities': True, 'consumer_cancel_notify': True, 'publisher_confirms': True}, 'cluster_name': 'rabbit@ifra04-pvc01', 'platform': 'Erlang/OTP', 'version': '3.6.6'}, mechanisms: ['PLAIN', 'AMQPLAIN'], locales: [u'en_US'] _on_start /usr/lib/python2.7/site-packages/amqp/connection.py:369
There is likely more lines in that log and it's being cutoff, can you find that log entry in the log file and show us the entire log line from that file?