nagios log server alert configuration

[geremew]

I have Nagios log server, and need to receive alert notification for critical events from all network devices and servers,i have tried to integrate with Nagiosxi and receive some alert,
10.x.x.x is our NagiosXi, which is integrated with the log server. Dhcp event occurs on an other server(10.y.y.y), but from the email it did not indicate as dhcp critical event happened on 10.y.y.y server, rather it indicates as it happens on 10.x.x.x which is our NagiosXi (monitoring server.)
And it would be also difficult to configure alert for every events,
Could you help me on such issues please?
here is the sample alert from Nagiosxi for DHCP events

Alert from Nagios monitoring server

Nagios has detected a problem with this service.


Service: Dhcp
Host: 10.x.x.x
Address: 10.x.x.x
State: CRITICAL
Error description:
CRITICAL: 374 matching entries found
Date/Time: 2019-03-07 09:39:47

[npolovenko]

Hello, @geremew. What particular logs are you importing to the log server from XI and how are you importing these logs? Could you upload the logstash configuration in this thread?

Untitled.png

The issue could be related to logstash not properly parsing hostnames from the logs.

[geremew]

Hello dears, here is the logstash configuration.



#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Sat, 09 Mar 2019 10:34:51 +0300
#

#
# Global inputs
#

input {
syslog {
type => 'syslog'
port => 5544
}
tcp {
type => 'eventlog'
port => 3515
codec => json
}
tcp {
type => 'import_raw'
tags => 'import_raw'
port => 2056
}
tcp {
type => 'import_json'
tags => 'import_json'
port => 2057
codec => json
}
}

#
# Local inputs
#



#
# Global filters
#

filter {
if [program] == 'apache_access' {
grok {
match => [ 'message', '%{COMBINEDAPACHELOG}']
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM dd HH:mm:ss', 'ISO8601' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}

if [program] == 'apache_error' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}
}

#
# Local filters
#



#
# Global outputs
#



#
# Local outputs
#

[npolovenko]

@geremew, Could you clarify, are you importing nagios.log files from the XI server to the Log Server? Are you using the Syslog input on the LS? Can you PM me one of the nagios.log files you're sending?

[geremew]

we have integrated NagiosXi with log server. We are importing from log server to the NagiosXi to get alert notification by using NRDP.
i am afraid,I am not clear for your question about syslog input.

here is the the log file message.
"CISE_System_Statistics 0000063923 2 1 PID: 12983\; Wifi Setup Helper Container=disabled\; pxGrid Infrastructure Service=disabled\; pxGrid Publisher Subscriber Service=disabled\; pxGrid Connection Manager=disabled\; pxGrid Controller=disabled\; PassiveID WMI Service=disabled\; PassiveID Syslog Service=disabled\; PassiveID API Service=disabled\; PassiveID Agent Service=disabled\; PassiveID Endpoint Service=disabled\; PassiveID SPAN Service=disabled\; DHCP Server (dhcpd)=disabled\; DNS Server (named)=disabled,"

[cdienger]

To clarify - you're running the check_nagioslogserver.php plugin on the XI server, correct? This plugin will query a NLS server to find events and will trigger a WARNING or CRITICAL if the number of returned events match a threshold given to the plugin. We would expect 10.x.x.x to be the IP address of the NLS server - can you doublecheck this value and also the ip address of the XI and NLS machine? This would be expected behavior. While the email will not contain the IP address of the DHCP server, the query can be tailor to only search for events from the DHCP server. To help with that we would need a copy of the query that is being run now and a screenshot from the NLS dashboard showing the details of the DHCP events you wish to monitor.

[geremew]

10.x.x.x is the IP address of the NagiosXi server.

the query is the following:

{
"name": "DHCP",
"raw": "{\"query\":{\"filtered\":{\"query\":{\"bool\":{\"should\":[{\"query_string\":{\"query\":\"*dhcp\"}},{\"query_string\":{\"query\":\"*\"}}]}},\"filter\":{\"bool\":{\"must\":[{\"range\":{\"@timestamp\":{\"from\":1551700026572,\"to\":1551786426573}}}]}}}}}",
"services": "{\"query\":{\"list\":{\"0\":{\"query\":\"*dhcp\",\"alias\":\"\",\"color\":\"#4D89F9\",\"id\":0,\"pin\":false,\"type\":\"lucene\",\"enable\":true},\"1\":{\"id\":1,\"color\":\"#EAB839\",\"alias\":\"\",\"pin\":false,\"type\":\"lucene\",\"enable\":true,\"query\":\"*\"}},\"ids\":[0,1]},\"filter\":{\"list\":{\"0\":{\"type\":\"time\",\"field\":\"@timestamp\",\"from\":\"now-24h\",\"to\":\"now\",\"mandate\":\"must\",\"active\":true,\"alias\":\"\",\"id\":0}},\"ids\":[0]}}",
"created_by": "nagiosadmin",
"created_id": "1",
"show_everyone": 1,
"imported": 0
}


==========================================================================================================================

and the events is the following.
"CISE_System_Statistics 0000088279 2 1 PID: 12983\; Wifi Setup Helper Container=disabled\; pxGrid Infrastructure Service=disabled\; pxGrid Publisher Subscriber Service=disabled\; pxGrid Connection Manager=disabled\; pxGrid Controller=disabled\; PassiveID WMI Service=disabled\; PassiveID Syslog Service=disabled\; PassiveID API Service=disabled\; PassiveID Agent Service=disabled\; PassiveID Endpoint Service=disabled\; PassiveID SPAN Service=disabled\; DHCP Server (dhcpd)=disabled\; DNS Server (named)=disabled, "

[cdienger]

Please PM me a profile from the XI machine. It can be collected from Admin > System Config > System Profile > Download Profile.

[geremew]

helle dears,
I have attached the profile from XI machine.
regards

[cdienger]

From where are you running the queries? The profile provided doesn't seem to have a query for dhcp logs. If I missed it, please let us know what the name of the check is in XI.