Page 4 of 5

Re: All Log Entries on same facility/priority/severity

Posted: Mon Dec 08, 2014 3:13 pm
by tmckay
i changed the owner and perms, which changed the modified date to today, however, when i applied the configuration, it didnt change the modified date. It is having an effect on the input, as i quit receiving logs from my ASA, as soon as i apply it. i don't really care about the ASA logs at this point, but i did issue a "verify" against the Global Config, and it came back with this error. The filter i have active is the one suggested above.

Re: All Log Entries on same facility/priority/severity

Posted: Mon Dec 08, 2014 5:51 pm
by sreinhardt
Could you post the filters you have again from the flat file. If they have changed to hold a 133 lines, per the error, they are vastly different than what was uploaded before and likely contain what we need to get this working, minus the current error. :) At the moment though, I can't see what is wrong with the filters file, as it only has about 20 lines.

Re: All Log Entries on same facility/priority/severity

Posted: Wed Dec 17, 2014 9:32 am
by tmckay
The files from /usr/local/nagioslogserver/logstash/etc/conf.d are attached.

Re: All Log Entries on same facility/priority/severity

Posted: Wed Dec 17, 2014 1:53 pm
by scottwilkerson
Actually we need you to go to Administration -> Global configuration and click "View" -> "All files combined", then post that.

thanks!

Re: All Log Entries on same facility/priority/severity

Posted: Wed Dec 17, 2014 2:41 pm
by tmckay
file attached, and contents listed below.




#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 17 Dec 2014 14:39:46 -0500
#

#
# Global Configuration
#

input {
tcp {
type => 'eventlog'
port => 3515
codec => json {
charset => 'CP1252'
}
}
tcp {
type => 'import_raw'
tags => 'import_raw'
port => 2056
}
tcp {
type => 'import_json'
tags => 'import_json'
port => 2057
codec => json
}
udp {
port => 514
type => "cisco"
}
}

filter {
if [type] == "cisco" {
grok {
match => ["message", "^<%{POSINT:syslog_pri}>%{POSINT:seqnum}: %{WORD:host}: %{CISCOTIMESTAMP:timestamp}: %%{CISCOTAG:ciscotag}: %{GREEDYDATA:cisco_message}"]
}
# Parse the syslog severity and facility
syslog_pri { }

# Extract fields from the each of the detailed message types
# The patterns provided below are included in core of LogStash 1.2.0.
grok {
match => [
"cisco_message", "%{CISCOFW106001}",
"cisco_message", "%{CISCOFW106006_106007_106010}",
"cisco_message", "%{CISCOFW106014}",
"cisco_message", "%{CISCOFW106015}",
"cisco_message", "%{CISCOFW106021}",
"cisco_message", "%{CISCOFW106023}",
"cisco_message", "%{CISCOFW106100}",
"cisco_message", "%{CISCOFW110002}",
"cisco_message", "%{CISCOFW302010}",
"cisco_message", "%{CISCOFW302013_302014_302015_302016}",
"cisco_message", "%{CISCOFW302020_302021}",
"cisco_message", "%{CISCOFW305011}",
"cisco_message", "%{CISCOFW313001_313004_313008}",
"cisco_message", "%{CISCOFW313005}",
"cisco_message", "%{CISCOFW402117}",
"cisco_message", "%{CISCOFW402119}",
"cisco_message", "%{CISCOFW419001}",
"cisco_message", "%{CISCOFW419002}",
"cisco_message", "%{CISCOFW500004}",
"cisco_message", "%{CISCOFW602303_602304}",
"cisco_message", "%{CISCOFW710001_710002_710003_710005_710006}",
"cisco_message", "%{CISCOFW713172}",
"cisco_message", "%{CISCOFW733100}"
]
}
date {
match => ["timestamp",
"MMM dd HH:mm:ss",
"MMM d HH:mm:ss",
"MMM dd yyyy HH:mm:ss",
"MMM d yyyy HH:mm:ss"
]
}

}

#
# Local Configuration
#

Re: All Log Entries on same facility/priority/severity

Posted: Wed Dec 17, 2014 6:09 pm
by scottwilkerson
In your Cisco filter you are missing a ending }

replace with

Code: Select all

if [type] == "cisco" {
    grok {
      match => ["message", "^<%{POSINT:syslog_pri}>%{POSINT:seqnum}: %{WORD:host}: %{CISCOTIMESTAMP:timestamp}: %%{CISCOTAG:ciscotag}: %{GREEDYDATA:cisco_message}"]
    }
    # Parse the syslog severity and facility
    syslog_pri { }
    
    # Extract fields from the each of the detailed message types
    # The patterns provided below are included in core of LogStash 1.2.0.
    grok {
    match => [
    "cisco_message", "%{CISCOFW106001}",
    "cisco_message", "%{CISCOFW106006_106007_106010}",
    "cisco_message", "%{CISCOFW106014}",
    "cisco_message", "%{CISCOFW106015}",
    "cisco_message", "%{CISCOFW106021}",
    "cisco_message", "%{CISCOFW106023}",
    "cisco_message", "%{CISCOFW106100}",
    "cisco_message", "%{CISCOFW110002}",
    "cisco_message", "%{CISCOFW302010}",
    "cisco_message", "%{CISCOFW302013_302014_302015_302016}",
    "cisco_message", "%{CISCOFW302020_302021}",
    "cisco_message", "%{CISCOFW305011}",
    "cisco_message", "%{CISCOFW313001_313004_313008}",
    "cisco_message", "%{CISCOFW313005}",
    "cisco_message", "%{CISCOFW402117}",
    "cisco_message", "%{CISCOFW402119}",
    "cisco_message", "%{CISCOFW419001}",
    "cisco_message", "%{CISCOFW419002}",
    "cisco_message", "%{CISCOFW500004}",
    "cisco_message", "%{CISCOFW602303_602304}",
    "cisco_message", "%{CISCOFW710001_710002_710003_710005_710006}",
    "cisco_message", "%{CISCOFW713172}",
    "cisco_message", "%{CISCOFW733100}"
    ]
    }
    date {
    match => ["timestamp",
    "MMM dd HH:mm:ss",
    "MMM d HH:mm:ss",
    "MMM dd yyyy HH:mm:ss",
    "MMM d yyyy HH:mm:ss"
    ]
    }
}

Re: All Log Entries on same facility/priority/severity

Posted: Fri Dec 19, 2014 2:25 pm
by tmckay
That allowed the configuration to verify after i added that. i'll check and see if we are receiving correct facility/priority/severity notices after applying this.

Re: All Log Entries on same facility/priority/severity

Posted: Fri Dec 19, 2014 3:40 pm
by abrist
Great, keep us in the loop!

Re: All Log Entries on same facility/priority/severity

Posted: Fri Dec 26, 2014 10:14 am
by tmckay
I'm still having the same result. Everything is coming through as severity 5, regardless of how it is coded in the Cisco syslog message. i'm attaching an expanded view of how the entry appears.

Re: All Log Entries on same facility/priority/severity

Posted: Fri Dec 26, 2014 11:11 am
by scottwilkerson
The current message (seen in the message field) doesn't match the grok pattern specified at the beginning of your cisco filter

Code: Select all

^<%{POSINT:syslog_pri}>%{POSINT:seqnum}: %{WORD:host}: %{CISCOTIMESTAMP:timestamp}: %%{CISCOTAG:ciscotag}: %{GREEDYDATA:cisco_message}
Which is why the item has a tag added "_grokparsefailure"

I had given the example earlier based on a log line you have given earlier in the thread, however it would be best to test these on
http://grokdebug.herokuapp.com/