All Log Entries on same facility/priority/severity

[tmckay]

i changed the owner and perms, which changed the modified date to today, however, when i applied the configuration, it didnt change the modified date. It is having an effect on the input, as i quit receiving logs from my ASA, as soon as i apply it. i don't really care about the ASA logs at this point, but i did issue a "verify" against the Global Config, and it came back with this error. The filter i have active is the one suggested above.

[sreinhardt]

Could you post the filters you have again from the flat file. If they have changed to hold a 133 lines, per the error, they are vastly different than what was uploaded before and likely contain what we need to get this working, minus the current error. At the moment though, I can't see what is wrong with the filters file, as it only has about 20 lines.

[tmckay]

The files from /usr/local/nagioslogserver/logstash/etc/conf.d are attached.

[scottwilkerson]

Actually we need you to go to Administration -> Global configuration and click "View" -> "All files combined", then post that.

thanks!

[tmckay]

file attached, and contents listed below.




#
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Wed, 17 Dec 2014 14:39:46 -0500
#

#
# Global Configuration
#

input {
tcp {
type => 'eventlog'
port => 3515
codec => json {
charset => 'CP1252'
}
}
tcp {
type => 'import_raw'
tags => 'import_raw'
port => 2056
}
tcp {
type => 'import_json'
tags => 'import_json'
port => 2057
codec => json
}
udp {
port => 514
type => "cisco"
}
}

filter {
if [type] == "cisco" {
grok {
match => ["message", "^<%{POSINT:syslog_pri}>%{POSINT:seqnum}: %{WORD:host}: %{CISCOTIMESTAMP:timestamp}: %%{CISCOTAG:ciscotag}: %{GREEDYDATA:cisco_message}"]
}
# Parse the syslog severity and facility
syslog_pri { }

# Extract fields from the each of the detailed message types
# The patterns provided below are included in core of LogStash 1.2.0.
grok {
match => [
"cisco_message", "%{CISCOFW106001}",
"cisco_message", "%{CISCOFW106006_106007_106010}",
"cisco_message", "%{CISCOFW106014}",
"cisco_message", "%{CISCOFW106015}",
"cisco_message", "%{CISCOFW106021}",
"cisco_message", "%{CISCOFW106023}",
"cisco_message", "%{CISCOFW106100}",
"cisco_message", "%{CISCOFW110002}",
"cisco_message", "%{CISCOFW302010}",
"cisco_message", "%{CISCOFW302013_302014_302015_302016}",
"cisco_message", "%{CISCOFW302020_302021}",
"cisco_message", "%{CISCOFW305011}",
"cisco_message", "%{CISCOFW313001_313004_313008}",
"cisco_message", "%{CISCOFW313005}",
"cisco_message", "%{CISCOFW402117}",
"cisco_message", "%{CISCOFW402119}",
"cisco_message", "%{CISCOFW419001}",
"cisco_message", "%{CISCOFW419002}",
"cisco_message", "%{CISCOFW500004}",
"cisco_message", "%{CISCOFW602303_602304}",
"cisco_message", "%{CISCOFW710001_710002_710003_710005_710006}",
"cisco_message", "%{CISCOFW713172}",
"cisco_message", "%{CISCOFW733100}"
]
}
date {
match => ["timestamp",
"MMM dd HH:mm:ss",
"MMM d HH:mm:ss",
"MMM dd yyyy HH:mm:ss",
"MMM d yyyy HH:mm:ss"
]
}

}

#
# Local Configuration
#

[scottwilkerson]

In your Cisco filter you are missing a ending }

replace with

Code: Select all

if [type] == "cisco" {
    grok {
      match => ["message", "^<%{POSINT:syslog_pri}>%{POSINT:seqnum}: %{WORD:host}: %{CISCOTIMESTAMP:timestamp}: %%{CISCOTAG:ciscotag}: %{GREEDYDATA:cisco_message}"]
    }
    # Parse the syslog severity and facility
    syslog_pri { }
    
    # Extract fields from the each of the detailed message types
    # The patterns provided below are included in core of LogStash 1.2.0.
    grok {
    match => [
    "cisco_message", "%{CISCOFW106001}",
    "cisco_message", "%{CISCOFW106006_106007_106010}",
    "cisco_message", "%{CISCOFW106014}",
    "cisco_message", "%{CISCOFW106015}",
    "cisco_message", "%{CISCOFW106021}",
    "cisco_message", "%{CISCOFW106023}",
    "cisco_message", "%{CISCOFW106100}",
    "cisco_message", "%{CISCOFW110002}",
    "cisco_message", "%{CISCOFW302010}",
    "cisco_message", "%{CISCOFW302013_302014_302015_302016}",
    "cisco_message", "%{CISCOFW302020_302021}",
    "cisco_message", "%{CISCOFW305011}",
    "cisco_message", "%{CISCOFW313001_313004_313008}",
    "cisco_message", "%{CISCOFW313005}",
    "cisco_message", "%{CISCOFW402117}",
    "cisco_message", "%{CISCOFW402119}",
    "cisco_message", "%{CISCOFW419001}",
    "cisco_message", "%{CISCOFW419002}",
    "cisco_message", "%{CISCOFW500004}",
    "cisco_message", "%{CISCOFW602303_602304}",
    "cisco_message", "%{CISCOFW710001_710002_710003_710005_710006}",
    "cisco_message", "%{CISCOFW713172}",
    "cisco_message", "%{CISCOFW733100}"
    ]
    }
    date {
    match => ["timestamp",
    "MMM dd HH:mm:ss",
    "MMM d HH:mm:ss",
    "MMM dd yyyy HH:mm:ss",
    "MMM d yyyy HH:mm:ss"
    ]
    }
}

[tmckay]

That allowed the configuration to verify after i added that. i'll check and see if we are receiving correct facility/priority/severity notices after applying this.

[abrist]

Great, keep us in the loop!

[tmckay]

I'm still having the same result. Everything is coming through as severity 5, regardless of how it is coded in the Cisco syslog message. i'm attaching an expanded view of how the entry appears.

[scottwilkerson]

The current message (seen in the message field) doesn't match the grok pattern specified at the beginning of your cisco filter

Code: Select all

^<%{POSINT:syslog_pri}>%{POSINT:seqnum}: %{WORD:host}: %{CISCOTIMESTAMP:timestamp}: %%{CISCOTAG:ciscotag}: %{GREEDYDATA:cisco_message}

Which is why the item has a tag added "_grokparsefailure"

I had given the example earlier based on a log line you have given earlier in the thread, however it would be best to test these on
http://grokdebug.herokuapp.com/