Page 2 of 3
Re: IP Charts with Geolocation
Posted: Fri Jul 17, 2015 1:02 pm
by networkeng
Ok, the verification failed. It is taking the first input fine.
Code: Select all
udp {
port => 5555
type => "cisco-asa"
}
However, it does not seem to like the second input:
Code: Select all
if type == "cisco-asa" {
# Split the syslog part and Cisco tag out of the message
grok {
match => ["message", "%{CISCO_TAGGED_SYSLOG} %{GREEDYDATA:cisco_message}"]
}
# Parse the syslog severity and facility
syslog_pri { }
# Parse the date from the "timestamp" field to the "@timestamp" field
date {
match => ["timestamp",
"MMM dd HH:mm:ss",
"MMM d HH:mm:ss",
"MMM dd yyyy HH:mm:ss",
"MMM d yyyy HH:mm:ss"
]
timezone => "America/New_York"
}
# Clean up redundant fields if parsing was successful
if "_grokparsefailure" not in [tags] {
mutate {
rename => ["cisco_message", "message"]
remove_field => ["timestamp"]
}
}
# Extract fields from the each of the detailed message types
# The patterns provided below are included in Logstash since 1.2.0
grok {
match => [
"message", "%{CISCOFW106001}",
"message", "%{CISCOFW106006_106007_106010}",
"message", "%{CISCOFW106014}",
"message", "%{CISCOFW106015}",
"message", "%{CISCOFW106021}",
"message", "%{CISCOFW106023}",
"message", "%{CISCOFW106100}",
"message", "%{CISCOFW110002}",
"message", "%{CISCOFW302010}",
"message", "%{CISCOFW302013_302014_302015_302016}",
"message", "%{CISCOFW302020_302021}",
"message", "%{CISCOFW305011}",
"message", "%{CISCOFW313001_313004_313008}",
"message", "%{CISCOFW313005}",
"message", "%{CISCOFW402117}",
"message", "%{CISCOFW402119}",
"message", "%{CISCOFW419001}",
"message", "%{CISCOFW419002}",
"message", "%{CISCOFW500004}",
"message", "%{CISCOFW602303_602304}",
"message", "%{CISCOFW710001_710002_710003_710005_710006}",
"message", "%{CISCOFW713172}",
"message", "%{CISCOFW733100}"
]
}
}
I have verified that it is listening on 5555, just can't get that second input to verify.
Re: IP Charts with Geolocation
Posted: Fri Jul 17, 2015 1:06 pm
by jolson
Ah, that is because it's not an input - but it's a filter. You will need to place your second block:
Code: Select all
if type == "cisco-asa" {
# Split the syslog part and Cisco tag out of the message
grok {
match => ["message", "%{CISCO_TAGGED_SYSLOG} %{GREEDYDATA:cisco_message}"]
}
# Parse the syslog severity and facility
syslog_pri { }
# Parse the date from the "timestamp" field to the "@timestamp" field
date {
match => ["timestamp",
"MMM dd HH:mm:ss",
"MMM d HH:mm:ss",
"MMM dd yyyy HH:mm:ss",
"MMM d yyyy HH:mm:ss"
]
timezone => "America/New_York"
}
# Clean up redundant fields if parsing was successful
if "_grokparsefailure" not in [tags] {
mutate {
rename => ["cisco_message", "message"]
remove_field => ["timestamp"]
}
}
# Extract fields from the each of the detailed message types
# The patterns provided below are included in Logstash since 1.2.0
grok {
match => [
"message", "%{CISCOFW106001}",
"message", "%{CISCOFW106006_106007_106010}",
"message", "%{CISCOFW106014}",
"message", "%{CISCOFW106015}",
"message", "%{CISCOFW106021}",
"message", "%{CISCOFW106023}",
"message", "%{CISCOFW106100}",
"message", "%{CISCOFW110002}",
"message", "%{CISCOFW302010}",
"message", "%{CISCOFW302013_302014_302015_302016}",
"message", "%{CISCOFW302020_302021}",
"message", "%{CISCOFW305011}",
"message", "%{CISCOFW313001_313004_313008}",
"message", "%{CISCOFW313005}",
"message", "%{CISCOFW402117}",
"message", "%{CISCOFW402119}",
"message", "%{CISCOFW419001}",
"message", "%{CISCOFW419002}",
"message", "%{CISCOFW500004}",
"message", "%{CISCOFW602303_602304}",
"message", "%{CISCOFW710001_710002_710003_710005_710006}",
"message", "%{CISCOFW713172}",
"message", "%{CISCOFW733100}"
]
}
}
In the 'fitlers' section. Please do so, remove the bad input, and run verify once more.
Re: IP Charts with Geolocation
Posted: Fri Jul 17, 2015 1:51 pm
by networkeng
Okay, that makes sense. I removed erroneous input and created a filter with the code, but it is still throwing an error during verification.
Code: Select all
if type == "cisco-asa" {
# Split the syslog part and Cisco tag out of the message
grok {
match => ["message", "%{CISCO_TAGGED_SYSLOG} %{GREEDYDATA:cisco_message}"]
}
# Parse the syslog severity and facility
syslog_pri { }
# Parse the date from the "timestamp" field to the "@timestamp" field
date {
match => ["timestamp",
"MMM dd HH:mm:ss",
"MMM d HH:mm:ss",
"MMM dd yyyy HH:mm:ss",
"MMM d yyyy HH:mm:ss"
]
timezone => "America/New_York"
}
# Clean up redundant fields if parsing was successful
if "_grokparsefailure" not in [tags] {
mutate {
rename => ["cisco_message", "message"]
remove_field => ["timestamp"]
}
}
# Extract fields from the each of the detailed message types
# The patterns provided below are included in Logstash since 1.2.0
grok {
match => [
"message", "%{CISCOFW106001}",
"message", "%{CISCOFW106006_106007_106010}",
"message", "%{CISCOFW106014}",
"message", "%{CISCOFW106015}",
"message", "%{CISCOFW106021}",
"message", "%{CISCOFW106023}",
"message", "%{CISCOFW106100}",
"message", "%{CISCOFW110002}",
"message", "%{CISCOFW302010}",
"message", "%{CISCOFW302013_302014_302015_302016}",
"message", "%{CISCOFW302020_302021}",
"message", "%{CISCOFW305011}",
"message", "%{CISCOFW313001_313004_313008}",
"message", "%{CISCOFW313005}",
"message", "%{CISCOFW402117}",
"message", "%{CISCOFW402119}",
"message", "%{CISCOFW419001}",
"message", "%{CISCOFW419002}",
"message", "%{CISCOFW500004}",
"message", "%{CISCOFW602303_602304}",
"message", "%{CISCOFW710001_710002_710003_710005_710006}",
"message", "%{CISCOFW713172}",
"message", "%{CISCOFW733100}"
]
}
}
Re: IP Charts with Geolocation
Posted: Fri Jul 17, 2015 2:06 pm
by jolson
My apologies - I gave you some bad syntax. I have tested and updated the syntax, the following should work for you:
input:
Code: Select all
udp {
port => 5555
type => "cisco-asa"
}
Code: Select all
if [type] == "cisco-asa" {
# Split the syslog part and Cisco tag out of the message
grok {
match => ["message", "%{CISCO_TAGGED_SYSLOG} %{GREEDYDATA:cisco_message}"]
}
# Parse the syslog severity and facility
syslog_pri { }
# Parse the date from the "timestamp" field to the "@timestamp" field
date {
match => ["timestamp",
"MMM dd HH:mm:ss",
"MMM d HH:mm:ss",
"MMM dd yyyy HH:mm:ss",
"MMM d yyyy HH:mm:ss"
]
timezone => "America/New_York"
}
# Clean up redundant fields if parsing was successful
if "_grokparsefailure" not in [tags] {
mutate {
rename => ["cisco_message", "message"]
remove_field => ["timestamp"]
}
}
# Extract fields from the each of the detailed message types
# The patterns provided below are included in Logstash since 1.2.0
grok {
match => [
"message", "%{CISCOFW106001}",
"message", "%{CISCOFW106006_106007_106010}",
"message", "%{CISCOFW106014}",
"message", "%{CISCOFW106015}",
"message", "%{CISCOFW106021}",
"message", "%{CISCOFW106023}",
"message", "%{CISCOFW106100}",
"message", "%{CISCOFW110002}",
"message", "%{CISCOFW302010}",
"message", "%{CISCOFW302013_302014_302015_302016}",
"message", "%{CISCOFW302020_302021}",
"message", "%{CISCOFW305011}",
"message", "%{CISCOFW313001_313004_313008}",
"message", "%{CISCOFW313005}",
"message", "%{CISCOFW402117}",
"message", "%{CISCOFW402119}",
"message", "%{CISCOFW419001}",
"message", "%{CISCOFW419002}",
"message", "%{CISCOFW500004}",
"message", "%{CISCOFW602303_602304}",
"message", "%{CISCOFW710001_710002_710003_710005_710006}",
"message", "%{CISCOFW713172}",
"message", "%{CISCOFW733100}"
]
}
}
Re: IP Charts with Geolocation
Posted: Fri Jul 17, 2015 2:50 pm
by networkeng
Okay, thanks. I have the Input and the Filter code added successfully. I have also configured two of our ASA firewalls to send their syslog data to UDP port 5555. However, I am unsure if I am receiving data on port 5555 or how to view this data in the dashboard. Syslog entries from the ASAs configured to send to UDP port 5555 does not show up in my dashboard.
Re: IP Charts with Geolocation
Posted: Mon Jul 20, 2015 9:24 am
by jolson
Did you open up UDP port 5555 on your Nagios Log Server firewall? If it is open, please try a TCPDump on port 5555 and verify that traffic is coming in properly.
Open UDP 5555:
Code: Select all
iptables -A INPUT -p udp --dport 5555 -j ACCEPT
service iptables save
TCPDump on port 5555:
Code: Select all
yum install tcpdump
tcpdump -n dst port 5555 and host x.x.x.x
*where x.x.x.x is the IP of one of your ASA devices.
Re: IP Charts with Geolocation
Posted: Tue Jul 21, 2015 11:28 am
by networkeng
Out of the office for training this week. Will check that when I am back in the office next week (27/Jul). Please leave this thread open for now.
Re: IP Charts with Geolocation
Posted: Tue Jul 21, 2015 12:13 pm
by jolson
The thread will be open when you come back, no worries.
Re: IP Charts with Geolocation
Posted: Tue Jul 28, 2015 9:36 am
by networkeng
Hi, I'm getting dumps for both hosts and they look like this:
#tcpdump -n dst port 5555 and host 10.10.x.x
09:26:30.678405 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 207
09:26:30.678530 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 206
09:26:34.688173 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 206
09:26:35.491080 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 191
09:26:35.702172 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 207
09:26:36.715663 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 207
09:26:37.088526 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 181
09:26:38.728200 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 207
09:26:38.728249 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 206
09:26:42.737468 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 207
09:26:42.737540 IP 10.10.x.x.syslog > 10.15.x.x.personal-agent: SYSLOG local4.warning, length: 206
However, when I run a search query in my dashboard I still can't find any entries after the time I switched to the new port (UDP 5555). The old entries (configured to send to TCP 5544) show up normally.
Re: IP Charts with Geolocation
Posted: Tue Jul 28, 2015 9:52 am
by jolson
The old entries (configured to send to TCP 5544) show up normally.
That could explain it. Try switching the new input to the tcp protocol - I assumed UDP.
Code: Select all
tcp {
port => 5555
type => "cisco-asa"
}