Page 3 of 5
Re: NLS Log from file not working
Posted: Thu Apr 09, 2015 2:40 pm
by jolson
I found what's causing the problem.
Code: Select all
"Description", "Last Name", "First Name", "Node UID", "Node Name", "L ocation", "Reader", "Card Number"]
Code: Select all
<%{NUMBER:number}>%{MONTH:month} %{MONTHDAY:day} %{TIME:time} %{HOST:hostname} %{GREEDYDATA:something}: %{DATA:Partiti on}[_,]+%{DATE_EU:date2} %{HOUR:hour2}:%{MINUTE:minute2},%{DATE_EU:date3} %{HOUR:hour3}:%{MINUTE:minute3},%{DATA:Description},%{DATA:LastName},%{DATA:First Name},%{NOTSPACE:NodeUID},%{DATA:NodeName},%{DATA:Location},%{DATA:Reader},%{NUMBER:CardNumber}$
Do you see what I see?
needs to be
needs to be
needs to be
I may have missed some. Please look over your configuration and ensure there's no whitespace that I may have missed.
Re: NLS Log from file not working
Posted: Thu Apr 09, 2015 4:13 pm
by kconti
Odd...I don't see any whitespace in my configuration. That might have just been a problem copying over to this support forum.
Code: Select all
filter {
if [type] == 'csvinput' {
csv {
columns => ["Partition", "Person ID", "Node Date/Time", "Date/Time", "Description", "Last Name", "First Name", "Node UID", "Node Name", "Location", "Reader", "Card Number"]
separator => ","
}
grok {
match => [ "message", "^<%{NUMBER:number}>%{MONTH:month} %{MONTHDAY:day} %{TIME:time} %{HOST:hostname} %{GREEDYDATA:something}: %{DATA:Partition}[_,]+%{DATE_EU:date2} %{HOUR:hour2}:%{MINUTE:minute2},%{DATE_EU:date3} %{HOUR:hour3}:%{MINUTE:minute3},%{DATA:Description},%{DATA:LastName},%{DATA:FirstName},%{NOTSPACE:NodeUID},%{DATA:NodeName},%{DATA:Location},%{DATA:Reader},%{NUMBER:CardNumber}$" ]
}
}
Re: NLS Log from file not working
Posted: Thu Apr 09, 2015 4:57 pm
by jolson
In that case, can you follow post 10 on page 2? I included a bunch of my configuration information - could you compare your configuration to mine?
Re: NLS Log from file not working
Posted: Fri Apr 10, 2015 8:40 am
by kconti
I matched up your configuration exatly...including the order of your inputs and filters. That still didn't work. However, I finally got the type "csvinput" to show up after I altered the /etc/rsyslog.conf
Original:
*.* @192.168.2.108:9001
*.* @192.168.2.108:5544
To this:
*.* @@192.168.2.108:9001
*.* @192.168.2.108:5544
I'm getting a few hits on the csvinput, which is definitely a step forward, but the data isn't coming over with it. I'm getting cvsinput parse and grok parse errors. I'll need to look into this more...
output_csvinput_ss.jpg
Re: NLS Log from file not working
Posted: Fri Apr 10, 2015 9:24 am
by jolson
Can you try using the example file that you sent me to see whether or not you get the same results I am getting?
Install your example file on your server in /root/ and type:
Code: Select all
bash setup-linux.sh -s NLSIP -p 9001 -f /root/csvtest.csv -t csvtag
Attached is the file I am referring to.
Re: NLS Log from file not working
Posted: Fri Apr 10, 2015 4:14 pm
by kconti
Unfortunately, I tried that before my last post. Still having the parsing issues. I went back and even tried copying and pasting your input and filter once again to make sure I didn't accidentally mess up anything during the on and off testing.
Anything after "First Name" does not show up as a field in NLS.
Missing: Node UID, Node Name, Location, Reader, Card Number.
I went back to see if anything looked funny near the configuration for First Name and Node UID and it looked fine. I did try changing your "NOSPACE" to "DATA" and that still didn't manage to fix it. I think I just need to read up more on GROK as some parts of it just doesn't seem to make sense to me.
Re: NLS Log from file not working
Posted: Mon Apr 13, 2015 9:12 am
by jolson
The filter that I defined for you was made very quickly, it's not optimal - could you show me a screenshot of what you are now receiving on Nagios Log Server?
I recommend reading the following post I made a little while ago for a better understanding of building your own custom filters:
http://support.nagios.com/forum/viewtop ... 37&t=32221
Re: NLS Log from file not working
Posted: Thu Apr 16, 2015 8:11 am
by kconti
Here are a couple screen shots. One of the limited fields list and the other of the current parse failed output.
output_parsefail_041615.jpg
limited hosts_ss.jpg
Re: NLS Log from file not working
Posted: Thu Apr 16, 2015 8:45 am
by kconti
Would the parse fail if the field is empty? Some of the fields don't get populated by all...and some fields are not needed all together for our reporting such as "Person ID", "Node Date/Time", and Node UID.
Re: NLS Log from file not working
Posted: Thu Apr 16, 2015 8:50 am
by kconti
Also, does the grok "match" need to be in the same order was the columns are listed, as they are not.