Help with searching for IP ranges

[ljorg]

Hi,

Totally new user here so my question may be very obvious. I've done a bit googling but couldn't find anything helpful.

I'm sending logs from pfSense to Nagios Log Server. I've got the grok somewhat sorted out so the lines get split into fields. I can search for things like interface or action ("block" or "pass"). But I would like to be able to search for IP ranges, preferably as a CIDR mask. I've read a bit up on elasticsearch and they seem to have something called "mask", but I can't figure out how to put it in a search. Also, I can't figure out the difference between "query" and "filter" ind the dashboard.

Can somebody point me to some relevant documentation specific to Nagios Log Server or just give me a few pointers to get me started?


Lars

[eloyd]

I can start with the query/filter difference.

A filter filters the data that you see. If you have a filter that says "color=red" for a collection of fruits and vegetables, then the visible data would only contain apples and tomatoes and red peppers, but not lettuce and potatoes.

A query queries the visible data for specific things. If you have a query that says "type=vegetable" then it will search all fruits and vegetables for just the vegetables.

Applying a filter first reduces the amount of data that a query has to search to find your results. So combining filters (color=red) with a query (type=vegetable) will only show the red pepper in our pretend data set.

The big reason for queries is that you can use them as data series on the dashboards to plot histograms, sparklines, and so forth. Filters filter the data but are not specifically viewable as discrete data sets while queries are.

I will try to answer more of your questions in another note.

[eloyd]

For my next note, I am going to suggest that you read up on Elasticsearch, which is the underlying technology used by Nagios Log Server to do its filtering and querying. Here is a good place to start: http://www.elasticsearch.org/guide/en/e ... y-dsl.html.

If you click on "queries" you can see all the types of queries that Elasticsearch can use (the examples are JSON based but you get the idea). "Range" is not what you are looking for to do an IP address check. But you can use the regexp to search for IP addresses, just be aware of the weirdness to the always-anchored queries. To search for 196.168.0/24, you would use a regexp of .*192\.168\.0\..* for instance. So long as your CIDR blocks can be converted to regexp, you can do it that way.

[ljorg]

"Range" is not what you are looking for to do an IP address check. But you can use the regexp to search for IP addresses, just be aware of the weirdness to the always-anchored queries. To search for 196.168.0/24, you would use a regexp of .*192\.168\.0\..* for instance. So long as your CIDR blocks can be converted to regexp, you can do it that way.

Thank you for an extremely quick reply (and thank you for explaining the difference between filter and query). regex could be a way, but it would be cumbersome for ranges that don't have pretty boundaries (like 10.4.84.0/21 for example).

Looking at the elasticsearch documentation it seems there is a CIDR-related "mask" keyword that can be used in a range expression. How would I execute a query like that in Nagios Log Server?


Lars

[eloyd]

Sorry, but I do not know of any current way to search via CIDR. If you can break your CIDR blocks into IP ranges, you can also search for [A.B.C.D TO W.X.Y.Z] for IP ranges. As an example you used, 10.4.84.0/21, the search would be: [10.4.80.0 TO 10.4.87.255]

[scottwilkerson]

Another possible option would be to filter the events on the way in by adding a logstash config filter and tagging the events, or adding a field using the cidr filter

http://logstash.net/docs/1.4.2/filters/cidr

And then searching for the added field.