nagios log server filter question

[ouyangxin]

Hi All,
I want to use nagios log server to filter nginx log and add fields like 'clientip response',my nginx log show at nagios log server:

Code: Select all

    <135>May  5 10:45:17 hostname poolname: 1.1.1.1 - [04/May/2017:17:19:02 +0800] "GET /Images/abc.gif HTTP/1.1" 200 1121 "http://www.abc.com/Styles/abc.css" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0" -"HIT"- - "0.000" [-]

I can get correct result run '/usr/local/nagioslogserver/logstash/bin/logstash -f stdin.conf' like following show:

Code: Select all

    "message" => "<135>May  5 10:45:17 hostname2 pool name: 1.1.1.1 - [04/May/2017:17:19:02 +0800] \"GET /Images/slidernav_bg.gif HTTP/1.1\" 200 1121 \"http://www.abc.com/Styles/sliderman.css\" \"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0\" -\"HIT\"- - \"0.000\" [-]",
                "@version" => "1",
              "@timestamp" => "2017-05-05T05:03:42.248Z",
                    "type" => "nginx",
                    "host" => "log-server",
                      "id" => "135",
                 "logtime" => "May  5 10:45:17",
                "hostname" => "hostname2",
                "poolname" => "pool name",
                  "client" => "1.1.1.1",
               "timeStamp" => "04/May/2017:17:19:02 +0800",
                  "method" => "GET",
                 "request" => "/Images/slidernav_bg.gif",
            "http_version" => "1.1",
                   "statu" => "200",
                   "bytes" => "1121",
                "referrer" => "http://www.abc.com/Styles/sliderman.css",
                   "agent" => "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0",
    "http_x_forwarded_for" => "-",
             "cache_statu" => "HIT",
                "duration" => "0.000"

but it did not work and 'tags' always show '_grokparsefailure_sysloginput' when i post this filter to nagios log server -> Administrator -> Global Configuration -> Filters

my stdin.conf :

Code: Select all

input {
    stdin {
        type => "nginx"
    }
}

filter {
    grok {
    match => [ "message" , "<%{NUMBER:id}>%{SYSLOGTIMESTAMP:logtime} %{WORD:hostname} %{WORD:poolname}\: %{IPV4:client} - \[%{HTTPDATE:timeStamp}\] \"%{WORD:method} %{URIPATHPARAM:request} HTTP/%{NUMBER:http_version}\" %{NUMBER:statu} (?:%{NUMBER:bytes}|-) \"%{DATA:referrer}\" \"%{DATA:agent}\" (?:%{NOTSPACE:http_x_forwarded_for}|-)\"%{WORD:cache_statu}\"(?:%{IPV4:server_ip}|-) (?:%{NUMBER:server_statu}|-) \"%{NUMBER:duration}\" \[(?:%{NUMBER:response_time}|-)\]"]
    }

}

    output{
        stdout{codec=>rubydebug}

[mcapra]

Are you running Nagios Log Server on the nginx box? We typically recommend all Nagios products get their own clean, minimal CentOS or RHEL machines to live on.

We also don't support flat-file modifications for Logstash. Nagios Log Server has it's own configuration management utility available via the GUI which overwrites those changes.