Nxlog can't start

[floki]

Done reading the documentation for Configuring Filters in NLS.
There's a part stated about Filter Condition:

Code: Select all

Filter Conditions
An filter can be restricted to certain logs by using a simple if statement:
if [program] == 'apache_error' {
grok {
}
mutate {
}
}
Here you can see that the [program] must be apache_error for these filters to be applied.
How does the log received by Logstash know that the program is apache_error? When you run the setup
script on your Linux server that is running Apache it will define the name as apache_error.
In Nagios Log Server navigate to Configure > Add Log Source and select Apache Server. Under the Run
the Script heading you can see the following line of code:
sudo bash setup-linux.sh -s nagios_log_server -p 5544 -f
"/var/log/httpd/error_log" -t apache_error
You can see that apache_error is being defined as part of the setup script. The syslog application on the
sending server will define these log entries as coming from apache_error.

Is there a way I can do such thing in Windows? So that NLS will recognize the Logs coming is an apache_error? Thanks a lot

[scottwilkerson]

You would want to look at a message coming in and filter based on name/value pairs, it might be something like

Code: Select all

if [SourceModuleType] == 'im_file' {
grok {
}
mutate {
}
}

[floki]

Thanks for that, here's my Global Config for NLS:

Code: Select all

# 
# Logstash Configuration File
# Dynamically created by Nagios Log Server
#
# DO NOT EDIT THIS FILE. IT WILL BE OVERWRITTEN.
#
# Created Fri, 11 Jan 2019 10:42:58 +0800
#

#
# Global inputs
#

input {
    syslog {
        type => 'syslog'
        port => 5544
    }
    tcp {
        type => 'eventlog'
        port => 3515
        codec => json {
            charset => 'CP1252'
        }
    }
    tcp {
        type => 'web_server_log'
        tags => 'web_server_log'
        port => 2056
    }
    tcp {
        type => 'import_json'
        tags => 'import_json'
        port => 2057
        codec => json
    }
    syslog {
        type => 'syslog-vmware'
        port => 514
    }
}

#
# Local inputs
#



#
# Global filters
#

filter {
    if [program] == 'test-email' {
        grok {
            match => { "message" => "%{DATESTAMP:Time_Snort}  \[.*?\] \[%{NUMBER:GID_Snort}:%{NUMBER:SID_Snort}:%{NUMBER:Rev_Snort}\] %{DATA:Alert_Snort} \[.*?\] (\[Classification: %{DATA:Class_Snort}\] )?\[Priority: %{NUMBER:Prio_Snort}\] \{%{WORD:Proto_Snort}\} %{IP:SrcIP_Snort}(:%{NUMBER:SrcPort_Snort})? -> %{IP:DestIP_Snort}(:%{NUMBER:DestPort_Snort})? \[.*?\] %{NUMBER:Thresh_Snort} hits in %{NUMBER:Period_Snort} %{WORD:Unit_Snort}" }
        }
    }
    if [host] == '10.109.196.226' {
        mutate {
            replace => { 'type' => 'syslog-esxi' }
        }
    }
    
    if [host] == '10.109.197.200' {
        mutate {
            replace => { 'type' => 'syslog-vcenter' }
        }
    }
    if [type] == 'web_server_log' {
        grok {
            match => [ 'message', '%{IPORHOST:clientip} %{HTTPDUSER:ident} %{HTTPDUSER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)']
        }
        date {
            match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM dd HH:mm:ss', 'ISO8601' ]
        }
        mutate {
            replace => [ 'program', 'apache_access' ]
        }
    }
     
    if [type] == 'web_server_log' {
        grok {
            match => [ 'message', '\[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \[pid %{POSINT:pid}(:tid %{NUMBER:tid})?\]( \(%{POSINT:proxy_errorcode}\)%{DATA:proxy_message}:)?( \[client %{IPORHOST:clientip}:%{POSINT:clientport}\])?( %{DATA:errorcode}:)? %{GREEDYDATA:message}']
        }
        mutate {
            replace => [ 'program', 'apache_error' ]
        }
    }
}

#
# Local filters
#



#
# Global outputs
#

output {
    if [program] == 'test-email' {
        file {
            codec => line { format => "Timestamp: %{Time_Snort} \nClassification: %{Class_Snort} \nPriority level: %{Prio_Snort} \nAlert: %{Alert_Snort} \nRule GID, SID, Revision: %{GID_Snort}, %{SID_Snort}, %{Rev_Snort} \nProtocol: %{Proto_Snort} \nSource Address, Port: %{SrcIP_Snort}, %{SrcPort_Snort} \nDestination Address, Port: %{DestIP_Snort}, %{DestPort_Snort} \nThreshold exceeded: %{Thresh_Snort} hits \nCheck period: %{Period_Snort} %{Unit_Snort}" }
            path => "/var/log/nls-alerts/security-alert"
        }
        exec {
            command => "/home/nagios/emailer.sh"
        }
    }
}

#
# Local outputs
#

Still getting Grok parse error I think the problem is that the config can't i dentify which is the access log and which is the error log. Configured the nxlog to send all logs coming from a directory to 2056 port

[floki]

The only solution I can think of is to open another port for Error logs. Which is a bad option since there might be another log file that I need to Monitor and end up opening a so many ports. Don't know how to make nagios log server know which one is access log & which one is error log

[floki]

Hello scottwilkerson!

it's working now! hahaha I think the trick is that I put on code ==> json to the input then Got it parse properly )

[scottwilkerson]

Hello scottwilkerson!

it's working now! hahaha I think the trick is that I put on code ==> json to the input then Got it parse properly )

Excellent!

Glad to hear it is working now

Locking thread