Nxlog can't start

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
floki
Posts: 65
Joined: Tue Dec 18, 2018 5:23 am

Nxlog can't start

Post by floki »

Hello guys,

Can't start nxlog from the host that i'm monitoring.
Here's the error logs comming from nxlog log:

Code: Select all

2019-01-08 02:46:46 ERROR Module missing at C:\Program Files (x86)\nxlog\conf\nxlog.conf:52
2019-01-08 02:46:46 ERROR Module missing at C:\Program Files (x86)\nxlog\conf\nxlog.conf:79
2019-01-08 02:46:46 ERROR path missing from route '2' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:74
2019-01-08 02:46:46 ERROR exiting...
*I'm monitoring access.log & error.log located at C:\Program Files (x86)\web_server_logs
*I want to use 2056/tcp port to send the logs to NLS
*I already stop the firewall on my NLS server (Centos7) & the Monitored Host (Windows Server 2016)

Here's my nxlog config:

Code: Select all

## See the nxlog reference manual at 
## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
 
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert
 
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
 
# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
#    Module xm_fileop
#</Extension>
 
<Extension json>
    Module      xm_json
</Extension>
 
<Extension syslog>
    Module xm_syslog
</Extension>
 
<Input internal>
    Module im_internal
</Input>
 
# Watch your own files
<Input file1>
    Module   im_file
    File     '%ROOT%\data\nxlog.log'
    SavePos  TRUE
    Exec     $Message = $raw_event;
</Input>
 
# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
    Module im_msvistalog
 
# Uncomment im_mseventlog for Windows XP/2000/2003
#   Module im_mseventlog
</Input>

# Watch your own files
<Input HTTPlog>
	Module   im_file
	File     'C:\Program Files (x86)\web_server_logs\access.log'
	SavePos  TRUE
	Exec     $Message = $raw_event;
</Input>

<Output out>
    Module om_tcp
    Host 172.16.16.29
    Port 3515
	
    Exec  $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
    Exec  $raw_event = to_json();
	
	# Uncomment for debug output
	# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
 
<Route 1>
    Path internal, file1, eventlog => out
</Route>

<Route 2>
    Path HTTPlog => out2
</Route>

<Output out2>
	Module om_tcp
	Host 172.16.16.29
	Port 2056
</Output>



which I copied from this post:
https://support.nagios.com/forum/viewto ... og#p177905

Pleaaaase help thanks a lot!
floki
Posts: 65
Joined: Tue Dec 18, 2018 5:23 am

Re: Nxlog can't start

Post by floki »

Deleted some spaces in config file and was able to run the nxlog service in monitored host. But still getting some errors in nxlog log file:

Code: Select all

2019-01-08 06:32:16 ERROR Module missing at C:\Program Files (x86)\nxlog\conf\nxlog.conf:22
2019-01-08 06:32:16 ERROR Module missing at C:\Program Files (x86)\nxlog\conf\nxlog.conf:35
2019-01-08 06:32:16 ERROR Module missing at C:\Program Files (x86)\nxlog\conf\nxlog.conf:52
2019-01-08 06:32:16 ERROR invalid keyword:   at C:\Program Files (x86)\nxlog\conf\nxlog.conf:63
2019-01-08 06:32:16 ERROR module 'file1' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:72
2019-01-08 06:32:16 ERROR module 'out' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:72
2019-01-08 06:32:16 ERROR route 1 is not functional without output modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:72
2019-01-08 06:32:16 ERROR module 'HTTPlog' is not declared at C:\Program Files (x86)\nxlog\conf\nxlog.conf:76
2019-01-08 06:32:16 ERROR route 2 is not functional without input modules, ignored at C:\Program Files (x86)\nxlog\conf\nxlog.conf:76
2019-01-08 06:32:16 WARNING no routes defined!
2019-01-08 06:32:16 WARNING not starting unused module internal
2019-01-08 06:32:16 WARNING not starting unused module eventlog
2019-01-08 06:32:16 WARNING not starting unused module out
2019-01-08 06:32:16 WARNING not starting unused module out2
2019-01-08 06:32:16 INFO nxlog-ce-2.9.1716 started
Also, can't get anything from port 2056 using tcpdump. Please help :)
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nxlog can't start

Post by scottwilkerson »

Can you post your current config after the changes?
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
User avatar
mcapra
Posts: 3739
Joined: Thu May 05, 2016 3:54 pm

Re: Nxlog can't start

Post by mcapra »

Double check the following paths you have in your configuration file:

Code: Select all

define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
Specifically, make sure C:\Program Files (x86)\nxlog\modules actually exists and is populated with the necessary module files. If you copy+pasted an exact configuration file from a different environment, the paths might not match up with your nxlog installation. Here's an example of the modules path on my lab machine's RPM based installation:

Code: Select all

$ ls -al /usr/libexec/nxlog/modules/*
/usr/libexec/nxlog/modules/extension:
total 420
drwxr-xr-x 2 root root   232 May  2  2018 .
drwxr-xr-x 6 root root    67 May  2  2018 ..
-rwxr-xr-x 1 root root 27296 Jul  5  2016 xm_charconv.so
-rwxr-xr-x 1 root root 27464 Jul  5  2016 xm_csv.so
-rwxr-xr-x 1 root root 19040 Jul  5  2016 xm_exec.so
-rwxr-xr-x 1 root root 62528 Jul  5  2016 xm_fileop.so
-rwxr-xr-x 1 root root 56000 Jul  5  2016 xm_gelf.so
-rwxr-xr-x 1 root root 64760 Jul  5  2016 xm_json.so
-rwxr-xr-x 1 root root 23368 Jul  5  2016 xm_kvp.so
-rwxr-xr-x 1 root root 18784 Jul  5  2016 xm_multiline.so
-rwxr-xr-x 1 root root 31520 Jul  5  2016 xm_perl.so
-rwxr-xr-x 1 root root 49616 Jul  5  2016 xm_syslog.so
-rwxr-xr-x 1 root root 10496 Jul  5  2016 xm_wtmp.so
-rwxr-xr-x 1 root root 19240 Jul  5  2016 xm_xml.so

/usr/libexec/nxlog/modules/input:
total 248
drwxr-xr-x 2 root root   226 May  2  2018 .
drwxr-xr-x 6 root root    67 May  2  2018 ..
-rwxr-xr-x 1 root root 23136 Jul  5  2016 im_dbi.so
-rwxr-xr-x 1 root root 18880 Jul  5  2016 im_exec.so
-rwxr-xr-x 1 root root 39648 Jul  5  2016 im_file.so
-rwxr-xr-x 1 root root 10528 Jul  5  2016 im_internal.so
-rwxr-xr-x 1 root root 10528 Jul  5  2016 im_kernel.so
-rwxr-xr-x 1 root root 14688 Jul  5  2016 im_mark.so
-rwxr-xr-x 1 root root  6120 Jul  5  2016 im_null.so
-rwxr-xr-x 1 root root 44008 Jul  5  2016 im_ssl.so
-rwxr-xr-x 1 root root 23040 Jul  5  2016 im_tcp.so
-rwxr-xr-x 1 root root 10592 Jul  5  2016 im_testgen.so
-rwxr-xr-x 1 root root 18848 Jul  5  2016 im_udp.so
-rwxr-xr-x 1 root root 14752 Jul  5  2016 im_uds.so

/usr/libexec/nxlog/modules/output:
total 232
drwxr-xr-x 2 root root   184 May  2  2018 .
drwxr-xr-x 6 root root    67 May  2  2018 ..
-rwxr-xr-x 1 root root  6272 Jul  5  2016 om_blocker.so
-rwxr-xr-x 1 root root 18944 Jul  5  2016 om_dbi.so
-rwxr-xr-x 1 root root 14688 Jul  5  2016 om_exec.so
-rwxr-xr-x 1 root root 27528 Jul  5  2016 om_file.so
-rwxr-xr-x 1 root root 52424 Jul  5  2016 om_http.so
-rwxr-xr-x 1 root root  6304 Jul  5  2016 om_null.so
-rwxr-xr-x 1 root root 40008 Jul  5  2016 om_ssl.so
-rwxr-xr-x 1 root root 23104 Jul  5  2016 om_tcp.so
-rwxr-xr-x 1 root root 18880 Jul  5  2016 om_udp.so
-rwxr-xr-x 1 root root 14752 Jul  5  2016 om_uds.so

/usr/libexec/nxlog/modules/processor:
total 260
drwxr-xr-x 2 root root    173 May  2  2018 .
drwxr-xr-x 6 root root     67 May  2  2018 ..
-rwxr-xr-x 1 root root  10760 Jul  5  2016 pm_blocker.so
-rwxr-xr-x 1 root root  27392 Jul  5  2016 pm_buffer.so
-rwxr-xr-x 1 root root  31232 Jul  5  2016 pm_evcorr.so
-rwxr-xr-x 1 root root  10528 Jul  5  2016 pm_filter.so
-rwxr-xr-x 1 root root  14720 Jul  5  2016 pm_norepeat.so
-rwxr-xr-x 1 root root   6304 Jul  5  2016 pm_null.so
-rwxr-xr-x 1 root root  48352 Jul  5  2016 pm_pattern.so
-rwxr-xr-x 1 root root 102424 Jul  5  2016 pm_transformer.so
As your machine is Windows, those paths will likely be populated with similarly named DLL files on a healthy nxlog installation.

For what it's worth, that config file provided in the OP validates just fine against my 2.9.1716 binary (assuming I correct the various module/cache/data paths for my lab machine):

Code: Select all

$ cat ~/test.conf

# some module path stuff up here

# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
#    Module xm_fileop
#</Extension>

<Extension json>
    Module      xm_json
</Extension>

<Extension syslog>
    Module xm_syslog
</Extension>

<Input internal>
    Module im_internal
</Input>

# Watch your own files
<Input file1>
    Module   im_file
    File     '%ROOT%/data/nxlog.log'
    SavePos  TRUE
    Exec     $Message = $raw_event;
</Input>


# Watch your own files
<Input HTTPlog>
   Module im_file
   File     '/tmp/access.log'
   SavePos  TRUE
   Exec     $Message = $raw_event;
</Input>

<Output out>
    Module om_tcp
    Host 172.16.16.29
    Port 3515

    Exec  $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
    Exec  $raw_event = to_json();

   # Uncomment for debug output
   # Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>

<Route 1>
    Path internal, file1 => out
</Route>

<Route 2>
    Path HTTPlog => out2
</Route>

<Output out2>
   Module om_tcp
   Host 172.16.16.29
   Port 2056
</Output>

$ /usr/bin/nxlog -v -c ~/test.conf
2019-01-08 10:22:26 INFO configuration OK

$ /usr/bin/nxlog --help
nxlog-ce-2.9.1716
 usage:  nxlog [-h/help] [-c/conf conffile] [-f] [-s/stop] [-v/verify]
   [-h] print help
   [-f] run in foreground, do not daemonize
   [-c conffile] specify an alternate config file
   [-r] reload configuration of a running instance
   [-s] send stop signal to a running nxlog
   [-v] verify configuration file syntax

Former Nagios employee
https://www.mcapra.com/
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nxlog can't start

Post by scottwilkerson »

One thing I noticed was in the original the out2 was below the route directives.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
floki
Posts: 65
Joined: Tue Dec 18, 2018 5:23 am

Re: Nxlog can't start

Post by floki »

Hi Guys

Thanks for the replies :) Anyway, here's my nxlog config:

Code: Select all

## See the nxlog reference manual at 
## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
 
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert
 
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
 
# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
#    Module xm_fileop
#</Extension>
 
<Extension json>
Module      xm_json
</Extension>
 
<Extension syslog>
Module xm_syslog
</Extension>
 
<Input internal>
Module im_internal
</Input>
 
# Watch your own files
<Input file1>
Module   im_file
File     '%ROOT%\data\nxlog.log'
SavePos  TRUE
Exec     $Message = $raw_event;
</Input>
 
# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
Module im_msvistalog
 
# Uncomment im_mseventlog for Windows XP/2000/2003
#   Module im_mseventlog
</Input>

# Watch your own files
<Input HTTPlog>
Module   im_file
File     'C:\Program Files (x86)\web_server_logs\access.log'
SavePos  TRUE
Exec     $Message = $raw_event;
</Input>

<Output out>
Module om_tcp
Host 172.16.16.29
Port 3515
	
Exec  $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec  $raw_event = to_json();

# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
 
<Route 1>
Path internal, file1, eventlog => out
</Route>

<Route 2>
Path HTTPlog => out2
</Route>

<Output out2>
Module om_tcp
Host 172.16.16.29
Port 2056
</Output>


as you can see, I just deleted the spaces in front of every config. I just don't understand why it doesn't see the modules, even though it's define at the start of the code.

I'll try to copy and paste the config above and see if it works. Thanks!
floki
Posts: 65
Joined: Tue Dec 18, 2018 5:23 am

Re: Nxlog can't start

Post by floki »

At last, was able to get logs:

1. I changed the log I want to monitor since the one that previously monitored was not changing (access.log) for testing purposes
2. Here's my current config right now on nxlog:

Code: Select all

## See the nxlog reference manual at 
## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
 
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert
 
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
 
# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
#    Module xm_fileop
#</Extension>
 
<Extension json>
    Module      xm_json
</Extension>
 
<Extension syslog>
    Module xm_syslog
</Extension>
 
<Input internal>
    Module im_internal
</Input>
 
# Watch your own files
<Input file1>
    Module   im_file
    File     '%ROOT%\data\nxlog.log'
    SavePos  TRUE
    Exec     $Message = $raw_event;
</Input>
 
# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
    Module im_msvistalog
 
# Uncomment im_mseventlog for Windows XP/2000/2003
#   Module im_mseventlog
</Input>

# Watch your own files
<Input HTTPlog>
    Module im_file
    File     'C:\Program Files (x86)\Nagios\NCPA\var\log\win32service_ncpalistener.log'
    SavePos  TRUE
    Exec     $Message = $raw_event;
</Input>

<Output out>
    Module om_tcp
    Host 172.16.16.29
    Port 3515
	
    Exec  $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
    Exec  $raw_event = to_json();
	
	# Uncomment for debug output
	# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
 
<Route 1>
    Path internal, file1, eventlog => out
</Route>

<Route 2>
    Path HTTPlog => out2
</Route>

<Output out2>
   Module om_tcp
   Host 172.16.16.29
   Port 2056
</Output>
3. I changed the configuration on /usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf to rename the import_raw
From:

Code: Select all

    tcp {
        type => 'import_raw'
        tags => 'import_raw'
        port => 2056
    }
To:

Code: Select all

   
    tcp {
        type => 'HTTPlog'
        tags => 'HTTPlog'
        port => 2056
    }
***Then I restarted logstash & elasticsearch

4. Then I changed the input config in web gui : Configure > Global config > Inputs
4.1. Clicked the '+' in import_raw, changed it to HTTPlog just like in step 3
4.2. Then save

Now I can see the logs I want to monitor with HTTPlog type in the Dashboard. I think step 3 is unnecessary though. Need your comment about that step. And also for the Firewall, should I allow 2056/tcp/udp outgoing on the Monitored host? (currently NLS & Monitored Host is off)

Thanks a lot
floki
Posts: 65
Joined: Tue Dec 18, 2018 5:23 am

Re: Nxlog can't start

Post by floki »

[Follow-up]
5. Can you help me figure out how to configure nxlog so that it will get all the log files inside the folder?
6. I tried to change the name of win32service_ncpalistener.log into *.log but had an error:

Code: Select all

ERROR apr_stat failed on file C:\Program Files (x86)\Nagios\NCPA\var\log\*.log; The filename, directory name, or volume label syntax is incorrect.  
Thanks :)
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: Nxlog can't start

Post by scottwilkerson »

floki wrote:3. I changed the configuration on /usr/local/nagioslogserver/logstash/etc/conf.d/000_inputs.conf to rename the import_raw
You should make any changes under Configure -> Global config -> Inputs, or it will get over written when you apply config in the future
floki wrote:5. Can you help me figure out how to configure nxlog so that it will get all the log files inside the folder?
6. I tried to change the name of win32service_ncpalistener.log into *.log but had an error:
You need to double the \ before the * because \* meand * literal

Code: Select all

File 'C:\Program Files (x86)\Nagios\NCPA\var\log\\*.log'
https://nxlog.co/docs/nxlog-ce/nxlog-re ... onfig_file
The backslash (\) is used to escape the wildcard characters. Unfortunately this is the same as the directory separator on Windows. Take this into account when specifying wildcarded filenames on this platform. Suppose that log files under the directory C:\test need to be monitored. Specifying the wildcard C:\test\*.log will not match because \* becomes a literal asterisk and the filename is treated as non-wildcarded. For this reason the directory separator needs to be escaped: C:\test\\*.log will match our files. C:\\test\\*.log will also work. When specifying the filename using double quotes, this would became C:\\test\\\\*.log because the backslash is also used as an escape character inside double quoted string literals. Filenames on Windows systems are treated case-insensitively, but case-sensitively on Unix/Linux.
Former Nagios employee
Creator:
ahumandesign.com
enneagrams.com
floki
Posts: 65
Joined: Tue Dec 18, 2018 5:23 am

Re: Nxlog can't start

Post by floki »

Thank you so much for that! Now the remaining task is to filter those logs. I've seen the default filter used for linux-apache-access & error logs

Code: Select all

if [program] == 'apache_access' {
    grok {
        match => [ 'message', '%{COMBINEDAPACHELOG}']
    }
    date {
        match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z', 'MMM dd HH:mm:ss', 'ISO8601' ]
    }
    mutate {
        replace => [ 'type', 'apache_access' ]
         convert => [ 'bytes', 'integer' ]
         convert => [ 'response', 'integer' ]
    }
}
 
if [program] == 'apache_error' {
    grok {
        match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}']
    }
    mutate {
        replace => [ 'type', 'apache_error' ]
    }
}
Can I also use this filter for Windows?
Locked