Display log file path field

[jgsupport]

Hi,

We would like to display the path log file location when viewing the events in the dashboard. Is this possible?
I can see for Windows Event Log events I can see Path field, but when I am pulling logs from a log file it doesnt have the "Path" field.

Currently it is showing only the following in JSON

{
"_index": "logstash-2021.09.13",
"_type": "eventlog",
"_id": "AqvmKjmtq-QHErH_ejZB",
"_score": null,
"_source": {
"EventReceivedTime": "2021-09-13 07:15:03",
"SourceModuleName": "log_file_1",
"SourceModuleType": "im_file",
"message": "this is the message in the log",
"@version": "1",
"@timestamp": "2021-09-12T21:15:03.362Z",
"host": "1.1.1.1",
"port": 123456,
"type": "eventlog"
},
"sort": [
null,
1631654103362
]
}

## See the nxlog reference manual at
## http://nxlog.org/nxlog-docs/en/nxlog-re ... anual.html

## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log

# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
# Module xm_fileop
#</Extension>

<Extension json>
Module xm_json
</Extension>

<Extension syslog>
Module xm_syslog
</Extension>

<Input internal>
Module im_internal
</Input>

# Watch your own files
<Input file1>
Module im_file
File '%ROOT%\data\nxlog.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>

<Input log_file_1>
Module im_file
File 'C:\temp\logfile.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>

# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
Module im_msvistalog

# Uncomment im_mseventlog for Windows XP/2000/2003
# Module im_mseventlog
</Input>

<Output out>
Module om_tcp
Host logserver.acme.com
Port 3515

Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();

# Uncomment for debug output
#Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>

<Route 1>
Path internal, file1, log_file_1, eventlog => out
</Route>

Where as in the JSON for Windows Event logs I can see

"Path": "C:\\Windows\\system32\\config\\systemprofile\\AppData\\Roaming\\Microsoft\\Windows\\Network Shortcuts",
"EventReceivedTime": "2021-09-15 09:39:51",
"SourceModuleName": "eventlog",
"SourceModuleType": "im_msvistalog",

Can the Path be available for both Windows and Linux servers?

[gsmith]

Hi

I am not sure I understand fully. I am going to get NLS running on a Windows machine
so I can check a few things.

Are you saying you want to know if a "path" field can be made available in:
{
"_index": "logstash-2021.09.13",
"_type": "eventlog",
"_id": "AqvmKjmtq-QHErH_ejZB",
"_score": null,
"_source": {
"EventReceivedTime": "2021-09-13 07:15:03",
"SourceModuleName": "log_file_1",
"SourceModuleType": "im_file",
"message": "this is the message in the log",
"@version": "1",
"@timestamp": "2021-09-12T21:15:03.362Z",
"host": "1.1.1.1",
"port": 123456,
"type": "eventlog"
},
"sort": [
null,
1631654103362
]
}


And in this case you would be expecting it's value to be "C:\temp" ?

Thanks

[jgsupport]

Hi,

Thank you for your reply.

What I need is when I check the logs in the dashboard of NLS, I want to be able to see the "path" of the log files from where the message is extracted from.

eg:

<Input log_file_1>
Module im_file
File 'C:\temp\logfile.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>

I want to be able to see C:\temp\logfile.log or just C:\temp how ever it works in the NLS dashboard when I am looking through the log entries.
I myself know exactly where the log messages are coming from because I setup the input in conf file and I know the SourceModuleName, but we have multiple sources from any given server so anyone else viewing the log messages on NLS wont know exactly while log file that message came from.
They can see SourceModuleName but they wont make sense of it.
The only really messy way I can think of is log into over 100 servers and change <Input log_file_1> in the conf file to <Input C_temp_logfile_log> or something. That way everyone can have an idea of which log file the messages came from.

I am trying to avoid that.

Thanks

[gsmith]

Hi

Ah, that makes total sense to me now. Thanks for clarifying.

I'll be able to dive into this either this afternoon or first thing in the morning.

Thanks!

[jgsupport]

Hi gsmith,

Have you had a chance to check?

Thanks

[ssax]

You could do this:

Code: Select all

<Input log_file_1>
Module im_file
File 'C:\temp\logfile.log'
SavePos TRUE
Exec $File 'C:\temp\logfile.log';
Exec $Message = $raw_event;
</Input>

Then restart the nxlog service.

[jgsupport]

I did try but how or where do I see the File$ output.
I can only see same output:

View: Table / JSON / Raw

Field Action Value Search
@timestamp 2021-09-22T02:50:33.445Z
@version 1
EventReceivedTime 2021-09-22 12:50:33
SourceModuleName trace_server1
SourceModuleType im_file
_id AXtLaXxT3eGKTvL9TT09
_index logstash-2021.09.22
_type eventlog
host server1.acme.local
message this is the message from log file
port 63089
tags dns
type eventlog

nxlog file
2021-09-22 13:48:10 ERROR Couldn't parse Exec block at C:\Program Files (x86)\nxlog\conf\nxlog.conf:68; couldn't parse statement at line 68, character 98 in C:\Program Files (x86)\nxlog\conf\nxlog.conf; Statement required, expression found
2021-09-22 13:48:10 ERROR module 'trace_server1' has configuration errors, not adding to route '1' at C:\Program Files (x86)\nxlog\conf\nxlog.conf:96
2021-09-22 13:48:10 WARNING not starting unused module trace_server1

Line 68
Exec $File 'C:\tracefile\trace_server1.log';

<Input trace_server1>
Module im_file
File 'C:\tracefile\trace_server1.log'
SavePos TRUE
Exec $File 'C:\tracefile\trace_server1.log';
Exec $Message = $raw_event;
</Input>

I changed to below an no longer get error in nxlog. But still cant find File field in NLS.

<Input trace_server1>
Module im_file
File 'C:\tracefile\trace_server1.log'
SavePos TRUE
Exec $File = 'C:\tracefile\trace_server1.log';
Exec $Message = $raw_event;
</Input>

[ssax]

Sorry, that was my mistake when integrating my test with yours. What you did is correct.

Code: Select all

<Input trace_server1>
Module im_file
File 'C:\tracefile\trace_server1.log'
SavePos TRUE
Exec $File = 'C:\tracefile\trace_server1.log';
Exec $Message = $raw_event;
</Input>

You should see a new field called File in the Log Server message (see the attached image).

Please attach or PM your full nxlog.conf.

[jgsupport]

Strangely enough I can see it now. Must of needed time to catch up on the logs after reboot or something.
Thank you so much for your help.

Case closed!

[ssax]

Glad to hear it! Locking the thread and marking as resolved, thanks for the update!