How to process Websphere Application server/HTTP server logs

[gimeb]

Currently, I am evaluating Logserver and would like to know the following:
1. My understanding that logs have to be in consistent (standard) format in order to be indexed by Elastic Search engine. For example apache logs or Windows event logs. I would like to know if SystemOut logs from Websphere Application server can be processed by logserver and what would be required to set them up. These are not in the standard log format and they reside on Windows and Linux machines. I have attached an example
2. In addition to the application server logs I also would like to process IBM HTTP server logs. The IBM HTTP server is an IBM repackaging of Apache. Therefore, they might be not in the same format. Please advise on how to accomplish that as well. The IBM HTTP server log is attached.

[tmcdonald]

While it can be a little tricky to understand at first, Logserver does in fact allow you to define your own patterns for log files:

https://assets.nagios.com/downloads/nag ... ilters.pdf

Pretty much any format can be defined as long as the logs are consistent. Take a look at that guide and let us know if you have any specific questions!

[gimeb]

I have reviewed the document and it is my understanding that I need to configure an Input first in order to add the Filter. I have configured the following in nxlog.conf and I do not see any HTTP logs. The nxlog.conf is attached. Please advise since the whole process is not clear to me. Do I have to learn Logstash first in order to use Logserver?

<Input HTTPlog>
Module im_file
File 'D:\IBMHTTPServer85\logs\access.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>


<Output out>
Module om_tcp
Host 136.133.236.12
Port 3515

Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();

# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>

<Route 1>
Path internal, file1, eventlog, HTTPlog => out
</Route>

[rkennedy]

Can you also post your nxlog log file, this way we can see what the error message is you're receiving.

[gimeb]

The attached is nxlog.log as you have requested

[rkennedy]

On your Nagios Log Server, do you have an input for TCP on port 3515? If so, are you allowing port 3515 through iptables?

[gimeb]

I have restarted nslog service on Windows and I do not see any longer error messages. I have attached new logs for your review. However, the dashboard still not showing access.logs for IBM HTP server. I have also modified the nxlog.conf as following. There results are still the same - no data on the dashbord. Please advise.

# Watch your own files
<Input apache_access>
Module im_file
File 'D:\IBMHTTPServer85\logs\access*.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>

<Route 1>
Path internal, file1, eventlog, apache_access => out
</Route>

[scottshen]

hi ,
you may open port tcp/udp 3515 in your WIN firewall to let packet through.

[rkennedy]

Thanks @scottshen!

Have you checked your iptables on the NLS machine, and your firewall on the windows machine?

Can you also paste the input rule on your NLS server that is corresponding to port 3515?

[gimeb]

As I have mentioned before the Windows event logs are successfully getting to the log serve and I can see them in the Dashboard. The Apache access logs are not. The Windows firewall is disabled. Please let me know how to proceed.