Page 2 of 5
Re: How to process Websphere Application server/HTTP server
Posted: Fri Feb 05, 2016 2:07 pm
by jolson
It'll be worth checking out your Logstash logs for any obvious errors. Please log into Nagios Log Server and check out the following:
Code: Select all
tail -n100 /var/log/logstash/logstash.log
I am willing to bet that your connection itself has no problems, but the logs aren't being interpreted properly or aren't being parsed by the JSON codec appropriately.
Would you give this nxlog configuration a try?
Code: Select all
## See the nxlog reference manual at
## http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.html
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.
#define ROOT C:\Program Files\nxlog
define ROOT C:\Program Files (x86)\nxlog
define CERT %ROOT%\cert
Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data
LogFile %ROOT%\data\nxlog.log
# Include fileop while debugging, also enable in the output module below
#<Extension fileop>
# Module xm_fileop
#</Extension>
<Extension json>
Module xm_json
</Extension>
<Extension syslog>
Module xm_syslog
</Extension>
<Input internal>
Module im_internal
</Input>
# Watch your own files
<Input file1>
Module im_file
File '%ROOT%\data\nxlog.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
Module im_msvistalog
# Uncomment im_mseventlog for Windows XP/2000/2003
# Module im_mseventlog
</Input>
# Watch your own files
<Input HTTPlog>
Module im_file
File 'D:\IBMHTTPServer85\logs\access.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
<Output out>
Module om_tcp
Host 136.133.236.12
Port 3515
Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
Exec $raw_event = to_json();
# Uncomment for debug output
# Exec file_write('%ROOT%\data\nxlog_output.log', $raw_event + "\n");
</Output>
<Route 1>
Path internal, file1, eventlog => out
</Route>
<Route 2>
Path HTTPlog => out2
</Route>
<Output out2>
Module om_tcp
Host 136.133.236.12
Port 2056
</Output>
You'll need to restart nxlog after the configuration is written. The idea is that you'll be sending your raw files to port 2056 (raw input) instead of sending those logs to the same port your Windows Event Logs come into.
I also want to answer an earlier question of yours:
Do I have to learn Logstash first in order to use Logserver?
Not necessarily, but I highly recommend it if you intend on maximizing the utility of Nagios Log Server. It takes a week or so to get used to - you might read through the following posts:
http://support.nagios.com/forum/viewtop ... 37&t=32221
http://support.nagios.com/forum/viewtop ... 68#p134768
Less long:
http://support.nagios.com/forum/viewtop ... 28#p137728
http://support.nagios.com/forum/viewtop ... 37&t=32548
Re: How to process Websphere Application server/HTTP server
Posted: Mon Feb 08, 2016 11:44 pm
by gimeb
I have implemented the nxlog configuration you have provided and restarted the service. However, the access.log is still not showing up. I have validated the current logstash.log and it is empty. I have attached the previous two logstash.logs for you review. The nxlog has the following entries: Please let me know on the next step.
2016-02-08 20:14:02 WARNING stopping nxlog service
2016-02-08 20:14:02 WARNING nxlog-ce received a termination request signal, exiting...
2016-02-08 20:14:17 INFO nxlog-ce-2.9.1347 started
2016-02-08 20:14:17 INFO connecting to 136.133.236.12:3515
2016-02-08 20:14:17 INFO connecting to 136.133.236.12:2056
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 4:14 pm
by hsmith
It may be worthwhile to find out if the logs are even hitting the server at this point.
Code: Select all
yum -y install tcpdump
tcpdump port 3515
Now generate some access.log traffic.
This command will show you all tcp traffic coming in on port 3515.
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 5:43 pm
by gimeb
The event logs are showing up on the dashboard. Therefore, I am assuming that they are reaching server on port 3515. Should I revert the nxlog.conf to the original configuration before running the dump since currently I have the nxlog.conf with your entries on port 2056. See below
<Route 2>
Path HTTPlog => out2
</Route>
<Output out2>
Module om_tcp
Host 136.133.236.12
Port 2056
</Output>
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 5:48 pm
by hsmith
I looked at the wrong spot, my apologies. You should run the tcpdump on port 2056, without reverting your nxlog configuration.
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 6:14 pm
by gimeb
I have access logs written to the access_20160209.log (i.e. every 24 hours the new logs is written with new date timestamp). Do I need to modify the nxlog.conf. See below
# Watch your own files
<Input HTTPlog>
Module im_file
File 'D:\IBMHTTPServer85\logs\access.log'
SavePos TRUE
Exec $Message = $raw_event;
</Input>
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 6:18 pm
by hsmith
Is there a log with just the name 'access.log' being generated, or does it go to the date format right away? If not, you'll need to wildcard it. access_* I would assume would work in your case.
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 7:27 pm
by gimeb
I have received the following output and I do not know what this means. I have also placed the wild card for the 'D:\IBMHTTPServer85\logs\access_*.log' in nxlog.conf and I do see entries in the access_20160210.log (see attached). The results are still the same - no access logs on the Dashboard
[root@X4LOGW01 logstash]# tcpdump port 2056
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:14:48.548068 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 699515732:699515819, ack 1104325821, win 513, length 87
16:14:48.548085 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 87, win 115, length 0
16:15:09.845616 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 87:174, ack 1, win 513, length 87
16:15:09.845634 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 174, win 115, length 0
16:16:56.381073 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 174:243, ack 1, win 513, length 69
16:16:56.381095 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 243, win 115, length 0
16:18:28.210303 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 243:312, ack 1, win 513, length 69
16:18:28.210321 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 312, win 115, length 0
16:19:49.466335 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 312:401, ack 1, win 513, length 89
16:19:49.466353 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 401, win 115, length 0
16:19:49.466461 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 401:610, ack 1, win 513, length 209
16:19:49.466467 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 610, win 123, length 0
16:19:55.557350 IP m4wsa800.mnao.net.51098 > x4logw01.mnao.net.omnisky: Flags [P .], seq 610:700, ack 1, win 513, length 90
16:19:55.557374 IP x4logw01.mnao.net.omnisky > m4wsa800.mnao.net.51098: Flags [. ], ack 700, win 123, length 0
Re: How to process Websphere Application server/HTTP server
Posted: Tue Feb 09, 2016 7:42 pm
by gimeb
Let me clarify my last reply. I do see the import_raw data now. However, it is not in the same format I have received from Linux server (default configuration of the Log server for apache). Please let me know
1. What was the issue? Was the update to the nxlog,conf file to include the wild card corrected the issue or it was something else
2. How to display the data in the same format as from Linux
3. How to include error.log as well
Windows via nxlog.conf
2016-02-09T16:16:56.381-08:00 136.133.236.147 import_raw 10.0.1.191 - - [09/Feb/2016:16:16:55 -0800] "HEAD / HTTP/1.1" 200 -
2016-02-09T16:15:09.845-08:00 136.133.236.147 import_raw 10.0.1.191 - - [09/Feb/2016:16:15:08 -0800] "POST /RDREntry/RDREntry HTTP/1.1" 500 90
2016-02-09T16:14:48.548-08:00 136.133.236.147 import_raw 10.0.1.191 - - [09/Feb/2016:16:14:36 -0800] "POST /RDREntry/RDREntry HTTP/1.1" 500 90
2016-02-09T16:13:28.444-08:00 136.133.236.147 import_raw 10.0.1.190 - - [09/Feb/2016:16:13:27 -0800] "HEAD / HTTP/1.1" 200 -
2016-02-09T16:11:57.166-08:00 136.133.236.147 import_raw 10.0.1.191 - - [09/Feb/2016:16:11:55 -0800] "HEAD / HTTP/1.1" 200 -
Linux
2016-02-03T16:49:33.000-08:00 136.133.231.211 apache_access 136.133.236.20 - - [26/Dec/2014:00:15:21 -0800] "GET /images/background.gif HTTP/1.1" 200 183099
2016-02-03T16:49:33.000-08:00 136.133.231.211 apache_access 136.133.236.20 - - [26/Dec/2014:00:23:24 -0800] "GET /images/help.gif HTTP/1.1" 200 210
2016-02-03T16:49:33.000-08:00 136.133.231.211 apache_access 136.133.98.173 - - [27/Dec/2014:16:34:04 -0800] "GET / HTTP/1.1" 200 3493
2016-02-03T16:49:33.000-08:00 136.133.231.211 apache_access 136.133.98.173 - - [27/Dec/2014:16:34:04 -0800] "GET /images/odot.gif HTTP/1.1" 200 49
2016-02-03T16:49:33.000-08:00 136.133.231.211 apache_access 136.133.98.173 - - [27/Dec/2014:16:49:10 -0800] "GET /images/help.gif HTTP/1.1" 304 -
2016-02-03T16:49:11.000-08:00 136.133.231.211 apache_error [Sat Dec 27 16:51:49 2014] [notice] Core file limit is 0; core dumps will be not be written for server crashes
2016-02-03T16:49:11.000-08:00 136.133.231.211 apache_error [Sat Dec 27 16:33:27 2014] [notice] Bld version: 8.5.5
2016-02-03T16:49:11.000-08:00 136.133.231.211 apache_error [Sat Dec 27 21:58:22 2014] [error] [client 136.133.236.20] File does not exist: /opt/IBM/HTTPServer85/htdocs/favicon.ico
2016-02-03T16:49:11.000-08:00 136.133.231.211 apache_error [Sat Dec 27 16:33:27 2014] [notice] WebSphere Plugins loaded.
Re: How to process Websphere Application server/HTTP server
Posted: Wed Feb 10, 2016 10:54 am
by hsmith
gimeb wrote:1. What was the issue? Was the update to the nxlog,conf file to include the wild card corrected the issue or it was something else
That I believe was the correct solution. If it was not pointing at the correct file, it is not going to get the correct data.
gimeb wrote:2. How to display the data in the same format as from Linux
Different versions of the Apahce software may look a little bit different. That's not something we have a ton of control over. May I ask what the end goal is? We may be able to come up with something else using filters.
gimeb wrote:3. How to include error.log as well
You'll need to modify the configuration again, change access_* to *.log. As long as they are in the same directory, that should work just fine.