Page 2 of 3
Re: Question on sending Apache Logs from Linux
Posted: Wed Mar 23, 2016 2:13 pm
by rkennedy
gimeb wrote:I seems that syslogs are getting processed. See below
2016-03-22T21:13:45.228-07:00 136.133.231.211 syslog <77>Mar 22 21:01:01 x3musaw80 run-parts(/etc/cron.hourly)[2914 finished 0anacron
However, the most resent entry from of ibm-logs is still show the old data from 10/Mar/2016. See below
2016-03-22T11:19:43.542-07:00 136.133.231.211 ibm-logs <133>Mar 22 11:07:01 x3musaw80 ibm-logs: 136.133.96.66 - - [10/Mar/2016:09:59:01 -0800] "GET /favicon.ico HTTP/1.1" 404 1920
Please advise on the next step
How large are your log files? Looking at the original file posted it looks like they were at the 2nd of march, your post here is now dated march 10th. As @hsmith mentioned, they may still be processing which will just take time.
The actual logs are showing the data from 3/22. See snippet below.
136.133.96.66 - - [22/Mar/2016:21:57:10 -0700] "POST /MusaWeb/calculatePayment.action HTTP/1.1" 200 242958
136.133.96.66 - - [22/Mar/2016:21:57:10 -0700] "GET /MusaWeb/javascript/config_snapshot.js HTTP/1.1" 304 -
136.133.96.66 - - [22/Mar/2016:21:57:10 -0700] "GET /MusaWeb/musa2/js/analytics/omniture/s.code.dev.js HTTP/1.1" 304 -
136.133.96.66 - - [22/Mar/2016:21:57:10 -0700] "GET /MusaWeb/musa2/js/lib/ui/effects.blind.js HTTP/1.1" 304 -
136.133.96.66 - - [22/Mar/2016:21:57:10 -0700] "GET /MusaWeb/musa2/js/lib/jquery_plugins/musa_paymentestimator.js HTTP/1.1" 304 -
136.133.96.66 - - [22/Mar/2016:21:57:11 -0700] "GET /MusaWeb/vehicleETCRestful.action?code=M3S&year=2015&format=json HTTP/1.1" 200 175
136.133.96.66 - - [22/Mar/2016:21:57:11 -0700] "GET /webservices/mx/incentivesV2ByZipJSON/92620 HTTP/1.1" 200 120
Just to clarify - are these logs appearing in NLS already, or only your access_log?
Re: Question on sending Apache Logs from Linux
Posted: Wed Mar 23, 2016 8:56 pm
by gimeb
When I select 7 day range I see the following on the top line
2016-03-22T11:19:43.542-07:00 136.133.231.211 ibm-logs <133>Mar 22 11:07:01 x3musaw80 ibm-logs: 136.133.96.66 - - [10/Mar/2016:09:59:01 -0800] "GET /favicon.ico HTTP/1.1" 404 19205
When I select 30 days I actually see the logs are getting processes and the top line is now starts with the following:
2016-03-22T11:19:43.542-07:00 136.133.231.211 ibm-logs <133>Mar 22 11:07:01 x3musaw80 ibm-logs: 136.133.96.66 - - [02/Mar/2016:08:32:29 -0800] "POST /MusaWeb/calculatePayment.action HTTP/1.1" 200 245769
There is no data from 3/22 [22/Mar/2016:21:57:10 -0700]
All my logs are having the following format: access_20160323.log. The new logs is getting generated every day with new time stamp
Re: Question on sending Apache Logs from Linux
Posted: Thu Mar 24, 2016 10:26 am
by rkennedy
chito wrote:Hello,
Does the file name contain date(YYYYMMDD)?
If so, I think you need to upgrade rsyslog v8.5 or later to use wild card.
http://www.slideshare.net/rainergerhard ... tor-imfile
Also, "$InputFileName" of rsyslog doesn't support wild card(*) in polling mode.
You have to use inotify mode. (You can not use script setup method.)
Please take a look at this response, you'll need to use a wildcard as the file names are always different.
Re: Question on sending Apache Logs from Linux
Posted: Fri Mar 25, 2016 7:02 pm
by gimeb
Please let me know the following.
1. My understanding based on the out of the command I have executed (see LinuxLogsNotShow_Commands.docx in my prior communication) the version I have is 5.8.10. Please confirm
2. Please let me know how to upgrade to V8.5 and would it cause any issues with NLS and logs it is already collecting from other Linus machines.
3. You have indicated that "You have to use inotify mode". Please let me know where I need to do this changes and the exact syntax
Re: Question on sending Apache Logs from Linux
Posted: Mon Mar 28, 2016 12:49 pm
by hsmith
gimeb wrote:1. My understanding based on the out of the command I have executed (see LinuxLogsNotShow_Commands.docx in my prior communication) the version I have is 5.8.10. Please confirm
Looks correct.
gimeb wrote:2. Please let me know how to upgrade to V8.5 and would it cause any issues with NLS and logs it is already collecting from other Linus machines.
http://www.rsyslog.com/doc/v8-stable/in ... kages.html. I have not seen any issues with it, I have it running on one of my test systems.
gimeb wrote:3. You have indicated that "You have to use inotify mode". Please let me know where I need to do this changes and the exact syntax
Take a look at this:
http://www.slideshare.net/rainergerhard ... tor-imfile
Re: Question on sending Apache Logs from Linux
Posted: Fri Apr 01, 2016 1:26 am
by gimeb
I have upgraded the rsyslog to V8. The output after the update - rsyslog.x86_64 0:8.17.0-1.el6
However, I have reviewed the link and I am still not clear of what I need to do to implement inotify mode and where. I am not Linux admin and any help would be appreciated.
Re: Question on sending Apache Logs from Linux
Posted: Fri Apr 01, 2016 1:52 am
by chito
Hello,
I am not Linux admin and a support person. So, I cannot be responsible....
Here is my conf file in my test environment.
----------------------------------------------------------------------------
/etc/rsyslog.conf
Add the following line:
Code: Select all
global(workDirectory="/var/lib/rsyslog")
* Create a new conf file.
Code: Select all
# Default Settings
$PrivDropToGroup adm
# Load Modules
module(load="imfile")
# rsyslog ruleset
ruleset(name="nagiosls") {
action(type="omfwd"
target="xxx.xxx.xxx.xxx" # NLS Server IP Address
port="5544"
protocol="tcp" )
}
# rsyslog Input Modules
input(type="imfile"
tag="xxxx:" # TAG info
file="/xxx/xxx/xxx.*" # File Path (you can use wild card(*))
persistStateInterval="20000" # Please tune the interval
severity="notice" # Please tune the level
ruleset="nagiosls")
------------------------------------------------------------------
I'm not sure whether the above sample is proper in your environment.
Please read rsyslog documentation.
http://www.rsyslog.com/doc/v8-stable/co ... mfile.html
I hope everything works out...
Re: Question on sending Apache Logs from Linux
Posted: Fri Apr 01, 2016 9:48 am
by hsmith
Thank you Chitose
Re: Question on sending Apache Logs from Linux
Posted: Fri Apr 01, 2016 6:27 pm
by gimeb
I have conf files in many directories and I do not know which ones I need to modify. I have attached them.
The /etc/rsyslog.d contains the files in attached 90-nagioslogserver_opt_IBM_HTTPServer85_logs_access.zip
The /etc/ contains rsyslog.conf and this file has the following line " #$WorkDirectory /var/lib/rsyslog # where to place spool files". Should I just un-comment it. However, the one you have provided is "global(workDirectory="/var/lib/rsyslog")"
The /var/lib/rsyslog contains files attached in nls-state-opt_IBM_HTTPServer85_logs_access_20160229.zip
Please let me know how to proceed with the changes
Re: Question on sending Apache Logs from Linux
Posted: Mon Apr 04, 2016 3:29 pm
by jolson
I have PM'd you, please check your inbox and respond when you have the time - thank you!