Page 1 of 3
apache log type missing from ubuntu 18.04
Posted: Wed Apr 15, 2020 6:43 am
by mannerr
Hello!
I created manually the rsyslog.conf in Ubuntu 18 agent from the log server documentation. The log is arrived successfully to logserver, but the fields are not OK in the apache_access type:
pic1.PNG
No response code, no program.
Whats wrong?
Thanks
Re: apache log type missing from ubuntu 18.04
Posted: Wed Apr 15, 2020 12:57 pm
by jbrunkow
Just so we're on the same page, can you specify which document you are following? Is it the one linked below?
https://assets.nagios.com/downloads/nag ... Server.pdf
It is possible that line of information is being sent to the Log Server, but is not being interpreted correctly by the filter. To view exactly what
grok filter is being applied to that field, navigate to
Configure on the top navigation bar > then click
Global Config on the left pane > and then click the plus sign to expand a view of a particular
grok filter.
Please refer to the following linked documentation for more detailed information on how to configure filters in
Nagios Log Server.
https://assets.nagios.com/downloads/nag ... ilters.pdf
Re: apache log type missing from ubuntu 18.04
Posted: Thu Apr 16, 2020 12:00 am
by mannerr
Yes, a document similar to the one on the server in the add log source / linux and add log source / apache options. With a manually edited syslog.conf, the automatic option returned an error. Attached the syslog.conf.
I use the default filters:
filters.png
My biggest problem is that, another apache server gives good results of the log with the same configuration:
results2.png
Thanks
Re: apache log type missing from ubuntu 18.04
Posted: Thu Apr 16, 2020 1:39 pm
by ssax
Did you change the LogFormat setting in your apache configs on the server sending it (NOT Log Server)?
The reason I ask is because I see
<133> on the beginning of the message and that's not normal from the default.
Re: apache log type missing from ubuntu 18.04
Posted: Thu Apr 16, 2020 2:07 pm
by mannerr
I didn't change logformats, I use the default apache2 package formats:
Code: Select all
user@webapp:~$ sudo grep -R LogFormat /etc/apache2
/etc/apache2/apache2.conf:LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
/etc/apache2/apache2.conf:LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
/etc/apache2/apache2.conf:LogFormat "%h %l %u %t \"%r\" %>s %O" common
/etc/apache2/apache2.conf:LogFormat "%{Referer}i -> %U" referer
/etc/apache2/apache2.conf:LogFormat "%{User-agent}i" agent
The acess log seems normal on the server:
Code: Select all
user@webapp:~$ tail -2 /var/log/apache2/access_ssl.log
84.2.x.xxx - - [16/Apr/2020:18:56:28 +0000] "GET /favicon.ico HTTP/1.1" 200 6359 "https://example.com/ex.pdf" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36"
84.2.x.xxx- - [16/Apr/2020:18:56:48 +0000] "-" 408 335 "-" "-"
Thanks
Re: apache log type missing from ubuntu 18.04
Posted: Thu Apr 16, 2020 2:24 pm
by mannerr
apache_access is <133> always, so this seems any type maybe. Real syslog has another number:
Code: Select all
2020-04-16T21:09:18.981+02:00 192.168.35.11 syslog <30>Apr 16 19:09:17 logoweb_app systemd[1]: Started Clean php session files.
2020-04-16T21:09:18.758+02:00 192.168.35.11 syslog <30>Apr 16 19:09:16 app systemd[1]: Starting Clean php session files...
2020-04-16T21:09:03.750+02:00 192.168.35.11 syslog <86>Apr 16 19:09:01 pp CRON[47327]: pam_unix(cron:session): session closed for user root
2020-04-16T21:09:03.749+02:00 192.168.35.11 syslog <78>Apr 16 19:09:01 app CRON[47328]: (root) CMD ( [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
2020-04-16T21:09:03.748+02:00 192.168.35.11 syslog <86>Apr 16 19:09:01 app CRON[47327]: pam_unix(cron:session): session opened for user root by (uid=0)
2020-04-16T21:08:44.358+02:00 192.168.35.11 syslog <133>Apr 16 19:08:42 app apache_access: 188.36.xxx.xxx - - [16/Apr/2020:19:08:39 +0000] "-" 408 3283 "-" "-"
Re: apache log type missing from ubuntu 18.04
Posted: Thu Apr 16, 2020 4:17 pm
by cdienger
The default syslog input looks like this:
Code: Select all
syslog {
type => 'syslog'
port => 5544
}
Make a and create a new input with it but using a different port. For example:
Code: Select all
syslog {
type => 'syslog'
port => 5545.
}
Then configure the Ubuntu machine to send the logs to this new port and let us know if that allows it to parse properly. I think the problem here is that the syslog format is slightly different between systems and once Logstash expects a specific format likes all incoming data on that input to follow it. Using the new input Logstash will hopefully recognize the format and parse the rest of the Apache logs correctly.
Re: apache log type missing from ubuntu 18.04
Posted: Fri Apr 17, 2020 2:55 am
by mannerr
Same situtation on 5545 port:
5545.PNG
inputs1.PNG
events1.PNG
Re: apache log type missing from ubuntu 18.04
Posted: Fri Apr 17, 2020 10:07 am
by cdienger
Try changing the type. I missed changing it in the last post and this is what Logstash associates with a given format:
Code: Select all
syslog {
type => 'apachetest'
port => 5545
}
Re: apache log type missing from ubuntu 18.04
Posted: Fri Apr 17, 2020 11:11 am
by mannerr
type ok, but same problem:
events2.PNG