apache log type missing from ubuntu 18.04

[mannerr]

Hello!

I created manually the rsyslog.conf in Ubuntu 18 agent from the log server documentation. The log is arrived successfully to logserver, but the fields are not OK in the apache_access type:

pic1.PNG

No response code, no program.
Whats wrong?

Thanks

[jbrunkow]

Just so we're on the same page, can you specify which document you are following? Is it the one linked below?
https://assets.nagios.com/downloads/nag ... Server.pdf

It is possible that line of information is being sent to the Log Server, but is not being interpreted correctly by the filter. To view exactly what grok filter is being applied to that field, navigate to Configure on the top navigation bar > then click Global Config on the left pane > and then click the plus sign to expand a view of a particular grok filter.

Please refer to the following linked documentation for more detailed information on how to configure filters in Nagios Log Server.
https://assets.nagios.com/downloads/nag ... ilters.pdf

[mannerr]

Yes, a document similar to the one on the server in the add log source / linux and add log source / apache options. With a manually edited syslog.conf, the automatic option returned an error. Attached the syslog.conf.
I use the default filters:

filters.png

My biggest problem is that, another apache server gives good results of the log with the same configuration:

results2.png

Thanks

[ssax]

Did you change the LogFormat setting in your apache configs on the server sending it (NOT Log Server)?

The reason I ask is because I see <133> on the beginning of the message and that's not normal from the default.

Code: Select all

grep -R LogFormat /etc/apache2

[mannerr]

I didn't change logformats, I use the default apache2 package formats:

Code: Select all

user@webapp:~$ sudo grep -R LogFormat /etc/apache2
/etc/apache2/apache2.conf:LogFormat "%v:%p %h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
/etc/apache2/apache2.conf:LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
/etc/apache2/apache2.conf:LogFormat "%h %l %u %t \"%r\" %>s %O" common
/etc/apache2/apache2.conf:LogFormat "%{Referer}i -> %U" referer
/etc/apache2/apache2.conf:LogFormat "%{User-agent}i" agent

The acess log seems normal on the server:

Code: Select all

user@webapp:~$ tail -2 /var/log/apache2/access_ssl.log
84.2.x.xxx - - [16/Apr/2020:18:56:28 +0000] "GET /favicon.ico HTTP/1.1" 200 6359 "https://example.com/ex.pdf" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36"
84.2.x.xxx- - [16/Apr/2020:18:56:48 +0000] "-" 408 335 "-" "-"

Thanks

[mannerr]

apache_access is <133> always, so this seems any type maybe. Real syslog has another number:

Code: Select all

2020-04-16T21:09:18.981+02:00	192.168.35.11	syslog	<30>Apr 16 19:09:17 logoweb_app systemd[1]: Started Clean php session files.
2020-04-16T21:09:18.758+02:00	192.168.35.11	syslog	<30>Apr 16 19:09:16 app systemd[1]: Starting Clean php session files...
2020-04-16T21:09:03.750+02:00	192.168.35.11	syslog	<86>Apr 16 19:09:01 pp CRON[47327]: pam_unix(cron:session): session closed for user root
2020-04-16T21:09:03.749+02:00	192.168.35.11	syslog	<78>Apr 16 19:09:01 app CRON[47328]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
2020-04-16T21:09:03.748+02:00	192.168.35.11	syslog	<86>Apr 16 19:09:01 app CRON[47327]: pam_unix(cron:session): session opened for user root by (uid=0)
2020-04-16T21:08:44.358+02:00	192.168.35.11	syslog	<133>Apr 16 19:08:42 app apache_access: 188.36.xxx.xxx - - [16/Apr/2020:19:08:39 +0000] "-" 408 3283 "-" "-"

[cdienger]

The default syslog input looks like this:

Code: Select all

    syslog {
        type => 'syslog'
        port => 5544
    }

Make a and create a new input with it but using a different port. For example:

Code: Select all

    syslog {
        type => 'syslog'
        port => 5545.
    }

Then configure the Ubuntu machine to send the logs to this new port and let us know if that allows it to parse properly. I think the problem here is that the syslog format is slightly different between systems and once Logstash expects a specific format likes all incoming data on that input to follow it. Using the new input Logstash will hopefully recognize the format and parse the rest of the Apache logs correctly.

[mannerr]

Same situtation on 5545 port:

5545.PNG

inputs1.PNG

events1.PNG

[cdienger]

Try changing the type. I missed changing it in the last post and this is what Logstash associates with a given format:

Code: Select all

syslog {
        type => 'apachetest'
        port => 5545
    }

[mannerr]

type ok, but same problem:

events2.PNG