syslog-ng instead of syslogd

This support forum board is for support questions relating to Nagios Log Server, our solution for managing and monitoring critical log data.
Locked
comfone
Posts: 127
Joined: Fri May 01, 2015 3:28 am

syslog-ng instead of syslogd

Post by comfone »

We have started to use syslog-ng instead of syslogd.
How can I configure or OpenSuse13.1 Server to send the syslog to our NagiosLogServer.
Thank you for your help.
Last edited by comfone on Thu Dec 03, 2015 2:02 pm, edited 1 time in total.
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: syslog-ng instead of syslogd

Post by jolson »

Using syslog-ng, your configuration should look something like this:

Add the following to /etc/syslog.conf:

Code: Select all

source s_src { unix-dgram("/dev/log"); internal(); file("/proc/kmsg"
   program_override("kernel"));
};

destination d_logstash { tcp("10.0.0.1" port(5544)); };

log { source(s_src); destination(d_logstash); };
Be sure to replace '10.0.0.1' with the IP address of a Nagios Log Server node. You're also free to change the port as you desire. After adding the above to your configuration file, be sure to restart syslog-ng.

Code: Select all

sudo /etc/init.d/syslog-ng restart
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: syslog-ng instead of syslogd

Post by scottwilkerson »

This file is also often located at /etc/syslog-ng/syslog-ng.conf

I am also adding documentation for syslog-ng to the next version of Log Server

Improving on this post, I would recommend settings like this, to be sure we are setting the syslog-protocol flag

Code: Select all

source s_nagios {
	system();
	internal();
};

destination d_nagios { 
    tcp("%hostname%" port(5544) flags(syslog-protocol)); 
};

log { 
	source(s_nagios); 
	destination(s_nagios); 
};
This time replacing %hostname% with the hostname of your Log Server cluster
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
comfone
Posts: 127
Joined: Fri May 01, 2015 3:28 am

Re: syslog-ng instead of syslogd

Post by comfone »

Thank you for your reply.
Could you please help me to forward apache2 logs to Nagios Log Server also using syslog-ng?
I would like to send all log files under /var/log/apache2/ to our Nagios Log Server and configure a filter to recognize them.
Thank you.
scottwilkerson
DevOps Engineer
Posts: 19396
Joined: Tue Nov 15, 2011 3:11 pm
Location: Nagios Enterprises
Contact:

Re: syslog-ng instead of syslogd

Post by scottwilkerson »

Something like this:

Code: Select all

source s_nagios {
	system();
	internal();
};

source s_apache2 {
   file("/var/log/apache2/*" flags(no-parse));
}; 

destination d_nagios { 
    tcp("%hostname%" port(5544) flags(syslog-protocol)); 
};

log { 
	source(s_nagios); 
	source(s_apache2); 
	destination(d_nagios); 
};
Former Nagios employee
Creator:
Human Design Website
Get Your Human Design Chart
comfone
Posts: 127
Joined: Fri May 01, 2015 3:28 am

Re: syslog-ng instead of syslogd

Post by comfone »

Hi Scott
Thank you for your reply.
Is there a default filter on the Nagios Log Server which I can apply for apache2 logs?
All your documentation is based on "rsyslog.conf" :(
jolson
Attack Rabbit
Posts: 2560
Joined: Thu Feb 12, 2015 12:40 pm

Re: syslog-ng instead of syslogd

Post by jolson »

The included filter can be viewed at 'Administration -> Global Configuration'. The filter should apply if your apache logs are detected appropriately. If not, you might find some of my older write-ups useful regarding filter construction:

http://support.nagios.com/forum/viewtop ... 37&t=32221
http://support.nagios.com/forum/viewtop ... 68#p134768
http://support.nagios.com/forum/viewtop ... 28#p137728
Twits Blog
Show me a man who lives alone and has a perpetually clean kitchen, and 8 times out of 9 I'll show you a man with detestable spiritual qualities.
comfone
Posts: 127
Joined: Fri May 01, 2015 3:28 am

Re: syslog-ng instead of syslogd

Post by comfone »

How can I make sure that they are detected?
Can I "flag" them in the syslong-ng.conf?
tmcdonald
Posts: 9117
Joined: Mon Sep 23, 2013 8:40 am

Re: syslog-ng instead of syslogd

Post by tmcdonald »

The closest I could find was this:

https://www.balabit.com/sites/default/f ... sages.html

That will just let you know if syslog-ng is working, not whether it has sent a particular message to NLS. That might be a question for the syslog-ng forums.
Former Nagios employee
comfone
Posts: 127
Joined: Fri May 01, 2015 3:28 am

Re: syslog-ng instead of syslogd

Post by comfone »

I have solved my problem as follow:
1. Create "Inputs" for Apache Access Logs
udp {
type => "apache_access_log"
port => 3333
}
2. Create "Inputs" for the Apache Error Logs
udp {
type => "apache_error_log"
port => 3334
}
3. Create following"Filter":
if [type] == 'apache_access_log' {
grok {
match => [ 'message', '%{COMMONAPACHELOG}']
}
geoip {
source => 'clientip'
}
date {
match => [ 'timestamp', 'dd/MMM/yyyy:HH:mm:ss Z' ]
}
mutate {
replace => [ 'type', 'apache_access' ]
convert => [ 'bytes', 'integer' ]
convert => [ 'response', 'integer' ]
}
}

if [type] == 'apache_error_log' {
grok {
match => [ 'message', '\[(?<timestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[.*:%{LOGLEVEL:loglevel}\] \[pid %{NUMBER:pid}\] \[client %{IP:clientip}:.*\] %{GREEDYDATA:errormsg}']
}
mutate {
replace => [ 'type', 'apache_error' ]
}
}

Thank you all for your help.
Cheers,
Locked