Overview
If one of your Linux hosts is running SELinux and rsyslog, you may be running into issues receiving logs from from this host on one of your Nagios Log Server nodes.
This article will show you how to resolve this problem.
In this article:
-
The Linux server with SELinux and rsyslog will be referred to as sending_server
-
The Nagios Log Server receiving the logs will be referred to as receiving_server
Problem Description
Execute the following command on the sending_server:
tail /var/log/audit/audit.log | grep syslog
The following output will indicate that you are experiencing the problem:
type=AVC msg=audit(1459307833.315:38): avc: denied { name_connect } for pid=1752 comm=72733A616374696F6E203120717565 dest=5544
scontext=unconfined_u:system_r:syslogd_t:s0 tcontext=system_u:object_r:port_t:s0 tclass=tcp_socket
type=SYSCALL msg=audit(1459307833.315:38): arch=c000003e syscall=42 success=no exit=-13 a0=2 a1=7fddc80016b0 a2=10 a3=40 items=0
ppid=1 pid=1752 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm=72733A616374696F6E203120717565
exe="/sbin/rsyslogd" subj=unconfined_u:system_r:syslogd_t:s0 key=(null)
Further diagnosis can be made using the semanage program which requires some python libraries to be installed:
RHEL 7 | CentOS 7
yum install -y policycoreutils-python
RHEL 8 | CentOS 8
yum install -y policycoreutils-python-utils
Ubuntu 16
apt-get install -y policycoreutils
Debian 9/10| Ubuntu 18/20
apt-get install -y policycoreutils-python-utils
Once the python libraries are installed, execute the following command:
semanage port -l | grep syslog
The command should output something similar to:
syslogd_port_t tcp 6514, 601
syslogd_port_t udp 514, 6514, 601
What is important here is that we know know what syslog ports SELinux will allow.
The resolution is to configure the sending_server to send logs on TCP port 6514 and the receiving_server to receive logs on on TCP port 6514. We are choosing 6514 as there are less changes required on the receiving_server.
Resolving The Problem
First step is to make changes to the receiving_server.
-
Open the Nagios Log Server web interface on the receiving_server.
-
Click the Configure menu at the top
-
Global (All Instances) > Global Config
-
Under Inputs click the Add Input button and select Custom
-
In the "Block Name" field type Syslog (SELinux)
-
In the blank space below the code you need to type (or copy and paste) is as follows:
-
tcp {
port => 6514
type => syslog
}
udp {
port => 6514
type => syslog
} -
Click the Save button
-
-
Under Filters click the Add Filter button and select Custom
-
In the "Block Name" field type Syslog (SELinux)
-
In the blank space below the code you need to type (or copy and paste) is as follows:
-
if [type] == "syslog" {
grok {
match => { "message" => "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
}
} -
Click the Save button
-
-
-
Config > Apply Configuration
-
Click the Apply button
- Click Yes, Apply Now
-
Wait while the configuration is applied to all the nodes in the cluster
-
Open an SSH session to the receiving_server and execute the following commands (depending on your OS):
iptables -I INPUT -p tcp --destination-port 6514 -j ACCEPT
iptables -I INPUT -p udp --destination-port 6514 -j ACCEPT
service iptables save
RHEL 7 +| CentOS 7 +
Add the firewall rules by executing the following commands:
firewall-cmd --zone=public --add-port=6514/udp
firewall-cmd --zone=public --add-port=6514/tcp
firewall-cmd --reload
Debian
Add the firewall rules by executing the following commands:
iptables -I INPUT -p udp --destination-port 6514 -j ACCEPT
iptables -I INPUT -p tcp --destination-port 6514 -j ACCEP
Ubuntu
Add the firewall rules by executing the following commands:
ufw allow proto udp from any to any port 6514
ufw allow proto tcp from any to any port 6514
ufw reload
This is all the changes required on the receiving_server.
Open an SSH session to the sending_server and edit the file:
vi /etc/rsyslog.d/99-nagioslogserver.conf
Change this line:
*.* @@receiving_server_address:5544 # NAGIOSLOGSERVER
To:
*.* @@receiving_server_address:6514 # NAGIOSLOGSERVER
Save the file.
Now you need to restart the rsyslogd daemon:
service rsyslog restart
This is all the changes required on the sending_server.
Test
Now that the changes have been made on both servers, you can easily test this by adding a test log to the sending_server's syslog.
In an SSH session on the sending_server execute the following command:
logger TroubleshootingTest
On the receiving_server log into Nagios Log Server and click the Dashboards menu.
In the default dashboard we can search for the test logs we generated.
In the Query field type:
TroubleshootingTest
Press Enter and you should see the results below in the "Events Over Time" and "All Events" panels.
Final Thoughts
For any support related questions please visit the Nagios Support Forums at: