Page 1 of 2
Cannot access web UI
Posted: Tue Nov 15, 2016 1:34 pm
by rkymtnhigh
When I got to work this morning, our remote Nagios instance seemed to be having problems.
The web UI gives me "This site can't be reached - took too long to respond" from both the public internet and from another server on it's LAN.
I SSH'd into the CentOS 6 server and made sure iptables and ip6tables were disabled.
I was not able to telnet to the nagios server from another internal address or over the public internet on 443.
I've restarted the nagios service as well as the httpd service with no change in behavior. cd /var
In the httpd error_log file I have this entry over and over. Doesn't seem to be related or triggered when attempting to access the webpage.
[Tue Nov 15 11:30:23.500932 2016] [:error] [pid 1899] [client ::1:34387] PHP Notice: Undefined variable: f2 in /usr/local/nagiosxi/html/includes/components/snmptrapsender/snmptrapsender.inc.php on line 485
[Tue Nov 15 11:30:43.858855 2016] [:error] [pid 1598] [client ::1:34392] PHP Notice: Undefined variable: f2 in /usr/local/nagiosxi/html/includes/components/snmptrapsender/snmptrapsender.inc.php on line 485
[Tue Nov 15 11:31:03.136705 2016] [:error] [pid 1594] [client ::1:34396] PHP Notice: Undefined variable: f2 in /usr/local/nagiosxi/html/includes/components/snmptrapsender/snmptrapsender.inc.php on line 485
[Tue Nov 15 11:31:23.496846 2016] [:error] [pid 1595] [client ::1:34402] PHP Notice: Undefined variable: f2 in /usr/local/nagiosxi/html/includes/components/snmptrapsender/snmptrapsender.inc.php on line 485
At this point I'm starting to suspect iptables is not actually disabled, as I can't see any other reason I wouldn't be able to connect.
Any input or advice is greatly appreciated.
Thank you,
RMH
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 1:41 pm
by avandemore
What is the output of iptables -L and netstat -plnt?
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 1:46 pm
by rkymtnhigh
Sure, here is iptables -L
Code: Select all
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
INPUT_direct all -- anywhere anywhere
INPUT_ZONES_SOURCE all -- anywhere anywhere
INPUT_ZONES all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
FORWARD_direct all -- anywhere anywhere
FORWARD_IN_ZONES_SOURCE all -- anywhere anywhere
FORWARD_IN_ZONES all -- anywhere anywhere
FORWARD_OUT_ZONES_SOURCE all -- anywhere anywhere
FORWARD_OUT_ZONES all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- anywhere anywhere
Chain FORWARD_IN_ZONES (1 references)
target prot opt source destination
FWDI_public all -- anywhere anywhere [goto]
FWDI_public all -- anywhere anywhere [goto]
Chain FORWARD_IN_ZONES_SOURCE (1 references)
target prot opt source destination
Chain FORWARD_OUT_ZONES (1 references)
target prot opt source destination
FWDO_public all -- anywhere anywhere [goto]
FWDO_public all -- anywhere anywhere [goto]
Chain FORWARD_OUT_ZONES_SOURCE (1 references)
target prot opt source destination
Chain FORWARD_direct (1 references)
target prot opt source destination
Chain FWDI_public (2 references)
target prot opt source destination
FWDI_public_log all -- anywhere anywhere
FWDI_public_deny all -- anywhere anywhere
FWDI_public_allow all -- anywhere anywhere
Chain FWDI_public_allow (1 references)
target prot opt source destination
Chain FWDI_public_deny (1 references)
target prot opt source destination
Chain FWDI_public_log (1 references)
target prot opt source destination
Chain FWDO_public (2 references)
target prot opt source destination
FWDO_public_log all -- anywhere anywhere
FWDO_public_deny all -- anywhere anywhere
FWDO_public_allow all -- anywhere anywhere
Chain FWDO_public_allow (1 references)
target prot opt source destination
Chain FWDO_public_deny (1 references)
target prot opt source destination
Chain FWDO_public_log (1 references)
target prot opt source destination
Chain INPUT_ZONES (1 references)
target prot opt source destination
IN_public all -- anywhere anywhere [goto]
IN_public all -- anywhere anywhere [goto]
Chain INPUT_ZONES_SOURCE (1 references)
target prot opt source destination
Chain INPUT_direct (1 references)
target prot opt source destination
Chain IN_public (2 references)
target prot opt source destination
IN_public_log all -- anywhere anywhere
IN_public_deny all -- anywhere anywhere
IN_public_allow all -- anywhere anywhere
Chain IN_public_allow (1 references)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:ssh ctstate NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:nrpe ctstate NEW
ACCEPT tcp -- anywhere anywhere tcp dpt:nsca ctstate NEW
Chain IN_public_deny (1 references)
target prot opt source destination
Chain IN_public_log (1 references)
target prot opt source destination
Chain OUTPUT_direct (1 references)
target prot opt source destination
And netstat -plnt
Code: Select all
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 1543/mysqld
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 504/rpcbind
tcp 0 0 127.0.0.1:8022 0.0.0.0:* LISTEN 1296/python
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1148/sshd
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 1221/postgres
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1510/master
tcp6 0 0 :::111 :::* LISTEN 504/rpcbind
tcp6 0 0 :::80 :::* LISTEN 1177/httpd
tcp6 0 0 :::22 :::* LISTEN 1148/sshd
tcp6 0 0 ::1:5432 :::* LISTEN 1221/postgres
tcp6 0 0 ::1:25 :::* LISTEN 1510/master
tcp6 0 0 :::443 :::* LISTEN 1177/httpd
tcp6 0 0 :::5666 :::* LISTEN 1153/xinetd
tcp6 0 0 :::5667 :::* LISTEN 1153/xinetd
Thank you very much.
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 1:57 pm
by avandemore
iptables is active.
To stop it, /etc/init.d/iptables stop
To disable it from starting at boot, chkconfig iptables off
Please retry the test with iptables stopped.
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 2:03 pm
by rkymtnhigh
When I run /etc/init.d/iptables stop, it gives me
Code: Select all
-bash: /etc/init.d/iptables: No such file or directory
When I run service iptables stop, it gives me
Code: Select all
Redirecting to /bin/systemctl stop iptables.service
When I run the chkconfig iptables off command it gives me
Code: Select all
Note: Forwarding request to 'systemctl disable iptables.service'.
There is no change in behavior after a reboot.
Thank you!
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 2:05 pm
by avandemore
What is the output of cat /etc/redhat-release, ls /etc/init.d/ and systemctl?
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 3:16 pm
by rkymtnhigh
Code: Select all
CentOS Linux release 7.1.1503 (Core)
ls /etc/init.d/
Code: Select all
ajaxterm functions nagios nagiosxi ndo2db netconsole network npcd pdagent README
systemctl
Code: Select all
UNIT LOAD ACTIVE SUB DESCRIPTION
proc-sys-fs-binfmt_misc.automount loaded active waiting Arbitrary Executable File Formats File System Automount Point
sys-devices-platform-serial8250-tty-ttyS1.device loaded active plugged /sys/devices/platform/serial8250/tty/ttyS1
sys-devices-platform-serial8250-tty-ttyS2.device loaded active plugged /sys/devices/platform/serial8250/tty/ttyS2
sys-devices-platform-serial8250-tty-ttyS3.device loaded active plugged /sys/devices/platform/serial8250/tty/ttyS3
sys-devices-pnp0-00:09-tty-ttyS0.device loaded active plugged /sys/devices/pnp0/00:09/tty/ttyS0
sys-devices-vbd\x2d768-block-xvda-xvda1.device loaded active plugged /sys/devices/vbd-768/block/xvda/xvda1
sys-devices-vbd\x2d768-block-xvda.device loaded active plugged /sys/devices/vbd-768/block/xvda
sys-devices-vif\x2d0-net-eth0.device loaded active plugged /sys/devices/vif-0/net/eth0
sys-module-configfs.device loaded active plugged /sys/module/configfs
sys-subsystem-net-devices-eth0.device loaded active plugged /sys/subsystem/net/devices/eth0
-.mount loaded active mounted /
dev-hugepages.mount loaded active mounted Huge Pages File System
dev-mqueue.mount loaded active mounted POSIX Message Queue File System
proc-fs-nfsd.mount loaded active mounted NFSD configuration filesystem
sys-kernel-config.mount loaded active mounted Configuration File System
sys-kernel-debug.mount loaded active mounted Debug File System
var-lib-nfs-rpc_pipefs.mount loaded active mounted RPC Pipe File System
brandbot.path loaded active waiting Flexible branding
systemd-ask-password-console.path loaded active waiting Dispatch Password Requests to Console Directory Watch
systemd-ask-password-wall.path loaded active waiting Forward Password Requests to Wall Directory Watch
session-1.scope loaded active running Session 1 of user centos
session-621.scope loaded active running Session 621 of user nagios
session-622.scope loaded active running Session 622 of user nagios
session-625.scope loaded active running Session 625 of user nagios
session-626.scope loaded active running Session 626 of user nagios
session-627.scope loaded active running Session 627 of user nagios
session-c1.scope loaded active abandoned Session c1 of user pdagent
ajaxterm.service loaded active running SYSV: Ajaxterm is a web-based terminal
auditd.service loaded active running Security Auditing Service
chronyd.service loaded active running NTP client/server
cloud-config.service loaded active exited Apply the settings specified in cloud-config
cloud-final.service loaded active exited Execute cloud user/final scripts
cloud-init-local.service loaded active exited Initial cloud-init job (pre-networking)
cloud-init.service loaded active exited Initial cloud-init job (metadata service crawler)
crond.service loaded active running Command Scheduler
dbus.service loaded active running D-Bus System Message Bus
firewalld.service loaded active running firewalld - dynamic firewall daemon
[email protected] loaded active running Getty on tty1
gssproxy.service loaded active running GSSAPI Proxy Daemon
httpd.service loaded active running The Apache HTTP Server
kmod-static-nodes.service loaded active exited Create list of required static device nodes for the current kernel
mariadb.service loaded active running MariaDB database server
nagios.service loaded active running LSB: Starts and stops the Nagios monitoring server
nagiosxi.service loaded active exited SYSV: NagiosXI subtasks
ndo2db.service loaded active running LSB: Nagios NDO2DB Initscript
network.service loaded active running LSB: Bring up/down networking
nfs-config.service loaded active exited Preprocess NFS configuration
npcd.service loaded active running SYSV: Visit the Website at http://sourceforge.net/projects/pnp4nagios/
pdagent.service loaded active exited LSB: PagerDuty Agent
polkit.service loaded active running Authorization Manager
postfix.service loaded active running Postfix Mail Transport Agent
postgresql.service loaded active running PostgreSQL database server
rhel-dmesg.service loaded active exited Dump dmesg to /var/log/dmesg
rhel-import-state.service loaded active exited Import network configuration from initramfs
rhel-readonly.service loaded active exited Configure read-only root support
rpcbind.service loaded active running RPC bind service
rsyslog.service loaded active running System Logging Service
[email protected] loaded active running Serial Getty on ttyS0
sshd.service loaded active running OpenSSH server daemon
sysstat.service loaded active exited Resets System Activity Logs
systemd-fsck-root.service loaded active exited File System Check on Root Device
systemd-journald.service loaded active running Journal Service
systemd-logind.service loaded active running Login Service
systemd-random-seed.service loaded active exited Load/Save Random Seed
systemd-remount-fs.service loaded active exited Remount Root and Kernel File Systems
systemd-sysctl.service loaded active exited Apply Kernel Variables
systemd-tmpfiles-setup-dev.service loaded active exited Create static device nodes in /dev
systemd-tmpfiles-setup.service loaded active exited Create Volatile Files and Directories
systemd-udev-trigger.service loaded active exited udev Coldplug all Devices
systemd-udevd.service loaded active running udev Kernel Device Manager
systemd-update-utmp.service loaded active exited Update UTMP about System Reboot/Shutdown
systemd-user-sessions.service loaded active exited Permit User Sessions
systemd-vconsole-setup.service loaded active exited Setup Virtual Console
tuned.service loaded active running Dynamic System Tuning Daemon
xinetd.service loaded active running Xinetd A Powerful Replacement For Inetd
-.slice loaded active active Root Slice
system-getty.slice loaded active active system-getty.slice
system-serial\x2dgetty.slice loaded active active system-serial\x2dgetty.slice
system.slice loaded active active System Slice
user-0.slice loaded active active user-0.slice
user-1000.slice loaded active active user-1000.slice
user-1001.slice loaded active active user-1001.slice
user-995.slice loaded active active user-995.slice
user.slice loaded active active User and Session Slice
dbus.socket loaded active running D-Bus System Message Bus Socket
rpcbind.socket loaded active running RPCbind Server Activation Socket
systemd-initctl.socket loaded active listening /dev/initctl Compatibility Named Pipe
systemd-journald.socket loaded active running Journal Socket
systemd-shutdownd.socket loaded active listening Delayed Shutdown Socket
systemd-udevd-control.socket loaded active running udev Control Socket
systemd-udevd-kernel.socket loaded active running udev Kernel Socket
basic.target loaded active active Basic System
cloud-config.target loaded active active Cloud-config availability
cryptsetup.target loaded active active Encrypted Volumes
getty.target loaded active active Login Prompts
local-fs-pre.target loaded active active Local File Systems (Pre)
local-fs.target loaded active active Local File Systems
multi-user.target loaded active active Multi-User System
network-online.target loaded active active Network is Online
network.target loaded active active Network
nfs-client.target loaded active active NFS client services
paths.target loaded active active Paths
remote-fs-pre.target loaded active active Remote File Systems (Pre)
remote-fs.target loaded active active Remote File Systems
slices.target loaded active active Slices
sockets.target loaded active active Sockets
swap.target loaded active active Swap
sysinit.target loaded active active System Initialization
timers.target loaded active active Timers
systemd-tmpfiles-clean.timer loaded active waiting Daily Cleanup of Temporary Directories
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
110 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
Thank you!
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 3:28 pm
by avandemore
So you are running CentOS 7, not 6.
To disable the service do this:
systemctl disable firewalld.service
and reboot, then test again.
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 4:08 pm
by rkymtnhigh
Oh my apologies! That command did the trick, thank you for all the help!
Re: Cannot access web UI
Posted: Tue Nov 15, 2016 4:24 pm
by avandemore
No worries, are we ok to lock this thread?