Acknowledge alert not stopping emails

[BradZynda]

Had a service alert for a cert expiration, clicked on it and added acknowledgement but it still sends email and is not showing the acknowledge in the section on the page:

Acknowledgements and Comments
No comments or acknowledgements.


I do see the statement in the audit log though...

Also if i go to the operations screen and click Hide Handled it is not hidden.

Thanks,
Brad

[dwhitfield]

It could be that you have two nagios processes running. The following should take care of that.

What's the output of the following? Please note that you will need to remove the "#" before each command line for the commands to work.
# df -i
# df -h

It's possible you are out of space. If that's the case, please follow https://assets.nagios.com/downloads/nag ... M-Disk.pdf

Please run through https://assets.nagios.com/downloads/nag ... tabase.pdf and report any errors. If you stop at any point, please know at which point you stop.

If the repair script and other instructions in the document do not resolve the issue, please continue.

Regarding the instructions below, if you do not have killall, you can install it via the following command:
# yum install psmisc

If psmisc is not in your repos, then instead you can check to make sure nagios is not running with
# ps -aef | grep nagios

If that document does not resolve your issue, please run the following commands in order and report any errors. You ***must*** use mariadb instead of mysqld in the commands below, ***if*** you have mariadb.
# service nagios stop
# service ndo2db stop
# service mysqld stop
# service crond stop
# service httpd stop
# killall -9 nagios
# killall -9 ndo2db
# rm -rf /usr/local/nagios/var/rw/nagios.cmd
# rm -rf /usr/local/nagios/var/nagios.lock
# rm -f /usr/local/nagios/var/ndo.sock
# rm -f /usr/local/nagios/var/ndo2db.lock
# rm -f /usr/local/nagiosxi/var/reconfigure_nagios.lock
# for i in `ipcs -q | grep nagios |awk '{print $2}'`; do ipcrm -q $i; done
# service ndo2db start
# service nagios start
# service mysqld start
# service crond start
# service httpd start

If that does not resolve the issue, can you PM me your Profile? You can download it by going to Admin > System Config > System Profile and click the ***Download Profile*** button towards the top. If for whatever reason you *cannot* download the profile, please put the output of View System Info (5.3.4+, Show Profile if older) in the thread (that will at least get us some info). This will give us access to many of the logs we would otherwise ask for individually. If security is a concern, you can unzip the profile take out what you like, and then zip it up again. We may end up needing something you remove, but we can ask for that specifically.

After you PM the profile, please update this thread. Updating this thread is the only way for it to show back up on our dashboard.

[BradZynda]

less than 1% use all the way down and no mounts greater than 43% so not out of space or inodes...

ran the repair DB script

MariaDB [(none)]> SELECT CONCAT('ALTER TABLE ', TABLE_SCHEMA, '.', TABLE_NAME, '
'> engine=MyISAM;') FROM information_schema.TABLES WHERE ENGINE = 'InnoDB';
+--------------------------------------------------------------------------+
| CONCAT('ALTER TABLE ', TABLE_SCHEMA, '.', TABLE_NAME, '
engine=MyISAM;') |
+--------------------------------------------------------------------------+
| ALTER TABLE nagiosxi.xi_incidents
engine=MyISAM; |
+--------------------------------------------------------------------------+
1 row in set (0.05 sec)

MariaDB [(none)]> quit
Bye
[root@nms1 ~]# mysqlcheck -f -r -u root -pnagiosxi --all-databases
mysql.columns_priv OK
mysql.db OK
mysql.event OK
mysql.func OK
mysql.help_category OK
mysql.help_keyword OK
mysql.help_relation OK
mysql.help_topic OK
mysql.host OK
mysql.ndb_binlog_index OK
mysql.plugin OK
mysql.proc OK
mysql.procs_priv OK
mysql.proxies_priv OK
mysql.servers OK
mysql.tables_priv OK
mysql.time_zone OK
mysql.time_zone_leap_second OK
mysql.time_zone_name OK
mysql.time_zone_transition OK
mysql.time_zone_transition_type OK
mysql.user OK
nagios.nagios_acknowledgements OK
nagios.nagios_commands OK
nagios.nagios_commenthistory OK
nagios.nagios_comments OK
nagios.nagios_configfiles OK
nagios.nagios_configfilevariables OK
nagios.nagios_conninfo OK
nagios.nagios_contact_addresses OK
nagios.nagios_contact_notificationcommands OK
nagios.nagios_contactgroup_members OK
nagios.nagios_contactgroups OK
nagios.nagios_contactnotificationmethods OK
nagios.nagios_contactnotifications OK
nagios.nagios_contacts OK
nagios.nagios_contactstatus OK
nagios.nagios_customvariables OK
nagios.nagios_customvariablestatus OK
nagios.nagios_dbversion OK
nagios.nagios_downtimehistory OK
nagios.nagios_eventhandlers OK
nagios.nagios_externalcommands OK
nagios.nagios_flappinghistory OK
nagios.nagios_host_contactgroups OK
nagios.nagios_host_contacts OK
nagios.nagios_host_parenthosts OK
nagios.nagios_hostchecks OK
nagios.nagios_hostdependencies OK
nagios.nagios_hostescalation_contactgroups OK
nagios.nagios_hostescalation_contacts OK
nagios.nagios_hostescalations OK
nagios.nagios_hostgroup_members OK
nagios.nagios_hostgroups OK
nagios.nagios_hosts OK
nagios.nagios_hoststatus OK
nagios.nagios_instances OK
nagios.nagios_logentries OK
nagios.nagios_notifications OK
nagios.nagios_objects OK
nagios.nagios_processevents OK
nagios.nagios_programstatus OK
nagios.nagios_runtimevariables OK
nagios.nagios_scheduleddowntime OK
nagios.nagios_service_contactgroups OK
nagios.nagios_service_contacts OK
nagios.nagios_service_parentservices OK
nagios.nagios_servicechecks OK
nagios.nagios_servicedependencies OK
nagios.nagios_serviceescalation_contactgroups OK
nagios.nagios_serviceescalation_contacts OK
nagios.nagios_serviceescalations OK
nagios.nagios_servicegroup_members OK
nagios.nagios_servicegroups OK
nagios.nagios_services OK
nagios.nagios_servicestatus OK
nagios.nagios_statehistory OK
nagios.nagios_systemcommands OK
nagios.nagios_timedeventqueue OK
nagios.nagios_timedevents OK
nagios.nagios_timeperiod_timeranges OK
nagios.nagios_timeperiods OK
nagiosql.tbl_command OK
nagiosql.tbl_contact OK
nagiosql.tbl_contactgroup OK
nagiosql.tbl_contacttemplate OK
nagiosql.tbl_domain OK
nagiosql.tbl_host OK
nagiosql.tbl_hostdependency OK
nagiosql.tbl_hostescalation OK
nagiosql.tbl_hostextinfo OK
nagiosql.tbl_hostgroup OK
nagiosql.tbl_hosttemplate OK
nagiosql.tbl_info OK
nagiosql.tbl_lnkContactToCommandHost OK
nagiosql.tbl_lnkContactToCommandService OK
nagiosql.tbl_lnkContactToContactgroup OK
nagiosql.tbl_lnkContactToContacttemplate OK
nagiosql.tbl_lnkContactToVariabledefinition OK
nagiosql.tbl_lnkContactgroupToContact OK
nagiosql.tbl_lnkContactgroupToContactgroup OK
nagiosql.tbl_lnkContacttemplateToCommandHost OK
nagiosql.tbl_lnkContacttemplateToCommandService OK
nagiosql.tbl_lnkContacttemplateToContactgroup OK
nagiosql.tbl_lnkContacttemplateToContacttemplate OK
nagiosql.tbl_lnkContacttemplateToVariabledefinition OK
nagiosql.tbl_lnkHostToContact OK
nagiosql.tbl_lnkHostToContactgroup OK
nagiosql.tbl_lnkHostToHost OK
nagiosql.tbl_lnkHostToHostgroup OK
nagiosql.tbl_lnkHostToHosttemplate OK
nagiosql.tbl_lnkHostToVariabledefinition OK
nagiosql.tbl_lnkHostdependencyToHost_DH OK
nagiosql.tbl_lnkHostdependencyToHost_H OK
nagiosql.tbl_lnkHostdependencyToHostgroup_DH OK
nagiosql.tbl_lnkHostdependencyToHostgroup_H OK
nagiosql.tbl_lnkHostescalationToContact OK
nagiosql.tbl_lnkHostescalationToContactgroup OK
nagiosql.tbl_lnkHostescalationToHost OK
nagiosql.tbl_lnkHostescalationToHostgroup OK
nagiosql.tbl_lnkHostgroupToHost OK
nagiosql.tbl_lnkHostgroupToHostgroup OK
nagiosql.tbl_lnkHosttemplateToContact OK
nagiosql.tbl_lnkHosttemplateToContactgroup OK
nagiosql.tbl_lnkHosttemplateToHost OK
nagiosql.tbl_lnkHosttemplateToHostgroup OK
nagiosql.tbl_lnkHosttemplateToHosttemplate OK
nagiosql.tbl_lnkHosttemplateToVariabledefinition OK
nagiosql.tbl_lnkServiceToContact OK
nagiosql.tbl_lnkServiceToContactgroup OK
nagiosql.tbl_lnkServiceToHost OK
nagiosql.tbl_lnkServiceToHostgroup OK
nagiosql.tbl_lnkServiceToServicegroup OK
nagiosql.tbl_lnkServiceToServicetemplate OK
nagiosql.tbl_lnkServiceToVariabledefinition OK
nagiosql.tbl_lnkServicedependencyToHost_DH OK
nagiosql.tbl_lnkServicedependencyToHost_H OK
nagiosql.tbl_lnkServicedependencyToHostgroup_DH OK
nagiosql.tbl_lnkServicedependencyToHostgroup_H OK
nagiosql.tbl_lnkServicedependencyToService_DS OK
nagiosql.tbl_lnkServicedependencyToService_S OK
nagiosql.tbl_lnkServiceescalationToContact OK
nagiosql.tbl_lnkServiceescalationToContactgroup OK
nagiosql.tbl_lnkServiceescalationToHost OK
nagiosql.tbl_lnkServiceescalationToHostgroup OK
nagiosql.tbl_lnkServiceescalationToService OK
nagiosql.tbl_lnkServicegroupToService OK
nagiosql.tbl_lnkServicegroupToServicegroup OK
nagiosql.tbl_lnkServicetemplateToContact OK
nagiosql.tbl_lnkServicetemplateToContactgroup OK
nagiosql.tbl_lnkServicetemplateToHost OK
nagiosql.tbl_lnkServicetemplateToHostgroup OK
nagiosql.tbl_lnkServicetemplateToServicegroup OK
nagiosql.tbl_lnkServicetemplateToServicetemplate OK
nagiosql.tbl_lnkServicetemplateToVariabledefinition OK
nagiosql.tbl_lnkTimeperiodToTimeperiod OK
nagiosql.tbl_logbook OK
nagiosql.tbl_mainmenu OK
nagiosql.tbl_service OK
nagiosql.tbl_servicedependency OK
nagiosql.tbl_serviceescalation OK
nagiosql.tbl_serviceextinfo OK
nagiosql.tbl_servicegroup OK
nagiosql.tbl_servicetemplate OK
nagiosql.tbl_session OK
nagiosql.tbl_session_locks OK
nagiosql.tbl_settings OK
nagiosql.tbl_submenu OK
nagiosql.tbl_timedefinition OK
nagiosql.tbl_timeperiod OK
nagiosql.tbl_user OK
nagiosql.tbl_variabledefinition OK
nagiosxi.xi_auditlog OK
nagiosxi.xi_commands OK
nagiosxi.xi_eventqueue OK
nagiosxi.xi_events OK
nagiosxi.xi_incidents
note : The storage engine for the table doesn't support repair
nagiosxi.xi_meta OK
nagiosxi.xi_options OK
nagiosxi.xi_sysstat OK
nagiosxi.xi_usermeta OK
nagiosxi.xi_users OK


[root@nms1 ~]# systemctl stop nagios
[root@nms1 ~]# systemctl stop nd02db
[root@nms1 ~]# systemctl stop ndo2db
[root@nms1 ~]# systemctl stop mariadb
[root@nms1 ~]# systemctl stop crond
[root@nms1 ~]# systemctl stop httpd
[root@nms1 ~]# killall -9 nagios
nagios: no process found
[root@nms1 ~]# killall -9 ndo2db
ndo2db: no process found
[root@nms1 ~]# rm -rf /usr/local/nagios/var/rw/nagios.cmd
[root@nms1 ~]# rm -rf /usr/local/nagios/var/nagios.lock
[root@nms1 ~]# rm -f /usr/local/nagios/var/ndo.sock
[root@nms1 ~]# rm -f /usr/local/nagios/var/ndo2db.lock
[root@nms1 ~]# rm -f /usr/local/nagiosxi/var/reconfigure_nagios.lock
[root@nms1 ~]# for i in `ipcs -q | grep nagios |awk '{print $2}'`; do ipcrm -q $i; done
[root@nms1 ~]# systemctl start ndo2db[root@nms1 ~]# systemctl start nagios
[root@nms1 ~]# systemctl start mariadb
[root@nms1 ~]# systemctl start crond
[root@nms1 ~]# systemctl start httpd

and

PROFILE BUILD FAILED

Array
(
)

CODE: 1

took it out of SELINUX enforcing and got the profile, also noticed out of selinux enforcing the acknowledge works...

Does this product not work with SELinux?

Thanks,
Brad

[dwhitfield]

Does this product not work with SELinux?

Certainly some customers have set up SELinux contexts that work, but we will not help you set up that context.

Out of the box, as you have found, no, it will not work with SELinux set to enforcing.

[BradZynda]

yikes

so no selinux, no 2 factor or radius support, passwords in plain text files and no help to even make it secure.. and a little

[dwhitfield]

no 2 factor

This is slated for XI 5.5.

no help to even make it secure

We have plenty of documentation on SSL and SSH (depending on the setting), and offer an offline installer. The developers have just made the decision they are going to leave SELinux contexts to the people that want them.