Alerts not going out

[matson-itops]

on nagios xi 5.8.1 email alerts are not going out

1. test email goes to matson.com address just fine
2. alerts are turned on with appropriate time period and contact
3. alert is showing read in console and notifications are enabled

what am I missing?

[ssax]

Please follow these instructions:

Please go to Admin > Manage Email Settings:
- Make sure Logging is checked
- Click the Update Settings button

Then run this tail command (and leave it running):

Code: Select all

tail -Fn0 /var/log/maillog /usr/local/nagiosxi/tmp/phpmailer.log /usr/local/nagiosxi/var/eventman.log

Then force a notification to be sent then send me the full output of the tail command above and any errors you see on the screen (please test via the host from the instructions below exactly).

After you've done the steps above PM me a FRESH copy of your profile.zip, you can download it from Admin > System Profile > Download Profile.

Additionally, please send the output of this command:
- NOTE: You may need to adjust the -h 127.0.0.1, the -uroot, and -pnagiosxi in the command if your DB is offloaded to another server and/or you've changed the root mysql password

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table

How to submit passive results for testing:

For Hosts
------------

Go to Home > Host Status:
- Find the Host and click on it
- Click the + tab
- Click the "Submit passive check result" link
- Select the Check Result and type in some text for the Check Output
- Click the Submit button

NOTE: By default, passive_host_checks_are_soft=0 is set in your /usr/local/nagios/etc/nagios.cfg, this differs from services in that when you submit a passive host check result it immeditely goes into a HARD state (and should send a notification if configured to) instead of going into a SOFT state like services do. (notifications are only sent on HARD states).

For Services
----------------

Go to Home > Service Status:
- Find the Service and click on it
- Click the + tab
- Note these two rows:

State Type: Hard
Current Check: 1 of 4

Those columns tell the current State Type and the Current Check number. In order to generate a notification for a service you will need to submit MULTIPLE problem check results (the number that you need to submit is determined by the last number in the Current Check column, that is the max_check_attempts setting). For services, when you submit a passive check result, each result that you submit will be a SOFT state until you submit enough to hit the Max Check Attempts setting that you've defined on the service, only then will the service enter a HARD problem state which will generate the notification (just remember, notifications are only sent on HARD states).

- Click the "Submit passive check result" link
- Select the Check Result and type in some text for the Check Output
- Click the Submit button
- Submit as many as you need, right after another, until the service enters the HARD state so that a notification will be sent

NOTE: When coming from a HARD problem state (whether we are talking about hosts or services) if you submit an OK passive result it should fire off a recovery notification after a single passive result has been submitted.

[matson-itops]

here is the info you requested

a) submitted passive check result successfully, but did NOT get notification or anything in the logs.

b) download system profile did not work . got the error below

PROFILE BUILD FAILED
Array
(
)
CODE: 1

c) below is the query results

$ echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('nagios', 'nagiosql', 'nagiosxi');" | mysql -h 127.0.0.1 -uroot -pnagiosxi --table
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------------------------------------------+------------+
| Table | Size in MB |
+--------------------------------------------+------------+
| nagios_acknowledgements | 0.00 |
| nagios_commands | 0.04 |
| nagios_commenthistory | 49.16 |
| nagios_comments | 0.27 |
| nagios_configfiles | 0.01 |
| nagios_configfilevariables | 0.02 |
| nagios_contactgroups | 0.00 |
| nagios_contactgroup_members | 0.01 |
| nagios_contacts | 0.02 |
| nagios_contactstatus | 0.01 |
| nagios_contact_addresses | 0.00 |
| nagios_contact_notificationcommands | 0.06 |
| nagios_customvariables | 0.48 |
| nagios_customvariablestatus | 0.41 |
| nagios_dbversion | 0.00 |
| nagios_downtimehistory | 0.94 |
| nagios_eventhandlers | 0.08 |
| nagios_externalcommands | 0.26 |
| nagios_flappinghistory | 5.33 |
| nagios_hostchecks | 0.00 |
| nagios_hostdependencies | 0.00 |
| nagios_hostescalations | 0.00 |
| nagios_hostescalation_contactgroups | 0.00 |
| nagios_hostescalation_contacts | 0.00 |
| nagios_hostgroups | 0.00 |
| nagios_hostgroup_members | 0.03 |
| nagios_hosts | 0.33 |
| nagios_hoststatus | 0.57 |
| nagios_host_contactgroups | 0.03 |
| nagios_host_contacts | 0.03 |
| nagios_host_parenthosts | 0.00 |
| nagios_instances | 0.00 |
| nagios_processevents | 0.06 |
| nagios_programstatus | 0.00 |
| nagios_runtimevariables | 0.00 |
| nagios_scheduleddowntime | 0.12 |
| nagios_servicechecks | 0.00 |
| nagios_servicedependencies | 0.00 |
| nagios_serviceescalations | 0.00 |
| nagios_serviceescalation_contactgroups | 0.00 |
| nagios_serviceescalation_contacts | 0.00 |
| nagios_servicegroups | 0.00 |
| nagios_servicegroup_members | 0.02 |
| nagios_services | 1.13 |
| nagios_servicestatus | 2.53 |
| nagios_service_contactgroups | 0.12 |
| nagios_service_contacts | 0.07 |
| nagios_service_parentservices | 0.00 |
| nagios_systemcommands | 0.05 |
| nagios_timeperiods | 0.01 |
| nagios_timeperiod_timeranges | 0.02 |
| tbl_contacttemplate | 0.03 |
| tbl_domain | 0.03 |
| tbl_hostdependency | 0.03 |
| tbl_hostescalation | 0.03 |
| tbl_hostextinfo | 0.03 |
| tbl_info | 0.17 |
| tbl_lnkContactgroupToContact | 0.02 |
| tbl_lnkContactgroupToContactgroup | 0.02 |
| tbl_lnkContacttemplateToCommandHost | 0.02 |
| tbl_lnkContacttemplateToCommandService | 0.02 |
| tbl_lnkContacttemplateToContactgroup | 0.02 |
| tbl_lnkContacttemplateToContacttemplate | 0.02 |
| tbl_lnkContacttemplateToVariabledefinition | 0.02 |
| tbl_lnkContactToCommandHost | 0.02 |
| tbl_lnkContactToCommandService | 0.02 |
| tbl_lnkContactToContactgroup | 0.02 |
| tbl_lnkContactToContacttemplate | 0.02 |
| tbl_lnkContactToVariabledefinition | 0.02 |
| tbl_lnkHostdependencyToHostgroup_DH | 0.02 |
| tbl_lnkHostdependencyToHostgroup_H | 0.02 |
| tbl_lnkHostescalationToContact | 0.02 |
| tbl_lnkHostescalationToContactgroup | 0.02 |
| tbl_lnkHosttemplateToContact | 0.02 |
| tbl_lnkHosttemplateToContactgroup | 0.02 |
| tbl_lnkHosttemplateToHost | 0.02 |
| tbl_lnkHosttemplateToHostgroup | 0.02 |
| tbl_lnkHosttemplateToHosttemplate | 0.02 |
| tbl_lnkHosttemplateToVariabledefinition | 0.02 |
| tbl_lnkHostToContact | 0.05 |
| tbl_lnkHostToContactgroup | 0.02 |
| tbl_lnkHostToHosttemplate | 0.05 |
| tbl_lnkHostToVariabledefinition | 0.05 |
| tbl_lnkServicedependencyToHostgroup_DH | 0.02 |
| tbl_lnkServicedependencyToHostgroup_H | 0.02 |
| tbl_lnkServicedependencyToHost_DH | 0.02 |
| tbl_lnkServicedependencyToHost_H | 0.02 |
| tbl_lnkServicedependencyToServicegroup_DS | 0.02 |
| tbl_lnkServicedependencyToServicegroup_S | 0.02 |
| tbl_lnkServiceescalationToContact | 0.02 |
| tbl_lnkServiceescalationToContactgroup | 0.02 |
| tbl_lnkServiceescalationToService | 0.02 |
| tbl_lnkServiceescalationToServicegroup | 0.02 |
| tbl_lnkServicegroupToService | 0.02 |
| tbl_lnkServicegroupToServicegroup | 0.02 |
| tbl_lnkServicetemplateToContact | 0.02 |
| tbl_lnkServicetemplateToContactgroup | 0.02 |
| tbl_lnkServicetemplateToServicegroup | 0.02 |
| tbl_lnkServicetemplateToServicetemplate | 0.02 |
| tbl_lnkServicetemplateToVariabledefinition | 0.02 |
| tbl_lnkServiceToContact | 0.08 |
| tbl_lnkServiceToContactgroup | 0.09 |
| tbl_lnkServiceToServicetemplate | 0.11 |
| tbl_lnkServiceToVariabledefinition | 0.13 |
| tbl_lnkTimeperiodToTimeperiod | 0.02 |
| tbl_mainmenu | 0.02 |
| tbl_servicedependency | 0.03 |
| tbl_serviceescalation | 0.03 |
| tbl_serviceextinfo | 0.03 |
| tbl_settings | 0.03 |
| tbl_submenu | 0.02 |
| tbl_variabledefinition | 0.23 |
| tbl_session | 0.02 |
| tbl_session_locks | 0.02 |
| tbl_lnkServicetemplateToHost | 0.02 |
| tbl_lnkServicetemplateToHostgroup | 0.02 |
| tbl_lnkServiceescalationToHost | 0.02 |
| tbl_lnkServiceescalationToHostgroup | 0.02 |
| tbl_lnkHostescalationToHost | 0.02 |
| tbl_lnkHostescalationToHostgroup | 0.02 |
| tbl_lnkHostgroupToHostgroup | 0.02 |
| tbl_permission | 0.02 |
| tbl_permission_inactive | 0.02 |
| xi_eventqueue | 0.03 |
| nagios_statehistory | 267.07 |
| nagios_contactnotifications | 128.97 |
| nagios_contactnotificationmethods | 160.56 |
| nagios_objects | 0.75 |
| nagios_timedeventqueue | 0.00 |
| nagios_timedevents | 0.00 |
| nagios_logentries | 0.00 |
| nagios_notifications | 135.63 |
| xi_auditlog | 1.94 |
| xi_commands | 0.02 |
| xi_events | 163.72 |
| xi_meta | 2812.98 |
| xi_mibs | 0.05 |
| xi_options | 0.03 |
| xi_sysstat | 0.03 |
| xi_usermeta | 0.14 |
| xi_users | 0.03 |
| xi_sessions | 0.03 |
| xi_auth_tokens | 0.03 |
| xi_deploy_jobs | 0.02 |
| xi_deploy_agents | 0.02 |
| xi_cmp_trapdata | 0.03 |
| xi_cmp_trapdata_log | 0.03 |
| xi_cmp_ccm_backups | 0.02 |
| xi_cmp_nagiosbpi_backups | 0.31 |
| xi_cmp_favorites | 0.03 |
| xi_cmp_scheduledreports_log | 0.02 |
| tbl_contact | 0.03 |
| tbl_contactgroup | 0.03 |
| tbl_host | 0.19 |
| tbl_hostgroup | 0.03 |
| tbl_lnkHostgroupToHost | 0.02 |
| tbl_lnkHostToHost | 0.02 |
| tbl_lnkHostToHostgroup | 0.05 |
| tbl_lnkHostdependencyToHost_DH | 0.02 |
| tbl_lnkHostdependencyToHost_H | 0.02 |
| tbl_lnkServiceToHost | 0.13 |
| tbl_lnkServicedependencyToService_DS | 0.02 |
| tbl_lnkServicedependencyToService_S | 0.02 |
| tbl_lnkServiceToHostgroup | 0.02 |
| tbl_lnkServiceToServicegroup | 0.02 |
| tbl_logbook | 0.02 |
| tbl_service | 1.52 |
| tbl_servicegroup | 0.03 |
| tbl_timeperiod | 0.03 |
| tbl_timedefinition | 0.05 |
| tbl_hosttemplate | 0.03 |
| tbl_servicetemplate | 0.03 |
| tbl_user | 0.03 |
| tbl_command | 0.06 |
+--------------------------------------------+------------+

[ssax]

You're likely hitting a known bug as these are way too large for temporary tables:

Code: Select all

| xi_events | 163.72 |
| xi_meta | 2812.98 |

Please run this command to fix it:

Code: Select all

echo "truncate table xi_events; truncate table xi_meta; truncate table xi_eventqueue;" | mysql -h 127.0.0.1 -uroot -pnagiosxi nagiosxi

Then test the notifications.

For the profile build failed message, follow this KB article:

https://support.nagios.com/kb/article.p ... ategory=44

[matson-itops]

thanks.
will allow some time to see if alerts are flowing, but looking better
will also work on fixing the reports as needed

[vtrac]

Great!! ..... Glad to hear issue is getting better ...