Timeout error occurred trying to start the MySQL Daemon

[AWS]

Fresh manual build of CentOS 5.6 and Nagios XI 2011R1.2, NOT SSL,
Complete noob to anything other than Win machines - sry, bear with me.

I followed the instructions at http://assets.nagios.com/downloads/nagi ... ctions.pdf for installing Nagios XI. I had no installation problems. All the scripts appeared to work correctly.

A couple of problems. When rebooting the server I am seeing that my MySQL daemon is failing to load:

Code: Select all

[root@nagios ~]# service mysqld restart
Stopping MySQL:                                            [  OK  ]
Timeout error occurred trying to start MySQL Daemon.
Starting MySQL:                                            [FAILED]

Also, when I am in the Nagios XI, Configure, Configuration Wizard, Monitoring Wizard, I complete my add new host wizard, click the button and get hours worth of "Waiting for configuration verification". I did find some help on this issue here but adjusting the time-out values and memory allocation per script to double their original values did not help.

On the Nagios XI, Admin, Administration screen, the "XI System Component Status" dashlet only has the top three items turned on, the other 8 items have never been "green". Is that a problem?

If seems that if I add a new host via the Nagios Core page, it will add (sometimes). Something seems to be wrong with my config but I do not know where to start to look. Help? Sometimes within the "Core" pages I read red wording that states "Warning: configuration file is out of date!" so I click the "Apply Configuration" button but that just goes on forever, too, like when I run the Monitoring Wizard.

Thanks

AWS

[tonyyarusso]

First possibility: When you installed, did you unpack and run things from /tmp, or someplace else (like /root)?

[AWS]

I am really thinking that it was /tmp because when this error began to show itself I reviewed the installation PDF and one of the commands it listed was to "cd /tmp". So I'm 95% sure I would have followed that step. One thing that is in the back of my head is, I thought that I found a page in Nagios that showed the owner or permission group associated to various components/files within the Nagios system. I remember seeing a lot of "root" entries and just a few "nagios" entries. I wish I could find that page again.

I have no attachment to this system and can rebuild if necessary.

[AWS]

... double post.

[AWS]

I'm pretty sure I ran the install from the /tmp directory because that step was part of the install PDF I followed. If you see something that suggests otherwise, let me know.

When I run the command to check to my config I get the following:

Code: Select all

[root@nagios ~]# /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg

Nagios Core 3.2.3
Copyright (c) 2009-2010 Nagios Core Development Team and Community Contributors
Copyright (c) 1999-2009 Ethan Galstad
Last Modified: 10-03-2010
License: GPL

Website: http://www.nagios.org
Reading configuration data...
Error in configuration file '/usr/local/nagios/etc/nagios.cfg' - Line 1 (NULL value)
   Error processing main config file!



***> One or more problems was encountered while processing the config files...

     Check your configuration file(s) to ensure that they contain valid
     directives and data defintions.  If you are upgrading from a previous
     version of Nagios, you should be aware that some variables/definitions
     may have been removed or modified in this version.  Make sure to read
     the HTML documentation regarding the config files, as well as the
     'Whats New' section to find out what has changed.

So then I ran a vi against /usr/local/nagios/etc/nagios.cfg looking for any "null" values and didn't see anything. But, admittingly, I don't know exactly what I'm looking for. Below is my config file in question.

Code: Select all

[root@nagios ~]# vi /usr/local/nagios/etc/nagios.cfg   
      1 MODIFIED
      2 admin_email=root@localhost
      3 admin_pager=root@localhost
      4 translate_passive_host_checks=1
      5 log_event_handlers=0
      6 use_large_installation_tweaks=1
      7 enable_environment_macros=1
      8
      9 # NDOUtils module
     10 broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg
     11
     12
     13 # PNP settings - bulk mode with NCPD
     14 process_performance_data=1
     15 # service performance data
     16 service_perfdata_file=/usr/local/nagios/var/service-perfdata
     17 service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tH        OSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tSERVICEOUTPUT::$SERVICEOUTPUT$
     18 service_perfdata_file_mode=a
     19 service_perfdata_file_processing_interval=15
     20 service_perfdata_file_processing_command=process-service-perfdata-file-bulk
     21 # host performance data
     22 host_perfdata_file=/usr/local/nagios/var/host-perfdata
     23 host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tHOSTOUTPUT::$HO        STOUTPUT$
     24 host_perfdata_file_mode=a
     25 host_perfdata_file_processing_interval=15
     26 host_perfdata_file_processing_command=process-host-perfdata-file-bulk
     27
     28
     29 # OBJECTS - UNMODIFIED
     30 #cfg_file=/usr/local/nagios/etc/objects/commands.cfg
     31 #cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
     32 #cfg_file=/usr/local/nagios/etc/objects/localhost.cfg
     33 #cfg_file=/usr/local/nagios/etc/objects/templates.cfg
     34 #cfg_file=/usr/local/nagios/etc/objects/timeperiods.cfg
     35
     36
     37 # STATIC OBJECT DEFINITIONS (THESE DON'T GET EXPORTED/IMPORTED BY NAGIOSQL)
     38 cfg_dir=/usr/local/nagios/etc/static
     39
     40 # OBJECTS EXPORTED FROM NAGIOSQL
     41 cfg_file=/usr/local/nagios/etc/contacttemplates.cfg
     42 cfg_file=/usr/local/nagios/etc/contactgroups.cfg
     43 cfg_file=/usr/local/nagios/etc/contacts.cfg
     44 cfg_file=/usr/local/nagios/etc/timeperiods.cfg
     45 cfg_file=/usr/local/nagios/etc/commands.cfg
     46 cfg_file=/usr/local/nagios/etc/hostgroups.cfg
     47 cfg_file=/usr/local/nagios/etc/servicegroups.cfg
     48 cfg_file=/usr/local/nagios/etc/hosttemplates.cfg
     49 cfg_file=/usr/local/nagios/etc/servicetemplates.cfg
     50 cfg_file=/usr/local/nagios/etc/servicedependencies.cfg
     51 cfg_file=/usr/local/nagios/etc/serviceescalations.cfg
     52 cfg_file=/usr/local/nagios/etc/hostdependencies.cfg
     53 cfg_file=/usr/local/nagios/etc/hostescalations.cfg
     54 cfg_file=/usr/local/nagios/etc/hostextinfo.cfg
     55 cfg_file=/usr/local/nagios/etc/serviceextinfo.cfg
     56 cfg_dir=/usr/local/nagios/etc/hosts
     57 cfg_dir=/usr/local/nagios/etc/services
     58
     59 # GLOBAL EVENT HANDLERS
     60 global_host_event_handler=xi_host_event_handler
     61 global_service_event_handler=xi_service_event_handler
     62
     63
     64
     65 # UNMODIFIED
     66 accept_passive_host_checks=1
     67 accept_passive_service_checks=1
     68 additional_freshness_latency=15
     69 auto_reschedule_checks=0
     70 auto_rescheduling_interval=30
     71 auto_rescheduling_window=180
     72 bare_update_check=0
     73 cached_host_check_horizon=15
     74 cached_service_check_horizon=15
     75 check_external_commands=1
     76 check_for_orphaned_hosts=1
     77 check_for_orphaned_services=1
     78 check_for_updates=1
     79 check_host_freshness=0
     80 check_result_path=/usr/local/nagios/var/spool/checkresults
     81 check_result_reaper_frequency=10
     82 check_service_freshness=1
     83 command_check_interval=-1
     84 command_file=/usr/local/nagios/var/rw/nagios.cmd
     85 daemon_dumps_core=0
     86 date_format=us
     87 debug_file=/usr/local/nagios/var/nagios.debug
     88 debug_level=0
     89 debug_verbosity=1
     90 enable_embedded_perl=1
     91 enable_event_handlers=1
     92 enable_flap_detection=1
     93 enable_notifications=1
     94 enable_predictive_host_dependency_checks=1
     95 enable_predictive_service_dependency_checks=1
     96 event_broker_options=-1
     97 event_handler_timeout=30
     98 execute_host_checks=1
     99 execute_service_checks=1
    100 external_command_buffer_slots=4096
    101 high_host_flap_threshold=20.0
    102 high_service_flap_threshold=20.0
    103 host_check_timeout=30
    104 host_freshness_check_interval=60
    105 host_inter_check_delay_method=s
    106 illegal_macro_output_chars=`~$&|'"<>
    107 illegal_object_name_chars=`~!$%^&*|'"<>?,()=
    108 interval_length=60
    109 lock_file=/usr/local/nagios/var/nagios.lock
    110 log_archive_path=/usr/local/nagios/var/archives
    111 log_external_commands=0
    112 log_file=/usr/local/nagios/var/nagios.log
    113 log_host_retries=1
    114 log_initial_states=0
    115 log_notifications=1
    116 log_passive_checks=0
    117 log_rotation_method=d
    118 log_service_retries=1
    119 low_host_flap_threshold=5.0
    120 low_service_flap_threshold=5.0
    121 max_check_result_file_age=3600
    122 max_check_result_reaper_time=30
    123 max_concurrent_checks=0
    124 max_debug_file_size=1000000
    125 max_host_check_spread=30
    126 max_service_check_spread=30
    127 nagios_group=nagios
    128 nagios_user=nagios
    129 notification_timeout=30
    130 object_cache_file=/usr/local/nagios/var/objects.cache
    131 obsess_over_hosts=0
    132 obsess_over_services=0
    133 ocsp_timeout=5
    134 p1_file=/usr/local/nagios/bin/p1.pl
    135 passive_host_checks_are_soft=0
    136 perfdata_timeout=5
    137 precached_object_file=/usr/local/nagios/var/objects.precache
    138 resource_file=/usr/local/nagios/etc/resource.cfg
    139 retained_contact_host_attribute_mask=0
    140 retained_contact_service_attribute_mask=0
    141 retained_host_attribute_mask=0
    142 retained_process_host_attribute_mask=0
    143 retained_process_service_attribute_mask=0
    144 retained_service_attribute_mask=0
    145 retain_state_information=1
    146 retention_update_interval=60
    147 service_check_timeout=60
    148 service_freshness_check_interval=60
    149 service_inter_check_delay_method=s
    150 service_interleave_factor=s
    151 sleep_time=0.25
    152 soft_state_dependencies=0
    153 state_retention_file=/usr/local/nagios/var/retention.dat
    154 status_file=/usr/local/nagios/var/status.dat
    155 status_update_interval=10
    156 temp_file=/usr/local/nagios/var/nagios.tmp
    157 temp_path=/tmp
    158 use_aggressive_host_checking=0
    159 use_embedded_perl_implicitly=1
    160 use_regexp_matching=0
    161 use_retained_program_state=1
    162 use_retained_scheduling_info=1
    163 use_syslog=1
    164 use_true_regexp_matching=0

Any help is greatly appreciated.

[tonyyarusso]

Does it literally start with "MODIFIED", not commented out or anything? Try just deleting that entire line and re-checking.

[AWS]

Thanks for your continued help.

Yes, the file literally started with the word "MODIFIED". I did remove it. When I was starting the vi session, I did get a warning that there was a swap file of the same name ...see below

Code: Select all

E325: ATTENTION
Found a swap file by the name "/usr/local/nagios/etc/.nagios.cfg.swp"
          owned by: root   dated: Wed May 18 13:16:52 2011
         file name: /usr/local/nagios/etc/nagios.cfg
          modified: YES
         user name: root   host name: nagios.sietg.local
        process ID: 301 (still running)
While opening file "/usr/local/nagios/etc/nagios.cfg"
             dated: Wed May 18 13:17:31 2011
      NEWER than swap file!

(1) Another program may be editing the same file.
    If this is the case, be careful not to end up with two
    different instances of the same file when making changes.
    Quit, or continue with caution.

(2) An edit session for this file crashed.
    If this is the case, use ":recover" or "vim -r /usr/local/nagios/etc/nagios.
cfg"
    to recover the changes (see ":help recovery").
    If you did this already, delete the swap file "/usr/local/nagios/etc/.nagios
.cfg.swp"
    to avoid this message.
"/usr/local/nagios/etc/nagios.cfg" 163L, 5753C
Press ENTER or type command to continue

I must have prematurely closed vi last time (?) so I pressed ENTER and continued. Removed "MODIFIED", saved and rechecked. Verify, aka "Pre-flight check", came back clean: 0 warnings, 0 errors. Am rebooting the machine now (because I don't know how to just restart Nagios...)

EDIT: Ok, great. Looks like MySQL is running. I can't see the bootup screen because I am remote, but running service mysqld status yields mysqld (pid 2752) is running.... Thanks for your help!

Now to find out why the waiting for verification configuration never stops... (new post)

Thanks again for the help,
AWS

[tonyyarusso]

Sounds good! Let us know if you need anything else.