Page 1 of 2

this is CRAZEE

Posted: Tue May 27, 2014 2:27 pm
by benhank
NOTE: this issue existed prior to today.

ok, my primary server has always been backued up using the same modified script.
My secondary and test machines have always had the primary's backup restored to them,with no errors or failures.
The machines have identical hardware amd os specs.
SO, on the test machine, when I do an apply config, it completes, but the changed/ newly added check still say:

Code: Select all

 Active	Yes 	Sync Missed
,BUT sometimes the check still works afterwards, sometimes it don't.
The Secondary:
@the write config :

Code: Select all

Write host configurations ...
Cannot open/overwrite the host configuration files (check the permissions)!
Write service configurations ...
Cannot open/overwrite service configuration files (check the permissions)!
Additionally:
Capture.JPG
But note: I CAN get the monitoring engine process to run. but after I do, if i try to enable or disable one of the parameters,say tick the notifications off X, then I get: the process is draggin its but to complete error

Re: this is CRAZEE

Posted: Tue May 27, 2014 2:58 pm
by scottwilkerson
What is the output of the following

Code: Select all

ls -ld /usr/local/nagios /usr/local/nagios/etc /usr/local/nagios/etc/hosts /usr/local/nagios/etc/services
also if you look in the directory, what do the file permissions look like?

Code: Select all

ls -l /usr/local/nagios/etc/services

Re: this is CRAZEE

Posted: Wed May 28, 2014 9:57 am
by benhank
here you go
this is from the secondary svr as it is the most critical to get running.

Code: Select all

drwxrwxr-x 8 root   root     4096 Dec  5  2012 /usr/local/nagios
drwsrwsr-x 7 apache nagios   4096 May 16 12:59 /usr/local/nagios/etc
drwsrwsrwt 2 apache nagios 118784 May 15 16:16 /usr/local/nagios/etc/hosts
drwsrwsrwt 2 apache nagios   4096 May 15 15:25 /usr/local/nagios/etc/services
[root@Lkennagiosp02 ~]#
and

Code: Select all

[root@Lkennagiosp02 ~]# ls -l /usr/local/nagios/etc/services
total 324
-rwxrwxrwt 1 apache nagios  910 May 19 11:40 APC_ups_Standard.cfg
-rwxrwxrwt 1 apache nagios 1127 May 19 11:40 atriusnsa_host_ping.cfg
-rwxrwxrwt 1 apache nagios 1127 May 19 11:40 atriusnsb_host_ping.cfg
-rwxrwxrwt 1 apache nagios  893 May 19 11:40 Avaya Error_MAJ.cfg
-rwxrwxrwt 1 apache nagios  893 May 19 11:40 Avaya Error_MIN.cfg
-rwxrwxrwt 1 apache nagios 1051 May 19 11:40 bds_service_check.cfg
-rwxrwxrwt 1 apache nagios  912 May 19 11:40 Check_APC_Verbose.cfg
-rwxrwxrwt 1 apache nagios  810 May 19 11:40 check_cisco_fru_fan.cfg
-rwxrwxrwt 1 apache nagios  808 May 19 11:40 check_cisco_fru_ps.cfg
-rwxrwxrwt 1 apache nagios 1120 May 19 11:40 check_dhcp_scope.cfg
-rwxrwxrwt 1 apache nagios  971 May 19 11:40 check_eventlog_EventID_1008.cfg
-rwxrwxrwt 1 apache nagios  971 May 19 11:40 check_eventlog_EventID_1009.cfg
-rwxrwxrwt 1 apache nagios  916 May 19 11:40 check_eventlog_EventID_2020.cfg
-rwxrwxrwt 1 apache nagios  928 May 19 11:40 check_eventlog_EventID_333.cfg
-rwxrwxrwt 1 apache nagios  970 May 19 11:40 check_eventlog_EventID_3621.cfg
-rwxrwxrwt 1 apache nagios  970 May 19 11:40 check_eventlog_EventID_3635.cfg
-rwxrwxrwt 1 apache nagios 1180 May 19 11:40 check_interface_table_core.cfg
-rwxrwxrwt 1 apache nagios 1132 May 19 11:40 check_interface_table_edge.cfg
-rwxrwxrwt 1 apache nagios  884 May 19 11:40 check_snmp_cisco_stack.cfg
-rwxrwxrwt 1 apache nagios  869 May 19 11:40 check_snmp_environment.cfg
-rwxrwxrwt 1 apache nagios  855 May 19 11:40 check_snmp_uptime.cfg
-rwxrwxrwt 1 apache nagios 1011 May 19 11:40 check_windows_time_external.cfg
-rwxrwxrwt 1 apache nagios 1161 May 19 11:40 check_windows_time_internal.cfg
-rwxrwxrwt 1 apache nagios 1101 May 19 11:40 ciscoworks_services.cfg
-rwxrwxrwt 1 apache nagios 1131 May 19 11:40 citrix_6.5_service_check.cfg
-rwxrwxrwt 1 apache nagios 1225 May 19 11:40 citrix_service_check.cfg
-rwxrwxrwt 1 apache nagios 1132 May 19 11:40 citrix_standard_service_check.cfg
-rwxrwxrwt 1 apache nagios 1056 May 19 11:40 dhcp_services.cfg
-rwxrwxrwt 1 apache nagios 1155 May 19 11:40 dmzatriusns.cfg
-rwxrwxrwt 1 apache nagios 1153 May 19 11:40 Exchange2003 Core Services.cfg
-rwxrwxrwt 1 apache nagios 1359 May 19 11:40 Exchange 2010 Core Services.cfg
-rwxrwxrwt 1 apache nagios 1138 May 19 11:40 Exchange 2010 Messages Pending Routing.cfg
-rwxrwxrwt 1 apache nagios 1161 May 19 11:40 Exchange 2010 Remote Queue Length.cfg
-rwxrwxrwt 1 apache nagios 1052 May 19 11:40 Exchange 2010 SMTP.cfg
-rwxrwxrwt 1 apache nagios 1751 May 19 11:40 Exchange Blacklist Status.cfg
-rwxrwxrwt 1 apache nagios 1260 May 19 11:40 Exchange Mailbox Core Services.cfg
-rwxrwxrwt 1 apache nagios 1007 May 19 11:40 harvestclient_process.cfg
-rwxrwxrwt 1 apache nagios  923 May 19 11:40 host_ping.cfg
-rwxrwxrwt 1 apache nagios 1033 May 19 11:40 iis_process_check.cfg
-rwxrwxrwt 1 apache nagios 1204 May 19 11:40 Liebert_UPS_Battery_Capacity.cfg
-rwxrwxrwt 1 apache nagios 1210 May 19 11:40 Liebert_UPS_Battery_Capacity_CON.cfg
-rwxrwxrwt 1 apache nagios 1140 May 19 11:40 Liebert_UPS_Connectivity.cfg
-rwxrwxrwt 1 apache nagios 1087 May 19 11:40 Liebert_UPS_Input_Voltage.cfg
-rwxrwxrwt 1 apache nagios 1097 May 19 11:40 Liebert_UPS_Model.cfg
-rwxrwxrwt 1 apache nagios 1108 May 19 11:40 Liebert_UPS_Runtime.cfg
-rwxrwxrwt 1 apache nagios 1220 May 19 11:40 Liebert_UPS_Serial_number.cfg
-rwxrwxrwt 1 apache nagios 1169 May 19 11:40 Liebert_UPS_TEMP.cfg
-rwxrwxrwt 1 apache nagios 2593 May 19 11:40 localhost.cfg
-rwxrwxrwt 1 apache nagios 1104 May 19 11:40 lync_service_check.cfg
-rwxrwxrwt 1 apache nagios  981 May 19 11:40 mapper_process.cfg
-rwxrwxrwt 1 apache nagios 1035 May 19 11:40 muse_modem_service_csi_111.cfg
-rwxrwxrwt 1 apache nagios 1035 May 19 11:40 muse_modem_service_csi_150.cfg
-rwxrwxrwt 1 apache nagios 1042 May 19 11:40 muse_modem_service_csi_89.cfg
-rwxrwxrwt 1 apache nagios 1946 May 19 11:40 myhealthadmin.atriushealth.org.cfg
-rwxrwxrwt 1 apache nagios 2094 May 19 11:40 myhealth.atriushealth.org.cfg
-rwxrwxrwt 1 apache nagios 1116 May 19 11:40 netbackup_client_service.cfg
-rwxrwxrwt 1 apache nagios 3278 May 19 11:40 owa.atriushealth.org.cfg
-rwxrwxrwt 1 apache nagios 1101 May 19 11:40 paloxp_services.cfg
-rwxrwxrwt 1 apache nagios 1176 May 19 11:40 rightfax_service_check.cfg
-rwxrwxrwt 1 apache nagios  866 Jan 25  2013 sccm_service_check.cfg
-rwxrwxrwt 1 apache nagios  998 May 19 11:40 sccm_services_check.cfg
-rwxrwxrwt 1 apache nagios 1225 May 19 11:40 security_services.cfg
-rwxrwxrwt 1 apache nagios 2290 May 19 11:40 Sherlock.cfg
-rwxrwxrwt 1 apache nagios 1129 May 19 11:40 SSL Certificate.cfg
-rwxrwxrwt 1 apache nagios 3987 May 19 11:40 webmail.healthonecare.org.cfg
-rwxrwxrwt 1 apache nagios 1267 May 19 11:40 windows_all_disk_usage.cfg
-rwxrwxrwt 1 apache nagios 1270 May 19 11:40 windows-cpuload.cfg
-rwxrwxrwt 1 apache nagios 1068 May 19 11:40 windows_disk_usage_C_drive.cfg
-rwxrwxrwt 1 apache nagios 1028 May 19 11:40 windows_disk_usage_D_drive.cfg
-rwxrwxrwt 1 apache nagios 1028 May 19 11:40 windows_disk_usage_F_drive.cfg
-rwxrwxrwt 1 apache nagios 1028 May 19 11:40 windows_disk_usage_H_drive.cfg
-rwxrwxrwt 1 apache nagios 1048 May 19 11:40 windows_disk_usage_M_drive.cfg
-rwxrwxrwt 1 apache nagios 1279 May 19 11:40 windows_memory.cfg
-rwxrwxrwt 1 apache nagios 1263 May 19 11:40 windows_nsclient.cfg
-rwxrwxrwt 1 apache nagios 1035 May 19 11:40 Windows_Services_Check.cfg
-rwxrwxrwt 1 apache nagios 1047 May 19 11:40 Windows_Services_wo_McShield.cfg
-rwxrwxrwt 1 apache nagios 1815 May 19 11:40 wkenmyhlthp01.healthone.org.cfg
-rwxrwxrwt 1 apache nagios 1822 May 19 11:40 wkenmyhlthp02.healthone.org.cfg
-rwxrwxrwt 1 apache nagios 1970 May 19 11:40 wkenorionp01.healthone.org.cfg
-rwxrwxrwt 1 apache nagios 1060 May 19 11:40 WKENPHFP01_PhoneFactor_Services.cfg
-rwxrwxrwt 1 apache nagios 1169 May 19 11:40 WKENPHFP01_SSL_Certificate.cfg
btw, I ran the

Code: Select all

usr/local/nagiosci/scripts/reset_config_perms.sh
yesterday too.

Re: this is CRAZEE

Posted: Wed May 28, 2014 10:37 am
by scottwilkerson
This permission is for sure wrong

Code: Select all

drwxrwxr-x 8 root   root     4096 Dec  5  2012 /usr/local/nagios
Lets run

Code: Select all

chown nagios.nagios /usr/local/nagios

Re: this is CRAZEE

Posted: Wed May 28, 2014 11:36 am
by benhank

Code: Select all

[root@Lkennagiosp02 ~]# chown nagios.nagios /usr/local/nagios
You have new mail in /var/spool/mail/root
[root@Lkennagiosp02 ~]# service nagios restart
Running configuration check...done.
Stopping nagios: No lock file found in /usr/local/nagios/var/nagios.lock
Starting nagios: done.
You have new mail in /var/spool/mail/root
[root@Lkennagiosp02 ~]# service mysqld  restart
Stopping mysqld:                                           [  OK  ]
Starting mysqld:                                           [  OK  ]
[root@Lkennagiosp02 ~]# service postgresql restart
Stopping postgresql service:                               [  OK  ]
Starting postgresql service:                               [  OK  ]
[root@Lkennagiosp02 ~]# service npcd restart
NPCD Stopped.
NPCD started.
[root@Lkennagiosp02 ~]# service ndo2db restart
Stopping ndo2db: done.
Starting ndo2db: done.
[root@Lkennagiosp02 ~]# ls -l /usr/local/nagios/etc/services
total 324
-rwxrwxrwt 1 apache nagios  910 May 19 11:40 APC_ups_Standard.cfg
-rwxrwxrwt 1 apache nagios 1127 May 19 11:40 atriusnsa_host_ping.cfg
-rwxrwxrwt 1 apache nagios 1127 May 19 11:40 atriusnsb_host_ping.cfg
-rwxrwxrwt 1 apache nagios  893 May 19 11:40 Avaya Error_MAJ.cfg
-rwxrwxrwt 1 apache nagios  893 May 19 11:40 Avaya Error_MIN.cfg
-rwxrwxrwt 1 apache nagios 1051 May 19 11:40 bds_service_check.cfg
-rwxrwxrwt 1 apache nagios  912 May 19 11:40 Check_APC_Verbose.cfg
-rwxrwxrwt 1 apache nagios  810 May 19 11:40 check_cisco_fru_fan.cfg
-rwxrwxrwt 1 apache nagios  808 May 19 11:40 check_cisco_fru_ps.cfg
-rwxrwxrwt 1 apache nagios 1120 May 19 11:40 check_dhcp_scope.cfg
-rwxrwxrwt 1 apache nagios  971 May 19 11:40 check_eventlog_EventID_1008.cfg
-rwxrwxrwt 1 apache nagios  971 May 19 11:40 check_eventlog_EventID_1009.cfg
-rwxrwxrwt 1 apache nagios  916 May 19 11:40 check_eventlog_EventID_2020.cfg
-rwxrwxrwt 1 apache nagios  928 May 19 11:40 check_eventlog_EventID_333.cfg
-rwxrwxrwt 1 apache nagios  970 May 19 11:40 check_eventlog_EventID_3621.cfg
-rwxrwxrwt 1 apache nagios  970 May 19 11:40 check_eventlog_EventID_3635.cfg
-rwxrwxrwt 1 apache nagios 1180 May 19 11:40 check_interface_table_core.cfg
-rwxrwxrwt 1 apache nagios 1132 May 19 11:40 check_interface_table_edge.cfg
-rwxrwxrwt 1 apache nagios  884 May 19 11:40 check_snmp_cisco_stack.cfg
-rwxrwxrwt 1 apache nagios  869 May 19 11:40 check_snmp_environment.cfg
-rwxrwxrwt 1 apache nagios  855 May 19 11:40 check_snmp_uptime.cfg
-rwxrwxrwt 1 apache nagios 1011 May 19 11:40 check_windows_time_external.cfg
-rwxrwxrwt 1 apache nagios 1161 May 19 11:40 check_windows_time_internal.cfg
-rwxrwxrwt 1 apache nagios 1101 May 19 11:40 ciscoworks_services.cfg
-rwxrwxrwt 1 apache nagios 1131 May 19 11:40 citrix_6.5_service_check.cfg
-rwxrwxrwt 1 apache nagios 1225 May 19 11:40 citrix_service_check.cfg
-rwxrwxrwt 1 apache nagios 1132 May 19 11:40 citrix_standard_service_check.cfg
-rwxrwxrwt 1 apache nagios 1056 May 19 11:40 dhcp_services.cfg
-rwxrwxrwt 1 apache nagios 1155 May 19 11:40 dmzatriusns.cfg
-rwxrwxrwt 1 apache nagios 1153 May 19 11:40 Exchange2003 Core Services.cfg
-rwxrwxrwt 1 apache nagios 1359 May 19 11:40 Exchange 2010 Core Services.cfg
-rwxrwxrwt 1 apache nagios 1138 May 19 11:40 Exchange 2010 Messages Pending Routing.cfg
-rwxrwxrwt 1 apache nagios 1161 May 19 11:40 Exchange 2010 Remote Queue Length.cfg
-rwxrwxrwt 1 apache nagios 1052 May 19 11:40 Exchange 2010 SMTP.cfg
-rwxrwxrwt 1 apache nagios 1751 May 19 11:40 Exchange Blacklist Status.cfg
-rwxrwxrwt 1 apache nagios 1260 May 19 11:40 Exchange Mailbox Core Services.cfg
-rwxrwxrwt 1 apache nagios 1007 May 19 11:40 harvestclient_process.cfg
-rwxrwxrwt 1 apache nagios  923 May 19 11:40 host_ping.cfg
-rwxrwxrwt 1 apache nagios 1033 May 19 11:40 iis_process_check.cfg
-rwxrwxrwt 1 apache nagios 1204 May 19 11:40 Liebert_UPS_Battery_Capacity.cfg
-rwxrwxrwt 1 apache nagios 1210 May 19 11:40 Liebert_UPS_Battery_Capacity_CON.cfg
-rwxrwxrwt 1 apache nagios 1140 May 19 11:40 Liebert_UPS_Connectivity.cfg
-rwxrwxrwt 1 apache nagios 1087 May 19 11:40 Liebert_UPS_Input_Voltage.cfg
-rwxrwxrwt 1 apache nagios 1097 May 19 11:40 Liebert_UPS_Model.cfg
-rwxrwxrwt 1 apache nagios 1108 May 19 11:40 Liebert_UPS_Runtime.cfg
-rwxrwxrwt 1 apache nagios 1220 May 19 11:40 Liebert_UPS_Serial_number.cfg
-rwxrwxrwt 1 apache nagios 1169 May 19 11:40 Liebert_UPS_TEMP.cfg
-rwxrwxrwt 1 apache nagios 2593 May 19 11:40 localhost.cfg
-rwxrwxrwt 1 apache nagios 1104 May 19 11:40 lync_service_check.cfg
-rwxrwxrwt 1 apache nagios  981 May 19 11:40 mapper_process.cfg
-rwxrwxrwt 1 apache nagios 1035 May 19 11:40 muse_modem_service_csi_111.cfg
-rwxrwxrwt 1 apache nagios 1035 May 19 11:40 muse_modem_service_csi_150.cfg
-rwxrwxrwt 1 apache nagios 1042 May 19 11:40 muse_modem_service_csi_89.cfg
-rwxrwxrwt 1 apache nagios 1946 May 19 11:40 myhealthadmin.atriushealth.org.cfg
-rwxrwxrwt 1 apache nagios 2094 May 19 11:40 myhealth.atriushealth.org.cfg
-rwxrwxrwt 1 apache nagios 1116 May 19 11:40 netbackup_client_service.cfg
-rwxrwxrwt 1 apache nagios 3278 May 19 11:40 owa.atriushealth.org.cfg
-rwxrwxrwt 1 apache nagios 1101 May 19 11:40 paloxp_services.cfg
-rwxrwxrwt 1 apache nagios 1176 May 19 11:40 rightfax_service_check.cfg
-rwxrwxrwt 1 apache nagios  866 Jan 25  2013 sccm_service_check.cfg
-rwxrwxrwt 1 apache nagios  998 May 19 11:40 sccm_services_check.cfg
-rwxrwxrwt 1 apache nagios 1225 May 19 11:40 security_services.cfg
-rwxrwxrwt 1 apache nagios 2290 May 19 11:40 Sherlock.cfg
-rwxrwxrwt 1 apache nagios 1129 May 19 11:40 SSL Certificate.cfg
-rwxrwxrwt 1 apache nagios 3987 May 19 11:40 webmail.healthonecare.org.cfg
-rwxrwxrwt 1 apache nagios 1267 May 19 11:40 windows_all_disk_usage.cfg
-rwxrwxrwt 1 apache nagios 1270 May 19 11:40 windows-cpuload.cfg
-rwxrwxrwt 1 apache nagios 1068 May 19 11:40 windows_disk_usage_C_drive.cfg
-rwxrwxrwt 1 apache nagios 1028 May 19 11:40 windows_disk_usage_D_drive.cfg
-rwxrwxrwt 1 apache nagios 1028 May 19 11:40 windows_disk_usage_F_drive.cfg
-rwxrwxrwt 1 apache nagios 1028 May 19 11:40 windows_disk_usage_H_drive.cfg
-rwxrwxrwt 1 apache nagios 1048 May 19 11:40 windows_disk_usage_M_drive.cfg
-rwxrwxrwt 1 apache nagios 1279 May 19 11:40 windows_memory.cfg
-rwxrwxrwt 1 apache nagios 1263 May 19 11:40 windows_nsclient.cfg
-rwxrwxrwt 1 apache nagios 1035 May 19 11:40 Windows_Services_Check.cfg
-rwxrwxrwt 1 apache nagios 1047 May 19 11:40 Windows_Services_wo_McShield.cfg
-rwxrwxrwt 1 apache nagios 1815 May 19 11:40 wkenmyhlthp01.healthone.org.cfg
-rwxrwxrwt 1 apache nagios 1822 May 19 11:40 wkenmyhlthp02.healthone.org.cfg
-rwxrwxrwt 1 apache nagios 1970 May 19 11:40 wkenorionp01.healthone.org.cfg
-rwxrwxrwt 1 apache nagios 1060 May 19 11:40 WKENPHFP01_PhoneFactor_Services.cfg
-rwxrwxrwt 1 apache nagios 1169 May 19 11:40 WKENPHFP01_SSL_Certificate.cfg
You have new mail in /var/spool/mail/root
[root@Lkennagiosp02 ~]# ls -ld /usr/local/nagios /usr/local/nagios/etc /usr/local/nagios/etc/hosts /usr/local/nagios/etc/services
drwxrwxr-x 8 nagios nagios   4096 Dec  5  2012 /usr/local/nagios
drwsrwsr-x 7 apache nagios   4096 May 16 12:59 /usr/local/nagios/etc
drwsrwsrwt 2 apache nagios 118784 May 15 16:16 /usr/local/nagios/etc/hosts
drwsrwsrwt 2 apache nagios   4096 May 15 15:25 /usr/local/nagios/etc/services
[root@Lkennagiosp02 ~]#
same prob =(

Re: this is CRAZEE

Posted: Wed May 28, 2014 1:52 pm
by lmiltchev
Run the following commands:

Code: Select all

cd /usr/local/nagiosxi/scripts
./reconfigure_nagios.sh &> reconfig.txt
Then also run the following command to begin capturing log output:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log &> cmd.txt
Attempt to Apply Configuration from the web interface. After the browser has returned some output to the screen, press Ctrl+C to stop the log tail, and post the cmd.txt file and the reconfig.txt that was generated by the above instructions (in code wraps).

Also, run the following command, and post the output:

Code: Select all

grep nag /etc/group

Re: this is CRAZEE

Posted: Wed May 28, 2014 2:36 pm
by benhank
after running the commands, a video of the scene in the matrix where neo gets that fedex package plays, And ends with "get the picture" =(

my apply config is still runnin after 8 mins. I ran this in another terminal

Code: Select all

grep nag /etc/group
nagios:x:500:apache,nagios
nagcmd:x:503:nagios,apache

Re: this is CRAZEE

Posted: Wed May 28, 2014 4:01 pm
by lmiltchev
Is apply config still running?

Re: this is CRAZEE

Posted: Wed May 28, 2014 4:21 pm
by benhank
yeah

Re: this is CRAZEE

Posted: Wed May 28, 2014 4:32 pm
by abrist
open a ticket and we will set up a remote session for tomorrow.