Nagios Support Forum

Support for Nagios products and services
https://support.nagios.com/forum/

Problem with nagios and ndoutils

https://support.nagios.com/forum/viewtopic.php?t=31593

Page 4 of 5

Re: Problem with nagios and ndoutils

Posted: Mon Mar 23, 2015 11:26 am
by tgriep
Could you run the following and post back the results?
The XXXXXXXX in the mysql command is your password defined for the mysql database

Code: Select all

ipcs -q
sysctl -a
ll /usr/local/nagios/var/
ll /usr/local/nagios
echo 'show tables;' | mysql  -u ndo -pXXXXXXXXXX ndo_nagios

Re: Problem with nagios and ndoutils

Posted: Tue Mar 24, 2015 5:47 am
by eFusion
ipcs -q

Code: Select all

Message Queues:
T           ID          KEY MODE        OWNER    GROUP   
q       196608  -1609064175 --rw------- nagios   nagios  
q        65537   2031591721 --rw------- nagios   nagios  
q        65538  -1726520089 --rw------- nagios   nagios  
q       131075   -266886895 --rw------- nagios   nagios  
q        65540   2048332635 --rw------- nagios   nagios  
q        65541  -1743318181 --rw------- nagios   nagios  
q        65542   -552099543 --rw------- nagios   nagios  
q        65543   1696031964 --rw------- nagios   nagios  
q        65544   2014826232 --rw------- nagios   nagios  
q        65545  -1743270152 --rw------- nagios   nagios  
q        65546  -1827175557 --rw------- nagios   nagios  
q        65547  -2112373967 --rw------- nagios   nagios  
q        65548   2098668042 --rw------- nagios   nagios  
q        65549   -971518216 --rw------- nagios   nagios  
q        65550   1008193272 --rw------- nagios   nagios  
q        65551   1897385720 --rw------- nagios   nagios  
q        65552  -2011705608 --rw------- nagios   nagios  
q        65553  -1911042312 --rw------- nagios   nagios  
q        65554   -535350147 --rw------- nagios   nagios  
q        65555  -1458101750 --rw------- nagios   nagios  
q        65556  -1273547651 --rw------- nagios   nagios  
q        65557    286733545 --rw------- nagios   nagios  
q        65558   -552132086 --rw------- nagios   nagios  
q        65559    303505930 --rw------- nagios   nagios
sysctl
output a very large - what needs to be?

ls -lG /usr/local/nagios/var/

Code: Select all

-rw-r--r--  1 nagios  nagios  138550 Mar 23 15:59 ndo2db.debug
ls -lG /usr/local/nagios/

Code: Select all

drwxrwxr-x  2 nagios  nagios  9 Feb 26 10:49 bin
drwxrwxr-x  2 nagios  nagios  3 Mar 23 17:49 var
echo 'show tables;' | mysql -u ndo -p ndo_nagios

Code: Select all

Tables_in_ndo_nagios
nagios_acknowledgements
nagios_commands
nagios_commenthistory
nagios_comments
nagios_configfiles
nagios_configfilevariables
nagios_conninfo
nagios_contact_addresses
nagios_contact_notificationcommands
nagios_contactgroup_members
nagios_contactgroups
nagios_contactnotificationmethods
nagios_contactnotifications
nagios_contacts
nagios_contactstatus
nagios_customvariables
nagios_customvariablestatus
nagios_dbversion
nagios_downtimehistory
nagios_eventhandlers
nagios_externalcommands
nagios_flappinghistory
nagios_host_contactgroups
nagios_host_contacts
nagios_host_parenthosts
nagios_hostchecks
nagios_hostdependencies
nagios_hostescalation_contactgroups
nagios_hostescalation_contacts
nagios_hostescalations
nagios_hostgroup_members
nagios_hostgroups
nagios_hosts
nagios_hoststatus
nagios_instances
nagios_logentries
nagios_notifications
nagios_objects
nagios_processevents
nagios_programstatus
nagios_runtimevariables
nagios_scheduleddowntime
nagios_service_contactgroups
nagios_service_contacts
nagios_service_parentservices
nagios_servicechecks
nagios_servicedependencies
nagios_serviceescalation_contactgroups
nagios_serviceescalation_contacts
nagios_serviceescalations
nagios_servicegroup_members
nagios_servicegroups
nagios_services
nagios_servicestatus
nagios_statehistory
nagios_systemcommands
nagios_timedeventqueue
nagios_timedevents
nagios_timeperiod_timeranges
nagios_timeperiods

Re: Problem with nagios and ndoutils

Posted: Tue Mar 24, 2015 10:41 am
by jdalrymple
I reviewed the last 3 and change pages of the thread - doesn't appear anyone has verified that selinux isn't running. Is it?

Code: Select all

[jdalrymple@localhost ~]$ sestatus
SELinux status:                 disabled

Re: Problem with nagios and ndoutils

Posted: Wed Mar 25, 2015 3:51 am
by eFusion
sestatus

Code: Select all

bash: sestatus: command not found
I have FreeBSD server.

Re: Problem with nagios and ndoutils

Posted: Wed Mar 25, 2015 11:56 am
by tgriep
Can you edit your ndomod.cfg file and change

Code: Select all

instance_name=default
to

Code: Select all

instance_name=localhost
Restart nagios and see if it works.

See if this file is created here after restarting.

Code: Select all

buffer_file=/usr/local/nagios/var/ndomod.tmp

Re: Problem with nagios and ndoutils

Posted: Thu Mar 26, 2015 2:33 am
by eFusion
Did as you said but file does not appear.
I created it and after the next restart he disappeared.

Re: Problem with nagios and ndoutils

Posted: Thu Mar 26, 2015 12:05 pm
by tgriep
Can you edit your ndomod.cfg file and remove the following lines. Save it out and restart the nagios service.

Code: Select all

acknowledgement_data=1
adaptive_contact_data=1
adaptive_host_data=1
adaptive_program_data=1
adaptive_service_data=1
aggregated_status_data=1
comment_data=1
contact_status_data=1
downtime_data=1
event_handler_data=1
external_command_data=1
flapping_data=1
host_check_data=1
host_status_data=1
log_data=1
main_config_data=1
notification_data=1
object_config_data=1
process_data=1
program_status_data=1
retention_data=1
service_check_data=1
service_status_data=1
statechange_data=1
system_command_data=1
timed_event_data=1
Wait some time and upload the following file to this post.

Code: Select all

/usr/local/nagios/var/ndo2db.debug
Also, get the last 200 lines from your nagios.log file and post it here.

Re: Problem with nagios and ndoutils

Posted: Fri Mar 27, 2015 2:45 am
by eFusion
ndo2db.debug

Code: Select all

INSERT INTO nagios_conninfo SET instance_id='2', connect_time=NOW(), last_checkin_time=NOW(), bytes_processed='0', lines_processed='0', entries_proc
354933.354198] [002.0] [pid=61963] SELECT object_id, objecttype_id, name1, name2 FROM nagios_objects WHERE instance_id='2'
354933.358868] [002.0] [pid=61963] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_programstatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
354933.360119] [002.0] [pid=61963] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_hoststatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
354933.364240] [002.0] [pid=61963] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_servicestatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
354933.364986] [002.0] [pid=61963] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_contactstatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
354933.365609] [002.0] [pid=61963] SELECT UNIX_TIMESTAMP(queued_time) AS latest_time FROM nagios_timedeventqueue WHERE instance_id='2' ORDER BY queued_time DESC LIMIT 0,1
354933.366219] [002.0] [pid=61963] SELECT UNIX_TIMESTAMP(entry_time) AS latest_time FROM nagios_comments WHERE instance_id='2' ORDER BY entry_time DESC LIMIT 0,1
354933.366924] [002.0] [pid=61963] UPDATE nagios_conninfo SET last_checkin_time=NOW(), bytes_processed='678', lines_processed='63', entries_processed='0' WHERE conninfo_id='18'
354933.367121] [002.0] [pid=61963] SELECT * FROM nagios_objects WHERE instance_id='2' AND objecttype_id='1' AND BINARY name1='don-dz49b-p0-n1-dlink-dcs-2103' AND name2 IS NULL
354933.412648] [002.0] [pid=61963] INSERT INTO nagios_objects SET instance_id='2', objecttype_id='1' , name1='don-dz49b-p0-n1-dlink-dcs-2103'
354933.412827] [002.0] [pid=61963] SELECT * FROM nagios_objects WHERE instance_id='2' AND objecttype_id='9' AND BINARY name1='24x7' AND name2 IS NULL
354933.413003] [002.0] [pid=61963] INSERT INTO nagios_objects SET instance_id='2', objecttype_id='9' , name1='24x7'
354933.413191] [002.0] [pid=61963] INSERT INTO nagios_hoststatus SET instance_id='2', host_object_id='31', status_update_time=FROM_UNIXTIME(1424941246), output='PING OK - Packet loss
354933.413972] [002.0] [pid=61963] DELETE FROM nagios_timedevents WHERE instance_id='2' AND scheduled_time<FROM_UNIXTIME(1427268533)
354933.414597] [002.0] [pid=61963] DELETE FROM nagios_systemcommands WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426750133)
354933.421145] [002.0] [pid=61963] DELETE FROM nagios_servicechecks WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426750133)
354933.421772] [002.0] [pid=61963] DELETE FROM nagios_hostchecks WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426750133)
354933.422411] [002.0] [pid=61963] DELETE FROM nagios_eventhandlers WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1424676533)
354933.423050] [002.0] [pid=61963] DELETE FROM nagios_externalcommands WHERE instance_id='2' AND entry_time<FROM_UNIXTIME(1424676533)
354933.423720] [002.0] [pid=61963] SELECT * FROM nagios_objects WHERE instance_id='2' AND objecttype_id='1' AND BINARY name1='don-dz49b-p0-n11-cisco-2851' AND name2 IS NULL
354933.430754] [002.0] [pid=61963] INSERT INTO nagios_objects SET instance_id='2', objecttype_id='1' , name1='don-dz49b-p0-n11-cisco-2851'
354933.430917] [002.0] [pid=61963] INSERT INTO nagios_hoststatus SET instance_id='2', host_object_id='33', status_update_time=FROM_UNIXTIME(1424941246), output='PING OK - Packet loss
355085.219624] [002.0] [pid=65713] SELECT instance_id FROM nagios_instances WHERE instance_name='localhost'
355085.219819] [002.0] [pid=65713] INSERT INTO nagios_conninfo SET instance_id='2', connect_time=NOW(), last_checkin_time=NOW(), bytes_processed='0', lines_processed='0', entries_proc
355085.220006] [002.0] [pid=65713] SELECT object_id, objecttype_id, name1, name2 FROM nagios_objects WHERE instance_id='2'
355085.220157] [002.0] [pid=65713] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_programstatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
355085.220254] [002.0] [pid=65713] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_hoststatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
355085.220445] [002.0] [pid=65713] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_servicestatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
355085.220563] [002.0] [pid=65713] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_contactstatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
355085.220663] [002.0] [pid=65713] SELECT UNIX_TIMESTAMP(queued_time) AS latest_time FROM nagios_timedeventqueue WHERE instance_id='2' ORDER BY queued_time DESC LIMIT 0,1
355085.220754] [002.0] [pid=65713] SELECT UNIX_TIMESTAMP(entry_time) AS latest_time FROM nagios_comments WHERE instance_id='2' ORDER BY entry_time DESC LIMIT 0,1
355085.220934] [002.0] [pid=65713] UPDATE nagios_conninfo SET last_checkin_time=NOW(), bytes_processed='677', lines_processed='63', entries_processed='0' WHERE conninfo_id='19'
355085.221132] [002.0] [pid=65713] SELECT * FROM nagios_objects WHERE instance_id='2' AND objecttype_id='1' AND BINARY name1='don-dz55-p0-n0-zyxel-es31244f' AND name2 IS NULL
355085.221319] [002.0] [pid=65713] INSERT INTO nagios_objects SET instance_id='2', objecttype_id='1' , name1='don-dz55-p0-n0-zyxel-es31244f'
355085.221518] [002.0] [pid=65713] INSERT INTO nagios_hoststatus SET instance_id='2', host_object_id='34', status_update_time=FROM_UNIXTIME(1424941246), output='PING OK - Packet loss
355085.222035] [002.0] [pid=65713] DELETE FROM nagios_timedevents WHERE instance_id='2' AND scheduled_time<FROM_UNIXTIME(1427268685)
355085.222150] [002.0] [pid=65713] DELETE FROM nagios_systemcommands WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426750285)
355085.222256] [002.0] [pid=65713] DELETE FROM nagios_servicechecks WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426750285)
355085.222352] [002.0] [pid=65713] DELETE FROM nagios_hostchecks WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426750285)
355085.222449] [002.0] [pid=65713] DELETE FROM nagios_eventhandlers WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1424676685)
355085.222550] [002.0] [pid=65713] DELETE FROM nagios_externalcommands WHERE instance_id='2' AND entry_time<FROM_UNIXTIME(1424676685)
439479.270718] [002.0] [pid=36141] SELECT instance_id FROM nagios_instances WHERE instance_name='localhost'
439479.331799] [002.0] [pid=36141] INSERT INTO nagios_conninfo SET instance_id='2', connect_time=NOW(), last_checkin_time=NOW(), bytes_processed='0', lines_processed='0', entries_proc
439479.340394] [002.0] [pid=36141] SELECT object_id, objecttype_id, name1, name2 FROM nagios_objects WHERE instance_id='2'
439479.362311] [002.0] [pid=36141] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_programstatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
439479.362947] [002.0] [pid=36141] SELECT UNIX_TIMESTAMP(status_update_time) AS latest_time FROM nagios_hoststatus WHERE instance_id='2' ORDER BY status_update_time DESC LIMIT 0,1
439479.365355] [002.0] [pid=36141] SELECT UNIX_TIMESTAMP(queued_time) AS latest_time FROM nagios_timedeventqueue WHERE instance_id='2' ORDER BY queued_time DESC LIMIT 0,1
439479.365920] [002.0] [pid=36141] SELECT UNIX_TIMESTAMP(entry_time) AS latest_time FROM nagios_comments WHERE instance_id='2' ORDER BY entry_time DESC LIMIT 0,1
439479.366592] [002.0] [pid=36141] UPDATE nagios_conninfo SET last_checkin_time=NOW(), bytes_processed='678', lines_processed='63', entries_processed='0' WHERE conninfo_id='20'
439479.366779] [002.0] [pid=36141] SELECT * FROM nagios_objects WHERE instance_id='2' AND objecttype_id='1' AND BINARY name1='don-fz46a-p0-n0-zyxel-es31244f' AND name2 IS NULL
439479.394063] [002.0] [pid=36141] INSERT INTO nagios_objects SET instance_id='2', objecttype_id='1' , name1='don-fz46a-p0-n0-zyxel-es31244f'
439479.394290] [002.0] [pid=36141] INSERT INTO nagios_hoststatus SET instance_id='2', host_object_id='35', status_update_time=FROM_UNIXTIME(1424941246), output='PING OK - Packet loss
439479.394879] [002.0] [pid=36141] DELETE FROM nagios_timedevents WHERE instance_id='2' AND scheduled_time<FROM_UNIXTIME(1427353079)
439479.395497] [002.0] [pid=36141] DELETE FROM nagios_systemcommands WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426834679)
439479.404019] [002.0] [pid=36141] DELETE FROM nagios_servicechecks WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426834679)
439479.404633] [002.0] [pid=36141] DELETE FROM nagios_hostchecks WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1426834679)
439479.405235] [002.0] [pid=36141] DELETE FROM nagios_eventhandlers WHERE instance_id='2' AND start_time<FROM_UNIXTIME(1424761079)
439479.405813] [002.0] [pid=36141] DELETE FROM nagios_externalcommands WHERE instance_id='2' AND entry_time<FROM_UNIXTIME(1424761079)
439479.406424] [002.0] [pid=36141] SELECT * FROM nagios_objects WHERE instance_id='2' AND objecttype_id='1' AND BINARY name1='don-ga5-p2-n0-dlink-des3200-26' AND name2 IS NULL
439479.406596] [002.0] [pid=36141] INSERT INTO nagios_objects SET instance_id='2', objecttype_id='1' , name1='don-ga5-p2-n0-dlink-des3200-26'
439479.406751] [002.0] [pid=36141] INSERT INTO nagios_hoststatus SET instance_id='2', host_object_id='36', status_update_time=FROM_UNIXTIME(1424941246), output='PING OK - Packet loss
nagios.log

Code: Select all

[1427439463] Event broker module '/usr/local/bin/ndomod-3x.o' initialized successfully.
[1427439464] Finished daemonizing... (New PID=35839)
[1427439479] ndomod: Successfully connected to data sink.  16607 items lost, 5000 queued items to flush.
[1427441948] ndomod: Error writing to data sink!  Some output may get lost.  4982 queued items to flush.
[1427441948] ndomod: Still unable to reconnect to data sink.  16607 items lost, 4982 queued items to flush.
[1427441948] Warning: A system time change of 0d 0h 41m 9s (forwards in time) has been detected.  Compensating...
[1427441951] Caught SIGTERM, shutting down...
[1427441951] Successfully shutdown... (PID=35839)
[1427441951] ndomod: Shutdown complete.
[1427441951] Event broker module '/usr/local/bin/ndomod-3x.o' deinitialized successfully.
[1427441952] Nagios 3.4.1 starting... (PID=73766)
[1427441952] Local time is Fri Mar 27 10:39:12 MSK 2015
[1427441952] LOG VERSION: 2.0
[1427441952] ndomod: NDOMOD 2.0.0 (02-28-2014) Copyright (c) 2009 Nagios Core Development Team and Community Contributors
[1427441952] ndomod: Could not open data sink!  I'll keep trying, but some output may get lost...
[1427441952] ndomod registered for process data
[1427441952] ndomod registered for timed event data
[1427441952] ndomod registered for log data'
[1427441952] ndomod registered for system command data'
[1427441952] ndomod registered for event handler data'
[1427441952] ndomod registered for notification data'
[1427441952] ndomod registered for service check data'
[1427441952] ndomod registered for host check data'
[1427441952] ndomod registered for comment data'
[1427441952] ndomod registered for downtime data'
[1427441952] ndomod registered for flapping data'
[1427441952] ndomod registered for program status data'
[1427441952] ndomod registered for host status data'
[1427441952] ndomod registered for service status data'
[1427441952] ndomod registered for adaptive program data'
[1427441952] ndomod registered for adaptive host data'
[1427441952] ndomod registered for adaptive service data'
[1427441952] ndomod registered for external command data'
[1427441952] ndomod registered for aggregated status data'
[1427441952] ndomod registered for retention data'
[1427441952] ndomod registered for contact data'
[1427441952] ndomod registered for contact notification data'
[1427441952] ndomod registered for acknowledgement data'
[1427441952] ndomod registered for state change data'
[1427441952] ndomod registered for contact status data'
[1427441952] ndomod registered for adaptive contact data'
[1427441952] Event broker module '/usr/local/bin/ndomod-3x.o' initialized successfully.
[1427441952] Finished daemonizing... (New PID=73767)
[1427441968] ndomod: Still unable to connect to data sink.  16608 items lost, 5000 queued items to flush.
[1427441973] SERVICE ALERT: don-co90-p2-n0-dlink-des3200-10;check_ping;CRITICAL;HARD;2;CRITICAL - Host Unreachable
[1427441973] SERVICE ALERT: don-ca12-p0-n0-dlink-des3200-28;check_ping;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1427441973] SERVICE ALERT: don-do8-p0-n2-dlink-des3200-28;check_ping;CRITICAL;HARD;1;CRITICAL - Host Unreachable
[1427441973] SERVICE ALERT: don-do8-p0-n0-dlink-des3200-28;check_ping;CRITICAL;HARD;1;CRITICAL - Host Unreachable
[1427441973] SERVICE ALERT: don-co56-p0-n0-dlink-des3200-28;check_ping;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1427441973] SERVICE ALERT: don-co96a-p0-n0-dlink-des3200-28;check_ping;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1427441973] SERVICE ALERT: don-co90-p0-n0-linksys-sps224g4;check_ping;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1427441973] SERVICE ALERT: don-co90-p1-n0-dlink-des3200-10;check_ping;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1427441973] SERVICE ALERT: don-co90-p0-n1-zyxel-es31244f;check_ping;CRITICAL;HARD;2;PING CRITICAL - Packet loss = 100%
[1427441973] SERVICE ALERT: don-du101-p0-n0-dlink-des3200-10;check_ping;CRITICAL;HARD;1;PING CRITICAL - Packet loss = 100%
[1427441984] ndomod: Still unable to connect to data sink.  18843 items lost, 5000 queued items to flush.
[1427441993] SERVICE ALERT: don-co90-p2-n0-dlink-des3200-10;check_ping;OK;HARD;2;PING OK - Packet loss = 0%, RTA = 2.58 ms
[1427441993] SERVICE ALERT: don-ca12-p0-n0-dlink-des3200-28;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 1.77 ms
[1427441993] SERVICE ALERT: don-do8-p0-n2-dlink-des3200-28;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 1.64 ms
[1427441993] SERVICE ALERT: don-do8-p0-n0-dlink-des3200-28;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 1.59 ms
[1427441993] SERVICE ALERT: don-co56-p0-n0-dlink-des3200-28;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 2.26 ms
[1427441993] SERVICE ALERT: don-co96a-p0-n0-dlink-des3200-28;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 1.66 ms
[1427441993] SERVICE ALERT: don-co90-p0-n0-linksys-sps224g4;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 2.52 ms
[1427441993] SERVICE ALERT: don-co90-p1-n0-dlink-des3200-10;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 3.55 ms
[1427441993] SERVICE ALERT: don-co90-p0-n1-zyxel-es31244f;check_ping;OK;HARD;2;PING OK - Packet loss = 0%, RTA = 2.34 ms
[1427441993] SERVICE ALERT: don-du101-p0-n0-dlink-des3200-10;check_ping;OK;HARD;1;PING OK - Packet loss = 0%, RTA = 2.53 ms
[1427441993] HOST ALERT: don-ca12-p0-n0-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.57 ms
[1427441993] HOST ALERT: don-ca12-p0-n1-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.82 ms
[1427441993] HOST ALERT: don-co90-p0-n0-linksys-sps224g4;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.49 ms
[1427441993] HOST ALERT: don-co56-p0-n0-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.35 ms
[1427441993] HOST NOTIFICATION: support;don-co56-p0-n0-dlink-des3200-28;UP;notify-host-by-sendmail;PING OK - Packet loss = 0%, RTA = 2.35 ms
[1427441993] HOST NOTIFICATION: noc-workhours;don-co56-p0-n0-dlink-des3200-28;UP;notify-host-by-sendmail;PING OK - Packet loss = 0%, RTA = 2.35 ms
[1427441993] HOST NOTIFICATION: maintenance;don-co56-p0-n0-dlink-des3200-28;UP;notify-host-by-sms-delay;PING OK - Packet loss = 0%, RTA = 2.35 ms
[1427441993] HOST ALERT: don-co56-p0-n1-dlink-des3200-26;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.12 ms
[1427441993] HOST ALERT: don-co90-p0-n1-zyxel-es31244f;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.76 ms
[1427441993] HOST ALERT: don-co90-p1-n0-dlink-des3200-10;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.43 ms
[1427441993] HOST ALERT: don-co90-p2-n0-dlink-des3200-10;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.64 ms
[1427441993] HOST ALERT: don-co96a-p0-n0-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.80 ms
[1427442000] ndomod: Still unable to connect to data sink.  19933 items lost, 5000 queued items to flush.
[1427442013] HOST ALERT: don-do8-p0-n2-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.60 ms
[1427442013] HOST ALERT: don-do8-p0-n0-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.88 ms
[1427442013] HOST ALERT: don-du101-p0-n0-dlink-des3200-10;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.47 ms
[1427442013] HOST ALERT: don-us17-p0-n0-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.78 ms
[1427442013] HOST ALERT: don-do8-p0-n1-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.62 ms
[1427442013] HOST ALERT: don-co115a-p0-n0-dlink-des3200-10;DOWN;HARD;1;PING CRITICAL - Packet loss = 100%
[1427442016] ndomod: Still unable to connect to data sink.  20670 items lost, 5000 queued items to flush.
[1427442032] ndomod: Still unable to connect to data sink.  20854 items lost, 5000 queued items to flush.
[1427442033] HOST ALERT: don-us17-p0-n1-dlink-des3200-28;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.66 ms
[1427442048] ndomod: Still unable to connect to data sink.  21541 items lost, 5000 queued items to flush.
[1427442053] HOST ALERT: don-kt13-p4-n1-dlink-des3200-18;DOWN;SOFT;1;CRITICAL - Host Unreachable
[1427442053] HOST ALERT: don-kt21-p1-n0-dlink-des3028;DOWN;SOFT;1;CRITICAL - Host Unreachable
[1427442064] ndomod: Still unable to connect to data sink.  22245 items lost, 5000 queued items to flush.
[1427442073] HOST ALERT: don-kt7-p1-n0-zyxel-es31244f;DOWN;SOFT;1;CRITICAL - Host Unreachable
[1427442073] HOST ALERT: don-kt13-p4-n0-zyxel-gs4012f;DOWN;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt13-p2-n0-zyxel-es31244f;DOWN;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt15-p1-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt13-p0-n0-linksys-sps224g4;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt17-p1-n0-zyxel-es31244f;DOWN;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt29-p1-n0-zyxel-es31244f;DOWN;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt27-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt19-p2-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt25-p1-n0-dlink-des3200-26;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442073] HOST ALERT: don-kt3-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442080] ndomod: Still unable to connect to data sink.  23024 items lost, 5000 queued items to flush.
[1427442093] HOST ALERT: don-kt7-p1-n0-zyxel-es31244f;UNREACHABLE;SOFT;2;CRITICAL - Host Unreachable
[1427442093] HOST ALERT: don-zv54-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;CRITICAL - Host Unreachable
[1427442093] HOST ALERT: don-zv59-p0-n0-dlink-des3200-26;UNREACHABLE;SOFT;1;CRITICAL - Host Unreachable
[1427442093] HOST ALERT: don-zv55-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-se19-p0-n0-dlink-des3200-26;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-se21-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-kt13-p4-n1-dlink-des3200-18;UNREACHABLE;SOFT;2;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-zv63-p0-n0-dlink-des3200-10;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-kt21-p1-n0-dlink-des3028;UNREACHABLE;SOFT;2;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-kt13-p2-n0-zyxel-es31244f;UNREACHABLE;SOFT;2;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-kt29-p1-n0-zyxel-es31244f;UNREACHABLE;SOFT;2;PING CRITICAL - Packet loss = 100%
[1427442093] HOST ALERT: don-kt17-p1-n0-zyxel-es31244f;UNREACHABLE;SOFT;2;PING CRITICAL - Packet loss = 100%
[1427442096] ndomod: Still unable to connect to data sink.  23783 items lost, 5000 queued items to flush.
[1427442112] ndomod: Still unable to connect to data sink.  23966 items lost, 5000 queued items to flush.
[1427442113] HOST ALERT: don-zv52-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;CRITICAL - Host Unreachable
[1427442113] HOST ALERT: don-pe119b-p3-n0-zyxel-es31244f;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.82 ms
[1427442113] HOST ALERT: don-pe119-p2-n0-dlink-des3028;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 2.07 ms
[1427442113] HOST ALERT: don-pe119a-p0-n0-dlink-des3028;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.94 ms
[1427442113] HOST ALERT: don-zv56-p0-n0-dlink-des3200-26;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442113] HOST ALERT: don-se17-p0-n0-zyxel-es31244f;UNREACHABLE;SOFT;1;PING CRITICAL - Packet loss = 100%
[1427442128] ndomod: Still unable to connect to data sink.  24730 items lost, 5000 queued items to flush.
[1427442133] HOST ALERT: don-kt21-p1-n0-dlink-des3028;UNREACHABLE;HARD;3;CRITICAL - Host Unreachable
[1427442133] HOST ALERT: don-kt13-p4-n1-dlink-des3200-18;UNREACHABLE;HARD;3;PING CRITICAL - Packet loss = 100%
[1427442144] ndomod: Still unable to connect to data sink.  25460 items lost, 5000 queued items to flush.

Re: Problem with nagios and ndoutils

Posted: Fri Mar 27, 2015 12:47 pm
by tgriep
Can you run the following and post back the results?

Code: Select all

echo 'show grants;' | mysql  -u ndo -pndo_nagios ndo_nagios
netstat -an
Can you post back the last 200 lines of your mysql log file?
If you have the firewall running on this system, can you disable it?
When you compiled the software, did you receive any errors?

Re: Problem with nagios and ndoutils

Posted: Mon Mar 30, 2015 2:41 am
by eFusion

Code: Select all

show grants;
+------------------------------------------------------------------------------------------------------------+
| Grants for ndo@localhost                                                                                   |
+------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'ndo'@'localhost' IDENTIFIED BY PASSWORD '*2C1BFBFF4C2C684091791638041511E6FFA0D08C' |
| GRANT ALL PRIVILEGES ON `ndo_nagios`.* TO 'ndo'@'localhost'                                                |
+------------------------------------------------------------------------------------------------------------+
2 rows in set (0.00 sec)
netstat -an

Code: Select all

Active Internet connections (including servers)
Proto Recv-Q Send-Q Local Address          Foreign Address        (state)
tcp4       0      0 193.109.164.26.52880   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.54130   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.36324   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.17867   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.57015   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.46239   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.12733   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.18381   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.37546   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.34392   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.43890   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.10106   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.13619   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.443     192.168.2.121.1603     TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.121.1602     TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.121.1601     TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.121.1600     TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.121.1599     TIME_WAIT
tcp4       0      0 193.109.164.26.56350   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.443     192.168.2.112.56963    TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.112.56962    TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.112.56960    TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.112.56961    TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.112.56959    TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.112.56958    TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.121.1598     TIME_WAIT
tcp4       0      0 193.109.164.26.3306    194.146.199.34.44506   TIME_WAIT
tcp4       0      0 193.109.164.26.3306    194.146.199.35.41389   TIME_WAIT
tcp4       0      0 193.109.164.26.31786   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.31785   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.31784   193.109.164.26.443     TIME_WAIT
tcp4       0      0 193.109.164.26.31783   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.1.4461       TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.1.4462       TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.1.4460       TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.1.4458       TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.1.4459       TIME_WAIT
tcp4       0      0 193.109.164.26.31782   91.196.230.235.389     TIME_WAIT
tcp4       0      0 193.109.164.26.443     192.168.2.1.4457       TIME_WAIT
tcp4       0      0 193.109.164.26.31781   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.443     192.168.2.121.1593     TIME_WAIT
tcp4       0      0 193.109.164.26.31780   91.196.230.235.389     TIME_WAIT
tcp4       0      0 193.109.164.26.31779   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.3306    193.109.164.26.31750   ESTABLISHED
tcp4       0      0 193.109.164.26.31750   193.109.164.26.3306    ESTABLISHED
tcp4       0      0 193.109.164.26.31558   91.196.230.235.389     TIME_WAIT
tcp4       0      0 193.109.164.26.31420   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.31317   91.196.230.235.389     ESTABLISHED
tcp4       0      0 193.109.164.26.3306    193.109.164.26.19657   ESTABLISHED
tcp4       0      0 193.109.164.26.19657   193.109.164.26.3306    ESTABLISHED
tcp4       0      0 193.109.164.26.64331   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.16359   91.196.230.235.389     TIME_WAIT
tcp4       0      0 193.109.164.26.12101   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.24724   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.29529   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.30528   193.109.164.93.389     TIME_WAIT
tcp4       0      0 193.109.164.26.30523   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.30522   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.30002   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.39096   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.63038   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.3306    193.109.164.15.55519   ESTABLISHED
tcp4       0      0 193.109.164.26.3306    *.*                    LISTEN
tcp4       0      0 193.109.164.26.11239   193.109.164.93.389     ESTABLISHED
tcp4       0      0 193.109.164.26.22      *.*                    LISTEN
tcp4       0      0 193.109.164.26.443     *.*                    LISTEN
tcp4       0      0 193.109.164.26.80      *.*                    LISTEN
tcp4       0      0 193.109.164.26.513     *.*                    LISTEN
tcp4       0      0 193.109.164.26.514     *.*                    LISTEN
tcp4       0      0 193.109.164.26.25      *.*                    LISTEN
udp4       0      0 193.109.164.26.43896   *.*                    
udp4       0      0 193.109.164.26.161     *.*                    
udp4       0      0 193.109.164.26.21822   *.*                    
udp4       0      0 193.109.164.26.514     *.*                    
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q    Inode     Conn     Refs  Nextref Addr
fffffe0293e7d5a0 stream      0      0        0 fffffe02271a4960        0        0 /tmp/mysql.sock
fffffe02271a4960 stream     31      0        0 fffffe0293e7d5a0        0        0
fffffe01f4d64b40 stream      0      0        0 fffffe002b9e9d20        0        0 /tmp/mysql.sock
fffffe002b9e9d20 stream      0      0        0 fffffe01f4d64b40        0        0
fffffe0037cae780 stream      0      0        0 fffffe0164e534b0        0        0 /tmp/mysql.sock
fffffe0164e534b0 stream      0      0        0 fffffe0037cae780        0        0
fffffe0254ea1870 stream      0      0        0 fffffe0254ea1e10        0        0 /tmp/mysql.sock
fffffe0254ea1e10 stream      0      0        0 fffffe0254ea1870        0        0
fffffe0082ba2960 stream      0      0        0 fffffe0065d444b0        0        0 /tmp/mysql.sock
fffffe0065d444b0 stream      0      0        0 fffffe0082ba2960        0        0
fffffe024d6ef690 stream      0      0        0 fffffe005945c0f0        0        0 /tmp/mysql.sock
fffffe005945c0f0 stream      0      0        0 fffffe024d6ef690        0        0
fffffe011a44bc30 stream   7831      0        0 fffffe00262b05a0        0        0 /usr/local/nagios/var/ndo.sock
fffffe00262b05a0 stream      0      0        0 fffffe011a44bc30        0        0
fffffe0034dc53c0 stream      0      0 fffffe004eab83c0        0        0        0 /usr/local/nagios/var/ndo.sock
fffffe028ea414b0 stream      0      0        0 fffffe015db661e0        0        0 /tmp/mysql.sock
fffffe015db661e0 stream      0      0        0 fffffe028ea414b0        0        0
fffffe0131d7fd20 stream      0      0        0 fffffe026e33f690        0        0 /tmp/mysql.sock
fffffe026e33f690 stream      0      0        0 fffffe0131d7fd20        0        0
fffffe0164e531e0 stream      0      0        0 fffffe0018ae6b40        0        0 /tmp/mysql.sock
fffffe0018ae6b40 stream      0      0        0 fffffe0164e531e0        0        0
fffffe001c5f5e10 stream      0      0        0 fffffe006556de10        0        0 /tmp/mysql.sock
fffffe006556de10 stream      0      0        0 fffffe001c5f5e10        0        0
fffffe014cac82d0 stream      0      0        0 fffffe000fe3c0f0        0        0 /tmp/mysql.sock
fffffe000fe3c0f0 stream      0      0        0 fffffe014cac82d0        0        0
fffffe0096d5ba50 stream      0      0        0 fffffe015c212d20        0        0 /tmp/mysql.sock
fffffe015c212d20 stream      0      0        0 fffffe0096d5ba50        0        0
fffffe01dded1d20 stream      0      0        0 fffffe002b9e9c30        0        0 /tmp/mysql.sock
fffffe002b9e9c30 stream      0      0        0 fffffe01dded1d20        0        0
fffffe018fd96b40 stream      0      0        0 fffffe025b8d0d20        0        0 /tmp/mysql.sock
fffffe025b8d0d20 stream      0      0        0 fffffe018fd96b40        0        0
fffffe0177278a50 stream      0      0        0 fffffe0146414000        0        0 /tmp/mysql.sock
fffffe0146414000 stream      0      0        0 fffffe0177278a50        0        0
fffffe0018ae6000 stream      0      0        0 fffffe02806d21e0        0        0 /tmp/mysql.sock
fffffe02806d21e0 stream      0      0        0 fffffe0018ae6000        0        0
fffffe015db653c0 stream      0      0        0 fffffe00b0e423c0        0        0 /tmp/mysql.sock
fffffe00b0e423c0 stream      0      0        0 fffffe015db653c0        0        0
fffffe009cda2e10 stream      0      0        0 fffffe01f4d643c0        0        0 /tmp/mysql.sock
fffffe01f4d643c0 stream      0      0        0 fffffe009cda2e10        0        0
fffffe000fe3c5a0 stream      0      0        0 fffffe006556d4b0        0        0 /tmp/mysql.sock
fffffe006556d4b0 stream      0      0        0 fffffe000fe3c5a0        0        0
fffffe008ecb0e10 stream      0      0        0 fffffe01f4d644b0        0        0 /tmp/mysql.sock
fffffe01f4d644b0 stream      0      0        0 fffffe008ecb0e10        0        0
fffffe017495a5a0 stream      0      0        0 fffffe01dded1c30        0        0 /tmp/mysql.sock
fffffe01dded1c30 stream      0      0        0 fffffe017495a5a0        0        0
fffffe0131d7f5a0 stream      0      0        0        0        0        0
fffffe000fb47c30 stream      0      0        0        0        0        0
fffffe0164e533c0 stream      0      0        0        0        0        0
fffffe0082ba2d20 stream      0      0        0        0        0        0
fffffe026513fd20 stream      0      0        0        0        0        0
fffffe0069f15780 stream      0      0        0        0        0        0
fffffe005b7b6a50 stream      0      0        0        0        0        0
fffffe0025169c30 stream      0      0        0        0        0        0
fffffe0237ff73c0 stream      0      0        0        0        0        0
fffffe016b7903c0 stream      0      0        0 fffffe009cda25a0        0        0
fffffe009cda25a0 stream      0      0        0 fffffe016b7903c0        0        0
fffffe00262b0a50 stream      0      0 fffffe00850bdd20        0        0        0 /tmp/mysql.sock
fffffe000fad1870 stream      0      0 fffffe01ae770b40        0        0        0 /var/agentx/master
fffffe00b0e42d20 dgram       0      0        0 fffffe000fe125a0        0 fffffe0037caec30
fffffe0037caec30 dgram       0      0        0 fffffe000fe125a0        0 fffffe018fd961e0
fffffe018fd961e0 dgram       0      0        0 fffffe000fe125a0        0 fffffe000fcd24b0
fffffe000fe3c1e0 dgram       0      0        0 fffffe000fe12780        0 fffffe000fcd23c0
fffffe000fcd23c0 dgram       0      0        0 fffffe000fe12780        0 fffffe000fcd25a0
fffffe000fcd24b0 dgram       0      0        0 fffffe000fe125a0        0        0
fffffe000fcd25a0 dgram       0      0        0 fffffe000fe12780        0        0
fffffe000fe12780 dgram       0      0 fffffe01ae63f000        0 fffffe000fe3c1e0        0 /var/run/logpriv
fffffe000fe125a0 dgram       0      0 fffffe01ae63f1e0        0 fffffe00b0e42d20        0 /var/run/log
In mysql.log only cacti queries not a ndomod or ndo2db.
About the errors I will not say, I do not remember.
All times are UTC-05:00
Page 4 of 5

Powered by phpBB® Forum Software © phpBB Limited