Nagios Support Forum

Posted: **Wed Apr 24, 2013 2:22 am**

Hi, every night we watched nagios process freezes, it must be because he isn't doing the log rotate (nagios.log file contains events for three days ago)
for example:
this is the header of Event Log

Code: Select all

Log File Navigation
Wed Apr 24 00:00:00 CEST 2013
to
Present..

and this is the event log, and in it we see the following events, in order by day

Code: Select all

[24-04-2013 09:13:09] SERVICE ALERT: MXB2PVR;PING;OK;SOFT;2;PING OK - Packet loss = 0%, RTA = 193.74 ms
. ..  .
 .. . .
[23-04-2013 02:53:19] SERVICE ALERT: ESUEACE1;PING;OK;SOFT;2;PING OK - Packet loss = 0%, RTA = 110.89 ms
..
...
 .
[22-04-2013 01:34:17] SERVICE ALERT: MXB2CZM;PING;OK;SOFT;2;PING OK - Packet loss = 20%, RTA = 237.17 ms

can i cut by hand and move/rename log to archives manually¿?

Posted: **Wed Apr 24, 2013 11:28 am**

What is your log rotation method?

Code: Select all

grep log_rotation_method /usr/local/nagios/etc/nagios.cfg

Have previous logs been rotated?

Code: Select all

grep log_archive_path /usr/local/nagios/etc/nagios.cfg
ls -la <path from previous command output above>

Posted: **Thu Apr 25, 2013 1:50 am**

abrist wrote:What is your log rotation method?
Code: Select all
grep log_rotation_method /usr/local/nagios/etc/nagios.cfg
Have previous logs been rotated?
Code: Select all
grep log_archive_path /usr/local/nagios/etc/nagios.cfg
ls -la <path from previous command output above>

the method is daily and the log archive path show a one log file per day (nagios-04-21-2013-00.log, nagios-04-20-2013-00.log,. . .)

Posted: **Thu Apr 25, 2013 2:32 pm**

fran.pastor wrote:the method is daily

is it just a d ?

fran.pastor wrote:and the log archive path show a one log file per day (nagios-04-21-2013-00.log, nagios-04-20-2013-00.log,. . .)

Do these files have contents? Specifically, the logs that are newer than Wed Apr 24 00:00:00 CEST 2013

Posted: **Wed May 15, 2013 2:18 am**

I created a script that monitors if the file status.dat is current, when it detects more than five minutes ago is not updated, rename the nagios.debug (debug_level=-1 ,debug_verbosity=2) and restart nagios. The last two nights has fallen back and these are the last lines of debug file:

day 1

Code: Select all

[1368482399.639388] [008.2] [pid=22822] No events to execute at the moment.  Idling for a bit...
[1368482399.639393] [001.0] [pid=22822] check_for_external_commands()
[1368482399.639398] [064.1] [pid=22822] Making callbacks (type 8)...
[1368482399.639403] [064.2] [pid=22822] Callback #1 (type 8) return code = 0
[1368482399.699038] [016.2] [pid=20265] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkJ8jN4F' to queue file '/usr/local/nagios/var/spool/checkresults/ccEyDlJ'...
[1368482399.706597] [016.2] [pid=20259] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check3GytQI' to queue file '/usr/local/nagios/var/spool/checkresults/cauYx8L'...
[1368482399.710496] [016.2] [pid=20263] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checktQKOZG' to queue file '/usr/local/nagios/var/spool/checkresults/cYHiwiK'...
[1368482399.713596] [016.2] [pid=20261] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkXTy3UH' to queue file '/usr/local/nagios/var/spool/checkresults/cCxsgeL'...
[1368482399.890312] [008.1] [pid=22822] ** Event Check Loop
[1368482399.890332] [008.1] [pid=22822] Next High Priority Event Time: Tue May 14 00:00:00 2013
[1368482399.890338] [008.1] [pid=22822] Next Low Priority Event Time:  Tue May 14 00:00:00 2013
[1368482399.890342] [008.1] [pid=22822] Current/Max Service Checks: 136/0
[1368482399.890380] [008.2] [pid=22822] No events to execute at the moment.  Idling for a bit...
[1368482399.890385] [001.0] [pid=22822] check_for_external_commands()
[1368482399.890390] [064.1] [pid=22822] Making callbacks (type 8)...
[1368482399.890396] [064.2] [pid=22822] Callback #1 (type 8) return code = 0
[1368482400.141305] [008.1] [pid=22822] ** Event Check Loop
[1368482400.141324] [008.1] [pid=22822] Next High Priority Event Time: Tue May 14 00:00:00 2013
[1368482400.141330] [008.1] [pid=22822] Next Low Priority Event Time:  Tue May 14 00:00:00 2013
[1368482400.141333] [008.1] [pid=22822] Current/Max Service Checks: 136/0
[1368482400.141351] [001.0] [pid=22822] handle_timed_event() start
[1368482400.141356] [064.1] [pid=22822] Making callbacks (type 8)...
[1368482400.141491] [064.1] [pid=22822] Making callbacks (type 9)...
[1368482400.141497] [064.2] [pid=22822] Callback #1 (type 9) return code = 0
[1368482400.234952] [016.2] [pid=20239] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkV2fXuO' to queue file '/usr/local/nagios/var/spool/checkresults/ckveQaT'...
[1368482400.240524] [016.2] [pid=20298] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkrceGD2' to queue file '/usr/local/nagios/var/spool/checkresults/cT6utk7'...
[1368482400.311436] [016.2] [pid=20195] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check5aGFje' to queue file '/usr/local/nagios/var/spool/checkresults/csLSYbj'...
[1368482400.325521] [016.2] [pid=20197] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkvhOljb' to queue file '/usr/local/nagios/var/spool/checkresults/ckxTWdg'...
[1368482400.332486] [016.2] [pid=20203] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkPYWgj8' to queue file '/usr/local/nagios/var/spool/checkresults/ckva0ed'...
[1368482400.359987] [016.2] [pid=20206] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkLy7rj5' to queue file '/usr/local/nagios/var/spool/checkresults/cJMiEja'...
[1368482400.363947] [016.2] [pid=20210] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkBZ0Vj2' to queue file '/usr/local/nagios/var/spool/checkresults/cx4YLk7'...
[1368482400.391618] [016.2] [pid=20212] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkbglJkZ' to queue file '/usr/local/nagios/var/spool/checkresults/c9q13p4'...
[1368482400.720692] [016.2] [pid=20312] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check5KIe3e' to queue file '/usr/local/nagios/var/spool/checkresults/cxw80Zk'...
[1368482400.724945] [016.2] [pid=20314] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkHGQ1Pg' to queue file '/usr/local/nagios/var/spool/checkresults/cthZuNm'...
[1368482401.093647] [016.2] [pid=20241] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check3FXIxN' to queue file '/usr/local/nagios/var/spool/checkresults/cE9d5sU'...
[1368482401.095569] [016.2] [pid=20243] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkFeTRAM' to queue file '/usr/local/nagios/var/spool/checkresults/c4AoxwT'...
[1368482401.177077] [016.2] [pid=20246] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkl1ueEL' to queue file '/usr/local/nagios/var/spool/checkresults/cRZR8MS'...
[1368482402.142332] [016.2] [pid=20318] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkBrEvqk' to queue file '/usr/local/nagios/var/spool/checkresults/ct18b6t'...
[1368482402.158529] [016.2] [pid=20316] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkHmf8Ci' to queue file '/usr/local/nagios/var/spool/checkresults/cjd2rls'...
[1368482402.159408] [016.2] [pid=20300] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check7RgWo4' to queue file '/usr/local/nagios/var/spool/checkresults/crjKo7d'...
[1368482402.173212] [016.2] [pid=20304] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check1aD9V7' to queue file '/usr/local/nagios/var/spool/checkresults/cBS6QGh'...
[1368482402.174422] [016.2] [pid=20302] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkTyzta6' to queue file '/usr/local/nagios/var/spool/checkresults/cPngnVf'...
[1368482402.184173] [016.2] [pid=20306] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkr2v3H9' to queue file '/usr/local/nagios/var/spool/checkresults/ctCgxuj'...
[1368482402.199747] [016.2] [pid=20308] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkL6aaub' to queue file '/usr/local/nagios/var/spool/checkresults/cpIZajl'...
[1368482402.292358] [016.2] [pid=20091] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check5AJtsi' to queue file '/usr/local/nagios/var/spool/checkresults/cK13wws'...
[1368482402.412546] [016.2] [pid=20310] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkvEizgd' to queue file '/usr/local/nagios/var/spool/checkresults/cxxT8Dn'...
[1368482403.294042] [016.2] [pid=19791] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkTXONlr' to queue file '/usr/local/nagios/var/spool/checkresults/cKg9x2D'...
[1368482403.415105] [016.2] [pid=20094] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check7BprFc' to queue file '/usr/local/nagios/var/spool/checkresults/cfrVQFp'...
[1368482404.100752] [016.2] [pid=19896] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkfODxuQ' to queue file '/usr/local/nagios/var/spool/checkresults/cdTDji5'...
[1368482404.235515] [016.2] [pid=20253] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkruo1HK' to queue file '/usr/local/nagios/var/spool/checkresults/calmGRZ'...
[1368482404.736675] [016.2] [pid=20256] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkHrK5LJ' to queue file '/usr/local/nagios/var/spool/checkresults/cd0y9e0'...
[1368482406.389813] [016.2] [pid=19986] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkXhhcPV' to queue file '/usr/local/nagios/var/spool/checkresults/cbLlLCg'...
[1368482406.391152] [016.2] [pid=19988] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkliQ0OP' to queue file '/usr/local/nagios/var/spool/checkresults/cP4rNCa'...
[1368482406.405990] [016.2] [pid=19990] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkBEzfPJ' to queue file '/usr/local/nagios/var/spool/checkresults/cN0dhz3'...
[1368482406.453287] [016.2] [pid=20098] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkR92HS6' to queue file '/usr/local/nagios/var/spool/checkresults/cP5mAQr'...
[1368482406.458694] [016.2] [pid=20103] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkbAyh60' to queue file '/usr/local/nagios/var/spool/checkresults/cOgs24l'...
[1368482407.459797] [016.2] [pid=20154] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkHzSmXX' to queue file '/usr/local/nagios/var/spool/checkresults/ctgSIyl'...
[1368482407.485117] [016.2] [pid=20156] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkHQcmSU' to queue file '/usr/local/nagios/var/spool/checkresults/cjPVOxi'...
[1368482420.190181] [016.2] [pid=19726] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkT4RDbV' to queue file '/usr/local/nagios/var/spool/checkresults/cnYSF8P'...
[1368482422.087712] [016.2] [pid=18133] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkbLBpdw' to queue file '/usr/local/nagios/var/spool/checkresults/cDt5D8v'...
[1368482446.233868] [016.2] [pid=19564] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check1iEFkO' to queue file '/usr/local/nagios/var/spool/checkresults/crupKvP'...

and day 2

Code: Select all

[1368568799.567129] [008.1] [pid=7253] Next Low Priority Event Time:  Wed May 15 00:00:00 2013
[1368568799.567135] [008.1] [pid=7253] Current/Max Service Checks: 119/0
[1368568799.567150] [008.2] [pid=7253] No events to execute at the moment.  Idling for a bit...
[1368568799.567156] [001.0] [pid=7253] check_for_external_commands()
[1368568799.567163] [064.1] [pid=7253] Making callbacks (type 8)...
[1368568799.567171] [064.2] [pid=7253] Callback #1 (type 8) return code = 0
[1368568799.818082] [008.1] [pid=7253] ** Event Check Loop
[1368568799.818120] [008.1] [pid=7253] Next High Priority Event Time: Wed May 15 00:00:00 2013
[1368568799.818130] [008.1] [pid=7253] Next Low Priority Event Time:  Wed May 15 00:00:00 2013
[1368568799.818135] [008.1] [pid=7253] Current/Max Service Checks: 119/0
[1368568799.818150] [008.2] [pid=7253] No events to execute at the moment.  Idling for a bit...
[1368568799.818156] [001.0] [pid=7253] check_for_external_commands()
[1368568799.818163] [064.1] [pid=7253] Making callbacks (type 8)...
[1368568799.818171] [064.2] [pid=7253] Callback #1 (type 8) return code = 0
[1368568800.069072] [008.1] [pid=7253] ** Event Check Loop
[1368568800.069103] [008.1] [pid=7253] Next High Priority Event Time: Wed May 15 00:00:00 2013
[1368568800.069113] [008.1] [pid=7253] Next Low Priority Event Time:  Wed May 15 00:00:00 2013
[1368568800.069119] [008.1] [pid=7253] Current/Max Service Checks: 119/0
[1368568800.069140] [001.0] [pid=7253] handle_timed_event() start
[1368568800.069146] [064.1] [pid=7253] Making callbacks (type 8)...
[1368568800.069293] [064.1] [pid=7253] Making callbacks (type 9)...
[1368568800.069302] [064.2] [pid=7253] Callback #1 (type 9) return code = 0
[1368568800.112119] [016.2] [pid=30080] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check0fHxN6' to queue file '/usr/local/nagios/var/spool/checkresults/cqpviFQ'...
[1368568800.136817] [016.2] [pid=29922] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkFC6RhL' to queue file '/usr/local/nagios/var/spool/checkresults/c31BDdv'...
[1368568800.138901] [016.2] [pid=29925] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkWdtQln' to queue file '/usr/local/nagios/var/spool/checkresults/cdu1Wh7'...
[1368568800.138944] [016.2] [pid=29909] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check8yiM4G' to queue file '/usr/local/nagios/var/spool/checkresults/cfc9S0q'...
[1368568800.139873] [016.2] [pid=29911] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check7kCm6i' to queue file '/usr/local/nagios/var/spool/checkresults/cKtSC22'...
[1368568800.178929] [016.2] [pid=29913] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkYAZh8U' to queue file '/usr/local/nagios/var/spool/checkresults/c9zsTaF'...
[1368568800.179740] [016.2] [pid=29917] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkzo1Vax' to queue file '/usr/local/nagios/var/spool/checkresults/cuLLFdh'...
[1368568800.182393] [016.2] [pid=29932] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check7rKaqZ' to queue file '/usr/local/nagios/var/spool/checkresults/cZMbltJ'...
[1368568800.183169] [016.2] [pid=29920] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check8KYie9' to queue file '/usr/local/nagios/var/spool/checkresults/c8zeBhT'...
[1368568800.187558] [016.2] [pid=29936] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkuCJGuB' to queue file '/usr/local/nagios/var/spool/checkresults/cm1hHyl'...
[1368568800.192718] [016.2] [pid=29940] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkpyorzd' to queue file '/usr/local/nagios/var/spool/checkresults/cnRJhEX'...
[1368568800.193560] [016.2] [pid=29944] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkqPBzEP' to queue file '/usr/local/nagios/var/spool/checkresults/cmstyJz'...
[1368568800.203014] [016.2] [pid=29947] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkBjSUJr' to queue file '/usr/local/nagios/var/spool/checkresults/cuMRqQb'...
[1368568800.224783] [016.2] [pid=29963] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkHVYlWF' to queue file '/usr/local/nagios/var/spool/checkresults/cEEkp6p'...
[1368568800.339175] [016.2] [pid=29961] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkWYeZP3' to queue file '/usr/local/nagios/var/spool/checkresults/ch5hCiO'...
[1368568800.788615] [016.2] [pid=30110] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checky9rfjF' to queue file '/usr/local/nagios/var/spool/checkresults/cI1xSWq'...
[1368568800.892066] [016.2] [pid=30124] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkN1zGTv' to queue file '/usr/local/nagios/var/spool/checkresults/cZlr7Nh'...
[1368568801.160871] [016.2] [pid=30120] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkzTMeG7' to queue file '/usr/local/nagios/var/spool/checkresults/clOkkiU'...
[1368568801.168705] [016.2] [pid=28626] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkY85uUK' to queue file '/usr/local/nagios/var/spool/checkresults/cW9DRxx'...
[1368568801.223015] [016.2] [pid=30112] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checknjzyom' to queue file '/usr/local/nagios/var/spool/checkresults/cVc2Ja9'...
[1368568801.233612] [016.2] [pid=30095] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkifAo9g' to queue file '/usr/local/nagios/var/spool/checkresults/cNXHiX3'...
[1368568801.264362] [016.2] [pid=30052] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkUmwPEO' to queue file '/usr/local/nagios/var/spool/checkresults/cYfgJxB'...
[1368568801.306844] [016.2] [pid=30128] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkvryj8T' to queue file '/usr/local/nagios/var/spool/checkresults/cz3a77G'...
[1368568801.499564] [016.2] [pid=30092] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check3ybV4z' to queue file '/usr/local/nagios/var/spool/checkresults/chAH1zn'...
[1368568801.522958] [016.2] [pid=30114] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkC2Gcu3' to queue file '/usr/local/nagios/var/spool/checkresults/coQfWWP'...
[1368568801.716824] [016.2] [pid=30126] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkGFjP0c' to queue file '/usr/local/nagios/var/spool/checkresults/c0ewd50'...
[1368568801.808561] [016.2] [pid=30122] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check00ySMO' to queue file '/usr/local/nagios/var/spool/checkresults/coxLa6C'...
[1368568801.819393] [016.2] [pid=30116] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkngiOtJ' to queue file '/usr/local/nagios/var/spool/checkresults/cbjrROx'...
[1368568801.819755] [016.2] [pid=30118] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkwGnTzq' to queue file '/usr/local/nagios/var/spool/checkresults/cYqg0Ue'...
[1368568802.043279] [016.2] [pid=30082] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/check7fumQN' to queue file '/usr/local/nagios/var/spool/checkresults/cp5iMLC'...
[1368568802.045159] [016.2] [pid=30084] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkqGHsTu' to queue file '/usr/local/nagios/var/spool/checkresults/cIcibPj'...
[1368568802.334379] [016.2] [pid=29443] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkCWkbBM' to queue file '/usr/local/nagios/var/spool/checkresults/cbMCShC'...
[1368568802.342903] [016.2] [pid=29445] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkLwCrBb' to queue file '/usr/local/nagios/var/spool/checkresults/csAvwj1'...
[1368568804.402561] [016.2] [pid=30054] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkVKYoFt' to queue file '/usr/local/nagios/var/spool/checkresults/cB8Y0Lo'...
[1368568804.436406] [016.2] [pid=29968] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkBB139T' to queue file '/usr/local/nagios/var/spool/checkresults/cZSQamP'...
[1368568804.437562] [016.2] [pid=29966] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkcuS52h' to queue file '/usr/local/nagios/var/spool/checkresults/cu1eofd'...
[1368568806.248555] [016.2] [pid=30090] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkqMbO0S' to queue file '/usr/local/nagios/var/spool/checkresults/ci4mfXS'...
[1368568806.298518] [016.2] [pid=30087] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkr39XWb' to queue file '/usr/local/nagios/var/spool/checkresults/cuJzw1b'...
[1368568809.208011] [016.2] [pid=29315] Moving temp check result file '/usr/local/nagios/var/spool/checkresults/checkOXnuc4' to queue file '/usr/local/nagios/var/spool/checkresults/cvZLBTb'...

Normally I'm solving opening the nagios.log from console and cutting / pasting in the log of correct day.
And it not happens every day, but most of days. Not find a clear pattern.

any ideas?

Posted: **Wed May 15, 2013 2:25 pm**

Does this system use syslog or a variant?

Posted: **Thu May 16, 2013 2:37 am**

abrist wrote:Does this system use syslog or a variant?

Yes, use syslogd (1.4.1), the operating system is Red Hat 5.7 (Tikanga). But I understand that Nagios has its own log rotate system, right? anyway, just installed syslog-ng and make it the default syslog.

This night down Again.
verbose changed to 1

Code: Select all

[1368655198.092308] [001.0] [pid=22347] handle_timed_event() start
[1368655198.092326] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.092333] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Tiberio1', Service: 'Check MEM', Options: 0, Latency: 0.092000 sec
[1368655198.092342] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.092348] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check MEM' on host 'Tiberio1': check options=0, latency=0.092000
[1368655198.092358] [001.0] [pid=22347] run_async_service_check()
[1368655198.092368] [001.0] [pid=22347] check_service_check_viability()
[1368655198.092373] [001.0] [pid=22347] check_time_against_period()
[1368655198.092392] [001.0] [pid=22347] check_service_dependencies()
[1368655198.092402] [016.0] [pid=22347] Checking service 'Check MEM' on host 'Tiberio1'...
[1368655198.092416] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.092431] [001.0] [pid=22347] process_macros_r()
[1368655198.092438] [001.0] [pid=22347] process_macros_r()
[1368655198.093114] [001.0] [pid=22347] handle_timed_event() end
[1368655198.093149] [001.0] [pid=22347] handle_timed_event() start
[1368655198.093165] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.093173] [008.0] [pid=22347] ** Service Check Event ==> Host: 'TUIE_trex_rg', Service: 'TBL SYSTEM - TREX DB', Options: 0, Latency: 0.093000 sec
[1368655198.093182] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.093187] [016.0] [pid=22347] Attempting to run scheduled check of service 'TBL SYSTEM - TREX DB' on host 'TUIE_trex_rg': check options=0, latency=0.093000
[1368655198.093197] [001.0] [pid=22347] run_async_service_check()
[1368655198.093206] [001.0] [pid=22347] check_service_check_viability()
[1368655198.093211] [001.0] [pid=22347] check_time_against_period()
[1368655198.093227] [001.0] [pid=22347] check_service_dependencies()
[1368655198.093243] [016.0] [pid=22347] Checking service 'TBL SYSTEM - TREX DB' on host 'TUIE_trex_rg'...
[1368655198.093257] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.093280] [001.0] [pid=22347] process_macros_r()
[1368655198.093288] [001.0] [pid=22347] process_macros_r()
[1368655198.095190] [001.0] [pid=22347] handle_timed_event() end
[1368655198.095229] [001.0] [pid=22347] handle_timed_event() start
[1368655198.095247] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.095259] [008.0] [pid=22347] ** Service Check Event ==> Host: 'TUIE_trex_rg', Service: 'TBL UEFINI - TREX DB', Options: 0, Latency: 0.095000 sec
[1368655198.095286] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.095295] [016.0] [pid=22347] Attempting to run scheduled check of service 'TBL UEFINI - TREX DB' on host 'TUIE_trex_rg': check options=0, latency=0.095000
[1368655198.095307] [001.0] [pid=22347] run_async_service_check()
[1368655198.095318] [001.0] [pid=22347] check_service_check_viability()
[1368655198.095323] [001.0] [pid=22347] check_time_against_period()
[1368655198.095341] [001.0] [pid=22347] check_service_dependencies()
[1368655198.095371] [016.0] [pid=22347] Checking service 'TBL UEFINI - TREX DB' on host 'TUIE_trex_rg'...
[1368655198.095382] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.095396] [001.0] [pid=22347] process_macros_r()
[1368655198.095403] [001.0] [pid=22347] process_macros_r()
[1368655198.096388] [001.0] [pid=22347] handle_timed_event() end
[1368655198.096439] [001.0] [pid=22347] handle_timed_event() start
[1368655198.096526] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.096537] [008.0] [pid=22347] ** Service Check Event ==> Host: 'TUIE_finan11i_rg', Service: 'TBL APPS_TS_UEFINI - TEFIN11i DB', Options: 0, Latency: 0.096000 sec
[1368655198.096550] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.096558] [016.0] [pid=22347] Attempting to run scheduled check of service 'TBL APPS_TS_UEFINI - TEFIN11i DB' on host 'TUIE_finan11i_rg': check options=0, latency=0.096000
[1368655198.096573] [001.0] [pid=22347] run_async_service_check()
[1368655198.096588] [001.0] [pid=22347] check_service_check_viability()
[1368655198.096596] [001.0] [pid=22347] check_time_against_period()
[1368655198.096622] [001.0] [pid=22347] check_service_dependencies()
[1368655198.096636] [016.0] [pid=22347] Checking service 'TBL APPS_TS_UEFINI - TEFIN11i DB' on host 'TUIE_finan11i_rg'...
[1368655198.096653] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.096677] [001.0] [pid=22347] process_macros_r()
[1368655198.096702] [001.0] [pid=22347] process_macros_r()
[1368655198.098342] [001.0] [pid=22347] handle_timed_event() end
[1368655198.098398] [001.0] [pid=22347] handle_timed_event() start
[1368655198.098419] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.098426] [008.0] [pid=22347] ** Service Check Event ==> Host: 'TUIE_finan11i_rg', Service: 'TBL APPS_TS_PEPSYI - TEFIN11i DB', Options: 0, Latency: 0.098000 sec
[1368655198.098436] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.098441] [016.0] [pid=22347] Attempting to run scheduled check of service 'TBL APPS_TS_PEPSYI - TEFIN11i DB' on host 'TUIE_finan11i_rg': check options=0, latency=0.098000
[1368655198.098452] [001.0] [pid=22347] run_async_service_check()
[1368655198.098461] [001.0] [pid=22347] check_service_check_viability()
[1368655198.098466] [001.0] [pid=22347] check_time_against_period()
[1368655198.098483] [001.0] [pid=22347] check_service_dependencies()
[1368655198.098492] [016.0] [pid=22347] Checking service 'TBL APPS_TS_PEPSYI - TEFIN11i DB' on host 'TUIE_finan11i_rg'...
[1368655198.098502] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.098517] [001.0] [pid=22347] process_macros_r()
[1368655198.098524] [001.0] [pid=22347] process_macros_r()
[1368655198.100230] [001.0] [pid=22347] handle_timed_event() end
[1368655198.100281] [001.0] [pid=22347] handle_timed_event() start
[1368655198.100296] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.100304] [008.0] [pid=22347] ** Service Check Event ==> Host: 'TUIE_trex_rg', Service: 'Proceso TREX DB', Options: 0, Latency: 0.100000 sec
[1368655198.100312] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.100317] [016.0] [pid=22347] Attempting to run scheduled check of service 'Proceso TREX DB' on host 'TUIE_trex_rg': check options=0, latency=0.100000
[1368655198.100329] [001.0] [pid=22347] run_async_service_check()
[1368655198.100337] [001.0] [pid=22347] check_service_check_viability()
[1368655198.100343] [001.0] [pid=22347] check_time_against_period()
[1368655198.100360] [001.0] [pid=22347] check_service_dependencies()
[1368655198.100368] [016.0] [pid=22347] Checking service 'Proceso TREX DB' on host 'TUIE_trex_rg'...
[1368655198.100378] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.100392] [001.0] [pid=22347] process_macros_r()
[1368655198.100404] [001.0] [pid=22347] process_macros_r()
[1368655198.101713] [001.0] [pid=22347] handle_timed_event() end
[1368655198.101748] [001.0] [pid=22347] handle_timed_event() start
[1368655198.101776] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.101783] [008.0] [pid=22347] ** Service Check Event ==> Host: 'TUIE_finan11i_rg', Service: 'lert ORA TEFIN11i DB', Options: 0, Latency: 0.101000 sec
[1368655198.101792] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.101797] [016.0] [pid=22347] Attempting to run scheduled check of service 'lert ORA TEFIN11i DB' on host 'TUIE_finan11i_rg': check options=0, latency=0.101000
[1368655198.101807] [001.0] [pid=22347] run_async_service_check()
[1368655198.101816] [001.0] [pid=22347] check_service_check_viability()
[1368655198.101821] [001.0] [pid=22347] check_time_against_period()
[1368655198.101841] [001.0] [pid=22347] check_service_dependencies()
[1368655198.101849] [016.0] [pid=22347] Checking service 'lert ORA TEFIN11i DB' on host 'TUIE_finan11i_rg'...
[1368655198.101860] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.101875] [001.0] [pid=22347] process_macros_r()
[1368655198.101882] [001.0] [pid=22347] process_macros_r()
[1368655198.102817] [001.0] [pid=22347] handle_timed_event() end
[1368655198.102859] [001.0] [pid=22347] handle_timed_event() start
[1368655198.102875] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.102882] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Tacus', Service: 'CPU Load', Options: 0, Latency: 0.102000 sec
[1368655198.102891] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.102896] [016.0] [pid=22347] Attempting to run scheduled check of service 'CPU Load' on host 'Tacus': check options=0, latency=0.102000
[1368655198.102906] [001.0] [pid=22347] run_async_service_check()
[1368655198.102915] [001.0] [pid=22347] check_service_check_viability()
[1368655198.102920] [001.0] [pid=22347] check_time_against_period()
[1368655198.102937] [001.0] [pid=22347] check_service_dependencies()
[1368655198.102950] [016.0] [pid=22347] Checking service 'CPU Load' on host 'Tacus'...
[1368655198.102961] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.102975] [001.0] [pid=22347] process_macros_r()
[1368655198.102982] [001.0] [pid=22347] process_macros_r()
[1368655198.102989] [001.0] [pid=22347] process_macros_r()
[1368655198.104320] [001.0] [pid=22347] handle_timed_event() end
[1368655198.104359] [001.0] [pid=22347] handle_timed_event() start
[1368655198.104373] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.104380] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Tiberio2', Service: 'Check AlertDB', Options: 0, Latency: 0.104000 sec
[1368655198.104389] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.104394] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check AlertDB' on host 'Tiberio2': check options=0, latency=0.104000
[1368655198.104404] [001.0] [pid=22347] run_async_service_check()
[1368655198.104412] [001.0] [pid=22347] check_service_check_viability()
[1368655198.104418] [001.0] [pid=22347] check_time_against_period()
[1368655198.104435] [001.0] [pid=22347] check_service_dependencies()
[1368655198.104449] [016.0] [pid=22347] Checking service 'Check AlertDB' on host 'Tiberio2'...
[1368655198.104460] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.104473] [001.0] [pid=22347] process_macros_r()
[1368655198.104480] [001.0] [pid=22347] process_macros_r()
[1368655198.105385] [001.0] [pid=22347] handle_timed_event() end
[1368655198.105430] [001.0] [pid=22347] handle_timed_event() start
[1368655198.105453] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.105464] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Tiberio2', Service: 'check_oracle_tbls_datos_atlas', Options: 0, Latency: 0.105000 sec
[1368655198.105476] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.105484] [016.0] [pid=22347] Attempting to run scheduled check of service 'check_oracle_tbls_datos_atlas' on host 'Tiberio2': check options=0, latency=0.105000
[1368655198.105516] [001.0] [pid=22347] run_async_service_check()
[1368655198.105529] [001.0] [pid=22347] check_service_check_viability()
[1368655198.105538] [001.0] [pid=22347] check_time_against_period()
[1368655198.105563] [001.0] [pid=22347] check_service_dependencies()
[1368655198.105576] [016.0] [pid=22347] Checking service 'check_oracle_tbls_datos_atlas' on host 'Tiberio2'...
[1368655198.105592] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.105614] [001.0] [pid=22347] process_macros_r()
[1368655198.105637] [001.0] [pid=22347] process_macros_r()
[1368655198.107516] [001.0] [pid=22347] handle_timed_event() end
[1368655198.107554] [001.0] [pid=22347] handle_timed_event() start
[1368655198.107569] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.107577] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Tiberio2', Service: 'check_oracle_tbls_indices_atlas', Options: 0, Latency: 0.107000 sec
[1368655198.107585] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.107591] [016.0] [pid=22347] Attempting to run scheduled check of service 'check_oracle_tbls_indices_atlas' on host 'Tiberio2': check options=0, latency=0.107000
[1368655198.107601] [001.0] [pid=22347] run_async_service_check()
[1368655198.107609] [001.0] [pid=22347] check_service_check_viability()
[1368655198.107614] [001.0] [pid=22347] check_time_against_period()
[1368655198.107633] [001.0] [pid=22347] check_service_dependencies()
[1368655198.107641] [016.0] [pid=22347] Checking service 'check_oracle_tbls_indices_atlas' on host 'Tiberio2'...
[1368655198.107652] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.107666] [001.0] [pid=22347] process_macros_r()
[1368655198.107673] [001.0] [pid=22347] process_macros_r()
[1368655198.109397] [001.0] [pid=22347] handle_timed_event() end
[1368655198.109440] [001.0] [pid=22347] handle_timed_event() start
[1368655198.109455] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.109462] [008.0] [pid=22347] ** Service Check Event ==> Host: 'virtual-altheon', Service: 'PING 10.162.238.3', Options: 0, Latency: 0.109000 sec
[1368655198.109471] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.109476] [016.0] [pid=22347] Attempting to run scheduled check of service 'PING 10.162.238.3' on host 'virtual-altheon': check options=0, latency=0.109000
[1368655198.109487] [001.0] [pid=22347] run_async_service_check()
[1368655198.109495] [001.0] [pid=22347] check_service_check_viability()
[1368655198.109500] [001.0] [pid=22347] check_time_against_period()
[1368655198.109517] [001.0] [pid=22347] check_service_dependencies()
[1368655198.109525] [016.0] [pid=22347] Checking service 'PING 10.162.238.3' on host 'virtual-altheon'...
[1368655198.109536] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.109544] [001.0] [pid=22347] process_macros_r()
[1368655198.109556] [001.0] [pid=22347] process_macros_r()
[1368655198.109566] [001.0] [pid=22347] process_macros_r()
[1368655198.109575] [001.0] [pid=22347] process_macros_r()
[1368655198.110874] [001.0] [pid=22347] handle_timed_event() end
[1368655198.110910] [001.0] [pid=22347] handle_timed_event() start
[1368655198.110928] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:58 2013
[1368655198.110935] [008.0] [pid=22347] ** Service Check Event ==> Host: 'virtual-altheon', Service: 'PING', Options: 0, Latency: 0.110000 sec
[1368655198.110944] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655198.110949] [016.0] [pid=22347] Attempting to run scheduled check of service 'PING' on host 'virtual-altheon': check options=0, latency=0.110000
[1368655198.110959] [001.0] [pid=22347] run_async_service_check()
[1368655198.110968] [001.0] [pid=22347] check_service_check_viability()
[1368655198.110973] [001.0] [pid=22347] check_time_against_period()
[1368655198.110991] [001.0] [pid=22347] check_service_dependencies()
[1368655198.111011] [016.0] [pid=22347] Checking service 'PING' on host 'virtual-altheon'...
[1368655198.111023] [001.0] [pid=22347] get_raw_command_line_r()
[1368655198.111037] [001.0] [pid=22347] process_macros_r()
[1368655198.111044] [001.0] [pid=22347] process_macros_r()
[1368655198.111050] [001.0] [pid=22347] process_macros_r()
[1368655198.111056] [001.0] [pid=22347] process_macros_r()
[1368655198.112639] [001.0] [pid=22347] handle_timed_event() end
[1368655198.112678] [001.0] [pid=22347] check_for_external_commands()
[1368655198.364318] [001.0] [pid=22347] check_for_external_commands()
[1368655198.615282] [001.0] [pid=22347] check_for_external_commands()
[1368655198.866258] [001.0] [pid=22347] check_for_external_commands()
[1368655199.117267] [001.0] [pid=22347] handle_timed_event() start
[1368655199.117283] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.117290] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano12', Service: 'Bedsonline R2', Options: 0, Latency: 0.117000 sec
[1368655199.117299] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.117305] [016.0] [pid=22347] Attempting to run scheduled check of service 'Bedsonline R2' on host 'Trajano12': check options=0, latency=0.117000
[1368655199.117315] [001.0] [pid=22347] run_async_service_check()
[1368655199.117325] [001.0] [pid=22347] check_service_check_viability()
[1368655199.117330] [001.0] [pid=22347] check_time_against_period()
[1368655199.117350] [001.0] [pid=22347] check_service_dependencies()
[1368655199.117360] [016.0] [pid=22347] Checking service 'Bedsonline R2' on host 'Trajano12'...
[1368655199.117372] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.117381] [001.0] [pid=22347] process_macros_r()
[1368655199.117388] [001.0] [pid=22347] process_macros_r()
[1368655199.117394] [001.0] [pid=22347] process_macros_r()
[1368655199.117400] [001.0] [pid=22347] process_macros_r()
[1368655199.118351] [001.0] [pid=22347] handle_timed_event() end
[1368655199.118386] [001.0] [pid=22347] handle_timed_event() start
[1368655199.118403] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.118410] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano11', Service: 'Hotelbeds', Options: 0, Latency: 0.118000 sec
[1368655199.118419] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.118424] [016.0] [pid=22347] Attempting to run scheduled check of service 'Hotelbeds' on host 'Trajano11': check options=0, latency=0.118000
[1368655199.118434] [001.0] [pid=22347] run_async_service_check()
[1368655199.118442] [001.0] [pid=22347] check_service_check_viability()
[1368655199.118447] [001.0] [pid=22347] check_time_against_period()
[1368655199.118464] [001.0] [pid=22347] check_service_dependencies()
[1368655199.118472] [016.0] [pid=22347] Checking service 'Hotelbeds' on host 'Trajano11'...
[1368655199.118485] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.118494] [001.0] [pid=22347] process_macros_r()
[1368655199.118501] [001.0] [pid=22347] process_macros_r()
[1368655199.118507] [001.0] [pid=22347] process_macros_r()
[1368655199.118512] [001.0] [pid=22347] process_macros_r()
[1368655199.120483] [001.0] [pid=22347] handle_timed_event() end
[1368655199.120518] [001.0] [pid=22347] handle_timed_event() start
[1368655199.120533] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.120540] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano14', Service: 'Hotelbeds', Options: 0, Latency: 0.120000 sec
[1368655199.120549] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.120554] [016.0] [pid=22347] Attempting to run scheduled check of service 'Hotelbeds' on host 'Trajano14': check options=0, latency=0.120000
[1368655199.120565] [001.0] [pid=22347] run_async_service_check()
[1368655199.120573] [001.0] [pid=22347] check_service_check_viability()
[1368655199.120578] [001.0] [pid=22347] check_time_against_period()
[1368655199.120611] [001.0] [pid=22347] check_service_dependencies()
[1368655199.120620] [016.0] [pid=22347] Checking service 'Hotelbeds' on host 'Trajano14'...
[1368655199.120630] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.120639] [001.0] [pid=22347] process_macros_r()
[1368655199.120645] [001.0] [pid=22347] process_macros_r()
[1368655199.120651] [001.0] [pid=22347] process_macros_r()
[1368655199.120657] [001.0] [pid=22347] process_macros_r()
[1368655199.122385] [001.0] [pid=22347] handle_timed_event() end
[1368655199.122419] [001.0] [pid=22347] handle_timed_event() start
[1368655199.122434] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.122441] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano12', Service: 'Hotelextras', Options: 0, Latency: 0.122000 sec
[1368655199.122449] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.122454] [016.0] [pid=22347] Attempting to run scheduled check of service 'Hotelextras' on host 'Trajano12': check options=0, latency=0.122000
[1368655199.122464] [001.0] [pid=22347] run_async_service_check()
[1368655199.122473] [001.0] [pid=22347] check_service_check_viability()
[1368655199.122478] [001.0] [pid=22347] check_time_against_period()
[1368655199.122495] [001.0] [pid=22347] check_service_dependencies()
[1368655199.122509] [016.0] [pid=22347] Checking service 'Hotelextras' on host 'Trajano12'...
[1368655199.122520] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.122528] [001.0] [pid=22347] process_macros_r()
[1368655199.122535] [001.0] [pid=22347] process_macros_r()
[1368655199.122551] [001.0] [pid=22347] process_macros_r()
[1368655199.122558] [001.0] [pid=22347] process_macros_r()
[1368655199.124260] [001.0] [pid=22347] handle_timed_event() end
[1368655199.124295] [001.0] [pid=22347] handle_timed_event() start
[1368655199.124309] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.124316] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano14', Service: 'Check Hotelbeds Search', Options: 0, Latency: 0.124000 sec
[1368655199.124325] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.124330] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check Hotelbeds Search' on host 'Trajano14': check options=0, latency=0.124000
[1368655199.124341] [001.0] [pid=22347] run_async_service_check()
[1368655199.124349] [001.0] [pid=22347] check_service_check_viability()
[1368655199.124354] [001.0] [pid=22347] check_time_against_period()
[1368655199.124372] [001.0] [pid=22347] check_service_dependencies()
[1368655199.124380] [016.0] [pid=22347] Checking service 'Check Hotelbeds Search' on host 'Trajano14'...
[1368655199.124391] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.124400] [001.0] [pid=22347] process_macros_r()
[1368655199.126111] [001.0] [pid=22347] handle_timed_event() end
[1368655199.126152] [001.0] [pid=22347] handle_timed_event() start
[1368655199.126166] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.126173] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano11', Service: 'Check Hotelbeds Search', Options: 0, Latency: 0.126000 sec
[1368655199.126182] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.126187] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check Hotelbeds Search' on host 'Trajano11': check options=0, latency=0.126000
[1368655199.126197] [001.0] [pid=22347] run_async_service_check()
[1368655199.126209] [001.0] [pid=22347] check_service_check_viability()
[1368655199.126216] [001.0] [pid=22347] check_time_against_period()
[1368655199.126238] [001.0] [pid=22347] check_service_dependencies()
[1368655199.126247] [016.0] [pid=22347] Checking service 'Check Hotelbeds Search' on host 'Trajano11'...
[1368655199.126258] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.126266] [001.0] [pid=22347] process_macros_r()
[1368655199.126934] [001.0] [pid=22347] handle_timed_event() end
[1368655199.126975] [001.0] [pid=22347] handle_timed_event() start
[1368655199.126992] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.126999] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano0', Service: 'Check Disk', Options: 0, Latency: 0.126000 sec
[1368655199.127008] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.127013] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check Disk' on host 'Trajano0': check options=0, latency=0.126000
[1368655199.127023] [001.0] [pid=22347] run_async_service_check()
[1368655199.127033] [001.0] [pid=22347] check_service_check_viability()
[1368655199.127038] [001.0] [pid=22347] check_time_against_period()
[1368655199.127055] [001.0] [pid=22347] check_service_dependencies()
[1368655199.127063] [016.0] [pid=22347] Checking service 'Check Disk' on host 'Trajano0'...
[1368655199.127073] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.127082] [001.0] [pid=22347] process_macros_r()
[1368655199.127089] [001.0] [pid=22347] process_macros_r()
[1368655199.129243] [001.0] [pid=22347] handle_timed_event() end
[1368655199.129288] [001.0] [pid=22347] handle_timed_event() start
[1368655199.129305] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.129312] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano12', Service: 'Check MEM', Options: 0, Latency: 0.129000 sec
[1368655199.129322] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.129327] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check MEM' on host 'Trajano12': check options=0, latency=0.129000
[1368655199.129337] [001.0] [pid=22347] run_async_service_check()
[1368655199.129346] [001.0] [pid=22347] check_service_check_viability()
[1368655199.129351] [001.0] [pid=22347] check_time_against_period()
[1368655199.129370] [001.0] [pid=22347] check_service_dependencies()
[1368655199.129379] [016.0] [pid=22347] Checking service 'Check MEM' on host 'Trajano12'...
[1368655199.129390] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.129405] [001.0] [pid=22347] process_macros_r()
[1368655199.129412] [001.0] [pid=22347] process_macros_r()
[1368655199.130245] [001.0] [pid=22347] handle_timed_event() end
[1368655199.130286] [001.0] [pid=22347] handle_timed_event() start
[1368655199.130302] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.130309] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Tiberio2', Service: 'check_oracle_tbls_system', Options: 0, Latency: 0.130000 sec
[1368655199.130318] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.130324] [016.0] [pid=22347] Attempting to run scheduled check of service 'check_oracle_tbls_system' on host 'Tiberio2': check options=0, latency=0.130000
[1368655199.130334] [001.0] [pid=22347] run_async_service_check()
[1368655199.130342] [001.0] [pid=22347] check_service_check_viability()
[1368655199.130347] [001.0] [pid=22347] check_time_against_period()
[1368655199.130365] [001.0] [pid=22347] check_service_dependencies()
[1368655199.130374] [016.0] [pid=22347] Checking service 'check_oracle_tbls_system' on host 'Tiberio2'...
[1368655199.130384] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.130398] [001.0] [pid=22347] process_macros_r()
[1368655199.130413] [001.0] [pid=22347] process_macros_r()
[1368655199.131280] [001.0] [pid=22347] handle_timed_event() end
[1368655199.131319] [001.0] [pid=22347] handle_timed_event() start
[1368655199.131334] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.131341] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Trajano18', Service: 'Check CPU', Options: 0, Latency: 0.131000 sec
[1368655199.131349] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.131354] [016.0] [pid=22347] Attempting to run scheduled check of service 'Check CPU' on host 'Trajano18': check options=0, latency=0.131000
[1368655199.131378] [001.0] [pid=22347] run_async_service_check()
[1368655199.131387] [001.0] [pid=22347] check_service_check_viability()
[1368655199.131392] [001.0] [pid=22347] check_time_against_period()
[1368655199.131411] [001.0] [pid=22347] check_service_dependencies()
[1368655199.131419] [016.0] [pid=22347] Checking service 'Check CPU' on host 'Trajano18'...
[1368655199.131429] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.131444] [001.0] [pid=22347] process_macros_r()
[1368655199.131456] [001.0] [pid=22347] process_macros_r()
[1368655199.132274] [001.0] [pid=22347] handle_timed_event() end
[1368655199.132308] [001.0] [pid=22347] handle_timed_event() start
[1368655199.132323] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.132330] [008.0] [pid=22347] ** Service Check Event ==> Host: 'BPI', Service: 'MDM', Options: 0, Latency: 0.132000 sec
[1368655199.132339] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.132344] [016.0] [pid=22347] Attempting to run scheduled check of service 'MDM' on host 'BPI': check options=0, latency=0.132000
[1368655199.132354] [001.0] [pid=22347] run_async_service_check()
[1368655199.132363] [001.0] [pid=22347] check_service_check_viability()
[1368655199.132368] [001.0] [pid=22347] check_time_against_period()
[1368655199.132385] [001.0] [pid=22347] check_service_dependencies()
[1368655199.132393] [016.0] [pid=22347] Checking service 'MDM' on host 'BPI'...
[1368655199.132404] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.132417] [001.0] [pid=22347] process_macros_r()
[1368655199.132424] [001.0] [pid=22347] process_macros_r()
[1368655199.133278] [001.0] [pid=22347] handle_timed_event() end
[1368655199.133323] [001.0] [pid=22347] handle_timed_event() start
[1368655199.133341] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.133348] [008.0] [pid=22347] ** Service Check Event ==> Host: 'BPI', Service: 'Bedsonline', Options: 0, Latency: 0.133000 sec
[1368655199.133357] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.133362] [016.0] [pid=22347] Attempting to run scheduled check of service 'Bedsonline' on host 'BPI': check options=0, latency=0.133000
[1368655199.133372] [001.0] [pid=22347] run_async_service_check()
[1368655199.133380] [001.0] [pid=22347] check_service_check_viability()
[1368655199.133385] [001.0] [pid=22347] check_time_against_period()
[1368655199.133403] [001.0] [pid=22347] check_service_dependencies()
[1368655199.133411] [016.0] [pid=22347] Checking service 'Bedsonline' on host 'BPI'...
[1368655199.133422] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.133440] [001.0] [pid=22347] process_macros_r()
[1368655199.133447] [001.0] [pid=22347] process_macros_r()
[1368655199.135040] [001.0] [pid=22347] handle_timed_event() end
[1368655199.135078] [001.0] [pid=22347] handle_timed_event() start
[1368655199.135093] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.135100] [008.0] [pid=22347] ** Service Check Event ==> Host: 'BPI', Service: 'XML', Options: 0, Latency: 0.135000 sec
[1368655199.135109] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.135114] [016.0] [pid=22347] Attempting to run scheduled check of service 'XML' on host 'BPI': check options=0, latency=0.135000
[1368655199.135123] [001.0] [pid=22347] run_async_service_check()
[1368655199.135132] [001.0] [pid=22347] check_service_check_viability()
[1368655199.135137] [001.0] [pid=22347] check_time_against_period()
[1368655199.135155] [001.0] [pid=22347] check_service_dependencies()
[1368655199.135164] [016.0] [pid=22347] Checking service 'XML' on host 'BPI'...
[1368655199.135175] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.135188] [001.0] [pid=22347] process_macros_r()
[1368655199.135199] [001.0] [pid=22347] process_macros_r()
[1368655199.136285] [001.0] [pid=22347] handle_timed_event() end
[1368655199.136347] [001.0] [pid=22347] handle_timed_event() start
[1368655199.136363] [008.0] [pid=22347] ** Timed Event ** Type: EVENT_SERVICE_CHECK, Run Time: Wed May 15 23:59:59 2013
[1368655199.136370] [008.0] [pid=22347] ** Service Check Event ==> Host: 'Octavio5', Service: 'PING', Options: 0, Latency: 0.136000 sec
[1368655199.136380] [001.0] [pid=22347] run_scheduled_service_check() start
[1368655199.136385] [016.0] [pid=22347] Attempting to run scheduled check of service 'PING' on host 'Octavio5': check options=0, latency=0.136000
[1368655199.136396] [001.0] [pid=22347] run_async_service_check()
[1368655199.136405] [001.0] [pid=22347] check_service_check_viability()
[1368655199.136410] [001.0] [pid=22347] check_time_against_period()
[1368655199.136428] [001.0] [pid=22347] check_service_dependencies()
[1368655199.136442] [016.0] [pid=22347] Checking service 'PING' on host 'Octavio5'...
[1368655199.136454] [001.0] [pid=22347] get_raw_command_line_r()
[1368655199.136469] [001.0] [pid=22347] process_macros_r()
[1368655199.136476] [001.0] [pid=22347] process_macros_r()
[1368655199.136482] [001.0] [pid=22347] process_macros_r()
[1368655199.138264] [001.0] [pid=22347] handle_timed_event() end
[1368655199.138314] [001.0] [pid=22347] check_for_external_commands()
[1368655199.389225] [001.0] [pid=22347] check_for_external_commands()
[1368655199.640200] [001.0] [pid=22347] check_for_external_commands()
[1368655199.891184] [001.0] [pid=22347] check_for_external_commands()
[1368655200.142192] [001.0] [pid=22347] handle_timed_event() start

and in dmesg has this notification

Code: Select all

nagios[22347]: segfault at 0000000100000001 rip 00000038ca079b60 rsp 00007fffcd8a6868 error 4

Posted: **Thu May 16, 2013 1:34 pm**

segfaults are never good. How many of those are in your logs over the past week?

Posted: **Fri May 17, 2013 2:53 am**

abrist wrote:segfaults are never good. How many of those are in your logs over the past week?

i don't known. at the moment have only two segfaults but I think I only see since enabled daemon_dumps_core=1 two days ago.

Code: Select all

[nagios@sa1itp02 nagios]$ dmesg | grep nagios
nagios[22347]: segfault at 0000000100000001 rip 00000038ca079b60 rsp 00007fffcd8a6868 error 4
nagios[11931]: segfault at 0000000100000001 rip 00000038ca079b60 rsp 00007fff0e70e6e8 error 4

Posted: **Fri May 17, 2013 2:29 pm**

Any chance you could post the logs from 12:59pm-1:02am for one of the nights that failed?

Nagios Support Forum

Nagios Core log rotate problem

Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem

Re: Nagios Core log rotate problem