Re: [Nagios-devel] nagios 3.2.3 localtime deadlock
Posted: Fri Oct 08, 2010 8:38 am
This is a multi-part message in MIME format.
--------------060403030606020204050100
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
On 10/08/2010 07:44 AM, Thomas Guyot-Sionnest wrote:
> At first glance it seems we might have quite some code to change in
> order to be 100% thread-safe:
> $ grep -RE '(asctime|ctime|gmtime|localtime)[[:space:]]*\(' base/|wc -l
> 77
> Although not all invocations are necessarily in threaded code. Anyone
> more experienced could confirm if this is the actual issue?
I'm afraid that I don't qualify, either, but I do have a question: Any
chance that the log rotation routines are particularly thread-poison-y?
Background: I hacked a bit of randomization into the check scheduling so
as to dissolve the "clusters" of concurrent checks I previously had, and
it turns out that *something* trashes the smoothing every midnight when
the log gets rotated, worse than even a widespread network outage's
checks-timeouts-and-notifications-storm does.
(See the attached graph; x =3D current time (small tickmarks every minute=
,
bigger ticks every 10m, first of them is midnight); y =3D (scheduled time
- current time), tick =3D 1m; pixel colors code for # of checks scheduled
at time y in the status.dat read at time x, black =3D 0, yellow =3D 51, s=
ee
spectrum in lower left corner.)
Ever since, I've been waiting for some inspiration how to debug this
further ... :-/
> Aieee!, so much IE spam... can't we just blacklist these words :p
(If IE does indeed conform to CSS3 now, it's good news at least. Last I
checked, it was still lacking :before and :after from CSS 2.0 - and
that's a standard from 1998, for cryin' out loud!)
Regards,
J. Bern
--=20
Jochen Bern, Systemingenieur --- LINworks GmbH
Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt
PGP (1024D/4096g) FP =3D D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C2=
7
Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202
Unternehmenssitz Weiterstadt, Gesch=E4ftsf=FChrer Metin Dogan, Oliver Mic=
hel
--------------060403030606020204050100
Content-Type: image/png;
name="SchedGraph.2010-10-07-23:55.png"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="SchedGraph.2010-10-07-23:55.png"
iVBORw0KGgoAAAANSUhEUgAAAcwAAAFpEAIAAADO+FPPAAAACXBIWXMAAABIAAAASABGyWs+
AAAACXZwQWcAAAHMAAABaQBgctKrAACAAElEQVR42u29T8guyXXf/+R3bRgCN4JJYAiIgGSQ
BYlADiFaJIsYpCxkUALRDEgLmSAtJIJEGBFf2QvJWAKpYdBCMngWEsIDluCOMNZivIhkRpiE
YbyIGGSGgEgwCZfAMFgEgvEQcH6LL5f3M/OdU2/1U93VdfrWd3F57vP2011df06d+nT1OX/r
gx/84Ac/+MHL1NTU1NTU1NTUVCr9f0cXYGpqampqampqauoa/UL0h/v379+/f//xxx9//PHH
jy5kDv3tv325XC5/9VdHl2PqaGXpCVnKOTU1NTX1KOsv//Iv//Iv//Kpp5566qmn/K9NRFYT
4Wg6qlTndgjGbOsxlaUnZCnn1NTU1Pias+RRNdDkyI45ER5bqrN25THbemRl6QlZyjk1NTU1
suYseVQNbLBHdsyJcHLZPTRmW4+pLD0hSzmnpqamxtecJfvXwAaO7JgT4eSye2jMth5ZWXpC
lnJOTU1Njaw5S/avgc2iFow5EU4uu4fGbOsxlaUnZCnn1NTU1Pias2TPGtjMkR1zIpxcdg+N
2dZT7Tprj52amprqqTlL9qyBjePIzomQOndXnm1dryx1de4eOzU1NTXVU33mvo0d2TEnwmPd
iCxOzFqN2dZjKlddnbXHTk1NTfXUtKV95r5dMnuN1njHuhG5nJip/TTauIg0e+zU1NRUu6Yt
lfae+3ZxZMdsvMll530dqzHHRaTZslNTU1PtmrZ077lvF0dWGq3xJped9zWCRhsXkWbLTk1N
TbVr2lJpv7lvR0d2zMabXHbe17Eac1xEmi07NTU11a5pS/eb+3Z0ZKXRGm9y2XlfI2i0cRFp
tuzU1NRUu6YtlfaY+3Z3ZMdsvMll530dqzHHRaTZslNTU1PtmrZ0j7lvd0dWGq3xJped9zWC
RhsXkWbLTk1NTbVr2lJp27mvkyM7ZuNNLjvv61iNOS4izZadmpqaate0pdvOfZ0cWWm0xptc
dt7XCBptXESaLTs1NTXVrmlLpa3mvq6O7JiNN7nsvK9jNea4iDRbdmpqaqpd05ZuNfd1dWSl
0Rpvctl5XyNotHERabbs1NTUVLumLZXa574DHNkxG29y2Xlfx2rMcRFptuzU1NRUu6YtbZ/7
DnBkpdEab3LZeV8jaLRxEWm27NTU1FS7pi2VWua+wxzZMRtvctl5X8dqzHERabbs1NTUVLum
LW2Z+w5zZKXRGm9y2XlfI2i0cRFptuzU1NRUu6Ytla6b+w52ZMdsvMll530dqzHHRaTZslNT
U1Ptmrb0urnvYEdWGq3xJped9zWCRhsXkWbLTk1NTbVr2lJp7dw3hCM7ZuNNLjvv61iNOS4i
zZadmpqaate0pWvnviEcWWm0xptcdt7XCBptXESaLTs1NTXVrmlLpfq5byBHdszGm1x23tex
GnNcRJotOzU1NdWuaUvr576BHFlptMabXHbe1wgabVxEmi07NTU11a5pS6WauW84R3bMxptc
dt7XsRpzXESaLTs1NTXVrmlLa+a+4RxZabTGm1x23tcIGm1cRJotOzU1NdWuaUtrNKgjO2bj
TS477+tYjTkuIs2WnZqammrXtKVlDerISqM13uSy875G0GjjItJs2ampqal2TVta1tCO7JiN
l8WNyKVZq/Uac1xEmi07NTU11a5p
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]
--------------060403030606020204050100
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
On 10/08/2010 07:44 AM, Thomas Guyot-Sionnest wrote:
> At first glance it seems we might have quite some code to change in
> order to be 100% thread-safe:
> $ grep -RE '(asctime|ctime|gmtime|localtime)[[:space:]]*\(' base/|wc -l
> 77
> Although not all invocations are necessarily in threaded code. Anyone
> more experienced could confirm if this is the actual issue?
I'm afraid that I don't qualify, either, but I do have a question: Any
chance that the log rotation routines are particularly thread-poison-y?
Background: I hacked a bit of randomization into the check scheduling so
as to dissolve the "clusters" of concurrent checks I previously had, and
it turns out that *something* trashes the smoothing every midnight when
the log gets rotated, worse than even a widespread network outage's
checks-timeouts-and-notifications-storm does.
(See the attached graph; x =3D current time (small tickmarks every minute=
,
bigger ticks every 10m, first of them is midnight); y =3D (scheduled time
- current time), tick =3D 1m; pixel colors code for # of checks scheduled
at time y in the status.dat read at time x, black =3D 0, yellow =3D 51, s=
ee
spectrum in lower left corner.)
Ever since, I've been waiting for some inspiration how to debug this
further ... :-/
> Aieee!, so much IE spam... can't we just blacklist these words :p
(If IE does indeed conform to CSS3 now, it's good news at least. Last I
checked, it was still lacking :before and :after from CSS 2.0 - and
that's a standard from 1998, for cryin' out loud!)
Regards,
J. Bern
--=20
Jochen Bern, Systemingenieur --- LINworks GmbH
Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt
PGP (1024D/4096g) FP =3D D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C2=
7
Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202
Unternehmenssitz Weiterstadt, Gesch=E4ftsf=FChrer Metin Dogan, Oliver Mic=
hel
--------------060403030606020204050100
Content-Type: image/png;
name="SchedGraph.2010-10-07-23:55.png"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="SchedGraph.2010-10-07-23:55.png"
iVBORw0KGgoAAAANSUhEUgAAAcwAAAFpEAIAAADO+FPPAAAACXBIWXMAAABIAAAASABGyWs+
AAAACXZwQWcAAAHMAAABaQBgctKrAACAAElEQVR42u29T8guyXXf/+R3bRgCN4JJYAiIgGSQ
BYlADiFaJIsYpCxkUALRDEgLmSAtJIJEGBFf2QvJWAKpYdBCMngWEsIDluCOMNZivIhkRpiE
YbyIGGSGgEgwCZfAMFgEgvEQcH6LL5f3M/OdU2/1U93VdfrWd3F57vP2011df06d+nT1OX/r
gx/84Ac/+MHL1NTU1NTU1NTUVCr9f0cXYGpqampqampqauoa/UL0h/v379+/f//xxx9//PHH
jy5kDv3tv325XC5/9VdHl2PqaGXpCVnKOTU1NTX1KOsv//Iv//Iv//Kpp5566qmn/K9NRFYT
4Wg6qlTndgjGbOsxlaUnZCnn1NTU1Pias+RRNdDkyI45ER5bqrN25THbemRl6QlZyjk1NTU1
suYseVQNbLBHdsyJcHLZPTRmW4+pLD0hSzmnpqamxtecJfvXwAaO7JgT4eSye2jMth5ZWXpC
lnJOTU1Njaw5S/avgc2iFow5EU4uu4fGbOsxlaUnZCnn1NTU1Pias2TPGtjMkR1zIpxcdg+N
2dZT7Tprj52amprqqTlL9qyBjePIzomQOndXnm1dryx1de4eOzU1NTXVU33mvo0d2TEnwmPd
iCxOzFqN2dZjKlddnbXHTk1NTfXUtKV95r5dMnuN1njHuhG5nJip/TTauIg0e+zU1NRUu6Yt
lfae+3ZxZMdsvMll530dqzHHRaTZslNTU1PtmrZ077lvF0dWGq3xJped9zWCRhsXkWbLTk1N
TbVr2lJpv7lvR0d2zMabXHbe17Eac1xEmi07NTU11a5pS/eb+3Z0ZKXRGm9y2XlfI2i0cRFp
tuzU1NRUu6YtlfaY+3Z3ZMdsvMll530dqzHHRaTZslNTU1PtmrZ0j7lvd0dWGq3xJped9zWC
RhsXkWbLTk1NTbVr2lJp27mvkyM7ZuNNLjvv61iNOS4izZadmpqaate0pdvOfZ0cWWm0xptc
dt7XCBptXESaLTs1NTXVrmlLpa3mvq6O7JiNN7nsvK9jNea4iDRbdmpqaqpd05ZuNfd1dWSl
0Rpvctl5XyNotHERabbs1NTUVLumLZXa574DHNkxG29y2Xlfx2rMcRFptuzU1NRUu6YtbZ/7
DnBkpdEab3LZeV8jaLRxEWm27NTU1FS7pi2VWua+wxzZMRtvctl5X8dqzHERabbs1NTUVLum
LW2Z+w5zZKXRGm9y2XlfI2i0cRFptuzU1NRUu6Ytla6b+w52ZMdsvMll530dqzHHRaTZslNT
U1Ptmrb0urnvYEdWGq3xJped9zWCRhsXkWbLTk1NTbVr2lJp7dw3hCM7ZuNNLjvv61iNOS4i
zZadmpqaate0pWvnviEcWWm0xptcdt7XCBptXESaLTs1NTXVrmlLpfq5byBHdszGm1x23tex
GnNcRJotOzU1NdWuaUvr576BHFlptMabXHbe1wgabVxEmi07NTU11a5pS6WauW84R3bMxptc
dt7XsRpzXESaLTs1NTXVrmlLa+a+4RxZabTGm1x23tcIGm1cRJotOzU1NdWuaUtrNKgjO2bj
TS477+tYjTkuIs2WnZqammrXtKVlDerISqM13uSy875G0GjjItJs2ampqal2TVta1tCO7JiN
l8WNyKVZq/Uac1xEmi07NTU11a5p
...[email truncated]...
This post was automatically imported from historical nagios-devel mailing list archives
Original poster: [email protected]