Possible issue with check_http
Posted: Tue Dec 02, 2014 9:23 am
Hello,
We are trying to track down a constantly, wrongly, failing HTTP content check (~ 1 out of 4 checks fails). check_http fails to find the search string in the HTML body, even though the website is never down and perfectly visible in a browser.
Command line output of a looped check_http call to demonstrate (abundant spaces are due to Putty):
I get the same behaviour on our QA server.
However, I can't reproduce this with wget (or my desktop browser). Command line output of looped wget:
Going deeper using verbose output on check_http I see some weird stuff. Output of a positive check (string found):
Output of a false negative check (string not found):
(seach for "58e" and you will see the weirdness)
Any ideas? 
Feel free to run your own tests against citycard.gent.be, it's a public website. But give us a yell and the IP address where you are doing/have done the checks from, so we can discern you from actual DOS attempts
We are trying to track down a constantly, wrongly, failing HTTP content check (~ 1 out of 4 checks fails). check_http fails to find the search string in the HTML body, even though the website is never down and perfectly visible in a browser.
Command line output of a looped check_http call to demonstrate (abundant spaces are due to Putty):
Code: Select all
# while true; do /usr/local/nagios/libexec/check_http -H citycard.gent.be --onredirect=follow -s "Gelieve uw gebruikers"; sleep 0.1; done
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.011770s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4017 by tes in 0.012 second response time |time=0.012453s;;;0.000000 size=4017B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009682s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.011 second response time |time=0.011414s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.011886s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4017 by tes in 0.013 second response time |time=0.012871s;;;0.000000 size=4017B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.013 second response time |time=0.012743s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.010452s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009769s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.014 second response time |time=0.013729s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.012432s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.035 second response time |time=0.035191s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009626s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.009047s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4024 by tes in 0.010 second response time |time=0.010092s;;;0.000000 size=4024B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.011787s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.013 second response time |time=0.012674s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.010080s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009692s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4017 by tes in 0.010 second response time |time=0.009694s;;;0.000000 size=4017B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4017 by tes in 0.012 second response time |time=0.011920s;;;0.000000 size=4017B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.019 second response time |time=0.018932s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.012123s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4017 by tes in 0.009 second response time |time=0.009104s;;;0.000000 size=4017B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.009046s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.011 second response time |time=0.011385s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.011932s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009577s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009566s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.009360s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.013 second response time |time=0.012608s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.013 second response time |time=0.012559s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.009494s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.008857s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.011 second response time |time=0.011351s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4024 by tes in 0.013 second response time |time=0.012801s;;;0.000000 size=4024B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.011 second response time |time=0.011330s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.009041s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009864s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.011925s;;;0.000000 size=4010B;;;0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4017 by tes in 0.012 second response time |time=0.012074s;;;0.000000 size=4017B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.012 second response time |time=0.012378s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.013 second response time |time=0.012991s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.009 second response time |time=0.009251s;;;0.000000 size=4010B;;;0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.010319s;;;0.000000 size=4010B;;;0
^C
However, I can't reproduce this with wget (or my desktop browser). Command line output of looped wget:
Code: Select all
# while true; do echo -n "Check: "; wget -O - http://citycard.gent.be 2>/dev/null | grep -i "Gelieve uw gebruikers"; sleep 0.1; done
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
Check: Gelieve uw gebruikersnaam en wachtwoord in te geven.
^C
Code: Select all
# /usr/local/nagios/libexec/check_http -v -H citycard.gent.be --onredirect=follow -s "Gelieve uw gebruikers"
option f:4
GET / HTTP/1.1
User-Agent: check_http/v2.0.3 (nagios-plugins 2.0.3)
Connection: close
Host: citycard.gent.be
Accept: */*
http://citycard.gent.be:80/ is 4010 characters
STATUS: HTTP/1.1 200 OK
**** HEADER ****
Cache-Control: private
Content-Type: text/html; charset=utf-8
Set-Cookie: ASP.NET_SessionId=fvzaz313shjetax1quir1zcb; path=/; HttpOnly
X-AspNetMvc-Version: 3.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 02 Dec 2014 13:28:58 GMT
Connection: close
Set-Cookie: BIGipServerGENIIS02_82_pool=739784202.20992.0000; path=/
Set-Cookie: TS01f1aa4e=015730da4e24ad3b754254603149f805c89164b8ab298fb9764ee710fbfe03b5b6a11c311f9222d197ffc922f85b51d35bb423ab8d3920a7d0e7e5a68f4c7ca6397fe006b5; Path=/
Transfer-Encoding: chunked
**** CONTENT ****
d6e
<!DOCTYPE html>
<html>
<head>
<title>Aanmelden</title>
<link href="/Content/Site.css" rel="stylesheet" type="text/css" />
<script src="/Scripts/jquery-1.5.1.min.js" type="text/javascript"></script>
</head>
<body>
<div class="page">
<div id="header">
<div id="header-title">
<h1>ReCreateX</h1>
</div>
<div id="logindisplay">
<!DOCTYPE html>
</div>
</div>
<div id="main">
<script src="/Scripts/jquery.validate.min.js" type="text/javascript"></script>
<script src="/Scripts/jquery.validate.unobtrusive.min.js" type="text/javascript"></script>
<script type="text/javascript">
$(function () {
$('#UserName').focus();
});
</script>
<form action="/LogOn/LogOn" method="post"> <fieldset>
<legend>Aanmelden</legend>
<p>
<div class="validation-summary-valid" data-valmsg-summary="true"><span>Aanmelden mislukt. Gelieve de fouten te corrigeren en opnieuw te proberen.</span>
<ul><li style="display:none"></li>
</ul></div>
</p>
<p class="description">
Gelieve uw gebruikersnaam en wachtwoord in te geven.
</p>
<table class="tableFullWidth">
<tr>
<td valign="top">
<img src="/Content/Images/login.png" alt="Login" />
</td>
<td>
<table class="tableFullWidth">
<tr>
<td class="tdlabel">
Gebruikersnaam
</td>
<td class="tdfield">
<input data-val="true" data-val-required="Het veld 'Gebruikersnaam' is vereist." id="UserName" name="UserName" type="text" value="" />
</td>
</tr>
<tr>
<td class="tdlabel">
Wachtwoord
</td>
<td class="tdfield">
<input data-val="true" data-val-required="Het veld 'Wachtwoord' is vereist." id="Password" name="Password" type="password" />
</td>
</tr>
<tr>
<td>
</td>
<td class="tdlabel">
<input data-val="true" data-val-required="The RememberMe field is required." id="RememberMe" name="RememberMe" type="checkbox" value="true" /><input name="RememberMe" type="hidden" value="false" />
<label for="RememberMe">Onthoud mij?</label>
</td>
</tr>
<tr>
<td>
<input type="submit" value="Aanmelden" />
</td>
<td class="tdlabel">
<a href="/ForgotPassword">Wachtwoord vergeten?</a>
</td>
</tr>
</table>
</td>
</tr>
</table>
</fieldset>
</form>
</div>
</div>
</body>
</html>
0
HTTP OK: HTTP/1.1 200 OK - 4010 bytes in 0.010 second response time |time=0.009949s;;;0.000000 size=4010B;;;0
(seach for "58e" and you will see the weirdness)
Code: Select all
# /usr/local/nagios/libexec/check_http -v -H citycard.gent.be --onredirect=follow -s "Gelieve uw gebruikers"
option f:4
GET / HTTP/1.1
User-Agent: check_http/v2.0.3 (nagios-plugins 2.0.3)
Connection: close
Host: citycard.gent.be
Accept: */*
http://citycard.gent.be:80/ is 4024 characters
STATUS: HTTP/1.1 200 OK
**** HEADER ****
Cache-Control: private
Content-Type: text/html; charset=utf-8
Set-Cookie: ASP.NET_SessionId=4wrjxfgllmwqddqwrrzrlter; path=/; HttpOnly
X-AspNetMvc-Version: 3.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Tue, 02 Dec 2014 13:28:59 GMT
Connection: close
Set-Cookie: BIGipServerGENIIS02_82_pool=739784202.20992.0000; path=/
Set-Cookie: TS01f1aa4e=015730da4e2e552fc67d16f8b19bedefa6e54672526a5b26e75ff5961ded5ecb5199dc522cf86a48273a535c3d50845251d966106352e61cf7b7e208d7be245a1a309a1a14; Path=/
Transfer-Encoding: chunked
**** CONTENT ****
467
<!DOCTYPE html>
<html>
<head>
<title>Aanmelden</title>
<link href="/Content/Site.css" rel="stylesheet" type="text/css" />
<script src="/Scripts/jquery-1.5.1.min.js" type="text/javascript"></script>
</head>
<body>
<div class="page">
<div id="header">
<div id="header-title">
<h1>ReCreateX</h1>
</div>
<div id="logindisplay">
<!DOCTYPE html>
</div>
</div>
<div id="main">
<script src="/Scripts/jquery.validate.min.js" type="text/javascript"></script>
<script src="/Scripts/jquery.validate.unobtrusive.min.js" type="text/javascript"></script>
<script type="text/javascript">
$(function () {
$('#UserName').focus();
});
</script>
<form action="/LogOn/LogOn" method="post"> <fieldset>
<legend>Aanmelden</legend>
<p>
<div class="validation-summary-valid" data-valmsg-summary="true"><span>Aanmelden mislukt. Gelieve de fouten te corrigeren en opnieuw te proberen.</span>
<ul><li style="display:none"></li>
</ul></div>
</p>
<p class="description">
Gel
58e
ieve uw gebruikersnaam en wachtwoord in te geven.
</p>
<table class="tableFullWidth">
<tr>
<td valign="top">
<img src="/Content/Images/login.png" alt="Login" />
</td>
<td>
<table class="tableFullWidth">
<tr>
<td class="tdlabel">
Gebruikersnaam
</td>
<td class="tdfield">
<input data-val="true" data-val-required="Het veld 'Gebruikersnaam' is vereist." id="UserName" name="UserName" type="text" value="" />
</td>
</tr>
<tr>
<td class="tdlabel">
Wachtwoord
</td>
<td class="tdfield">
<input data-val="true" data-val-required="Het veld 'Wachtwoord' is vereist." id="Password" name="Password" type="password" />
</td>
</tr>
<tr>
<td>
</td>
<td class="tdlabel">
379
<input data-val="true" data-val-required="The RememberMe field is required." id="RememberMe" name="RememberMe" type="checkbox" value="true" /><input name="RememberMe" type="hidden" value="false" />
<label for="RememberMe">Onthoud mij?</label>
</td>
</tr>
<tr>
<td>
<input type="submit" value="Aanmelden" />
</td>
<td class="tdlabel">
<a href="/ForgotPassword">Wachtwoord vergeten?</a>
</td>
</tr>
</table>
</td>
</tr>
</table>
</fieldset>
</form>
</div>
</div>
</body>
</html>
0
HTTP CRITICAL: HTTP/1.1 200 OK - string 'Gelieve uw gebruikers' not found on 'http://citycard.gent.be:80/' - 4024 bytes in 0.013 second response time |time=0.012724s;;;0.000000 size=4024B;;;0
Feel free to run your own tests against citycard.gent.be, it's a public website. But give us a yell and the IP address where you are doing/have done the checks from, so we can discern you from actual DOS attempts