Inconsistency in a script

[peter.zanetti]

So I have put together a simple script to read a file and delete hosts in nagios to automate the inventory we keep.

delete-hosts.sh

It works as designed, however when I run it sometimes it doesn't work the first time. I wait 5 - 10 minutes and run it again and it works. Does anyone have an idea as to why it doesn't work the first time sometimes?

[tmcdonald]

So I have put together a simple script to read a file and delete hosts in nagios to automate the inventory we keep.

I hate to be the bearer of bad news, but this is not only generally outside of what we can support (debugging custom scripts) but direct database manipulation is not something we can condone, endorse, or support. It is incredibly easy to break things when doing direct inserts/deletes into the DB, and for this reason we have a "Do at your own risk" policy with the understanding that any resulting damage is out of scope.

Update: I might have been a bit trigger-happy on the reply, didn't see how simple the script really is. I do still need to stand by the "Be incredibly careful when working around the web interface" statement, however. We need to be very clear in that regard.

That being said, how long is the delete-hosts.csv file typically? 10 lines? 100 lines? Do those hosts ever have services attached to them? Is someone perhaps doing an Apply Config while this is running? Nothing sticks out as being overtly wrong here, or obviously introducing a delay. One thing to note is that the nagiosql_delete_host.php script, at least to my recent memory, may not have been updated along with the rest of XI so there might be some old/broken code in there somewhere.

[peter.zanetti]

We have not put this into production yet and have just been testing in our dev environment. the delete-host.csv file will be about 300 devices the first time this script is used in production, but after that should be no more that 20-25 per day. These hosts do not have services attached to them, they are inheriting the services from the host group they belong to. No one else is in the development box when I am running these tests so an Apply Config should not be running while the tests are going on.

Also just for testing purposes the delete-host.csv file is only 5 devices.

[ssax]

Do you receive any error output, warnings, or anything when it doesn't work?

Do you see anything in your apache logs?

Code: Select all

/var/log/httpd/error_log

OR

/var/log/httpd/ssl_error_log

Do you see any messages in this file after you're done?

Code: Select all

/usr/local/nagiosxi/scripts/nagiosql.delete.host

[peter.zanetti]

I just ran the delete script and it did not work.

Here is the /var/log/httpd/error_log

Code: Select all

[Tue Aug 25 14:22:54 2015] [error] [client ::1] PHP Notice:  Undefined index: ccm_language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:54 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:54 2015] [error] [client ::1] PHP Notice:  Undefined index: ccm_language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:54 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:54 2015] [error] [client ::1] PHP Notice:  Undefined index: ccm_language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:54 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:55 2015] [error] [client ::1] PHP Notice:  Undefined index: ccm_language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:55 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:55 2015] [error] [client ::1] PHP Notice:  Undefined index: ccm_language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:55 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:56 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 710
[Tue Aug 25 14:22:56 2015] [error] [client ::1] PHP Notice:  Undefined index: language in /usr/local/nagiosxi/html/includes/components/ccm/includes/common_functions.inc.php on line 711

There is nothing in the /var/log/httpd/ssl_error_log

and here is the file /usr/local/nagiosxi/scripts/nagiosql.delete.host

Code: Select all

more /usr/local/nagiosxi/scripts/nagiosql.delete.host
        <!DOCTYPE html>
        <!-- Produced by Nagios XI. Copyright (c) 2008-2015 Nagios Enterprises, LLC (www.nagios.com). All Rights Reserved. -->
        <!-- Powered by the Nagios Synthesis Framework -->

        <html>

    <head>
        <meta http-equiv="X-UA-Compatible" content="IE=Edge"/>
                <title>Nagios XI - Nagios Core Config Manager</title>
        <meta name="ROBOTS" content="NOINDEX, NOFOLLOW">
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>


            <link rel="shortcut icon" href="http://localhost/nagiosxi/images/favicon.ico" type="image/ico"/>
    <link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/css/jquery.autocomplete.css'/>

    <script type='text/javascript'>
        //javascript:alert(document.documentMode);

        var base_url = "http://localhost/nagiosxi/";
        var backend_url = "http%3A%2F%2Flocalhost%2Fnagiosxi%2Fincludes%2Fcomponents%2Fccm%2Findex.php";
        var ajax_helper_url = "http://localhost/nagiosxi/ajaxhelper.php";
        var ajax_proxy_url = "http://localhost/nagiosxi/ajaxproxy.php";
        var suggest_url = "http://localhost/nagiosxi/suggest.php";
        var request_uri = "%2Fnagiosxi%2Fincludes%2Fcomponents%2Fccm%2F";
        var permalink_base = "http://localhost/nagiosxi/includes/components/ccm/?&type=host&cmd=delete&id=1724";
        var demo_mode =0;
        var nsp_str = "c4b48baa049185b7b44f0d62fbac0e4f";
    </script>

            <!-- Bootstrap for 2014 UI -->
        <link rel="stylesheet" href="http://localhost/nagiosxi/includes/css/bootstrap.min.css?2014R2.7" type="text/css" />

    <!-- Adding Font-Awesome for all themes -->
    <link rel="stylesheet" href="http://localhost/nagiosxi/includes/css/font-awesome.min.css?2014R2.7" type="text/css" />

    <!-- main jquery libraries -->
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/jquery/jquery-1.8.2.min.js?2014R2.7'></script>
    <link type="text/css" href="http://localhost/nagiosxi/includes/js/jquery/css/smoothness/jquery-ui-1.9.0.custom.min.css?2014R2.7" rel="stylesheet"/>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/jquery/jquery.colorBlend.js'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/jquery/jquery.timers-1.1.3.js'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/jquery/jquery-ui-1.9.0.custom.min.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/jquery/jquery.searchabledropdown.custom.js'></script>

    <!-- colorpicker -->
    <link rel="stylesheet" href="http://localhost/nagiosxi/includes/js/jquery/colorpicker/css/colorpicker.css" type="text/css" />
    <script type="text/javascript" src="http://localhost/nagiosxi/includes/js/jquery/colorpicker/js/colorpicker.js"></script>

    <!-- clipboard plugin -->
    <script type="text/javascript" src="http://localhost/nagiosxi/includes/js/jquery/jquery.zclip.min.js"></script>

    <!-- XI JS Scripts -->
        <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/core_2014.js?2014R2.7'></script>
        <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/commands.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/views.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/dashboards.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/dashlets.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/tables.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/users.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/perfdata.js?2014R2.7'></script>
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/wizards.js?2014R2.7'></script>

    <!-- XI CSS -->
        <link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/css/nagiosxi_2014.css?2014R2.7' />

    <!-- Highcharts Graphing Library -->
    <script type="text/javascript" src="http://localhost/nagiosxi//includes/js/highcharts/highcharts.js?416"></script>
            <script type="text/javascript" src="http://localhost/nagiosxi//includes/js/highcharts/highcharts-more.js?416"></script>
            <script type="text/javascript" src="http://localhost/nagiosxi//includes/js/highcharts/modules/exporting.js?416"></script>
    <!-- D3 Graphing Library -->
    <script type='text/javascript' src='http://localhost/nagiosxi/includes/js/d3/d3.v3.min.js?2014R2.7'></script>

    <script type='text/javascript' src='http://localhost/nagiosxi/includes/dashlets/gauges/js/gauge.js'></script>
<link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/dashlets/internethealthreport/internethealthreport.css' />
<link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/dashlets/internettrafficreport/internettrafficreport.css' />
<link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/dashlets/rss_dashlet/rss_dashlet.css' />
<link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/dashlets/sansrisingports/sansrisingports.css' />
<link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/dashlets/worldtimeserver/worldtimeserver.css' />
                <link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/components/ccm/css/style.css?2.3.3' /><script type="text/javascript" src="http://localhost/nagiosxi/i
ncludes/components/ccm/javascript/main_js.js?2.3.3"></script>
          <script type="text/javascript">
          var NAGIOSXI=true
          </script><script type="text/javascript" src="http://localhost/nagiosxi/includes/components/graphexplorer/includes/graphexplorerinclude.js"></script><script type='text/javascript' src='htt
p://localhost/nagiosxi/includes/components/helpsystem/helpsysteminclude.js?0.5'></script><link rel='stylesheet' type='text/css' href='http://localhost/nagiosxi/includes/components/helpsystem/helpsy
stem.css?0.5' />    </head>


    <body  class=' child' >


    <div  class="childpage"><!-- page-->

    <div id ="childheader" class="childhead" >

<!--- CHILD HEADER START -->

<div id="child_popup_layer">
    <div id="child_popup_content">
        <div id="child_popup_close">
            <a id="close_child_popup_link" style="display: inline-block;" title="Close"><i class="fa fa-times" style="font-size: 16px;"></i></a>
        </div>
        <div id="child_popup_container">
        </div>
    </div>
</div>

<!--- CHILD HEADER END -->    </div><!--header -->


            <div id="childcontentthrobber"><img src='http://localhost/nagiosxi/images/throbber1.gif'/></div>
        <div id="screen-overlay"></div>
    <div id="loginMsgDiv" style="display: none;">
        <span class='deselect'>
            <div >
                Login Required!            </div>
        </span>
    </div>

    <div id='loginDiv'>
        <h3>Nagios CCM Login</h3>
        <form id='loginForm' action='index.php' method='post'>
            <label for='username'>Username: </label><br />
            <input type='text' name='username' id='username' size='20' /><br /><br />
            <label for='password'>Password</label><br />
            <input type='password' name='password' id='password' size='20' /><br /><br />
            <input type='hidden' name='loginSubmitted' value='true' />
            <input type='hidden' name='menu' value='invisible' />
            <input class='ccmbutton' type='submit' name='submit' id='submit' value='Login' />
        </form>
    </div>

    <!-- CHILD FOOTER START -->


<!-- CHILD FOOTER END -->
    </div><!--page-->

    <noframes>
        <!-- This page requires a web browser which supports frames. -->
        <h2>Nagios XI</h2>
        <p align="center">
            <a href="http://www.nagios.com/">www.nagios.com</a><br>
            Copyright (c) 2009-2012 Nagios Enterprises, LLC<br>
        </p>
        <p>
            <i>Note: These pages require a browser which supports frames</i>
        </p>
    </noframes>



    </body>
</html>

[tmcdonald]

It works as designed, however when I run it sometimes it doesn't work the first time.

Could you clarify specifically what you mean when you say it does not work? Do you mean the deleted hosts are not removed from the CCM or they are not removed from the Host Detail page? Both?

Is there any output when you run the script? It really should be producing something since the nagiosql_delete_host.php script does so.

[lmiltchev]

Let step back for a bit. A host can be deleted ONLY if all of the services on this particular host are deleted first.

You will need to add the following line to your script:

Code: Select all

/usr/local/nagiosxi/scripts/nagiosql_delete_service.php --config=$Name

The delete will still fail if you have any other unresolved relationships - host or service dependencies, etc.

Also, do the names that you grab from the "delete-hosts.csv" match the config names in "/usr/local/nagios/etc/hosts" and "/usr/local/nagios/etc/services"?

[peter.zanetti]

It works as designed, however when I run it sometimes it doesn't work the first time.

Could you clarify specifically what you mean when you say it does not work? Do you mean the deleted hosts are not removed from the CCM or they are not removed from the Host Detail page? Both?

Is there any output when you run the script? It really should be producing something since the nagiosql_delete_host.php script does so.

When I say It does not work I mean the script runs and completes fine as if it has worked but the devices never leave core or host detail and I can still search for them. I then run the script for a second time and it deletes the devices out. When I kick the script off manually there is output and I watch it go through each device one by one and then apply the config. When it kicks off as a crontab I don't see anything.

Let step back for a bit. A host can be deleted ONLY if all of the services on this particular host are deleted first.

You will need to add the following line to your script:

Code: Select all

/usr/local/nagiosxi/scripts/nagiosql_delete_service.php --config=$Name

The delete will still fail if you have any other unresolved relationships - host or service dependencies, etc.

Also, do the names that you grab from the "delete-hosts.csv" match the config names in "/usr/local/nagios/etc/hosts" and "/usr/local/nagios/etc/services"?

None of the devices I'm trying to delete have services attached to them, we are using hostgroups to inherit services.

[jdalrymple]

When I run that script it offers console output regarding what's going on:

Code: Select all

URL: http://localhost/nagiosxi/includes/components/ccm/
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'type=host&cmd=delete&id=30' -O nagiosql.delete.host
--2015-08-27 10:34:59--  http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.delete.host”

    [ <=>                                                                                           ] 38,346      --.-K/s   in 0.02s

2015-08-27 10:34:59 (1.60 MB/s) - “nagiosql.delete.host” saved [38346]

You should be getting some of the same I would assume. Are you getting anything different from one iteration to the next when it succeeds?

[peter.zanetti]

It looks very similar to that regardless of if the devices are actually removed or not.