Long Apply Config Time with Large Imports

[Deantwo]

In an attempt to make the system more automated I have created a little bash script that I can call externally to create an new host using specific templates.
The script works nicely and is not really my issue, the problem is that with the amount of hosts I am importing at a time is making "Apply Configuration" run for a rather long time.

I add about 850 files to the /usr/local/nagios/etc/import folder at a time, this is in order to override existing host configurations with possible changes. Then I apply the configuration using NagiosXI's web interface.
I would use /usr/local/nagiosxi/scripts/reconfigure_nagios.sh, but since it takes so long to complete I decided against it for now because it would freeze my application while it waits for it to finish.

So my question is, is there anyway I can speed this up?
Of course now I am thinking up ways of how I can improve this on my end, but still curious about if there is anything that can be done on the Nagios side.

It is kinda hilarious how many dots the "Apply Configuration" screen will append. Does it make one for each imported file?

Apply Slowly.png

[lmiltchev]

When you are importing so many configs at a time, apply configuration will take a long time. I believe each dot corresponds to each second. Most probably, apply configuration is timing out.

Can you show us the resource limits in the "/etc/php.ini" file? Run the following command and show the output:

Code: Select all

grep 'max_execution_time =\|max_input_time =\|memory_limit\|max_input_vars =' /etc/php.ini

[Deantwo]

Can you show us the resource limits in the "/etc/php.ini" file? Run the following command and show the output:

Code: Select all

grep 'max_execution_time =\|max_input_time =\|memory_limit\|max_input_vars =' /etc/php.ini

Code: Select all

[root@localhost ~]# grep 'max_execution_time =\|max_input_time =\|memory_limit\|max_input_vars =' /etc/php.ini
max_execution_time = 30
max_input_time = 60
memory_limit = 128M
[root@localhost ~]#

Would increasing this maybe make it faster?

I didn't really say how long it actually takes to apply the configuration with all the import files, I am guessing it is like 5 minutes or so, but I haven't really timed it.

[ssax]

As long as it completes the only thing you can really do is add faster disks, more RAM, more CPU, etc. Actually, you might be able to setup a RAMDisk and mount it to the /usr/local/nagios/etc/import directory but I haven't tested it yet, but that will only speed up the import. The apply configuration imports the data into the DB, exports the data to files, and then restarts the nagios service. You could see where it's taking the most time by running this tail command:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log

Then apply configuration again and watch the tail output and it will show you what it's doing.


Thank you

[Deantwo]

You could see where it's taking the most time by running this tail command:

Code: Select all

tail -f /usr/local/nagiosxi/var/cmdsubsys.log

Then apply configuration again and watch the tail output and it will show you what it's doing.

Most I see stuff like this repeated over and over:

Code: Select all

IMPORTING /usr/local/nagios/etc/import/10.x.x.1.cfg
CMDLINE:
/usr/bin/wget --load-cookies=nagiosql.cookies http://localhost/nagiosxi/includes/components/ccm/ --no-check-certificate --post-data 'cmd=admin&type=import&importsubmitted=true&chbOverwrite=1&subForm=Import&selImportFile[]=/usr/local/nagios/etc/import/10.x.x.1.cfg' -O nagiosql.import.monitoring
--2016-08-09 08:21:18--  http://localhost/nagiosxi/includes/components/ccm/
Resolving localhost... ::1, 127.0.0.1
Connecting to localhost|::1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: “nagiosql.import.monitoring”

     0K .......... .......... .......... .......... .......... 88.0K
    50K .......... .......... .......... .......... .......... 43.7M
   100K .......... .......... .......... .......... .......... 47.3M
   150K .......... .......... ..                               1016K=0.6s

2016-08-09 08:21:19 (291 KB/s) - “nagiosql.import.monitoring” saved [176237]

Of course it is not for the same file more than once.

So this is where it is loading the files into the database I guess?

[tgriep]

One thing you can try is to do some changes in the php.ini file and see if that helps.
Can you edit the /etc/php.ini file and change the following from

Code: Select all

max_execution_time = 30
max_input_time = 60
memory_limit = 128M

to

Code: Select all

max_execution_time = 60
max_input_time = 120
memory_limit = 512M

Save the file and restart apache by running

Code: Select all

service httpd restart

Try this to see if it helps out.