Core Config Manager takes near 3 hours to write configuratio

[virgilio.cervantes]

Hi there,
we are evaluating nagios xi and with 1476 hosts and 8791 Core Config Manager spends near 3 hours to write the configuration.
the host is not overloaded is a 4 cores and 4GB vmware VM in a also not overloaded server.
any ideas?
could you please provide a list of commands to generate a report i can run it and send you the results.

Regards,

[abrist]

You may need to increase the the memory and timeout limits in the file /etc/php.ini given how large the installation is. You should also read the system requirements document for XI, as you could be pushing the server depending how often your 8000+ checks happen.

php.ini changes:
http://support.nagios.com/wiki/index.ph ... _Completes
After these changes, do not forget to restart the apache server:

Code: Select all

service httpd restart

Hardware requirements:
http://assets.nagios.com/downloads/nagi ... ements.pdf

Further troubleshooting can be pursued by doing a running tail of /var/log/httpd/error_log and then trying to write out the configuration.

Code: Select all

tail -f /var/log/httpd/error_log

Click "Apply Configuration"
Post the errors here.

[slansing]

In addition to abrist's requests please install the component I have attached via:

Admin > Manage Components > Browse "Find the zip file" > Upload

Then Navigate to Admin > Profile > Download Profile, download and attach the file which pops up to your reply on this thread.

[virgilio.cervantes]

Hi There,

the php time limits was obviously increased, about memory, there is no memory errors in httpd logs, but i will double check.
current php.ini related setup
memory_limit = 512M
max_execution_time = 30000

with above setting the time was 2:30 near 3 hours.
we have a service definition per service, so we have 8791 services definitions, this is must for us and is not negotiable.
i am reviewing the report, and it contains sensitive information, i need to review it carefully.

[scottwilkerson]

According to our recommendations we would recommend more Cores and more RAM for this many services
http://assets.nagios.com/downloads/nagi ... ements.pdf

In addition to that, what kind of I/O load in on this server?

[abrist]

If you wish to not post the information on a public forum, feel free to pm one of the nagios support staff with the logs/profile.

[virgilio.cervantes]

Hi Guys,

the server is a VM 4 vcpus 4GB RAM and 10Gb disk
is the same configuration that we used with nagios core, with the same amount of hosts and services.
both machines are fully dedicated to nagios.

as we want to compare the performance among other new features, we used the same virtual hardware configuration.
IO on the vmware server is well balanced as is monitored all the time to avoid suffer IO degradation.

the issue is the way that naqiosql write the configuration files, i found the httpd process is performing millions of queries to DB in order to build configuration nagios.
i found thousand of queries like this
SELECT * FROM `tbl_variabledefinition` LEFT JOIN `tbl_lnkServiceToVariabledefinition` ON `id` = `idSlave`\n WHERE `idMaster`=6086 ORDER BY `name`", 180) = 180
i am trying to rebuild the configuration with nagios down, so the server is totally dedicated to build the configuration files, with the same results.
check the number of calls performed in 3 minute, each write is the query to DB and each write is the answer.

[root@localhost share]# time strace -c -s 4040 -p 24229
Process 24229 attached - interrupt to quit
^CProcess 24229 detached
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
51.62 0.004182 2 2091 read
36.85 0.002985 1 2091 write
11.53 0.000934 0 2091 poll
------ ----------- ----------- --------- --------- ----------------
100.00 0.008101 6273 total

real 3m33.209s
user 0m0.080s
sys 0m0.829s


as this rebuild is done by a single thread there is no benefit in have 4 or more cpu cores.

as with every change we need to wait almost 3 hours, I think that your product is not ready for us.
that is a shame, cause the rest of features are awesome.

[scottwilkerson]

I've never seen a healthy server with that quantity of hosts/service take over 5 minutes to apply configuration, something else must be amiss.

Please send profile one you have stripped out any sensitive info for us to take a look at. Please feel free to PM it to one of our staff (marked in green)