Long Delay for Scheduled Downtimes for Non-admin Users

[onegative]

G 'Day Support,

I am trying to figure out why I have an extremely long delay for populating and eventually completing the Scheduled Downtime screen when attempting to schedule a blackout. If I use an Admin account the returned page completes fairly quick but all non-admin users take a long a minute or more to complete. I have two recorded sessions showing real-time against the system that I can provide the actually delay. I am using Host Templates and a Contact Group to specify host access to the non-admin users. This delay happens with different teams but essentially in the exact same manner...long delay to complete the host list and complete the downtime form.

Please watch the two attached videos for the exact difference in time. Unfortunately my session recorder only does swf files which are not allowed to be attached to this forum software. So you will have to download them from my Google Share at URL https://drive.google.com/drive/folders/ ... sp=sharing

Please let me know what you think and perhaps a way to troubleshoot this deeper and determine if there is a way to increase the resulting form completion for non-admin users.

As always thanks for your help and support,
Danny

[npolovenko]

Hello, @onegative. Please run the following command to test the database tables for corruption:

mysqlcheck -r -f -uroot -pnagiosxi --all-databases --use_frm

Then run the following command in the command line:

tail -f /var/log/*error_log*

And while it's still running please log in as the non-admin user, go to the schedule downtime menu and while its hanging look for any errors in the console.

[onegative]

Hey npolovenko,

So I performed the actions requested. I did not see any obvious corruptions reported in the mysqlcheck but there were info and warnings but unsure if they contribute to the long delay. I attached the results of that command in this ticket. I also tailed the error_log for apache and the mariadb.log while running Scheduled Downtime form as a non-admin user but saw no errors in either. I can see from top that mysql's cpu utilization increases during this time and I am hoping that the sql query itself is not the issue based on the way I use Contact Groups as my access control for servers between teams. But seeing the initial populated list of hosts then delay that just doesn't seem to make sense. I also attached the whole error_log which is very small. Anyway take a gander at the results and let me know if you see anything concerning.

I really appreciate your help,
Danny

[npolovenko]

@onegative, Let's try increasing the setting in the /etc/php.ini file.
https://support.nagios.com/kb/article/n ... e-611.html
Most importantly "max_input_vars" and "memory_limit". You should increase max input vars to 5000 and double the existing memory limit value. Then run the following command to restart apache:

service httpd restart

Let me know if this change increases the speed for the non-admin user.

[onegative]

Hey npolovenko,

Well that really shouldn't make a difference...as I stated initially if I run the form as an Admin it returns all hosts within 10-15 seconds so I don't see how a performance setting would effect returned values if the admin who can see everything returns more objects where a specific team (Contact Group) which sees less objects takes 4 or 5 times as long.

But in the interest of satisfying your request I inspected the php.ini setting:

Current values:
; How many GET/POST/COOKIE input variables may be accepted
; max_input_vars = 1000
max_input_vars = 5000

; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 512M

Modified values:
; How many GET/POST/COOKIE input variables may be accepted
; max_input_vars = 1000
max_input_vars = 10000

; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 1024M

Saw no difference in the non-admin user object returns.

Again maybe if I knew what the actual SQL SELECT call is between an Admin user and a Non-Admin user perhaps I could execute each from mysql to see if it is due to SQL performance or something internal to the PHP???

Let me know and thanks,
Danny

[npolovenko]

@ onegative, Are you on the latest version of XI? How many hosts and services are you monitoring with this XI server?
Whenever you click on the Schedule Downtime menu as a no-nadmin user XI checks to verify that the user has access to all the objects he may apply the command to.
There were some improvements to user permissions and performance in the latest versions of XI. I highly recommend upgrading to XI 5.5.9.

[onegative]

Hey npolovenko,

Yea I am not there yet, still at 5.4.7...I am not big fan of upgrading every time a new patch level is released but in this case I am willing to try.

I will perform an upgrade to my test systems early next week and verify the upgrade process before I update Production.

Question, is there any specific things like upgrading intermediate levels before I reach 5.5.9 or should it recognize the leap of versions automatically?

Please leave the ticket unlocked until I can perform the necessary upgrades to test thereafter.

Thanks for your assistance,
Danny

[npolovenko]

@onegative, You can just upgrade to the latest version right away.

Before you upgrade please create a backup of your XI or take a snapshot of the VM:
https://assets.nagios.com/downloads/nag ... ios-XI.pdf

I suggest upgrading using the script in the command line:
https://assets.nagios.com/downloads/nag ... ctions.pdf

I will leave this open.

[npolovenko]

@onegative, You can just upgrade to the latest version right away.

Before you upgrade please create a backup of your XI or take a snapshot of the VM:
https://assets.nagios.com/downloads/nag ... ios-XI.pdf

I suggest upgrading using the script in the command line:
https://assets.nagios.com/downloads/nag ... ctions.pdf

I will leave this open.

[onegative]

@npolovenko,

I attempted to upgrade one of my test systems manually as suggested using the following commands:

Code: Select all

cd /tmp
rm -rf nagiosxi xi*.tar.gz
wget http://assets.nagios.com/downloads/nagiosxi/xi-latest.tar.gz
tar xzf xi-latest.tar.gz
cd nagiosxi
./upgrade

It got to this point and failed...I am not running nrpe on this server...so what does this mean?
How do I get around this error?
/bin/install -c -m 755 -d /usr/local/nagios/bin
/bin/install -c -m 755 ../uninstall /usr/local/nagios/bin/nrpe-uninstall
/bin/install -c -m 755 nrpe /usr/local/nagios/bin
/bin/install -c -m 755 -d /usr/lib/tmpfiles.d
/bin/install -c -m 644 ../startup/tmpfile.conf /usr/lib/tmpfiles.d/nrpe.conf
make[1]: Leaving directory `/tmp/nagiosxi/subcomponents/nrpe/nrpe-3.2.1/src'
/bin/install -c -m 644 startup/default-service /usr/lib/systemd/system/nrpe.service
mv: cannot stat ‘/usr/local/nagios/etc/nrpe.cfg’: No such file or directory
[root@dcom-nagiosxi-t2 nagiosxi]#