Fusion - Web Accounts Disabled

[P201420]

This morning accessing fusion - site would not load
rebooted from command line
web page back up

can not login - no existing account can login - account disabled message
including nagiosadmin.

Ideas?

Thanks

[benjaminsmith]

Hello @P2201420,

Can you post a screenshot of the exact error message, so I can look it up in the code? Also, there's the possiblibly of database issues as the site would not load, please post the database and apache logs for us to review. Thanks.

Code: Select all

tail -n 25 /var/log/mariadb/mariadb.log
# or
tail -n 25 /var/log/mysqld.log

#Apache logs
tail -n 25 /var/log/httpd/*error_log

[P201420]

fusion disabled login.PNG

here is screen shot

[P201420]

[root@ny17u3030 ~]# tail -n 25 /var/log/mariadb/mariadb.log
:0(__GI_raise)[0x7f75fa4da207]
:0(__GI_abort)[0x7f75fa4db8f8]
/usr/libexec/mysqld(+0x81add0)[0x55e7edfeedd0]
/usr/libexec/mysqld(+0x7cb42d)[0x55e7edf9f42d]
/usr/libexec/mysqld(+0x7d6bf6)[0x55e7edfaabf6]
/usr/libexec/mysqld(+0x76611e)[0x55e7edf3a11e]
/usr/libexec/mysqld(+0x83765f)[0x55e7ee00b65f]
/usr/libexec/mysqld(+0x83adea)[0x55e7ee00edea]
/usr/libexec/mysqld(+0x83b07d)[0x55e7ee00f07d]
/usr/libexec/mysqld(+0x7103e4)[0x55e7edee43e4]
/usr/libexec/mysqld(+0x6f658c)[0x55e7edeca58c]
/usr/libexec/mysqld(_ZN7handler12ha_write_rowEPh+0xb6)[0x55e7edc80806]
/usr/libexec/mysqld(_Z12write_recordP3THDP5TABLEP12st_copy_info+0x1d0)[0x55e7edb2de10]
/usr/libexec/mysqld(_Z12mysql_insertP3THDP10TABLE_LISTR4ListI4ItemERS3_IS5_ES6_S6_15enum_duplicatesb+0xcc4)[0x55e7edb312d4]
/usr/libexec/mysqld(_Z21mysql_execute_commandP3THD+0x2a6e)[0x55e7edb4907e]
/usr/libexec/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x125)[0x55e7edb4d445]
/usr/libexec/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1750)[0x55e7edb4f4a0]
/usr/libexec/mysqld(_Z24do_handle_one_connectionP3THD+0x1c2)[0x55e7edc01a22]
/usr/libexec/mysqld(handle_one_connection+0x4a)[0x55e7edc01aca]
pthread_create.c:0(start_thread)[0x7f75fbda7dd5]
/lib64/libc.so.6(clone+0x6d)[0x7f75fa5a1ead]

Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (0x7f7530005f58): REPLACE INTO polled_extras (polled_data_id, poll_key, extra_value) VALUES ('631669', 'das[root@ny17u3030 ~]#

[P201420]

[root@ny17u3030 ~]# tail -n 25 /var/log/httpd/*error_log
[Mon Oct 21 11:23:56.268084 2019] [:error] [pid 10400] [client 172.19.77.90:61130] PHP Warning: file_put_contents(): Only 0 of 69 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Mon Oct 21 11:23:56.268108 2019] [:error] [pid 10400] [client 172.19.77.90:61130] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Mon Oct 21 11:23:56.271744 2019] [:error] [pid 10400] [client 172.19.77.90:61130] PHP Warning: file_put_contents(): Only 0 of 123 bytes written, possibly out of fr ee disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Mon Oct 21 11:23:56.271781 2019] [:error] [pid 10400] [client 172.19.77.90:61130] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Mon Oct 21 11:23:56.271824 2019] [:error] [pid 10400] [client 172.19.77.90:61130] PHP Warning: file_put_contents(): Only 0 of 69 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Mon Oct 21 11:23:56.271849 2019] [:error] [pid 10400] [client 172.19.77.90:61130] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, p[Tue Oct 22 10:47: 16.060253 2019] [mpm_prefork:notice] [pid 9861] AH00170: caught SIGWINCH, shutting down gracefully
[Tue Oct 22 10:47:33.469683 2019] [suexec:notice] [pid 9867] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Tue Oct 22 10:47:33.487606 2019] [auth_digest:notice] [pid 9867] AH01757: generating secret for digest authentication ...
[Tue Oct 22 10:47:33.488072 2019] [lbmethod_heartbeat:notice] [pid 9867] AH02282: No slotmem from mod_heartmonitor
[Tue Oct 22 10:47:33.591135 2019] [mpm_prefork:notice] [pid 9867] AH00163: Apache/2.4.6 (CentOS) PHP/5.4.16 configured -- resuming normal operations
[Tue Oct 22 10:47:33.591187 2019] [core:notice] [pid 9867] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'
[Tue Oct 22 10:47:49.413322 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 123 bytes written, possibly out of fr ee disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Tue Oct 22 10:47:49.413395 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Tue Oct 22 10:47:49.413470 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 69 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Tue Oct 22 10:47:49.413493 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Tue Oct 22 10:47:49.416566 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 123 bytes written, possibly out of fr ee disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Tue Oct 22 10:47:49.416601 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Tue Oct 22 10:47:49.416641 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 69 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Tue Oct 22 10:47:49.416666 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Tue Oct 22 10:47:49.417019 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 123 bytes written, possibly out of fr ee disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Tue Oct 22 10:47:49.417047 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Tue Oct 22 10:47:49.417084 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 69 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 573
[Tue Oct 22 10:47:49.417108 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: file_put_contents(): Only 0 of 36 bytes written, possibly out of fre e disk space in /usr/local/nagiosfusion/html/includes/classes/db.inc.php on line 574
[Tue Oct 22 10:47:49.431572 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: Unknown: write failed: No space left on device (28) in Unknown on li ne 0
[Tue Oct 22 10:47:49.431593 2019] [:error] [pid 10391] [client 172.19.77.90:61839] PHP Warning: Unknown: Failed to write session data (files). Please verify that th [root@ny17u3030 ~]#

[P201420]

[root@ny17u3030 ~]# cd /usr/local/nagiosfusion/html/includes/classes/
[root@ny17u3030 classes]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/centos-root 254798528 244040584 0 100% /
devtmpfs 16373960 0 16373960 0% /dev
tmpfs 16385796 0 16385796 0% /dev/shm
tmpfs 16385796 11932 16373864 1% /run
tmpfs 16385796 0 16385796 0% /sys/fs/cgroup
/dev/sda1 999320 201204 729304 22% /boot
tmpfs 3277160 0 3277160 0% /run/user/0
tmpfs 3277160 0 3277160 0% /run/user/1000
[root@ny17u3030 classes]# ^C
[root@ny17u3030 classes]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 243G 233G 0 100% /
devtmpfs 16G 0 16G 0% /dev
tmpfs 16G 0 16G 0% /dev/shm
tmpfs 16G 12M 16G 1% /run
tmpfs 16G 0 16G 0% /sys/fs/cgroup
/dev/sda1 976M 197M 713M 22% /boot
tmpfs 3.2G 0 3.2G 0% /run/user/0
tmpfs 3.2G 0 3.2G 0% /run/user/1000
[root@ny17u3030 classes]#

[P201420]

is there a cleanup option from command line?

[lmiltchev]

Check to see if you have any backups in "/store/backups/nagiosfusion". If you have bunch of them, you could probably delete some of the older ones to clear some space. You can also delete some/all logs under /var/log. To find the 10 largest files in /var/log, you could run:

Code: Select all

du -a /var/log | sort -n -r | head -n 10

After you clear some room, you should try adding more disk space. Is this a physical machine or a VM? If it is a VM, you could try to follow the steps, outlined in the document below:

https://assets.nagios.com/downloads/nag ... M-Disk.pdf

It is for XI, but it should work for Fusion as well. You will need to use the manual process though. Don't try to use the automated way (via script) as it is for XI! Clone the VM or exported as a OVA before trying to resize it (just in case something goes wrong).

Let us know if this helped.

[P201420]

i resized it via those instructions when it was first built
250 GB

its a vm

Log files are only a few mb - but I deleted them anyway.,

this directory is empty
/store/backups/nagiosfusion

any other ideas on how to clean enough up?

had to reboot again and still cant login

[lmiltchev]

Hmm, I wonder if you set your log level to "debug" or "trace", and you have tons of logs in "/usr/local/nagiosfusion/var/log" directory. Can you cd into this directory and see what kind of logs you have in there?