Nagios Fusion Mariadb keeps dying

This support forum board is for questions relating to Nagios Fusion.
Locked
hbouma
Posts: 483
Joined: Tue Feb 27, 2018 9:31 am

Nagios Fusion Mariadb keeps dying

Post by hbouma »

Our Mariadb instance keeps crashing with

Code: Select all

Feb 23 14:38:58 SERVERXXXXX kernel: Out of memory: Kill process 14670 (poll_subsys.php) score 17 or sacrifice child
Feb 23 14:38:58 SERVERXXXXX kernel: Killed process 14670 (poll_subsys.php) total-vm:860164kB, anon-rss:482552kB, file-rss:0kB, shmem-rss:0kB
For reference, please see https://support.nagios.com/forum/viewto ... 17&t=51412 and https://support.nagios.com/forum/viewto ... 17&t=50902 and https://support.nagios.com/forum/viewto ... 17&t=49945 along with Ticket 100103

Any help would be appreciated. At this point, we have moved the dbmaint script from running every 5 minutes to every 30 minutes to every hour to allow it to complete properly. Unfortunately, every time the Nagios Fusion system pages one of our Nagios XI instances, the MariaDB instance crashes again.

We have also dropped the innodb_buffer_pool_size from the 6GB previously recommended to us down to 4GB and also upped the RAM on the server to 25GB.
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: Nagios Fusion Mariadb keeps dying

Post by ssax »

How large are the DB tables getting?

Code: Select all

echo "SELECT table_name AS 'Table', round(((data_length + index_length) / 1024 / 1024), 2) 'Size in MB' FROM information_schema.TABLES WHERE table_schema IN ('fusion');" | mysql -uroot -pfusion --table
Get that output first but did you run the truncate_polled to clean it up?

Code: Select all

cd /usr/local/nagiosfusion/scripts/
./truncate_polled.php
hbouma
Posts: 483
Joined: Tue Feb 27, 2018 9:31 am

Re: Nagios Fusion Mariadb keeps dying

Post by hbouma »

I had previously run the truncate polled as part of my troubleshooting from the previous issues.

Code: Select all

+-----------------+------------+
| Table           | Size in MB |
+-----------------+------------+
| auth_tokens     |       0.03 |
| commands        |       0.02 |
| dashboards      |       0.08 |
| dashlets        |       0.28 |
| dashlets_params |       0.08 |
| log             |       2.02 |
| meta            |       0.11 |
| options         |       0.06 |
| polled_averages |       0.03 |
| polled_data     |       1.02 |
| polled_deltas   |       0.03 |
| polled_extras   |     870.03 |
| polling_lock    |       0.02 |
| servers         |       0.03 |
| sysstat         |       0.03 |
| users           |       0.09 |
| users_servers   |       0.08 |
| views           |       0.14 |
+-----------------+------------+
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: Nagios Fusion Mariadb keeps dying

Post by ssax »

Ok, I sent a reply to the original ticket for you to schedule a remote session, we are going to have a developer on with us.
hbouma
Posts: 483
Joined: Tue Feb 27, 2018 9:31 am

Re: Nagios Fusion Mariadb keeps dying

Post by hbouma »

Well, we were able to get the server to stay up by increasing the RAM to 36GB, but I have noticed a new issue. All polls of the end users seem to be failing. The fusion.log file is saying:

Code: Select all

[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 14:57:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:07:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:22:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:22:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:22:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:22:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:22:02] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:36:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:36:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:36:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:36:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:36:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:36:03] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
[2019-02-27 16:37:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USERNAME, bailing out
I am also seeing the following in the dberrors.log

Code: Select all

8: exec_query(DESCRIBE servers, ''): Failed to execute 'DESCRIBE servers'SQL: [16] DESCRIBE servers
Params:  0
7: exec(): SQLSTATE[HY000]: General error: 2013 Lost connection to MySQL server during querySQL: [16] DESCRIBE servers
Params:  0
8: exec_query(DESCRIBE servers, ''): Failed to execute 'DESCRIBE servers'SQL: [16] DESCRIBE servers
Params:  0

Mariadb is up and running. The services on the server seem stable.
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: Nagios Fusion Mariadb keeps dying

Post by ssax »

First, please send the output of this command:

Code: Select all

ps aux
After you've grabbed that output, click the Admin menu item at the top of the screen, then when the page loads, under the "Subsys Status", scroll down and you will see Polling Locks: #, click the little Gear icon next to it and click the Clear Polling Locks button and see if that resolves it.
hbouma
Posts: 483
Joined: Tue Feb 27, 2018 9:31 am

Re: Nagios Fusion Mariadb keeps dying

Post by hbouma »

There are currently 4 poll locks. Those have been cleared.

Code: Select all

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.1  0.0 191716  3876 ?        Ss   Feb28   1:23 /usr/lib/systemd/systemd --switched-root --system --deserialize 22
root         2  0.0  0.0      0     0 ?        S    Feb28   0:00 [kthreadd]
root         3  0.0  0.0      0     0 ?        S    Feb28   0:46 [ksoftirqd/0]
root         5  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/0:0H]
root         7  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/0]
root         8  0.0  0.0      0     0 ?        S    Feb28   0:00 [rcu_bh]
root         9  0.1  0.0      0     0 ?        S    Feb28   2:46 [rcu_sched]
root        10  0.0  0.0      0     0 ?        S<   Feb28   0:00 [lru-add-drain]
root        11  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/0]
root        12  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/1]
root        13  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/1]
root        14  0.0  0.0      0     0 ?        S    Feb28   0:47 [ksoftirqd/1]
root        16  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/1:0H]
root        17  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/2]
root        18  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/2]
root        19  0.0  0.0      0     0 ?        S    Feb28   0:48 [ksoftirqd/2]
root        21  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/2:0H]
root        22  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/3]
root        23  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/3]
root        24  0.0  0.0      0     0 ?        S    Feb28   0:48 [ksoftirqd/3]
root        26  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/3:0H]
root        27  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/4]
root        28  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/4]
root        29  0.0  0.0      0     0 ?        S    Feb28   0:49 [ksoftirqd/4]
root        31  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/4:0H]
root        32  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/5]
root        33  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/5]
root        34  0.0  0.0      0     0 ?        S    Feb28   0:49 [ksoftirqd/5]
root        36  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/5:0H]
root        37  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/6]
root        38  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/6]
root        39  0.0  0.0      0     0 ?        S    Feb28   0:49 [ksoftirqd/6]
root        41  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/6:0H]
root        42  0.0  0.0      0     0 ?        S    Feb28   0:00 [watchdog/7]
root        43  0.0  0.0      0     0 ?        S    Feb28   0:00 [migration/7]
root        44  0.0  0.0      0     0 ?        S    Feb28   0:48 [ksoftirqd/7]
root        46  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/7:0H]
root        48  0.0  0.0      0     0 ?        S    Feb28   0:00 [kdevtmpfs]
root        49  0.0  0.0      0     0 ?        S<   Feb28   0:00 [netns]
root        50  0.0  0.0      0     0 ?        S    Feb28   0:00 [khungtaskd]
root        51  0.0  0.0      0     0 ?        S<   Feb28   0:00 [writeback]
root        52  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kintegrityd]
root        53  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root        54  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root        55  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root        56  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kblockd]
root        57  0.0  0.0      0     0 ?        S<   Feb28   0:00 [md]
root        58  0.0  0.0      0     0 ?        S<   Feb28   0:00 [edac-poller]
root        59  0.0  0.0      0     0 ?        S<   Feb28   0:00 [watchdogd]
root        65  0.0  0.0      0     0 ?        S    Feb28   0:38 [kswapd0]
root        66  0.0  0.0      0     0 ?        SN   Feb28   0:00 [ksmd]
root        67  0.0  0.0      0     0 ?        SN   Feb28   0:03 [khugepaged]
root        68  0.0  0.0      0     0 ?        S<   Feb28   0:00 [crypto]
root        76  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kthrotld]
root        78  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kmpath_rdacd]
root        79  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kaluad]
root        81  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kpsmoused]
root        83  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ipv6_addrconf]
root        98  0.0  0.0      0     0 ?        S<   Feb28   0:00 [deferwq]
root       132  0.0  0.0      0     0 ?        S    Feb28   0:08 [kauditd]
root       863  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ata_sff]
root       890  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_0]
root       898  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_0]
root       900  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_1]
root       902  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_1]
root       987  0.0  0.0      0     0 ?        S<   Feb28   0:00 [nfit]
root      1091  0.0  0.0      0     0 ?        S    07:24   0:00 [kworker/7:0]
root      1141  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_2]
root      1156  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_2]
root      1166  0.0  0.0      0     0 ?        S    07:10   0:00 [kworker/1:0]
root      1170  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_3]
root      1171  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_3]
root      1173  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_4]
root      1183  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_4]
root      1185  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_5]
root      1200  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_5]
root      1211  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_6]
root      1220  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_6]
root      1231  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_7]
root      1240  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_7]
root      1241  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_8]
root      1243  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_8]
root      1261  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_9]
root      1267  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_9]
root      1275  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_10]
root      1284  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_10]
root      1287  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_11]
root      1288  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_11]
root      1289  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_12]
root      1290  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_12]
root      1291  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_13]
root      1292  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_13]
root      1294  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_14]
root      1299  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_14]
root      1318  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_15]
root      1333  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_15]
root      1354  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_16]
root      1356  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_16]
root      1365  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_17]
root      1367  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_17]
root      1375  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ttm_swap]
root      1383  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_18]
root      1389  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_18]
root      1392  0.0  0.0      0     0 ?        S    Feb28   0:00 [irq/16-vmwgfx]
root      1409  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_19]
root      1415  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_19]
root      1420  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_20]
root      1434  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_20]
root      1445  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_21]
root      1456  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_21]
root      1462  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_22]
root      1463  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_22]
root      1470  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_23]
root      1471  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_23]
root      1472  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_24]
root      1473  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_24]
root      1474  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_25]
root      1475  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_25]
root      1476  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_26]
root      1477  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_26]
root      1478  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_27]
root      1479  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_27]
root      1480  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_28]
root      1515  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_28]
root      1521  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_29]
root      1528  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_29]
root      1533  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_30]
root      1545  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_30]
root      1552  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_31]
root      1621  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_31]
root      1700  0.0  0.0      0     0 ?        S    Feb28   0:00 [scsi_eh_32]
root      1716  0.0  0.0      0     0 ?        S<   Feb28   0:00 [scsi_tmf_32]
root      1719  0.0  0.0      0     0 ?        S<   Feb28   0:00 [vmw_pvscsi_wq_3]
root      3577  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/0:1H]
root      3632  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      3633  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      3646  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/5:1H]
root      3648  0.0  0.0      0     0 ?        S    07:25   0:00 [kworker/0:0]
root      3650  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      3651  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      3653  0.0  0.0      0     0 ?        S    07:25   0:00 [kworker/3:1]
root      3663  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      3664  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
apache    3671  0.0  0.0 547100 22936 ?        S    07:25   0:00 /usr/sbin/httpd -DFOREGROUND
root      3709  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-0-8]
root      3710  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      3742  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-2-8]
root      3743  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      3792  0.0  0.0  72260 35808 ?        Ss   Feb28   0:06 /usr/lib/systemd/systemd-journald
root      3821  0.0  0.0 485288   756 ?        Ss   Feb28   0:00 /usr/sbin/lvmetad -f
root      3823  0.0  0.0  44636  1328 ?        Ss   Feb28   0:00 /usr/lib/systemd/systemd-udevd
root      4377  0.0  0.0      0     0 ?        S    06:57   0:00 [kworker/7:2]
root      4802  0.0  0.0      0     0 ?        S    06:14   0:00 [kworker/6:0]
root      5408  0.0  0.0      0     0 ?        S    06:58   0:00 [kworker/3:0]
root      6130  0.0  0.0      0     0 ?        S    03:26   0:00 [kworker/3:2]
root      6324  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      6325  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      6327  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      6337  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      6947  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/6:1H]
root      7026  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/sda1-8]
root      7027  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7033  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7034  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7037  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7038  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7041  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7043  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7047  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7048  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7052  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7055  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7060  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7072  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7079  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7092  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7106  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/7:1H]
root      7121  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-4-8]
root      7122  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7127  0.0  0.0      0     0 ?        S    Feb28   0:03 [jbd2/dm-3-8]
root      7128  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7180  0.0  0.0      0     0 ?        S    Feb28   0:03 [jbd2/dm-8-8]
root      7189  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7195  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-9-8]
root      7196  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7209  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-10-8]
root      7210  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7219  0.0  0.0      0     0 ?        S    Feb28   0:01 [jbd2/dm-7-8]
root      7220  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7229  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7230  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7234  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7235  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7237  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7241  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7246  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7255  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7260  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7262  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7269  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7270  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7275  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kdmflush]
root      7278  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-6-8]
root      7284  0.0  0.0      0     0 ?        S<   Feb28   0:00 [bioset]
root      7289  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7302  0.2  0.0      0     0 ?        S    Feb28   2:57 [jbd2/dm-5-8]
root      7303  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7327  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/1:1H]
root      7353  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-12-8]
root      7354  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7410  0.0  0.0      0     0 ?        S    Feb28   0:01 [jbd2/dm-15-8]
root      7416  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7417  0.0  0.0      0     0 ?        S    Feb28   0:00 [jbd2/dm-17-8]
root      7419  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7427  0.0  0.0      0     0 ?        S    Feb28   0:14 [jbd2/dm-16-8]
root      7428  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7441  0.0  0.0      0     0 ?        S    Feb28   0:01 [jbd2/dm-18-8]
root      7442  0.0  0.0      0     0 ?        S<   Feb28   0:00 [ext4-rsv-conver]
root      7498  0.1  0.0  55520   700 ?        S<sl Feb28   2:46 /sbin/auditd
apache    7501  0.1  0.0 544696 20084 ?        S    07:27   0:00 /usr/sbin/httpd -DFOREGROUND
polkitd   7521  0.0  0.0 625220  3600 ?        Ssl  Feb28   0:09 /usr/lib/polkit-1/polkitd --no-debug
root      7524  0.0  0.0  21688  1124 ?        Ss   Feb28   0:05 /usr/sbin/irqbalance --foreground
root      7525  0.0  0.0 259512  2772 ?        Ss   Feb28   0:00 /usr/sbin/sssd -i --logger=files
root      7528  0.0  0.0  99664  3064 ?        Ss   Feb28   0:00 /usr/bin/VGAuthService -s
root      7529  0.0  0.0 300820  4124 ?        Ss   Feb28   0:59 /usr/bin/vmtoolsd
dbus      7531  0.0  0.0  70648  2428 ?        Ss   Feb28   0:31 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation
ntp       7538  0.0  0.0  59520  1728 ?        Ss   Feb28   0:04 /usr/sbin/ntpd -u ntp:ntp -u ntp:ntp -p /var/run/ntpd.pid -g -x
root      7561  0.0  0.0 339444  9272 ?        S    Feb28   0:02 /usr/libexec/sssd/sssd_be --domain ddmi.intra.renhsc.com --uid 0 --gid 0 --logger=files
root      7562  0.0  0.0 358272  8400 ?        Ssl  Feb28   0:04 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
root      7563  0.0  0.0 277776  4128 ?        S    Feb28   0:02 /usr/libexec/sssd/sssd_nss --uid 0 --gid 0 --logger=files
root      7564  0.0  0.0 247132  3612 ?        S    Feb28   0:04 /usr/libexec/sssd/sssd_pam --uid 0 --gid 0 --logger=files
root      7565  0.0  0.0  36772  1652 ?        Ss   Feb28   0:13 /usr/lib/systemd/systemd-logind
root      7568  0.0  0.0 126288  1120 ?        Ss   Feb28   0:01 /usr/sbin/crond -n
root      7569  0.0  0.0  25904   756 ?        Ss   Feb28   0:00 /usr/sbin/atd -f
root      7590  0.0  0.0 473556  6588 ?        Ssl  Feb28   0:05 /usr/sbin/NetworkManager --no-daemon
apache    7593  0.1  0.0 543732 19536 ?        S    07:27   0:00 /usr/sbin/httpd -DFOREGROUND
root      7707  0.0  0.0      0     0 ?        S    04:10   0:00 [kworker/5:2]
root      8010  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/2:1H]
root      8071  0.0  0.0 421100 15016 ?        Ss   Feb28   0:09 /usr/sbin/httpd -DFOREGROUND
root      8072  0.0  0.0 573928  6008 ?        Ssl  Feb28   0:08 /usr/bin/python2 -Es /usr/sbin/tuned -l -P
root      8075  0.0  0.0  52716  1732 ?        Ss   Feb28   0:00 /usr/sbin/oddjobd -n -p /var/run/oddjobd.pid -t 300
root      8082  0.0  0.0 115708   620 ?        Ss   Feb28   0:00 /usr/bin/rhsmcertd
root      8084  0.0  0.0 112860  3156 ?        Ss   Feb28   0:00 /usr/sbin/sshd -D
root      8086  0.0  0.0 393704  3540 ?        Ssl  Feb28   0:31 /usr/sbin/rsyslogd -n
root      8121  0.0  0.0 107968   572 ?        Ss   Feb28   0:00 rhnsd
root      8154  0.0  0.0 110092   676 tty1     Ss+  Feb28   0:00 /sbin/agetty --noclear tty1 linux
mysql     8291  0.0  0.0 113312  1312 ?        Ss   Feb28   0:00 /bin/sh /usr/bin/mysqld_safe --basedir=/usr
apache    8846  0.0  0.0 540736 16160 ?        S    07:28   0:00 /usr/sbin/httpd -DFOREGROUND
root      8905  0.0  0.0      0     0 ?        S    06:30   0:00 [kworker/2:0]
mysql     8925  115 21.1 11425108 7793316 ?    Sl   Feb28 1616:54 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mariadb/mariadb.log --open-files-limit=20480 --pid-file=/var/run/mariadb/mariadb.pid --socket=/var/lib/mysql/mysql.sock
nagios    8937  0.0  0.0 278888  2036 ?        S    Feb28   0:14 /usr/local/ncpa/ncpa_passive --start
nagios    8939  0.0  0.0 354852 30556 ?        Sl   Feb28   1:18 /usr/local/ncpa/ncpa_listener --start
root      9077  0.0  0.0      0     0 ?        S    07:13   0:00 [kworker/5:0]
root      9096  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/4:1H]
tidal     9350  0.1  0.2 4354852 89492 ?       Sl   Feb28   1:41 /bin/java -cp /opt/TIDAL/Agent/lib/TAgent.jar:/opt/TIDAL/Agent/lib/agutil.jar -Xms16m -Xmx48m JAgent agent=SERVEDRNAME port=5912 logdays=7 OSLnk.InActTimeout=600 path=/opt/TIDAL/Agent bin=/opt/TIDAL/Agent/lib/LINUX stdout=n jobstoptree=y jobkilltree=y agplatform=LINUX calccpuload=y encryptonly=n sslvldcrt=n sslvldhst= sslvldhstpw= sshvldhst= sftpumask=113 jobexec64=n multftpstd=y
root      9774  0.0  0.0      0     0 ?        S<   Feb28   0:00 [kworker/3:1H]
root     10664  0.0  0.0 210252  2428 ?        S    07:29   0:00 /usr/sbin/CROND -n
root     10665  0.0  0.0 210252  2428 ?        S    07:29   0:00 /usr/sbin/CROND -n
root     10666  0.0  0.0 210252  2428 ?        S    07:29   0:00 /usr/sbin/CROND -n
root     10667  0.0  0.0 210252  2428 ?        S    07:29   0:00 /usr/sbin/CROND -n
root     10668  0.0  0.0 210252  2428 ?        S    07:29   0:00 /usr/sbin/CROND -n
nagios   10669  0.0  0.0 113176  1220 ?        Ss   07:29   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosfusion/cron/poll_subsys.php --max-time=60 --master-poll >>/usr/local/nagiosfusion/var/log/poll_subsys.log 2>&1
nagios   10670  1.8  0.0 340528 17364 ?        S    07:29   0:00 /usr/bin/php -q /usr/local/nagiosfusion/cron/poll_subsys.php --max-time=60 --master-poll
nagios   10671  0.0  0.0 113176  1216 ?        Ss   07:29   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosfusion/cron/auth_subsys.php --max-time=60 >>/usr/local/nagiosfusion/var/log/auth_subsys.log 2>&1
nagios   10672  0.0  0.0 113176  1216 ?        Ss   07:29   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosfusion/cron/log_subsys.php --max-time=60 >>/usr/local/nagiosfusion/var/log/log_subsys.log 2>&1
nagios   10673  0.0  0.0 113176  1220 ?        Ss   07:29   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosfusion/cron/sysstat_subsys.php --max-time=60 >>/usr/local/nagiosfusion/var/log/sysstat_subsys.log 2>&1
nagios   10674  0.0  0.0 113176  1220 ?        Ss   07:29   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosfusion/cron/cmd_subsys.php --max-time=60 >>/usr/local/nagiosfusion/var/log/cmd_subsys.log 2>&1
nagios   10675  0.1  0.0 340696 17092 ?        S    07:29   0:00 /usr/bin/php -q /usr/local/nagiosfusion/cron/log_subsys.php --max-time=60
nagios   10676  0.1  0.0 340688 17172 ?        S    07:29   0:00 /usr/bin/php -q /usr/local/nagiosfusion/cron/sysstat_subsys.php --max-time=60
nagios   10677  0.1  0.0 340272 16868 ?        S    07:29   0:00 /usr/bin/php -q /usr/local/nagiosfusion/cron/cmd_subsys.php --max-time=60
nagios   10678  0.5  0.0 340492 16900 ?        S    07:29   0:00 /usr/bin/php -q /usr/local/nagiosfusion/cron/auth_subsys.php --max-time=60
root     10725  0.0  0.0      0     0 ?        S    07:29   0:00 [kworker/4:1]
apache   10734  0.1  0.0 542448 17184 ?        S    07:29   0:00 /usr/sbin/httpd -DFOREGROUND
apache   10740  0.1  0.0 542448 17176 ?        S    07:29   0:00 /usr/sbin/httpd -DFOREGROUND
apache   10741  0.0  0.0 433348  7832 ?        S    07:29   0:00 /usr/sbin/httpd -DFOREGROUND
root     10826  0.1  0.0 187968  6088 ?        Ss   07:29   0:00 sshd: USER [priv]
root     10858  0.0  0.0 313708 21584 ?        Sl   02:02   0:00 /usr/bin/python /usr/libexec/rhsmd
root     11371  0.0  0.0 210252  2424 ?        S    07:00   0:00 /usr/sbin/CROND -n
nagios   11373  0.0  0.0 113176  1220 ?        Ss   07:00   0:00 /bin/sh -c /usr/bin/php -q /usr/local/nagiosfusion/cron/dbmaint_subsys.php >>/usr/local/nagiosfusion/var/log/dbmaint_subsys.log 2>&1
nagios   11381  0.1  0.0 353648 21436 ?        S    07:00   0:01 /usr/bin/php -q /usr/local/nagiosfusion/cron/dbmaint_subsys.php
root     11393  0.0  0.0      0     0 ?        S    07:00   0:00 [kworker/6:2]
USER   11513  0.0  0.0 187968  2512 ?        S    07:29   0:00 sshd: USER@pts/0
USER   11514  0.2  0.0 126260  2736 pts/0    Ss   07:29   0:00 -bash
root     11772  0.0  0.0 274560  4804 pts/0    S    07:29   0:00 sudo su -
root     11968  0.0  0.0 225632  2380 pts/0    S    07:29   0:00 su -
root     11974  0.2  0.0 116340  2984 pts/0    S    07:29   0:00 -bash
nagios   12732  6.0  0.0 352600 20756 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=3 --user=USER
nagios   12759  4.8  0.0 352600 20760 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=3 --user=USER
nagios   12771 17.4  0.1 404584 72988 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=1 --user=USER
nagios   12786  5.7  0.0 352464 20592 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12788  5.5  0.0 352464 20592 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12791  6.0  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12796  6.0  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12800  5.5  0.0 352464 20592 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12803  5.5  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12809  5.7  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12811  4.7  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12815  5.5  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12820  5.7  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12830  6.0  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=8 --user=USER
nagios   12845 10.5  0.1 387920 56000 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=5 --user=USER
nagios   12850 13.2  0.1 402188 70388 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=5 --user=USER
nagios   12894  9.0  0.0 356240 24548 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=7 --user=USER
nagios   12901  8.0  0.0 352464 20588 ?        S    07:29   0:00 /usr/bin/php /usr/local/nagiosfusion/cron/poll_subsys.php --server=7 --user=USER
root     13146  0.0  0.0 165656  1936 pts/0    R+   07:29   0:00 ps aux
root     13265  0.0  0.0      0     0 ?        S    07:14   0:00 [kworker/0:1]
root     16534  0.0  0.0      0     0 ?        S    07:02   0:00 [kworker/u16:0]
root     20271  0.0  0.0      0     0 ?        S    04:30   0:00 [kworker/2:1]
root     23610  0.0  0.0      0     0 ?        S    07:20   0:00 [kworker/0:2]
root     23857  0.0  0.0      0     0 ?        S    06:37   0:00 [kworker/4:0]
apache   26977  0.1  0.0 545904 21296 ?        S    07:21   0:00 /usr/sbin/httpd -DFOREGROUND
root     30163  0.0  0.0      0     0 ?        S    04:20   0:00 [kworker/1:2]
root     31610  0.0  0.0      0     0 ?        S    07:23   0:00 [kworker/4:2]
apache   31615  0.1  0.0 545904 21756 ?        S    07:23   0:00 /usr/sbin/httpd -DFOREGROUND
apache   31616  0.1  0.0 545908 21732 ?        S    07:23   0:00 /usr/sbin/httpd -DFOREGROUND
root     32677  0.0  0.0      0     0 ?        S    07:23   0:00 [kworker/u16:2]

EDIT: Issue is still happening even after the change. As of a half hour later, the fusion.log is filled with:

Code: Select all

[2019-03-01 07:54:03] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:54:03] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:54:03] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:54:03] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:54:03] [SYSTEM] [WARN] [SERVERA]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERB]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:02] [SYSTEM] [WARN] [SERVERC]: poll_subsys(): Lock exists for user:USER, bailing out
[2019-03-01 07:56:03] [SYSTEM] [WARN] [SERVERD]: poll_subsys(): Lock exists for user:USER, bailing out
And the dberrors.log is filled with:

Code: Select all

7: exec(): SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transactionSQL: [268] DELETE
                                FROM
                                polled_extras
                                WHERE
                                     polled_data_id NOT IN
                                    (SELECT polled_data_id FROM polled_data)
Params:  0
8: exec_query(DELETE
                                FROM
                                polled_extras
                                WHERE
                                     polled_data_id NOT IN
                                    (SELECT polled_data_id FROM polled_data), ''): Failed to execute 'DELETE
                                FROM
                                polled_extras
                                WHERE
                                     polled_data_id NOT IN
                                    (SELECT polled_data_id FROM polled_data)'SQL: [268] DELETE
                                FROM
                                polled_extras
                                WHERE
                                     polled_data_id NOT IN
                                    (SELECT polled_data_id FROM polled_data)
Params:  0
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: Nagios Fusion Mariadb keeps dying

Post by ssax »

Please schedule a remote from the link I sent in the ticket so that we can look at this on a remote.

Thank you
hbouma
Posts: 483
Joined: Tue Feb 27, 2018 9:31 am

Re: Nagios Fusion Mariadb keeps dying

Post by hbouma »

Booking complete.

Thank you
ssax
Dreams In Code
Posts: 7682
Joined: Wed Feb 11, 2015 12:54 pm

Re: Nagios Fusion Mariadb keeps dying

Post by ssax »

Great, I see it on my calendar for tomorrow, I will send the connection details out shortly before we start.
Locked