Enormous Daily files in /usr/local/nagios/var/archives (~1G)

[wrobj0]

Yes, I see many hosts that are no longer visible in the UI.

For instance:

Code: Select all

MariaDB [nagios]> select * from nagios_objects where name1 like '%DELETED%' ;
+-----------+-------------+---------------+---------------+---------------------------------------------+-----------+
| object_id | instance_id | objecttype_id | name1         | name2                                       | is_active |
+-----------+-------------+---------------+---------------+---------------------------------------------+-----------+
|     14242 |           1 |             1 | TO-BE-DELETED |                                             |         0 |
|     14243 |           1 |             2 | TO-BE-DELETED | Ping                                        |         0 

Interestingly, that is not in the other table.

Code: Select all

MariaDB [nagios]> select * from nagios_hosts where display_name like '%TO-BE-DELETED%' ;
Empty set (0.00 sec)

The centos3 object shown in my previous reply is in this table, though.

Code: Select all

MariaDB [nagios]> select * from nagios_objects where name1 like '%centos%' ;
+-----------+-------------+---------------+---------+-------------------------------------------+-----------+
| object_id | instance_id | objecttype_id | name1   | name2                                     | is_active |
+-----------+-------------+---------------+---------+-------------------------------------------+-----------+
|     12004 |           1 |             1 | centos3 |                                           |         0 |
|     12005 |           1 |             2 | centos3 | Load                                      |         0 |

[gsmith]

Hi,

That is what should be happening. The deleted hosts should be removed from nagios_hosts and any reference to them
in nagios_object should have his is_active flag set to 0, which yours shows.

I did come across this:

Code: Select all

[1620925162] Error: Unable to create temp file '/usr/local/nagios/var/nagios.tmptma4uF' for writing status data: No space left on device
[1620925171] Error: Unable to create temp file '/usr/local/nagios/var/nagios.tmpOyHcj1' for writing status data: No space left on device
[1620925177] Warning: File '/usr/local/nagios/var/service-perfdata' could not be opened - service performance data will not be written to file!
[1620925177] Warning: File '/usr/local/nagios/var/host-perfdata' could not be opened - host performance data will not be written to file!

and

Code: Select all

Total files in /usr/local/nagios/var/spool/perfdata/
13728

Could you check those locations and clean them up. Then get another System Profile please:
To send us your system profile.
1. Login to the Nagios XI GUI using a web browser.
2. Click the "Admin" > "System Profile" Menu
3. Click the "Download Profile" button
4. Save the profile.zip file and share this in a private message and then reply to this post to bring it up in the queue.

Thanks

[wrobj0]

Those errors are actually from May. 1620925171 == Thu May 13 2021 11:59:31 GMT-0500 (Central Daylight Time)

Is there a way to use standard timezones instead of the epoch? It's really annoying to have logs timestamped this way.

Code: Select all

75G   56G   20G  75% /var

There are even more files in perfdata now, but I don't know what can/should be deleted.

Code: Select all

0 directories, 17396 files

I should mention that I updated to 5.8.4 the other day. I was hoping that there might be an unknown bug causing the issues we're having (we also have problems with emails not sending correctly, but I haven't opened a thread for that yet).

EDIT: The PM with the profile is sitting in my outbox, so I'm not sure if it sent correctly. I tried sending it to gsmith and ssax since you've both worked on the issue.

[gsmith]

Hi,

Please run this command and give me the output:

Code: Select all

ls -l /usr/local/nagios/share/perfdata

Thanks

[wrobj0]

Did you get the PM I sent you?

[gsmith]

Hi,

I was mistaken. I received the original request for the System Profile, but I never got the output of:

Code: Select all

ls -l /usr/local/nagios/share/perfdata

Please send me that. And if you get that out to me today (Monday), run it again Tuesday morning and
send me that output.

Thanks

[wrobj0]

I was out sick a couple of days this week. I just sent the list to you

[gsmith]

Hi there,

I need three things please:

1. a fresh system profile
To send us your system profile.
1. Login to the Nagios XI GUI using a web browser.
2. Click the "Admin" > "System Profile" Menu
3. Click the "Download Profile" button
4. Save the profile.zip file and share this in a private message and then reply to this post to bring it up in the queue.

2. the latest log file from /usr/local/nagios/var/archives - compress it first please. If you can't attach it
to a pm then send me your email address and I'll have dropbox send you a link. I'll also send you a public
key (and instructions) so you can encrypt the file

3. a dump of the nagios database if youare allowed to.
mysqldump -u root -p nagios > nagiosbckup.sql
let me know the size of this file if you can't attach it to a pm

Thanks