Page 1 of 1

NLS runs slowly out of disk space - how to replace?

Posted: Thu Mar 03, 2022 2:26 am
by mafamgm
Hello,

I have an NLS that is slowly running out of space. Currently there are about 300 GB free.

I have already added a 2nd server, which has significantly more disk space available, as a 2nd cluster node.

I have noticed that no virtual IP is created.

Now I have 2 questions.

What is the cluster functionality for if no virtual IP is created for the cluster?
All devices that I have configured, send their log data now continue only to the first node.

How can I replace the first server that runs out of disk space with a newer one with more disk space without loosing any data?

Thanks a lot

Re: NLS runs slowly out of disk space - how to replace?

Posted: Thu Mar 03, 2022 3:59 pm
by gsmith
Hi

It sounds like there is a configuration problem. The two nodes should be using approximately the same
storage space. The functionality is such that all the data is captured in 5 pieces, aka shards. And in a
cluster shards 1,3,5 live on the master, and shards 2 and 4 live on the secondary. In addition to that,
shards 1,3,5 should be replicated to the secondary while shards 2 and 4 should be replicated to the master.

On the master node please run:

Code: Select all

tar cvzf /tmp/logstshmstr.tar.gz /usr/local/nagioslogserver/var/*
and on the secondary node please run:

Code: Select all

tar cvzf /tmp/logstshsec.tar.gz /usr/local/nagioslogserver/var/*
then please send the resulting files to me:
/tmp/logstshmstr.tar.gz
/tmp/logstshsec.tar.gz

Thanks

Re: NLS runs slowly out of disk space - how to replace?

Posted: Fri Mar 04, 2022 2:29 am
by mafamgm
Hi,
According to the "Total Elasticsearch Disk Usage" display, the master consumes 2.397 GiB of disk space and the slave consumes 585 GiB

The master generally has less disk space physically available than the new slave node

I send you the data.

Thanks

Re: NLS runs slowly out of disk space - how to replace?

Posted: Mon Mar 07, 2022 1:42 pm
by gsmith
Hi,

1. Do you have a repository setup?
2. If so do both nodes have access to it?
3. On each node go to Admin, Snapshots & Maintenance, and please take a screenshot of the
Maintenance and Repository Settings tab.

Thank you

Re: NLS runs slowly out of disk space - how to replace?

Posted: Tue Mar 08, 2022 3:25 am
by mafamgm
Hi,

i've added the screenshots.

They have a snapshot repos, but it's not a shared one.
Every node has it's own local repos configured.

Thanks

Re: NLS runs slowly out of disk space - how to replace?

Posted: Tue Mar 08, 2022 3:01 pm
by gsmith
Hi,

It seems like your two nodes aren't really clustered. There is not supposed to be a virtual IP created, the master node
will delegate compute and storage to a slave node when things are configured properly.

Please open a ticket at:
https://info.nagios.com/answer-hub-access-new-users

This is the new location of our ticketing system. If you need help logging in/registering please let me know.

Thank you!

Re: NLS runs slowly out of disk space - how to replace?

Posted: Wed Mar 09, 2022 4:03 am
by mafamgm
Ok, will create a Ticket.

Thanks.