Page 1 of 1
Most recent state PARTIAL
Posted: Mon Mar 02, 2020 7:08 pm
by kmartin
From time to time under snapshots and maintenance is see Most Recent State = PARTIAL. Almost all of them are SUCCESS, however am wondering what PARTIAL means and what are ramifications.
Re: Most recent state PARTIAL
Posted: Tue Mar 03, 2020 5:35 pm
by jdunitz
Hello, Ken!
The partial state means that one or more indices added to the snapshot didn't have all primary shards available.
You can still restore from a partial snapshot, but you might need to use older snapshots to restore any missing indices.
This is something that just happens from time to time. If you're getting a lot of partial snapshots, that might be worth looking into.
What percentage of partials are you getting?
If it's a lot, we might want to have a look at your system profile.
--Jeffrey
Re: Most recent state PARTIAL
Posted: Tue Mar 10, 2020 12:10 pm
by kmartin
Hey Jeffrey,
8 out of 41 or almost 20%. Is that a lot?
Ken
Re: Most recent state PARTIAL
Posted: Wed Mar 11, 2020 10:05 am
by jdunitz
That's a bit higher than we'd expect. It's worth looking into, so would you mind sending a profile, and also a screenshot of your snapshots status? That will help get us started and we can go from there.
Thanks!
Re: Most recent state PARTIAL
Posted: Wed Mar 11, 2020 4:25 pm
by kmartin
Here's the system profile and screencap of snapshots.
Re: Most recent state PARTIAL
Posted: Fri Mar 13, 2020 5:17 pm
by cdienger
Thanks. I've downloaded the data and made it available for the team. Can you also click the Restore button for the 2020.03.08 logstash and provide a screenshot of that please?
Re: Most recent state PARTIAL
Posted: Mon Mar 23, 2020 2:48 pm
by kmartin
Restore not successful. NagiosLS state was green when restore attempted. It complains about the index already being open, but it is not.
Re: Most recent state PARTIAL
Posted: Tue Mar 24, 2020 11:08 am
by cdienger
It looks like around the time of that particular failure the disk was pretty full:
Code: Select all
low disk watermark [85%] exceeded on
high disk watermark [90%] exceeded on
I would recommend increasing the disk size to help avoid these issues and also using the NCPA client to monitor disk space:
https://support.nagios.com/kb/article/n ... i-857.html