Page 2 of 3
Re: Long apply configurations
Posted: Wed Dec 07, 2016 10:14 am
by avandemore
No I don't need to see the full output. The behavior you're seeing is normal as stall is when the file state is used, and the burst is when Nagios starts collecting data again.
What's concerning is that you are reporting Core shows hosts/services as down during an apply config. This is expected in XI, not Core when Core is set to use retention data.
I believe you answered this already, but I would like to confirm this. When doing an Apply Config the host and service status at http://<Your Nagios install>/nagios/ shows states incorrectly?
If that is true, a remote may be needed to see more of this behavior.
Re: Long apply configurations
Posted: Wed Dec 07, 2016 10:48 am
by WillemDH
Avandemore,
Just tested this one more time. yes, I do see incorrect states in Nagios Core when applying configuration.
I'll pm you some screenshots.
Somewhere between the incorrect status'es, I also saw these two errors during my tests:
Grtz
Willem
Re: Long apply configurations
Posted: Wed Dec 07, 2016 11:36 am
by avandemore
Ok, that clears things up. The items in your screen shots appear to either have been in that state for some time or are directly related to the apply config.
So this is the same behavior as the other thread. Same thing applies as there. The Apply Config process is IO bound.
Things like a Linux RAMDisk or
rrdcached can help. Ultimately, the solution needs to be something which greatly increase the IOPS of Nagios and NDO related processes and doesn't bog down under memory pressure. Linux RAMDisk doesn't fulfill this need. If all else is insufficient something like this might help:
https://www.sandisk.com/business/datace ... lash/sx350
http://www.thessdreview.com/our-reviews ... 3-storage/
This is not a recommendation or endorsement of those specific products, they are simply examples of ways to get more IOPS for your system.
Re: Long apply configurations
Posted: Wed Dec 07, 2016 2:54 pm
by WillemDH
Sorry Avandemore but my Nagios XI server is virtual and the the disks of the datastore are already on tier 1 RAID 50 with SSD. We are not seeing any disk io related performance issues, almost no iowait and read / write time.
I'm indeed already using a ramdisk.
Except maybe for rrdcached and offloading mysql I don't see any other options to improve performance. What is the downside of rrdcached?
To be honest if you ask me, the issue of acknowledged and downtimed hosts and services popping up is a Nagios issue related to non optimal programming.
I can understand better io would lessen the time needed to apply configuration, but a longer apply configuration shouldnt mean acknowledged and downtimed hosts and services have to show up.
Re: Long apply configurations
Posted: Wed Dec 07, 2016 4:02 pm
by BanditBBS
WillemDH wrote:Sorry Avandemore but my Nagios XI server is virtual and the the disks of the datastore are already on tier 1 RAID 50 with SSD. We are not seeing any disk io related performance issues, almost no iowait and read / write time.
I'm indeed already using a ramdisk.
Except maybe for rrdcached and offloading mysql I don't see any other options to improve performance. What is the downside of rrdcached?
To be honest if you ask me, the issue of acknowledged and downtimed hosts and services popping up is a Nagios issue related to non optimal programming.
I can understand better io would lessen the time needed to apply configuration, but a longer apply configuration shouldnt mean acknowledged and downtimed hosts and services have to show up.
+1 I see exact same stuff and have same opinion as Willem!
Re: Long apply configurations
Posted: Wed Dec 07, 2016 4:34 pm
by avandemore
WillemDH wrote:Nagios XI server is virtual
That is irrelevant
BanditBBS wrote: We are not seeing any disk io related performance issues
Let's explore this a little, see below.
WillemDH wrote:disks of the datastore are already on tier 1 RAID 50 with SSD
As already mentioned, IOPS are important here, until you can tell me what IOPS the related block device does on random read and writes we're stuck in the same place.
WillemDH wrote:What is the downside of rrdcached?
RRD files would then be processed in intervals, not FIFO as they are without rrdcached. This means there may be more latency in displaying performance data. Also the daemon will be using its own resources eg RAM, CPU, and periodically increasing disk IO.
https://oss.oetiker.ch/rrdtool/doc/rrdcached.en.html
WillemDH wrote:but a longer apply configuration shouldnt mean acknowledged and downtimed hosts and services have to show up
I'll try again to rephrase this. Core doesn't have a longer apply configuration delay. XI does because it gets data from the DB. The DB is overloaded until NDO is caught up, at that point XI would be current again. This is bottlenecked by disk IO.
Right before an Apply Configuration run the following:
After the Apply Configuration has finished kill that process, compress the file and attach or PM it me.
Re: Long apply configurations
Posted: Wed Dec 07, 2016 4:44 pm
by WillemDH
Here you go:
Code: Select all
Linux 2.6.32-642.6.2.el6.x86_64 (nagiosserver) 12/07/2016 _x86_64_ (6 CPU)
12/07/2016 10:38:06 PM
avg-cpu: %user %nice %system %iowait %steal %idle
12.76 0.00 3.45 1.58 0.00 82.22
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.31 1188.00 19.74 464.69 1134.81 13221.68 29.64 0.05 0.11 2.51 0.01 0.27 13.17
dm-0 0.00 0.00 0.06 0.07 0.45 0.60 8.00 0.00 25.76 4.86 41.54 1.09 0.01
dm-1 0.00 0.00 20.00 1652.64 1134.34 13221.08 8.58 2.42 1.40 2.58 1.39 0.08 13.17
12/07/2016 10:38:07 PM
avg-cpu: %user %nice %system %iowait %steal %idle
13.90 0.00 5.19 1.17 0.00 79.73
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 10093.00 0.00 107.00 0.00 81600.00 762.62 1.72 16.07 0.00 16.07 0.91 9.70
dm-1 0.00 0.00 0.00 10200.00 0.00 81600.00 8.00 193.85 19.01 0.00 19.01 0.01 9.70
12/07/2016 10:38:08 PM
avg-cpu: %user %nice %system %iowait %steal %idle
6.53 0.00 3.35 0.00 0.00 90.12
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 1.00 0.00 1.00 1.00 0.20
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.00 1.00 0.00 1.00 0.50 0.20
12/07/2016 10:38:09 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.61 0.00 3.86 0.00 0.00 79.53
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 8.00 0.00 24.00 0.00 256.00 10.67 0.07 2.88 0.00 2.88 0.38 0.90
dm-1 0.00 0.00 0.00 32.00 0.00 256.00 8.00 0.09 2.75 0.00 2.75 0.28 0.90
12/07/2016 10:38:10 PM
avg-cpu: %user %nice %system %iowait %steal %idle
20.53 0.00 3.84 0.00 0.00 75.63
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 1.50 0.00 1.50 1.50 0.30
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.01 1.50 0.00 1.50 0.75 0.30
12/07/2016 10:38:11 PM
avg-cpu: %user %nice %system %iowait %steal %idle
11.17 0.00 4.17 0.17 0.00 84.50
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 4.00 0.00 8.00 0.00 96.00 12.00 0.02 2.25 0.00 2.25 2.00 1.60
dm-1 0.00 0.00 0.00 12.00 0.00 96.00 8.00 0.03 2.50 0.00 2.50 1.33 1.60
12/07/2016 10:38:12 PM
avg-cpu: %user %nice %system %iowait %steal %idle
18.76 0.00 4.36 0.00 0.00 76.88
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 58.00 0.00 13.00 0.00 568.00 43.69 0.01 0.92 0.00 0.92 0.62 0.80
dm-1 0.00 0.00 0.00 71.00 0.00 568.00 8.00 0.12 1.69 0.00 1.69 0.11 0.80
12/07/2016 10:38:13 PM
avg-cpu: %user %nice %system %iowait %steal %idle
4.68 0.00 3.01 0.17 0.00 92.14
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 1.50 0.00 1.50 1.50 0.30
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.01 1.50 0.00 1.50 0.75 0.30
12/07/2016 10:38:14 PM
avg-cpu: %user %nice %system %iowait %steal %idle
8.19 0.00 2.68 0.00 0.00 89.13
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 8.00 0.00 80.00 10.00 0.01 1.25 0.00 1.25 0.62 0.50
dm-1 0.00 0.00 0.00 10.00 0.00 80.00 8.00 0.01 1.20 0.00 1.20 0.50 0.50
12/07/2016 10:38:15 PM
avg-cpu: %user %nice %system %iowait %steal %idle
7.19 0.00 3.34 0.00 0.00 89.46
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 2.00 0.00 2.00 2.00 0.20
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 2.00 0.00 2.00 1.00 0.20
12/07/2016 10:38:16 PM
avg-cpu: %user %nice %system %iowait %steal %idle
14.88 0.00 4.52 0.17 0.00 80.43
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 25.00 0.00 9.00 0.00 272.00 30.22 0.01 0.78 0.00 0.78 0.67 0.60
dm-1 0.00 0.00 0.00 34.00 0.00 272.00 8.00 0.03 0.76 0.00 0.76 0.18 0.60
12/07/2016 10:38:17 PM
avg-cpu: %user %nice %system %iowait %steal %idle
29.08 0.00 13.45 1.51 0.00 55.97
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 11848.00 0.00 120.00 0.00 95744.00 797.87 2.43 20.24 0.00 20.24 1.07 12.80
dm-1 0.00 0.00 0.00 11968.00 0.00 95744.00 8.00 283.62 23.70 0.00 23.70 0.01 12.80
12/07/2016 10:38:18 PM
avg-cpu: %user %nice %system %iowait %steal %idle
15.72 0.00 8.53 0.00 0.00 75.75
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 14.00 0.00 3.00 0.00 136.00 45.33 0.01 2.00 0.00 2.00 2.00 0.60
dm-1 0.00 0.00 0.00 17.00 0.00 136.00 8.00 0.05 2.94 0.00 2.94 0.35 0.60
12/07/2016 10:38:19 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.31 0.00 6.75 0.17 0.00 66.78
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 40.00 0.00 83.00 0.00 984.00 11.86 0.39 4.66 0.00 4.66 0.25 2.10
dm-1 0.00 0.00 0.00 123.00 0.00 984.00 8.00 0.56 4.54 0.00 4.54 0.17 2.10
12/07/2016 10:38:20 PM
avg-cpu: %user %nice %system %iowait %steal %idle
22.82 0.00 3.19 0.00 0.00 73.99
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 1.50 0.00 1.50 1.50 0.30
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.01 1.50 0.00 1.50 0.75 0.30
12/07/2016 10:38:21 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.75 0.00 4.52 0.00 0.00 69.73
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 3.00 0.00 48.00 16.00 0.00 1.33 0.00 1.33 1.33 0.40
dm-1 0.00 0.00 0.00 6.00 0.00 48.00 8.00 0.01 1.33 0.00 1.33 0.67 0.40
12/07/2016 10:38:22 PM
avg-cpu: %user %nice %system %iowait %steal %idle
39.09 0.00 8.39 0.50 0.00 52.01
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5212.00 0.00 67.00 0.00 42232.00 630.33 1.01 15.12 0.00 15.12 1.03 6.90
dm-1 0.00 0.00 0.00 5279.00 0.00 42232.00 8.00 113.17 21.44 0.00 21.44 0.01 6.90
12/07/2016 10:38:23 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.92 0.00 4.18 0.00 0.00 69.90
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 16.00 0.00 6.00 0.00 176.00 29.33 0.01 0.83 0.00 0.83 0.67 0.40
dm-1 0.00 0.00 0.00 22.00 0.00 176.00 8.00 0.02 0.91 0.00 0.91 0.18 0.40
12/07/2016 10:38:24 PM
avg-cpu: %user %nice %system %iowait %steal %idle
36.41 0.00 5.87 3.02 0.00 54.70
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5464.00 0.00 1130.00 0.00 52752.00 46.68 21.90 19.38 0.00 19.38 0.21 23.50
dm-1 0.00 0.00 0.00 6594.00 0.00 52752.00 8.00 136.69 20.73 0.00 20.73 0.04 23.50
12/07/2016 10:38:25 PM
avg-cpu: %user %nice %system %iowait %steal %idle
36.89 0.00 7.01 0.00 0.00 56.09
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5.00 0.00 7.00 0.00 96.00 13.71 0.01 0.71 0.00 0.71 0.71 0.50
dm-1 0.00 0.00 0.00 12.00 0.00 96.00 8.00 0.01 0.75 0.00 0.75 0.42 0.50
12/07/2016 10:38:26 PM
avg-cpu: %user %nice %system %iowait %steal %idle
33.67 0.00 4.52 0.17 0.00 61.64
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 1.00 0.00 1.00 1.00 0.20
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.00 1.00 0.00 1.00 0.50 0.20
12/07/2016 10:38:27 PM
avg-cpu: %user %nice %system %iowait %steal %idle
40.20 0.00 10.55 0.67 0.00 48.58
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7084.00 0.00 74.00 0.00 57264.00 773.84 1.30 17.57 0.00 17.57 0.93 6.90
dm-1 0.00 0.00 0.00 7158.00 0.00 57264.00 8.00 154.27 21.55 0.00 21.55 0.01 6.90
12/07/2016 10:38:28 PM
avg-cpu: %user %nice %system %iowait %steal %idle
39.93 0.00 5.37 0.17 0.00 54.53
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 3.00 0.00 48.00 16.00 0.00 1.33 0.00 1.33 1.33 0.40
dm-1 0.00 0.00 0.00 6.00 0.00 48.00 8.00 0.01 1.33 0.00 1.33 0.67 0.40
12/07/2016 10:38:29 PM
avg-cpu: %user %nice %system %iowait %steal %idle
39.50 0.00 5.55 0.00 0.00 54.96
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 173.00 0.00 8.00 0.00 1448.00 181.00 0.02 2.00 0.00 2.00 0.75 0.60
dm-1 0.00 0.00 0.00 181.00 0.00 1448.00 8.00 0.64 3.55 0.00 3.55 0.03 0.60
12/07/2016 10:38:30 PM
avg-cpu: %user %nice %system %iowait %steal %idle
27.35 0.00 4.70 0.00 0.00 67.95
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 3.00 0.00 32.00 10.67 0.00 1.00 0.00 1.00 0.67 0.20
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.00 1.00 0.00 1.00 0.50 0.20
12/07/2016 10:38:31 PM
avg-cpu: %user %nice %system %iowait %steal %idle
24.66 0.00 5.87 0.00 0.00 69.46
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 31.00 0.00 8.00 0.00 312.00 39.00 0.02 2.25 0.00 2.25 2.12 1.70
dm-1 0.00 0.00 0.00 39.00 0.00 312.00 8.00 0.14 3.72 0.00 3.72 0.44 1.70
12/07/2016 10:38:32 PM
avg-cpu: %user %nice %system %iowait %steal %idle
24.07 0.00 6.06 0.17 0.00 69.70
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 661.00 0.00 26.00 0.00 5496.00 211.38 0.08 3.08 0.00 3.08 1.69 4.40
dm-1 0.00 0.00 0.00 687.00 0.00 5496.00 8.00 3.81 5.55 0.00 5.55 0.06 4.40
12/07/2016 10:38:33 PM
avg-cpu: %user %nice %system %iowait %steal %idle
18.29 0.00 5.37 0.34 0.00 76.01
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 11977.00 0.00 127.00 0.00 96832.00 762.46 2.09 16.43 0.00 16.43 1.53 19.40
dm-1 0.00 0.00 0.00 12104.00 0.00 96832.00 8.00 243.75 20.14 0.00 20.14 0.02 19.40
12/07/2016 10:38:34 PM
avg-cpu: %user %nice %system %iowait %steal %idle
10.35 0.00 3.01 2.84 0.00 83.81
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 154.00 0.00 1252.00 0.00 11184.00 8.93 26.51 20.15 0.00 20.15 0.16 20.40
dm-1 0.00 0.00 0.00 1566.00 0.00 12528.00 8.00 29.36 17.92 0.00 17.92 0.13 20.40
12/07/2016 10:38:35 PM
avg-cpu: %user %nice %system %iowait %steal %idle
39.36 0.00 10.47 23.99 0.00 26.18
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5664.00 59.00 9678.00 472.00 122632.00 12.64 144.11 14.79 22.42 14.75 0.10 100.00
dm-1 0.00 0.00 61.00 15348.00 488.00 122784.00 8.00 322.36 20.89 21.70 20.88 0.06 100.10
12/07/2016 10:38:36 PM
avg-cpu: %user %nice %system %iowait %steal %idle
50.93 0.00 21.25 2.87 0.00 24.96
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 483.00 154.00 1598.00 1232.00 16816.00 10.30 8.65 5.71 1.31 6.13 0.34 59.70
dm-1 0.00 0.00 152.00 1915.00 1216.00 15320.00 8.00 10.71 5.92 1.30 6.29 0.29 59.70
12/07/2016 10:38:37 PM
avg-cpu: %user %nice %system %iowait %steal %idle
42.81 0.00 12.52 0.00 0.00 44.67
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 1.50 0.00 1.50 1.00 0.20
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.01 1.25 0.00 1.25 0.75 0.30
12/07/2016 10:38:38 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.48 0.00 8.38 2.68 0.00 60.47
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 13614.00 0.00 140.00 0.00 110032.00 785.94 3.48 24.89 0.00 24.89 1.52 21.30
dm-1 0.00 0.00 0.00 13754.00 0.00 110032.00 8.00 354.86 25.80 0.00 25.80 0.02 21.30
12/07/2016 10:38:39 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.71 0.00 5.38 0.00 0.00 68.91
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5.00 0.00 3.00 0.00 64.00 21.33 0.01 1.00 0.00 1.00 1.67 0.50
dm-1 0.00 0.00 0.00 9.00 0.00 72.00 8.00 0.01 0.89 0.00 0.89 0.56 0.50
12/07/2016 10:38:40 PM
avg-cpu: %user %nice %system %iowait %steal %idle
21.94 0.00 5.19 1.84 0.00 71.02
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 145.00 0.00 84.00 0.00 1832.00 21.81 0.38 4.50 0.00 4.50 1.83 15.40
dm-1 0.00 0.00 0.00 228.00 0.00 1824.00 8.00 1.29 5.65 0.00 5.65 0.68 15.40
12/07/2016 10:38:41 PM
avg-cpu: %user %nice %system %iowait %steal %idle
35.62 0.00 5.02 0.17 0.00 59.20
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5.00 0.00 9.00 0.00 112.00 12.44 0.01 1.11 0.00 1.11 0.44 0.40
dm-1 0.00 0.00 0.00 14.00 0.00 112.00 8.00 0.02 1.14 0.00 1.14 0.36 0.50
12/07/2016 10:38:42 PM
avg-cpu: %user %nice %system %iowait %steal %idle
29.08 0.00 7.39 0.17 0.00 63.36
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 11.00 0.00 11.00 0.00 176.00 16.00 0.02 2.00 0.00 2.00 1.82 2.00
dm-1 0.00 0.00 0.00 22.00 0.00 176.00 8.00 0.04 1.82 0.00 1.82 0.91 2.00
12/07/2016 10:38:43 PM
avg-cpu: %user %nice %system %iowait %steal %idle
20.57 0.00 4.85 0.00 0.00 74.58
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 3.00 0.00 48.00 16.00 0.00 0.67 0.00 0.67 0.67 0.20
dm-1 0.00 0.00 0.00 6.00 0.00 48.00 8.00 0.00 0.67 0.00 0.67 0.33 0.20
12/07/2016 10:38:44 PM
avg-cpu: %user %nice %system %iowait %steal %idle
22.82 0.00 5.37 0.00 0.00 71.81
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 1.00 0.00 1.00 1.00 0.10
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 1.00 0.00 1.00 0.50 0.10
12/07/2016 10:38:45 PM
avg-cpu: %user %nice %system %iowait %steal %idle
30.03 0.00 5.20 0.17 0.00 64.60
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 128.00 0.00 19.00 0.00 1176.00 61.89 0.04 2.05 0.00 2.05 1.74 3.30
dm-1 0.00 0.00 0.00 147.00 0.00 1176.00 8.00 0.55 3.75 0.00 3.75 0.22 3.30
12/07/2016 10:38:46 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.92 0.00 7.53 2.17 0.00 64.38
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7048.00 0.00 76.00 0.00 56992.00 749.89 1.38 18.09 0.00 18.09 1.99 15.10
dm-1 0.00 0.00 0.00 7124.00 0.00 56992.00 8.00 163.96 23.01 0.00 23.01 0.02 15.10
12/07/2016 10:38:47 PM
avg-cpu: %user %nice %system %iowait %steal %idle
31.83 0.00 4.86 0.00 0.00 63.32
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 3.00 0.00 48.00 16.00 0.00 1.00 0.00 1.00 1.00 0.30
dm-1 0.00 0.00 0.00 6.00 0.00 48.00 8.00 0.01 1.00 0.00 1.00 0.50 0.30
12/07/2016 10:38:48 PM
avg-cpu: %user %nice %system %iowait %steal %idle
36.53 0.00 5.56 0.17 0.00 57.74
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 7.00 0.00 104.00 14.86 0.02 3.29 0.00 3.29 3.14 2.20
dm-1 0.00 0.00 0.00 13.00 0.00 104.00 8.00 0.05 3.69 0.00 3.69 1.69 2.20
12/07/2016 10:38:49 PM
avg-cpu: %user %nice %system %iowait %steal %idle
64.82 0.00 7.20 0.17 0.00 27.81
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 10.00 0.00 18.00 0.00 224.00 12.44 0.02 1.33 0.00 1.33 1.06 1.90
dm-1 0.00 0.00 0.00 28.00 0.00 224.00 8.00 0.04 1.36 0.00 1.36 0.71 2.00
12/07/2016 10:38:50 PM
avg-cpu: %user %nice %system %iowait %steal %idle
32.89 0.00 6.18 0.17 0.00 60.77
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 10.00 1.00 8.00 8.00 144.00 16.89 0.01 1.00 0.00 1.12 1.00 0.90
dm-1 0.00 0.00 1.00 18.00 8.00 144.00 8.00 0.02 1.16 0.00 1.22 0.47 0.90
12/07/2016 10:38:51 PM
avg-cpu: %user %nice %system %iowait %steal %idle
21.25 0.00 6.41 13.32 0.00 59.02
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1346.00 0.00 8378.00 0.00 77576.00 9.26 136.78 16.17 0.00 16.17 0.11 95.90
dm-1 0.00 0.00 0.00 9874.00 0.00 78992.00 8.00 159.01 15.95 0.00 15.95 0.10 95.90
12/07/2016 10:38:52 PM
avg-cpu: %user %nice %system %iowait %steal %idle
27.59 0.00 5.69 1.84 0.00 64.88
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 108.00 0.00 1116.00 0.00 10008.00 8.97 18.72 17.92 0.00 17.92 0.13 14.50
dm-1 0.00 0.00 0.00 1074.00 0.00 8592.00 8.00 20.77 20.75 0.00 20.75 0.14 14.50
12/07/2016 10:38:53 PM
avg-cpu: %user %nice %system %iowait %steal %idle
18.52 0.00 3.20 0.00 0.00 78.28
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 1.00 0.00 1.00 1.00 0.10
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 1.00 0.00 1.00 0.50 0.10
12/07/2016 10:38:54 PM
avg-cpu: %user %nice %system %iowait %steal %idle
31.49 0.00 5.03 0.00 0.00 63.48
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 8.00 0.00 112.00 14.00 0.01 1.00 0.00 1.00 0.88 0.70
dm-1 0.00 0.00 0.00 14.00 0.00 112.00 8.00 0.01 0.93 0.00 0.93 0.50 0.70
12/07/2016 10:38:55 PM
avg-cpu: %user %nice %system %iowait %steal %idle
27.05 0.00 6.18 0.17 0.00 66.61
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 7.00 0.00 80.00 11.43 0.01 1.14 0.00 1.14 0.86 0.60
dm-1 0.00 0.00 0.00 10.00 0.00 80.00 8.00 0.01 1.20 0.00 1.20 0.60 0.60
12/07/2016 10:38:56 PM
avg-cpu: %user %nice %system %iowait %steal %idle
22.18 0.00 4.87 0.50 0.00 72.44
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7217.00 0.00 72.00 0.00 58312.00 809.89 1.25 17.33 0.00 17.33 0.75 5.40
dm-1 0.00 0.00 0.00 7289.00 0.00 58312.00 8.00 145.47 19.96 0.00 19.96 0.01 5.40
12/07/2016 10:38:57 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.81 0.00 5.53 0.17 0.00 65.49
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2030.00 0.00 45.00 0.00 16600.00 368.89 0.42 9.29 0.00 9.29 1.67 7.50
dm-1 0.00 0.00 0.00 2075.00 0.00 16600.00 8.00 21.99 10.60 0.00 10.60 0.04 7.50
12/07/2016 10:38:58 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.46 0.00 5.40 0.00 0.00 69.14
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 6.00 0.00 96.00 16.00 0.01 1.17 0.00 1.17 1.17 0.70
dm-1 0.00 0.00 0.00 12.00 0.00 96.00 8.00 0.01 1.25 0.00 1.25 0.67 0.80
12/07/2016 10:38:59 PM
avg-cpu: %user %nice %system %iowait %steal %idle
38.80 0.00 8.70 0.00 0.00 52.51
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 4.00 0.00 6.00 0.00 80.00 13.33 0.01 0.83 0.00 0.83 0.67 0.40
dm-1 0.00 0.00 0.00 10.00 0.00 80.00 8.00 0.01 0.80 0.00 0.80 0.40 0.40
12/07/2016 10:39:00 PM
avg-cpu: %user %nice %system %iowait %steal %idle
17.20 0.00 6.91 0.00 0.00 75.89
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 2.00 0.00 24.00 12.00 0.01 4.00 0.00 4.00 4.00 0.80
dm-1 0.00 0.00 0.00 3.00 0.00 24.00 8.00 0.01 3.00 0.00 3.00 2.67 0.80
12/07/2016 10:39:01 PM
avg-cpu: %user %nice %system %iowait %steal %idle
39.16 0.00 12.27 0.17 0.00 48.40
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1106.00 0.00 103.00 0.00 9672.00 93.90 0.69 6.64 0.00 6.64 0.62 6.40
dm-1 0.00 0.00 0.00 1210.00 0.00 9680.00 8.00 11.69 9.66 0.00 9.66 0.05 6.60
12/07/2016 10:39:02 PM
avg-cpu: %user %nice %system %iowait %steal %idle
21.02 0.00 9.83 2.37 0.00 66.78
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5413.00 0.00 1181.00 0.00 52752.00 44.67 21.45 18.16 0.00 18.16 0.15 17.50
dm-1 0.00 0.00 0.00 6593.00 0.00 52744.00 8.00 130.62 19.81 0.00 19.81 0.03 17.50
12/07/2016 10:39:03 PM
avg-cpu: %user %nice %system %iowait %steal %idle
31.66 0.00 8.04 0.17 0.00 60.13
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 8.00 0.00 10.00 0.00 144.00 14.40 0.02 1.90 0.00 1.90 1.80 1.80
dm-1 0.00 0.00 0.00 18.00 0.00 144.00 8.00 0.04 2.06 0.00 2.06 1.00 1.80
12/07/2016 10:39:04 PM
avg-cpu: %user %nice %system %iowait %steal %idle
23.31 0.00 5.24 0.00 0.00 71.45
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 1.00 2.00 8.00 32.00 13.33 0.01 2.33 0.00 3.50 2.33 0.70
dm-1 0.00 0.00 1.00 4.00 8.00 32.00 8.00 0.01 2.80 0.00 3.50 1.40 0.70
12/07/2016 10:39:05 PM
avg-cpu: %user %nice %system %iowait %steal %idle
20.07 0.00 5.56 0.00 0.00 74.37
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 31.00 0.00 5.00 0.00 288.00 57.60 0.01 2.40 0.00 2.40 2.20 1.10
dm-1 0.00 0.00 0.00 36.00 0.00 288.00 8.00 0.15 4.11 0.00 4.11 0.31 1.10
12/07/2016 10:39:06 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.77 0.00 7.91 2.36 0.00 62.96
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7992.00 51.00 463.00 408.00 67632.00 132.37 4.44 8.63 2.65 9.29 0.71 36.70
dm-1 0.00 0.00 51.00 8457.00 408.00 67656.00 8.00 265.12 31.16 2.63 31.33 0.04 36.70
12/07/2016 10:39:07 PM
avg-cpu: %user %nice %system %iowait %steal %idle
37.48 0.00 8.07 0.84 0.00 53.61
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1587.00 71.00 773.00 568.00 18888.00 23.05 5.42 6.43 0.61 6.97 0.58 49.00
dm-1 0.00 0.00 71.00 2358.00 568.00 18864.00 8.00 31.70 13.05 0.62 13.43 0.20 49.00
12/07/2016 10:39:08 PM
avg-cpu: %user %nice %system %iowait %steal %idle
24.28 0.00 5.73 0.00 0.00 69.98
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 2.00 0.00 2.00 2.00 0.20
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 2.00 0.00 2.00 1.00 0.20
12/07/2016 10:39:09 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.04 0.00 6.05 0.00 0.00 68.91
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 4.00 0.00 56.00 14.00 0.01 1.75 0.00 1.75 1.75 0.70
dm-1 0.00 0.00 0.00 7.00 0.00 56.00 8.00 0.01 1.43 0.00 1.43 1.00 0.70
12/07/2016 10:39:10 PM
avg-cpu: %user %nice %system %iowait %steal %idle
19.39 0.00 4.72 0.00 0.00 75.89
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 5.00 0.00 88.00 17.60 0.01 1.60 0.00 1.60 1.60 0.80
dm-1 0.00 0.00 0.00 11.00 0.00 88.00 8.00 0.01 1.27 0.00 1.27 0.73 0.80
12/07/2016 10:39:11 PM
avg-cpu: %user %nice %system %iowait %steal %idle
30.25 0.00 6.89 0.50 0.00 62.35
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1999.00 0.00 47.00 0.00 16368.00 348.26 0.40 8.47 0.00 8.47 0.87 4.10
dm-1 0.00 0.00 0.00 2046.00 0.00 16368.00 8.00 33.57 16.41 0.00 16.41 0.02 4.10
12/07/2016 10:39:12 PM
avg-cpu: %user %nice %system %iowait %steal %idle
15.35 0.00 4.22 0.17 0.00 80.27
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 304.00 0.00 12.00 0.00 2528.00 210.67 0.03 2.67 0.00 2.67 0.58 0.70
dm-1 0.00 0.00 0.00 316.00 0.00 2528.00 8.00 1.31 4.15 0.00 4.15 0.02 0.70
12/07/2016 10:39:13 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.94 0.00 4.55 0.00 0.00 68.52
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7.00 0.00 11.00 0.00 144.00 13.09 0.01 1.18 0.00 1.18 1.00 1.10
dm-1 0.00 0.00 0.00 18.00 0.00 144.00 8.00 0.02 1.11 0.00 1.11 0.61 1.10
12/07/2016 10:39:14 PM
avg-cpu: %user %nice %system %iowait %steal %idle
18.55 0.00 4.72 0.00 0.00 76.73
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 2.00 0.00 2.00 2.00 0.20
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 2.00 0.00 2.00 1.00 0.20
12/07/2016 10:39:15 PM
avg-cpu: %user %nice %system %iowait %steal %idle
32.83 0.00 5.39 0.17 0.00 61.62
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 8.00 0.00 112.00 14.00 0.01 1.38 0.00 1.38 1.25 1.00
dm-1 0.00 0.00 0.00 14.00 0.00 112.00 8.00 0.02 1.36 0.00 1.36 0.71 1.00
12/07/2016 10:39:16 PM
avg-cpu: %user %nice %system %iowait %steal %idle
23.61 0.00 6.91 0.51 0.00 68.97
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7063.00 0.00 84.00 0.00 57176.00 680.67 1.51 17.98 0.00 17.98 0.65 5.50
dm-1 0.00 0.00 0.00 7147.00 0.00 57176.00 8.00 139.65 19.54 0.00 19.54 0.01 5.50
12/07/2016 10:39:17 PM
avg-cpu: %user %nice %system %iowait %steal %idle
20.10 0.00 4.22 0.00 0.00 75.68
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 8.00 0.00 88.00 11.00 0.01 1.88 0.00 1.88 1.00 0.80
dm-1 0.00 0.00 0.00 11.00 0.00 88.00 8.00 0.02 1.64 0.00 1.64 0.73 0.80
12/07/2016 10:39:18 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.84 0.00 5.70 0.17 0.00 68.29
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 6.00 0.00 64.00 10.67 0.01 1.33 0.00 1.33 1.00 0.60
dm-1 0.00 0.00 0.00 8.00 0.00 64.00 8.00 0.01 1.38 0.00 1.38 0.75 0.60
12/07/2016 10:39:19 PM
avg-cpu: %user %nice %system %iowait %steal %idle
18.29 0.00 4.87 0.00 0.00 76.85
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 34.00 0.00 10.00 0.00 352.00 35.20 0.02 1.90 0.00 1.90 1.50 1.50
dm-1 0.00 0.00 0.00 44.00 0.00 352.00 8.00 0.15 3.32 0.00 3.32 0.34 1.50
12/07/2016 10:39:20 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.69 0.00 6.38 0.00 0.00 64.93
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 7.00 0.00 80.00 11.43 0.01 1.14 0.00 1.14 0.86 0.60
dm-1 0.00 0.00 0.00 10.00 0.00 80.00 8.00 0.01 1.00 0.00 1.00 0.60 0.60
12/07/2016 10:39:21 PM
avg-cpu: %user %nice %system %iowait %steal %idle
17.59 0.00 5.36 0.17 0.00 76.88
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 42.00 0.00 8.00 0.00 400.00 50.00 0.01 1.38 0.00 1.38 0.62 0.50
dm-1 0.00 0.00 0.00 50.00 0.00 400.00 8.00 0.09 1.86 0.00 1.86 0.10 0.50
12/07/2016 10:39:22 PM
avg-cpu: %user %nice %system %iowait %steal %idle
42.18 0.00 10.08 2.86 0.00 44.87
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 354.00 122.00 984.00 976.00 10696.00 10.55 2.90 2.62 1.36 2.77 0.51 56.20
dm-1 0.00 0.00 122.00 1341.00 976.00 10728.00 8.00 3.92 2.67 1.37 2.78 0.38 56.10
12/07/2016 10:39:23 PM
avg-cpu: %user %nice %system %iowait %steal %idle
29.82 0.00 8.54 0.34 0.00 61.31
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 127.00 48.00 355.00 384.00 3864.00 10.54 0.89 2.20 0.62 2.42 0.52 20.90
dm-1 0.00 0.00 48.00 479.00 384.00 3832.00 8.00 1.26 2.39 0.62 2.56 0.40 21.00
12/07/2016 10:39:24 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.91 0.00 4.54 0.17 0.00 66.39
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 8.00 0.00 8.00 0.00 128.00 16.00 0.01 1.25 0.00 1.25 1.25 1.00
dm-1 0.00 0.00 0.00 16.00 0.00 128.00 8.00 0.02 1.25 0.00 1.25 0.62 1.00
12/07/2016 10:39:25 PM
avg-cpu: %user %nice %system %iowait %steal %idle
38.63 0.00 8.70 0.00 0.00 52.68
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 12.00 0.00 144.00 12.00 0.01 1.25 0.00 1.25 1.00 1.20
dm-1 0.00 0.00 0.00 18.00 0.00 144.00 8.00 0.02 1.28 0.00 1.28 0.67 1.20
12/07/2016 10:39:26 PM
avg-cpu: %user %nice %system %iowait %steal %idle
23.95 0.00 5.90 0.51 0.00 69.65
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 10819.00 0.00 112.00 0.00 87448.00 780.79 1.77 15.85 0.00 15.85 0.62 7.00
dm-1 0.00 0.00 0.00 10931.00 0.00 87448.00 8.00 200.13 18.31 0.00 18.31 0.01 7.00
12/07/2016 10:39:27 PM
avg-cpu: %user %nice %system %iowait %steal %idle
29.34 0.00 6.24 0.00 0.00 64.42
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 137.00 0.00 15.00 0.00 1216.00 81.07 0.03 1.93 0.00 1.93 1.27 1.90
dm-1 0.00 0.00 0.00 152.00 0.00 1216.00 8.00 0.45 2.98 0.00 2.98 0.12 1.90
12/07/2016 10:39:28 PM
avg-cpu: %user %nice %system %iowait %steal %idle
25.04 0.00 4.71 0.17 0.00 70.08
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 5.00 0.00 5.00 0.00 80.00 16.00 0.01 1.60 0.00 1.60 1.60 0.80
dm-1 0.00 0.00 0.00 10.00 0.00 80.00 8.00 0.02 1.60 0.00 1.60 0.80 0.80
12/07/2016 10:39:29 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.76 0.00 8.80 0.00 0.00 62.44
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 5.00 0.00 64.00 12.80 0.01 1.40 0.00 1.40 1.20 0.60
dm-1 0.00 0.00 0.00 8.00 0.00 64.00 8.00 0.01 1.38 0.00 1.38 0.75 0.60
12/07/2016 10:39:30 PM
avg-cpu: %user %nice %system %iowait %steal %idle
14.38 0.00 8.05 0.00 0.00 77.57
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3.00 0.00 3.00 0.00 48.00 16.00 0.00 1.33 0.00 1.33 1.33 0.40
dm-1 0.00 0.00 0.00 6.00 0.00 48.00 8.00 0.01 1.33 0.00 1.33 0.67 0.40
12/07/2016 10:39:31 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.77 0.00 6.23 0.17 0.00 66.84
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 105.00 0.00 33.00 0.00 1104.00 33.45 0.09 2.64 0.00 2.64 0.42 1.40
dm-1 0.00 0.00 0.00 138.00 0.00 1104.00 8.00 0.53 3.84 0.00 3.84 0.11 1.50
12/07/2016 10:39:32 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.16 0.00 3.37 0.00 0.00 80.47
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 130.00 0.00 13.00 0.00 1144.00 88.00 0.02 1.62 0.00 1.62 0.46 0.60
dm-1 0.00 0.00 0.00 143.00 0.00 1144.00 8.00 0.41 2.88 0.00 2.88 0.04 0.60
12/07/2016 10:39:33 PM
avg-cpu: %user %nice %system %iowait %steal %idle
30.55 0.00 5.34 0.17 0.00 63.94
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 6.00 0.00 8.00 0.00 112.00 14.00 0.01 1.12 0.00 1.12 1.00 0.80
dm-1 0.00 0.00 0.00 14.00 0.00 112.00 8.00 0.02 1.14 0.00 1.14 0.57 0.80
12/07/2016 10:39:34 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.26 0.00 4.35 0.00 0.00 67.39
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 33.00 0.00 11.00 0.00 352.00 32.00 0.02 1.73 0.00 1.73 1.45 1.60
dm-1 0.00 0.00 0.00 44.00 0.00 352.00 8.00 0.14 3.25 0.00 3.25 0.36 1.60
12/07/2016 10:39:35 PM
avg-cpu: %user %nice %system %iowait %steal %idle
17.82 0.00 3.19 0.00 0.00 78.99
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 2.00 0.00 2.00 2.00 0.20
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 2.00 0.00 2.00 1.00 0.20
12/07/2016 10:39:36 PM
avg-cpu: %user %nice %system %iowait %steal %idle
28.57 0.00 6.05 0.50 0.00 64.87
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7085.00 0.00 76.00 0.00 57288.00 753.79 1.37 18.08 0.00 18.08 0.74 5.60
dm-1 0.00 0.00 0.00 7161.00 0.00 57288.00 8.00 160.90 22.47 0.00 22.47 0.01 5.60
12/07/2016 10:39:37 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.02 0.00 3.88 9.61 0.00 70.49
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 3198.00 0.00 6662.00 0.00 76888.00 11.54 102.87 15.12 0.00 15.12 0.11 72.00
dm-1 0.00 0.00 0.00 9993.00 0.00 79944.00 8.00 151.49 14.75 0.00 14.75 0.07 72.00
12/07/2016 10:39:38 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.84 0.00 2.53 0.51 0.00 80.13
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 699.00 0.00 564.00 0.00 12096.00 21.45 7.06 16.35 0.00 16.35 0.10 5.60
dm-1 0.00 0.00 0.00 1130.00 0.00 9040.00 8.00 17.17 18.85 0.00 18.85 0.05 5.60
12/07/2016 10:39:39 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.01 0.00 4.03 0.17 0.00 69.80
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 4.00 0.00 4.00 0.00 64.00 16.00 0.01 1.25 0.00 1.25 1.25 0.50
dm-1 0.00 0.00 0.00 8.00 0.00 64.00 8.00 0.01 1.25 0.00 1.25 0.62 0.50
12/07/2016 10:39:40 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.55 0.00 4.37 0.00 0.00 69.08
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 4.00 0.00 40.00 10.00 0.01 1.75 0.00 1.75 1.50 0.60
dm-1 0.00 0.00 0.00 5.00 0.00 40.00 8.00 0.01 1.60 0.00 1.60 1.20 0.60
12/07/2016 10:39:41 PM
avg-cpu: %user %nice %system %iowait %steal %idle
21.64 0.00 3.52 0.00 0.00 74.83
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 68.00 0.00 35.00 0.00 824.00 23.54 0.11 3.14 0.00 3.14 0.23 0.80
dm-1 0.00 0.00 0.00 103.00 0.00 824.00 8.00 0.31 3.04 0.00 3.04 0.08 0.80
12/07/2016 10:39:42 PM
avg-cpu: %user %nice %system %iowait %steal %idle
31.88 0.00 5.03 0.17 0.00 62.92
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 4.00 0.00 11.00 0.00 112.00 10.18 0.02 1.55 0.00 1.55 1.27 1.40
dm-1 0.00 0.00 0.00 16.00 0.00 128.00 8.00 0.02 1.31 0.00 1.31 0.88 1.40
12/07/2016 10:39:43 PM
avg-cpu: %user %nice %system %iowait %steal %idle
21.20 0.00 5.51 0.17 0.00 73.12
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 9.00 0.00 11.00 0.00 168.00 15.27 0.02 1.73 0.00 1.73 1.45 1.60
dm-1 0.00 0.00 0.00 19.00 0.00 152.00 8.00 0.04 2.05 0.00 2.05 0.84 1.60
12/07/2016 10:39:44 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.97 0.00 2.69 0.00 0.00 80.34
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 2.00 0.00 2.00 2.00 0.20
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 2.00 0.00 2.00 1.00 0.20
12/07/2016 10:39:45 PM
avg-cpu: %user %nice %system %iowait %steal %idle
17.25 0.00 3.18 0.00 0.00 79.56
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 108.00 0.00 18.00 0.00 1008.00 56.00 0.06 3.11 0.00 3.11 2.83 5.10
dm-1 0.00 0.00 0.00 126.00 0.00 1008.00 8.00 0.69 5.48 0.00 5.48 0.40 5.10
12/07/2016 10:39:46 PM
avg-cpu: %user %nice %system %iowait %steal %idle
31.65 0.00 4.21 0.51 0.00 63.64
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 7069.00 0.00 83.00 0.00 57216.00 689.35 1.28 15.43 0.00 15.43 0.70 5.80
dm-1 0.00 0.00 0.00 7152.00 0.00 57216.00 8.00 148.54 20.77 0.00 20.77 0.01 5.80
12/07/2016 10:39:47 PM
avg-cpu: %user %nice %system %iowait %steal %idle
20.90 0.00 5.52 0.00 0.00 73.58
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 1.00 0.00 1.00 1.00 0.10
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 1.00 0.00 1.00 0.50 0.10
12/07/2016 10:39:48 PM
avg-cpu: %user %nice %system %iowait %steal %idle
18.43 0.00 4.19 0.00 0.00 77.39
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 4.00 0.00 5.00 0.00 72.00 14.40 0.01 1.20 0.00 1.20 1.00 0.50
dm-1 0.00 0.00 0.00 9.00 0.00 72.00 8.00 0.01 1.00 0.00 1.00 0.56 0.50
12/07/2016 10:39:49 PM
avg-cpu: %user %nice %system %iowait %steal %idle
17.28 0.00 3.02 0.17 0.00 79.53
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 0.50 0.00 0.50 0.50 0.10
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.00 0.50 0.00 0.50 0.25 0.10
12/07/2016 10:39:50 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.28 0.00 2.85 0.00 0.00 80.87
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 1.00 0.00 1.00 1.00 0.10
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 1.00 0.00 1.00 0.50 0.10
12/07/2016 10:39:51 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.67 0.00 3.54 0.00 0.00 79.80
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 35.00 1.00 6.00 8.00 328.00 48.00 0.01 1.14 1.00 1.17 0.86 0.60
dm-1 0.00 0.00 1.00 41.00 8.00 328.00 8.00 0.08 1.81 1.00 1.83 0.14 0.60
12/07/2016 10:39:52 PM
avg-cpu: %user %nice %system %iowait %steal %idle
16.28 0.00 3.36 0.00 0.00 80.37
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 2.00 0.00 2.00 0.00 32.00 16.00 0.00 0.50 0.00 0.50 0.50 0.10
dm-1 0.00 0.00 0.00 4.00 0.00 32.00 8.00 0.00 0.50 0.00 0.50 0.25 0.10
12/07/2016 10:39:53 PM
avg-cpu: %user %nice %system %iowait %steal %idle
21.75 0.00 4.55 26.31 0.00 47.39
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 571.00 31.00 3983.00 248.00 36352.00 9.12 138.44 33.45 27.84 33.49 0.24 95.80
dm-1 0.00 0.00 32.00 4690.00 256.00 37520.00 8.00 155.40 31.94 26.97 31.97 0.20 95.80
12/07/2016 10:39:54 PM
avg-cpu: %user %nice %system %iowait %steal %idle
38.38 0.00 8.59 23.40 0.00 29.63
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 1.00 767.00 52.00 5419.00 424.00 49440.00 9.11 144.94 26.98 21.69 27.03 0.18 100.00
dm-0 0.00 0.00 3.00 0.00 24.00 0.00 8.00 0.09 30.67 30.67 0.00 20.00 6.00
dm-1 0.00 0.00 51.00 6221.00 408.00 49768.00 8.00 165.91 26.85 20.94 26.90 0.16 100.00
12/07/2016 10:39:55 PM
avg-cpu: %user %nice %system %iowait %steal %idle
26.05 0.00 5.88 1.34 0.00 66.72
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 164.00 68.00 758.00 544.00 7528.00 9.77 5.92 9.03 1.81 9.68 0.53 44.00
dm-1 0.00 0.00 66.00 754.00 528.00 6032.00 8.00 7.10 10.77 1.86 11.55 0.54 44.00
12/07/2016 10:39:56 PM
avg-cpu: %user %nice %system %iowait %steal %idle
20.17 0.00 4.54 3.03 0.00 72.27
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 11859.00 0.00 434.00 0.00 98344.00 226.60 22.89 52.74 0.00 52.74 0.48 20.80
dm-1 0.00 0.00 0.00 12293.00 0.00 98344.00 8.00 1069.71 87.02 0.00 87.02 0.02 20.80
12/07/2016 10:39:57 PM
avg-cpu: %user %nice %system %iowait %steal %idle
17.45 0.00 2.52 0.00 0.00 80.03
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 1.00 0.00 1.00 0.00 16.00 16.00 0.00 1.00 0.00 1.00 1.00 0.10
dm-1 0.00 0.00 0.00 2.00 0.00 16.00 8.00 0.00 1.00 0.00 1.00 0.50 0.10
I'll try again to rephrase this. Core doesn't have a longer apply configuration delay. XI does because it gets data from the DB. The DB is overloaded until NDO is caught up, at that point XI would be current again. This is bottlenecked by disk IO.
Sorry but I just can't believe there is 'no way' to not show hosts / services that are in downtime regardless of the load / apply time..
Willem
Re: Long apply configurations
Posted: Wed Dec 07, 2016 4:58 pm
by avandemore
Re: Long apply configurations
Posted: Wed Dec 07, 2016 5:03 pm
by WillemDH
Code: Select all
dmsetup ls
VolGroup00-LogVol01 (253:0)
VolGroup00-LogVol00 (253:1)
Re: Long apply configurations
Posted: Wed Dec 07, 2016 5:28 pm
by avandemore
That's naming isn't helpful in tracking down what is what, try this:
Code: Select all
# ls -la /dev/mapper/*
# cat /proc/mounts
I'm seeing peak IOPS of approximately 16000/ps. This is far less than the hardware I referenced earlier. And if it's anywhere close to the theoretical maximum for the block devices you have all the evidence you need to show a disk IO limitation.