Azure VM DISK write performance is seems to be CAPPED at 200MB/s

Question

I have a WINDOWS 2016 server running in Azure.
C:\ NTFS
DATA drives 32TB ReFS 64K blocks

Whether the VM is
F8s_v2
or f32s_v2
or
DS5_v2

Standard HDD 32TB
or
Standard SSD 32TB
or
Performance SSD 32TB

AGGREGATE DISK WRITES do NOT EXCEED 200MB/s
Individual file writes do NOT exceed 15MB/s

See https://social.msdn.microsoft.com/Forums/en-US/575ac43a-20ea-4142-9885-b9bb53a9f202/disk-performance-of-dsseries-vms-premium-storage?forum=windowsazuredata
and
https://social.msdn.microsoft.com/Forums/en-US/b11c5a0e-dced-4b99-8960-d04cbb19c4ef/disk-performance-something-wrong-host-limitscaps-io-time?forum=WAVirtualMachinesforWindows

Answer 1

Diskspd results:

Command Line: amd64\diskspd.exe -c10G -b512K -d90 -Sh f:\diskspd-10G.tst

Input parameters:

    timespan:   1
    -------------
    duration: 90s
    warm up time: 5s
    cool down time: 0s
    random seed: 0
    path: 'f:\diskspd-10G.tst'
            think time: 0ms
            burst size: 0
            software cache disabled
            hardware write cache disabled, writethrough on
            performing read test
            block size: 524288
            using sequential I/O (stride: 524288)
            number of outstanding I/O operations: 2
            thread stride size: 0
            threads per file: 1
            using I/O Completion Ports
            IO priority: normal

System information:

    start time: 2020/06/10 21:47:36 UTC

Results for timespan 1:

---

actual test time: 90.00s
thread count: 1
proc count: 16

CPU | Usage | User | Kernel | Idle

0| 9.15%| 6.42%| 2.73%| 90.85%
1| 12.24%| 10.73%| 1.51%| 87.76%
2| 27.37%| 19.53%| 7.83%| 72.63%
3| 21.81%| 12.00%| 9.81%| 78.19%
4| 6.60%| 2.88%| 3.72%| 93.40%
5| 5.78%| 2.97%| 2.81%| 94.22%
6| 7.66%| 3.39%| 4.27%| 92.34%
7| 5.99%| 2.90%| 3.09%| 94.01%
8| 7.17%| 3.61%| 3.56%| 92.83%
9| 7.73%| 2.52%| 5.21%| 92.27%
10| 6.75%| 3.80%| 2.95%| 93.25%
11| 7.69%| 3.42%| 4.27%| 92.31%
12| 4.88%| 3.18%| 1.70%| 95.12%
13| 6.56%| 3.42%| 3.14%| 93.44%
14| 5.52%| 3.07%| 2.45%| 94.48%

15| 5.61%| 2.17%| 3.44%| 94.39%

avg.| 9.28%| 5.38%| 3.91%| 90.72%

Total IO

thread | bytes | I/Os | MiB/s | I/O per s | file

0 | 1228931072 | 2344 | 13.02 | 26.04 | f:\diskspd-10G.tst (10240MiB)

total: 1228931072 | 2344 | 13.02 | 26.04

Read IO

thread | bytes | I/Os | MiB/s | I/O per s | file

0 | 1228931072 | 2344 | 13.02 | 26.04 | f:\diskspd-10G.tst (10240MiB)

total: 1228931072 | 2344 | 13.02 | 26.04

Write IO

thread | bytes | I/Os | MiB/s | I/O per s | file

0 | 0 | 0 | 0.00 | 0.00 | f:\diskspd-10G.tst (10240MiB)

total: 0 | 0 | 0.00 | 0.00

13MB/s - UGH!

Answer 2

Hi,

Are you running this against a single disk? Or is this a storage space? Also using managed disks?

If I run performance tests I use much more workers and outstanding IO. Try increasing it to 4 workers and 12 outstanding IO. Your block size is also not realistic and is causing much more latency which lowers the performance.

I remember a blog from about a year ago (cannot find it anymore) were you need to up the workers and outstanding IO and not exceed 64k blocks size other wise you were stuck at 200mb... But this is from top of my mind, could be wrong about the numbers.

Try formatting your disk to 4k block size which is the recommended for REFS to align with underlying platform.