Freigeben über

SAPS ratings on Azure VMs – where to look and where you can get confused

  • Artikel

Moving SAP software to Azure, you need to decide which Azure VM types you want to use. One of the major criteria for selecting a certain VM is the throughput requirement of the SAP workload. SAP decided decades ago, to characterize the throughput in SAPS (SAP Application Performance Standard). To measure the SAPS a server or an Azure VM can deliver, server manufacturers or cloud service providers need to run the SAP SD Standard Application Benchmark. One hundred SAPS are defined as 2,000 fully business processed order line items per hour (https://www.sap.com/about/benchmark/measuring.html ).

As we certify Azure VMs for SAP NetWeaver and SAP HANA, one of the main pieces of information we need to deliver is the throughput measured in SAPS of the VM to be certified. SAP again, uses that data to document it for Azure in SAP note #1928533. This note is the official source to find the SAPS for SAP NetWeaver certified VMs. This note lists the number of vCPUs, memory and SAPS for each of the Azure VM types that are certified to run the SAP NetWeaver workload.

So where is the confusion then?

First area of confusion is created by the way publicly released SAP SD Standard Application benchmarks are documented by SAP in case of hyperscale clouds like Azure. When you follow this link, you are getting a list of SAP SD benchmarks which were released on Azure VMs. Note, we are not forced to release benchmarks conducted for certification of Azure VMs. As a result, you won’t find benchmarks released in this location for every certified Azure VM type. As you click onto a single record of a benchmark, data like this will be displayed:

image

And this is where the problem starts. Nowhere in this displayed data would you find the vCPU and memory definition of the VM that got tested. Instead you find the technical data of the host that the VM ran on. In terms of the number of CPUs and cores, the data is also not reflecting the real configuration of the host. But it does list the theoretical capacity for the case when Intel Hyperthreading is configured on the host. Means at the end, you can’t draw any conclusions on the number of vCPUs or the volume of memory for e.g. the M128s VM type (as shown above) out of the official SAP benchmark webpage. In order to get that data, you would need to check on the Azure pricing page or SAP note #1928533. So at the end, the data displayed on the official SAP benchmark webpage just confuses for the purpose of getting the SAPS of a specific Azure VM type for SAP sizing purposes. Though it displays the SAPS values of a certain VM type (which you can get in SAP note #1928533 as well), they however were reported with the host technical data; data that is unnecessary to size Azure VM infrastructure according to SAPS requirements.

Second area of confusion is created when you compare different VM types and calculate the SAPS/vCPU. Please note, we are always talking about vCPUs or CPU threads as a single unit that shows up in e.g. Windows Task Manager. We avoid the term ‘core’ for VMs on purpose because it has a very fixed meaning on bare-metal. In the Intel bare-metal world you can have one core that is representing one CPU thread in the case that the server is configured w/o Hyperthreading. Or you can have a core representing two CPU threads in case the server is configured with Hyperthreading enabled.

As you calculate the SAPS a single vCPU of an Azure VM can deliver as throughput, you can get to a somewhat surprising results when comparing different Azure VM types. E.g., let’s check DS14v2, which in SAP note #1928533 is reported with:

  • 16 vCPUs, 112GiB memory and 24180 SAPS.
  • We look at 24180 / 16 = 1511 SAPS per vCPU as throughput

As we move to DS16v3 which has the same number of vCPUs (though only 64GiB memory), you calculate only:

  • 17420 SAPS / 16 = 1088 SAPS throughput per CPU thread

This is kind of surprising since the DS16v3 is a more recent VM type and you would expect that performance or throughput per vCPU is improving all the time. Reason that this is not the case, is, that we introduced VM types that are running on hosts where Intel Hyperthreading is enabled. Means one physical processor core on the host server represents two CPU threads on the host. With that a single vCPU in an Azure VM is mapped into one of the two CPU threads of a hyperthreaded core on the host server. Hyperthreading on bare-metal server improves the overall throughput. But does not double the throughput as it does double the number of CPU threads of the host. The throughput improvement by Hyperthreading under classical SAP workload is ranging from 30-40%. As a result one core with two hyperthreaded CPU threads on the host will deliver 130-140% of the throughput the same processor core delivers without Hyperthreading. At the end, this means that a single CPU thread of a hyperthreaded core will deliver between 65-70% of what a non-hyperthreaded core delivers with a single CPU thread.

That is exactly the difference you are seeing between the SAPS a single vCPU delivers on a DS14v2 compared to a DS16v3.

So far, the following NetWeaver and/or SAP HANA certified Azure VM families are running on host hardware where Intel Hyperthreading is enabled:

  • D(S)v3
  • E(S)v3
  • M-Series

As mentioned earlier, the detailed data that is displayed by SAP on their benchmark webpage is not giving any indication whether Hyperthreading is enabled or not by just listing the theoretical capacity of the host. Indications whether Hyperthreading is enabled on the Azure hosts running certain VM families is published as well in the Azure pricing webpage. You can get indications like shown in the screenshot below:

clip_image004[4]

IMPORTANT:

Keep in mind that VM types have certain bandwidth limitations as well. In general, we can state that the smaller the VM in a VM family is, the smaller the storage and network bandwidth. In case of large VMs, like M128s or M128ms, or ES64v3 the VM is the only VM running on a host. As a result, it benefits from the complete network and storage bandwidth the host has available. In case of smaller VMs, the network and storage bandwidth need to be divided across multiple VM. Especially for SAP HANA, but also for SAP NetWeaver, it is vitally important that a VM running intensive workload does not steal CPU, memory, network and storage bandwidth from other VMs running on the same host. As a result, in sizing a VM, you also need to take the required network and storage bandwidth into account. Detailed data of network and storage throughput for each VM type can be found here:

We hope this clarifies aspects of the different SAPS per vCPU throughput calculations and measurements. And gives some explanations why the SAP benchmark webpages are not suited to check for sizing. Instead SAP note #1928533 should be consulted.