Azure VM vs. App Service and Azure SQL Performance

Question

I've found a smattering of similar articles, some older, some that you could no longer reply to, but I'm having this problem now and it is leading me to reconsider my entire migration plan.

We have a pretty straight forward app, 2 on premises bare metal servers, one is a SQL Server DB and the other is our ASP.NET web app running in IIS. Both servers are Win2k8, SQL Server 2008 on one and IIS 7.5 on the other. Hardware is 1st gen Core i7, 24GB RAM in each. They're old.

My plan was to migrate this all to Azure App Service + Azure SQL. I created a test environment and everything went pretty smoothly, so testing began.

I started with a query script that was processing intensive and tried to run this through SSMS. Right away, we noticed that the query ran slower in Azure SQL than our old on prem machine. Even when bumping up Azure SQL to the highest tier, we could not get this query to perform as fast as our on prem. We even tried using a comparable SQL version (2019) on a developer workstation which was still faster than Azure SQL. There are a LOT of variables here but after investigating extensively and discussing with Microsoft support, one of the MS techs said he believes that the overhead associated with the serverless solution and its high availability, low maintenance capabilities makes it lag slightly behind in performance compared to a bare metal or even a dedicated VM solution. At its best, performance was close, but still not what we'd expect. Our application is not a high volume, with millions of users, but thousands.

As for App Service, we also experienced slowness on a particular page which is heavily used in our app. Page load was substantially lower in App Service. With our awareness of the Azure SQL performance issue we experienced, we removed that from the equation but still experienced slow page load. We narrowed this down to Time To First Byte (TTFB) using several methods and provided info to MS Tech Support. We were never able to get the performance we are used to seeing from our old on prem hardware or from dedicated VMs.

Again, this is not an app used by millions, its a business app used by under 10 thousand users and concurrent users are less than that. The performance differences we're seeing was not huge, but much bigger than I'd ever have expected comparing old on prem hardware to the latest serverless technologies. It makes me wonder how any large scale app would ever use these services over setting up their own VM server farm.

Who is using App Services and Azure SQL over VMs? Even though we're small in comparison to other large web companies, we have some pages with performance issues and I can't imagine migrating to a serverless solution and shortly after experiencing worse performance problems requiring us to consider alternatives. In other words, we don't want to start out in a new environment where we already know of a performance setback.

Suggestions/thoughts?

Answer 1

@Larry L This would certainly be unexpected behavior as there are sites that process tens of thousands of requests every minute with hardly any issue. With that being said, there are a number of factors that can impact performance and as a result, we do not commit to certain performance levels. I should also add that the advice you received from the support engineer is likely more detailed than we can provide you here as they have the ability to access and review logs to understand where time is being spent on each request.

Some of the basic items that the support engineer likely had you verify is to make sure your web app and Azure SQL are both running in the exact same data center. All data centers are geographically a minimum of 500 miles apart and this can introduce enough latency to impact your site. Please note that one item in East US and another in East US 2 is not considered the same data center and risks introducing latency.

What exactly is your site trying to load? Pictures and videos? If so, you should consider moving those resources off your web app and store them on Azure CDN. Azure Web Apps makes use of network storage and while this is fast, sites with heavy pictures/videos can be optimized by offloading these files to Azure CDN.

Have you checked to see if your CPU or memory usage is high? Note that just because the documentation says standard tier or higher can run unlimited sites, there is a limitation at a certain point. Please ensure you aren't running lots of sites in a single app service plan.

Are you still working with the support engineer? What is the current status or final solution? If the issue was determined to be with your site code, they likely won't be able to assist much but I want to ensure they ruled out any platform issues impacting your site performance.