Upgraded our ExpressRoute circuit from 2Gb to 10Gb but stuck at 2Gb

Jason Bea 25

We have a 10Gb ExpressRoute circuit from our provider that we have had configured at 2Gb inside Azure. We now have a need for the full 10Gb bandwidth, so we have changed from 2Gb to 10Gb within the Azure ExpressRoute configuration, and changed our Virtual Network Gateway from ErGw2AZ to ErGw3AZ. The first change took about 15 minutes with no errors, so that tells me that there was capacity. The second change took almost an hour, which is normal from what I am reading. The problem is we are still only getting 2 Gbps. The port on our router is showing no errors, and showing 10 Gb. Our circuit provider is showing the circuit at 10Gb. Within Azure, it is also showing correctly at 10Gb and ErGw3AZ. Traffic is definitely routing through the ExpressRoute and back, so we are not seeing an asymmetric routing issue. Does anyone have an idea on what else to check? I'm wondering if MS didn't upgrade their side correctly even though it is showing 10Gb, but maybe I just overlooked something.

Jason Bea 25 Reputation points

2025-02-14T19:00:18.1433333+00:00

Here is the In bps metric for our ExpressRoute:

This is an iPerf test. We are pushing images across our ExpressRoute constantly at almost 2Gbps, so this is the speed I am getting with those images uploading:

It hits the 2Gb limit and will not go over at all.
Ganesh Patapati 3,920 Reputation points Microsoft Vendor

2025-02-17T15:22:20.24+00:00
Hello Jason Bea

Can you please try to verify whether your Internet Service provider (ISP) providing 10GB speed or not please let me know.

Should there be any follow-up questions or concerns, please let us know and we shall try to address them.
Ganesh Patapati 3,920 Reputation points Microsoft Vendor

2025-02-18T08:28:41.8+00:00
Hello Jason Bea

Please check your VM size, determine the maximum supported bandwidth for that particular VM size, and let me know.
Jason Bea 25 Reputation points

2025-02-18T13:06:04.1966667+00:00

We are not getting a cumulative 10 Gb, so there isn't a single VM involved. Our images go to blob storage, and no matter how much we push across, it hits exactly 2Gbps at most. So when we test with iPerf, we only get what is left of the 2Gbps cap that it seems to be hitting.
Ganesh Patapati 3,920 Reputation points Microsoft Vendor

2025-02-18T14:35:09.95+00:00

Jason Bea

Could you please share the architecture diagram for the end-to-end connectivity test? Additionally, is there a firewall behind the express route circuit? If so, are you using an Azure firewall or a third-party firewall?
Venkat V 550 Reputation points Microsoft Vendor

2025-02-20T07:07:48.8633333+00:00
Hi @Jason Bea

Here are the troubleshooting steps for the ExpressRoute bandwidth issue, including considerations for insufficient capacity and circuit recreation.

Verify the ExpressRoute Circuit: Ensure that the circuit is showing 10 Gbps provisioned in Azure and that both the circuit provisioning state and service provider provisioning state are marked as provisioned.

Confirm ExpressRoute Gateway Upgrade: Validate that the Virtual Network Gateway SKU has been successfully upgraded to ErGw3AZ and that it reflects a maximum bandwidth of 10 Gbps. If needed, restart the gateway to force reapplication.

Check ExpressRoute Peering Configuration: Verify that both private peering and Microsoft peering settings align with the 10 Gbps upgrade and are correctly configured.

4 . Check for Azure firewall or NVA bottlenecks. If traffic passes through an Azure Firewall, ensure it is Azure Firewall Premium (Standard is limited to 2 Gbps). If using an NVA (e.g., Palo Alto, Fortinet, Cisco), confirm that it supports 10 Gbps throughput and is not limiting bandwidth.

Review Load Balancer Configurations: Ensure any Azure Load Balancer in the traffic path is Standard SKU and correctly configured for high throughput.

Validate Provider Configuration: Although the circuit provider confirms 10 Gbps provisioning, request a detailed validation to ensure that the committed bandwidth settings have been updated correctly.

Force Azure to Revalidate the Configuration: If all settings appear correct but bandwidth remains capped at 2 Gbps, escalate to Microsoft by requesting backend re-provisioning of the ExpressRoute circuit.

Reference: About upgrading ExpressRoute circuit bandwidth

I hope this helps to resolve your issue.

If the above is unclear and/or you are unsure about something, add a comment below.
Venkat V 550 Reputation points Microsoft Vendor

2025-02-21T12:32:07.8966667+00:00

Hi @Jason Bea

I just wanted to follow up and see if you had a chance to review my response in the comment to your question. Please let us know if it was helpful, and feel free to reach out if you have any further queries.
Sarthak Agarwal 81 Reputation points Microsoft Employee

2025-02-24T06:00:08.66+00:00

Hi Jason Bea,

Can you please try and recreate connections between the upgraded ExRGW and the upgraded ExR from the Azure side?

Also, can you run mtr/winmtr tests to see there is no bottleneck hop between source and destination?

For a live on call troubleshooting session, if possible please do reach out to MS support.

Thanks,
Sarthak.

Accepted answer

Ganesh Patapati 3,920 Reputation points Microsoft Vendor

2025-02-14T06:27:02.5666667+00:00
Hi @Jason Bea

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

Could you please check and share the performance test results? like iPerf and PSPing can provide information about your network.

Refer: https://learn.microsoft.com/en-us/azure/expressroute/expressroute-troubleshooting-network-performance#tools

You can also check the metrics at Monitor Azure ExpressRoute in the documentation below.

Refer: https://learn.microsoft.com/en-us/azure/expressroute/monitor-expressroute#expressroute-metrics

Similarly, the Throughput tab plots the total throughput of ingress and egress traffic for the circuit in bits/second. You can also view throughput for individual connections and each type of configured peering.

Refer: https://learn.microsoft.com/en-us/azure/expressroute/expressroute-network-insights?source=recommendations#throughput

Refer: https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/expressroute/about-upgrade-circuit-bandwidth.md

I hope this helps!

Should there be any follow-up questions or concerns, please let us know and we shall try to address them.
Please sign in to rate this answer.
Jason Bea 25 Reputation points

2025-02-17T15:30:44.1333333+00:00

Our circuit provider portal was showing that the circuit was provisioned at 10Gb, but I did turn in a ticket with them anyway. They checked the circuit and verified that it is configured correctly at 10Gb.

Jason Bea 25 Reputation points

2025-02-27T16:04:02.2633333+00:00

In case anyone else runs into this exact same issue...this has been resolved by our circuit provider. We initially ordered a 10Gb ExpressRoute through them, but on the Azure side we started at 2Gb since that was what we needed at the time. When we created the ExpressRoute in Azure and obtained the service key, our circuit provider used that information and limited our 10Gb to 2Gb. I had them check the circuit when we first noticed the issue after upgrading the Azure side to 10Gb, but they stated that they verified everything was 10Gb on their side. My guess is that they never actually checked this liked they said since after telling them that Microsoft support wasn't seeing an issue, they checked again and found the 2Gb limit on their side. We are now getting the full 10Gb.

KapilAnanth-MSFT 48,761 Reputation points Microsoft Employee

2025-02-27T16:23:42.06+00:00

Hi @Jason Bea,

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this!

Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I have converted the discussion you had with Ganesh as an answer.

I would appreciacte if you could "Accept" the answer.

Issue:

ExpressRoute circuit bandwidth capped at 2Gb desptite upgrading to 10Gb

Solution:

Your Service Provider overlooked the upgrade to 10Gb.

Once you identified the issue, this was fixed and now you are getting the full potential

Cheers,

Kapil
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Upgraded our ExpressRoute circuit from 2Gb to 10Gb but stuck at 2Gb

0 additional answers

Your answer