Share via

ADF Managed Virtual Network unexpectedly incurring cost in other region

Thijs 0 Reputation points
2024-12-04T13:31:46.2433333+00:00

Untitled

Recently I have implemented denied public access, virtual networking and private endpoints for my SQL Servers, as well as Managed Virtual Networks in my ADF. Everything works as intended, but there was a quick spike in the cost for my ADF, which I noticed was billed for location US East upon further investigation. Even though all of my resources are hosted in EU West, except for the private DNS zones which are public.

Approximately 2.000.000 rows (~100MB) are exported daily from an Azure SQL VM (hosted in EU West) running SQL Server to my Azure SQL Server. I am using an Integration Runtime on the VM to connect to it. A few thousand rows are exported daily from another Azure SQL Server hosted in EU West.

Is this intended behaviour, or would this be a misconfiguration on my side?

Azure Cost Management
Azure Cost Management
A Microsoft offering that enables tracking of cloud usage and expenditures for Azure and other cloud providers.
2,833 questions
Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,572 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,044 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Pranay Reddy Madireddy 1,230 Reputation points Microsoft Vendor
    2024-12-13T21:23:25.9266667+00:00

    Hi Thijs

    Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

    Check the activity runs in your ADF pipelines for any unexpected or unnecessary tasks, such as:

    Duplicate data exports or retries.

    Activities that are taking longer than usual.

    Use Azure Monitor to check the performance and resource usage of your Integration Runtime. High CPU or memory usage might indicate problems or delays.

    Review the data transfer between your Azure SQL VM and Azure SQL Server. Only transfer the data you need and use incremental loads instead of full exports if you can.

    Make sure your Integration Runtime is set up correctly for your workload, including:

    Using a self-hosted runtime on a VM in the same region as your SQL resources.

    Configuring it to scale according to demand.

    If you have triggers set up for your pipelines, make sure they aren't firing more often than needed, as this could increase costs.

    If you're using data flow activities, check their settings. Since data flows can increase costs based on the amount of data, optimizing them can help reduce expenses.

    you have any further queries, do let us know.

    If the answer is helpful, please click "Accept Answer" and "Upvote it".

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.