Jaa

Serverless compute plane networking

  • Artikkeli

This guide introduces tools to secure network access between the compute resources in the Azure Databricks serverless compute plane and customer resources. To learn more about the control plane and the serverless compute plane, see Azure Databricks architecture overview.

To learn more about classic compute and serverless compute, see Types of compute.

Important

Starting December 4, 2024, Databricks will begin charging for networking costs on serverless workloads that connect to external resources. Billing will be implemented gradually, and you might not be charged until after December 4, 2024. You won’t be charged retroactively for usage before billing is enabled. After billing is enabled, you may be charged for:

  • Private connectivity to your resources over Private Link. Data processing charges for private connectivity to your resources over Private Link are waived indefinitely. Per-hour charges will apply.
  • Public connectivity to your resources over NAT gateway.
  • Data transfer charges incurred, such as when serverless compute and the target resource are in different regions.

Serverless compute plane networking overview

Serverless compute resources run in the serverless compute plane, which is managed by Azure Databricks. Account admins can configure secure connectivity between the serverless compute plane and their resources. This network connection is labeled as 2 on the diagram below:

Network connectivity overview diagram

Connectivity between the control plane and the serverless compute plane is always over the cloud network backbone and not the public internet. For more information on configuring security features on the other network connections in the diagram, see Networking.

What is serverless egress control?

Serverless egress control allows you to manage outbound network connections from your serverless compute resources.

Using network policies, you can:

  • Enhance security: Mitigate data exfiltration risks by restricting outbound connections.
  • Define precise rules: Control outbound connections by specifying allowed locations, connections, FQDNs, and Azure storage accounts.
  • Simplify management: Easily configure and manage egress policies in your serverless environment.

See What is serverless egress control?.

What is a network connectivity configuration (NCC)?

Serverless network connectivity is managed with network connectivity configurations (NCC). NCCs are account-level regional constructs that are used to manage private endpoints creation and firewall enablement at scale.

Account admins create NCCs in the account console and an NCC can be attached to one or more workspaces. An NCC enables firewalls and private endpoints:

  • Resource firewall enablement by subnets: An NCC enables Databricks-managed stable Azure service subnets for adding service endpoints to your resource firewalls for secure access to Azure resources from serverless SQL warehouses. When an NCC is attached to a workspace, serverless compute in that workspace uses one of those networks to connect the Azure resource using service endpoints. You can allow list those networks on your Azure resource firewall. The network rules are automatically added to the workspace storage account. See Configure a firewall for serverless compute access.
  • Private endpoints: When you add a private endpoint in an NCC, Azure Databricks creates a private endpoint request to your Azure resource. Once the request is accepted on the resource side, the private endpoint is used to access your Azure resource from the serverless compute plane. See Configure private connectivity from serverless compute.