Connect Stream Analytics jobs to resources in an Azure Virtual Network
Your Stream Analytics jobs make outbound connections to your input and output Azure resources to process data in real time and produce results. These input and output resources (for example, Azure Event Hubs and Azure SQL Database) could be behind an Azure firewall or in an Azure Virtual Network. Stream Analytics service operates from networks that can't be directly included in your network rules.
However, there are several ways to securely connect your Stream Analytics jobs to your input and output resources in such scenarios.
- Run your Azure Stream Analytics job in an Azure Virtual Network
- Use private endpoints in Stream Analytics clusters.
- Use Managed Identity authentication mode coupled with 'Allow trusted services' networking setting.
Your Stream Analytics job doesn't accept any inbound connection.
Run your Azure Stream Analytics job in an Azure Virtual Network
Virtual network support enables you to lock down access to Azure Stream Analytics to your virtual network infrastructure. This capability provides you with the benefits of network isolation and can be accomplished by deploying a containerized instance of your Azure Stream Analytics job inside your Virtual Network. Your virtual network injected job can then privately access your resources within the virtual network via:
- Private endpoints, which connect your virtual network injected ASA job to your data sources over private links powered by Azure Private Link.
- Service endpoints, which connect your data sources to your virtual network injected ASA job.
- Service tags, which allow or deny traffic to Azure Stream Analytics.
Currently, virtual network integration is only available in select regions. Visit this page for most recent list of virtual network enabled regions and how to request it in your region.
Private endpoints in Stream Analytics clusters.
Stream Analytics clusters is a single tenant dedicated compute cluster where you can run your Stream Analytics jobs. You can create managed private endpoints in your Stream Analytics cluster, which allows any jobs running on your cluster to make a secure outbound connection to your input and output resources.
The creation of private endpoints in your Stream Analytics cluster is a two step operation. This option is best suited for medium to large streaming workloads as the minimum size of a Stream Analytics cluster is 12 SU V2 or 36 SU V1s (SUs can be shared by different jobs in various subscriptions or environments like development, test, and production). For more information, see Azure Stream Analytics cluster.
Managed identity authentication with 'Allow trusted services' configuration
Some Azure services provide Allow trusted Microsoft services networking setting, which when enabled, allows your Stream Analytics jobs to securely connect to your resource using strong authentication. This option allows you to connect your jobs to your input and output resources without requiring a Stream Analytics cluster and private endpoints. Configuring your job to use this technique is a 2-step operation:
- Use Managed Identity authentication mode when configuring input or output in your Stream Analytics job.
- Grant your specific Stream Analytics jobs explicit access to your target resources by assigning an Azure role to the job's system-assigned managed identity.
Enabling Allow trusted Microsoft services doesn't grant blanket access to any job. It gives you full control of which specific Stream Analytics jobs can access your resources securely.
Your jobs can connect to the following Azure services using this technique:
- Blob Storage or Azure Data Lake Storage Gen2 - can be your job's storage account, streaming input or output.
- Azure Event Hubs - can be your job's streaming input or output.
If your jobs need to connect to other input or output types, you could write from Stream Analytics to Event Hubs output first and then to any destination of your choice using Azure Functions. If you want to directly write from Stream Analytics to other output types secured in a virtual network or firewall, then the only option is to use private endpoints in Stream Analytics clusters.