Integrate storage, database, webapp, and databricks into one VNET

P, John 200 Reputation points
2024-11-22T17:43:49.06+00:00

We are hardening our ML platform in Azure. Basically, we want to put all the Azure services ( ADLS, PostgreSQL DB, Web App, and Databricks cluster) in one virtural network so all the traffic will be inside the Azure network whenever possible (except the ingress collected data from on-premise network etc)

We plan to create different subnets for each services if needed:

  • two subnets for databrick cluster (VNET injection)
  • one subnet for the webapp service (VNET integration for outbound traffic)

The questions are:

Do we need to create dedicated subnet for ADLS storage and Database?

If not, how to route traffic between different Azure services? will the private endpoint be enough?

Thanks!

Azure Virtual Network
Azure Virtual Network
An Azure networking service that is used to provision private networks and optionally to connect to on-premises datacenters.
2,539 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,244 questions
Azure Web Application Firewall
Azure Database for PostgreSQL
{count} votes

1 answer

Sort by: Most helpful
  1. KapilAnanth-MSFT 47,286 Reputation points Microsoft Employee
    2024-11-25T12:40:23.8433333+00:00

    @P, John ,

    Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

    I am afraid I am not sure if your question is correct to begin with.

    NOTE :

    • Private EndPoint is different from VNET Integration.
    • Private EndPoint is used when you want to connect to the PaaS Service and expect the traffic to stay within the Microsoft backbone
      • The other way traffic (PaaS to resources in VNET via PE is not supported)
    • While VNET integration is used when you want the PaaS Services to connect to other resources that reside in the VNET.
      • The other way traffic (Resources in VNET to the PaaS via VNET integration is not supported)
    • i.e., one compliments the other

    Also, PE is considered as a resource in a VNET

    With that said, coming to your question,

    "Do we need to create dedicated subnet for ADLS storage and Database?"

    • Did you first check if ADLS storage and Database support VNET integration?
    • From my analysis, they only support PE
    • This makes sense as the above two never have to initiate traffic (outbound) - only receive traffic (inbound)

    To give you an overall summary of how to utilize VNET integration and PE to keep the traffic inside the MS Backbone,

    • If you want the PaaS Service to talk to resources in Azure VNET , enable VNET integration for this PaaS Service.
    • If you want the resources in Azure VNET to talk to PaaS Service , create a PE for this PaaS Service.
    • Now, combining these 2, if you want two PaaS Services to be able to talk to each other
      • Enable VNET integration for both of them
      • And also create PE for both of them.

    So,

    1 . When PaaS1 wants to talk to PaaS2

    • VNET integrated PaaS1 will initiate traffic from the delegated subnet to the PE of PaaS2

    2 . When PaaS2 wants to talk to PaaS1

    • VNET integrated PaaS2 will initiate traffic from the delegated subnet to the PE of PaaS1

    Please let us know if we can be of any further assistance here.

    Thanks,

    Kapil


    Please Accept an answer if correct.

    Original posters help the community find answers faster by identifying the correct answer.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.