This reference architecture illustrates how you can use Azure Arc to manage, govern, and secure servers across on-premises, multicloud, and edge scenarios. The architecture is based on the Azure Arc Jumpstart ArcBox for IT Pros implementation. ArcBox is a solution that provides an easy-to-deploy sandbox for all things Azure Arc. ArcBox for IT Pros is a version of ArcBox that's intended for users who want to experience Azure Arc-enabled server capabilities in a sandbox environment.
Architecture
Download a PowerPoint file of this architecture.
Components
The architecture consists of the following components:
- An Azure resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group.
- ArcBox workbook is an Azure Monitor workbook, which provides a single pane of glass for monitoring and reporting on ArcBox resources. The workbook acts as a flexible canvas for data analysis and visualization in the Azure portal, gathering information from several data sources from across ArcBox and combining them into an integrated interactive experience.
- Azure Monitor enables you to track performance and events for systems running in Azure, on-premises, or in other clouds.
- Azure Policy guest configuration can audit operating systems and machine configuration both for machines running in Azure and Azure Arc-enabled servers running on-premises or in other clouds.
- Azure Log Analytics is a tool in the Azure portal to edit and run log queries from data collected by Azure Monitor Logs and interactively analyze their results. You can use Log Analytics queries to retrieve records that match particular criteria, identify trends, analyze patterns, and provide various insights into your data.
- Microsoft Defender for Cloud is a cloud security posture management (CSPM) and cloud workload protection (CWP) solution. Defender for Cloud finds weak spots across your cloud configuration, helps strengthen the overall security posture of your environment, and can protect workloads across multicloud and hybrid environments from evolving threats.
- Microsoft Sentinel is a scalable, cloud-native, security information and event management (SIEM) solution, and security orchestration, automation, and response (SOAR) solution. Microsoft Sentinel provides intelligent security analytics and threat intelligence across the enterprise. It also provides a single solution for attack detection, threat visibility, proactive hunting, and threat response.
- Azure Arc-enabled servers enables you to connect Azure to your Windows and Linux machines hosted outside of Azure on your corporate network. When a server is connected to Azure, it becomes an Azure Arc-enabled server and is treated as a resource in Azure. Each Azure Arc-enabled server has a Resource ID, a managed system identity, and is managed as part of a resource group inside a subscription. Azure Arc-enabled servers benefit from standard Azure constructs such as inventory, policy, tags, and Azure Lighthouse.
- Hyper-V nested virtualization is used by Jumpstart ArcBox for IT Pros to host Windows and Linux Server virtual machines inside of an Azure virtual machine. This approach provides the same experience as using physical Windows Server machines, but without the hardware requirements.
- Azure Virtual Network provides a private network that enables components, such as virtual machines, within the Azure resource group to communicate.
Scenario details
Potential use cases
Typical uses for this architecture include:
- Organize, govern, and inventory large groups of virtual machines (VMs) and servers across multiple environments.
- Enforce organization standards and assess compliance at scale for all your resources anywhere with Azure Policy.
- Easily deploy supported VM extensions to Azure Arc-enabled servers.
- Configure and enforce Azure Policy for VMs and servers hosted across multiple environments.
Recommendations
The following recommendations apply for most scenarios. Follow these recommendations unless you have a specific requirement that overrides them.
Configure Azure Arc Connected Machine agent
You can connect any other physical or virtual machine running Windows or Linux to Azure Arc. Before onboarding machines, be sure to complete the Connected machine agent prerequisites, which includes registering the Azure resource providers for Azure Arc-enabled servers. To use Azure Arc to connect the machine to Azure, you need to install the Azure Connected Machine agent on each machine that you plan to connect using Azure Arc. For more information, see Overview of Azure Arc-enabled servers agent.
After you configure the Connected Machine agent, it sends a regular heartbeat message to Azure every five minutes. When the heartbeat isn't received, Azure assigns the machine Offline status, which is reflected in the portal within 15 to 30 minutes. When Azure receives a subsequent heartbeat message from the Connected Machine agent, its status automatically changes to Connected.
There are several options available in Azure to connect your Windows and Linux machines, including:
- Install manually: You can enable Azure Arc-enabled servers for one or more Windows or Linux machines in your environment by using the Windows Admin Center or by performing a set of steps manually.
- Install by using a script: You can perform automated agent installation by running a template script that you download from the Azure portal.
- Connect machines at scale by using a service principal: To onboard at scale, use a service principal and deploy via your organizations existing automation.
- Install by using Windows PowerShell DSC.
Consult the Azure Connected Machine agent deployment options for comprehensive documentation on the various deployment options available.
Enable Azure Policy guest configuration
Azure Arc-enabled servers support Azure Policy at the Azure resource management layer, and also within the individual server machine using guest configuration policies. Azure Policy guest configuration can audit settings inside a machine, both for machines running in Azure and Azure Arc-enabled servers. For example, you can audit settings such as:
- Operating system configuration
- Application configuration or presence
- Environment settings
There are several Azure Policy built-in definitions for Azure Arc. These policies provide auditing and configuration settings for both Windows and Linux-based machines.
Enable Azure Update Manager and change tracking
It's important that you adopt an update management process for Azure Arc-enabled servers by enabling the following components:
- Use Azure Update Manager to manage, assess, and govern the installation of Windows and Linux updates across all servers.
- Use change tracking and inventory for Azure Arc-enabled servers to:
- Determine what software is installed in your environment.
- Collect and observe inventory for software, files, Linux daemons, Windows services, and Windows Registry keys.
- Track the configurations of your machines to pinpoint operational problems across your environment and better understand the state of your machines.
Monitor Azure Arc-enabled servers
Use Azure Monitor to monitor your VMs, Azure Virtual Machine Scale Sets, and Azure Arc machines at scale. Use Azure Monitor to:
- Analyze the performance and health of your Windows and Linux VMs.
- Monitor VM processes and dependencies on other resources and external processes.
- Monitor performance and application dependencies for VMs that are hosted on-premises or in another cloud provider.
The Azure Monitor agent should be automatically deployed to Azure Arc-enabled Windows and Linux servers, through Azure Policy. Review and understand how the Azure Monitor agent operates and collects data before deployment.
Design and plan your Azure Monitor Logs workspace deployment. The workspace is the container where data is collected, aggregated, and analyzed. An Azure Monitor Logs workspace represents a geographical location of your data, data isolation, and scope for configurations like data retention. Use a single Azure Monitor Logs workspace as described in the management and monitoring best practices of Cloud Adoption Framework for Azure.
Secure Azure Arc-enabled servers
Use Azure role-based access control (RBAC) to control and manage the permissions for managed identities on Azure Arc-enabled servers and to perform periodic access reviews for these identities. Control privileged user roles to prevent system-managed identities from being misused to gain unauthorized access to Azure resources.
- Consider using Azure Key Vault to manage certificates on your Azure Arc-enabled servers. You can use the key vault VM extension to manage the certificate lifecycle on Windows and Linux machines.
- Connect Azure Arc-enabled servers to Defender for Cloud. Use Defender for Cloud to collect the security-related configurations and event logs that you need to recommend actions and improve your overall Azure security posture.
- Connect Azure Arc-enabled servers to Microsoft Sentinel. Use Microsoft Sentinel to collect security-related events and correlate them with other data sources.
Validate network topology
The Connected Machine agent for Linux and Windows communicates outbound securely to Azure Arc over TCP port 443. The Connected Machine agent can connect to the Azure control plane using the following methods:
- Direct connection to Azure public endpoints, optionally from behind a firewall or a proxy server.
- Azure Private Link using a Private Link Scope model to allow multiple servers or machines to communicate with their Azure Arc resources using a single private endpoint.
Consult Network topology and connectivity for Azure Arc-enabled servers for comprehensive networking guidance for your Azure Arc-enabled servers implementation.
Considerations
These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that can be used to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.
Reliability
Reliability ensures your application can meet the commitments you make to your customers. For more information, see Design review checklist for Reliability.
- In most cases, the location you select when you create the installation script should be the Azure region geographically closest to your machine's location. The rest of the data is stored within the Azure geography containing the region you specify, which might also affect your choice of region if you have data residency requirements. If an outage affects the Azure region to which your machine is connected, the outage doesn't affect the Azure Arc-enabled server. However, management operations using Azure might not be available.
- If the Azure connected machine agent stops sending heartbeats to Azure or goes offline, you can't perform operational tasks on it. So, you must develop a plan for notifications and responses.
- Set up resource health alerts to get notified in near real-time when resources have a change in their health status. And define a monitoring and alerting policy in Azure Policy that identifies unhealthy Azure Arc-enabled servers.
- Extend your current backup solution to Azure, or easily configure our application-aware replication and application-consistent backup that scales based on your business needs. The centralized management interface for Azure Backup and Azure Site Recovery makes it simple to define policies to natively protect, monitor, and manage your Azure Arc-enabled Windows and Linux servers.
- Review the business continuity and disaster recovery guidance to determine whether your enterprise requirements are met.
- For other reliability considerations for your solution, see Reliability design principles in the Well-Architected Framework.
Security
Security provides assurances against deliberate attacks and the abuse of your valuable data and systems. For more information, see Design review checklist for Security.
- Appropriate Azure RBAC should be managed for Azure Arc-enabled servers. To onboard machines, you must be a member of the Azure Connected Machine Onboarding role. To read, modify, re-onboard, and delete a machine, you must be a member of the Azure Connected Machine Resource Administrator role.
- Defender for Cloud can monitor on-premises systems, Azure VMs, and VMs hosted by other cloud providers. Enable Microsoft Defender for servers for all subscriptions that contain Azure Arc-enabled servers for security baseline monitoring, security posture management, and threat protection.
- Microsoft Sentinel can help simplify data collection across different sources, including Azure, on-premises solutions, and across clouds using built-in connectors.
- You can use Azure Policy to manage security policies across your Azure Arc-enabled servers, including implementing security policies in Defender for Cloud. A security policy defines the desired configuration of your workloads and helps ensure you're complying with the security requirements of your company or regulators. Defender for Cloud policies are based on policy initiatives created in Azure Policy.
- To limit which extensions can be installed on your Azure Arc-enabled server, you can configure the lists of extensions you wish to allow and block on the server. The extension manager evaluates all requests to install, update, or upgrade extensions against the allowlist and blocklist to determine if the extension can be installed on the server.
- Azure Private Link allows you to securely link Azure PaaS services to your virtual network using private endpoints. You can connect your on-premises or multicloud servers with Azure Arc and send all traffic over an Azure ExpressRoute or site-to-site VPN connection instead of using public networks. You can use a Private Link Scope model to allow multiple servers or machines to communicate with their Azure Arc resources using a single private endpoint.
- Consult Azure Arc-enabled servers security overview for a comprehensive overview of the security features in Azure Arc-enabled server.
- For other security considerations for your solution, see Security design principles in the Well-Architected Framework.
Cost Optimization
Cost Optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Design review checklist for Cost Optimization.
- Azure Arc control plane functionality is provided at no extra cost. This includes support for resource organization through Azure management groups and tags, and access control through Azure RBAC. Azure services used in conjunction to Azure Arc-enabled servers incur costs according to their usage.
- See Cost governance for Azure Arc-enabled servers for more Azure Arc cost optimization guidance.
- For other cost optimization considerations for your solution, see Cost Optimization design principles in the Well-Architected Framework.
- Use the Azure pricing calculator to estimate costs.
- When deploying the Jumpstart ArcBox for IT Pros reference implementation for this architecture, keep in mind ArcBox resources generate Azure Consumption charges from the underlying Azure resources. These resources include core compute, storage, networking, and auxiliary services.
Operational Excellence
Operational Excellence covers the operations processes that deploy an application and keep it running in production. For more information, see Design review checklist for Operational Excellence.
- Automate the deployment of your Azure Arc-enabled servers environment. The reference implementation of this architecture is fully automated using a combination of Azure ARM templates, VM extensions, Azure Policy configurations, and PowerShell scripts. You can also reuse these artifacts for your own deployments. For more information, see Automation disciplines for Azure Arc-enabled servers.
- There are several options available in Azure to automate the onboarding of Azure Arc-enabled servers. To onboard at scale, use a service principal and deploy via your organizations existing automation platform.
- VM extensions can be deployed to Azure Arc-enabled servers to simplify the management of hybrid servers throughout their lifecycle. Consider automating the deployment of VM extensions via Azure Policy when managing servers at scale.
- Enable patch and Update Management in your onboarded Azure Arc-enabled servers to ease OS lifecycle management.
- See Azure Arc Jumpstart Unified Operations Use Cases to learn about additional operational excellence scenarios for Azure Arc-enabled servers.
- For other operational excellence considerations for your solution, see Operational Excellence design principles in the Well-Architected Framework.
Performance Efficiency
Performance Efficiency is the ability of your workload to scale to meet the demands placed on it by users in an efficient manner. For more information, see Design review checklist for Performance Efficiency.
- Before configuring your machines with Azure Arc-enabled servers, you should review the Azure Resource Manager subscription limits and resource group limits to plan for the number of machines to be connected.
- A phased deployment approach as described in the deployment guide can help you determine the resource capacity requirements for your implementation.
- Use Azure Monitor to collect data directly from your Azure Arc-enabled servers into an Azure Monitor Logs workspace for detailed analysis and correlation. Review the deployment options for the Azure Monitor agent.
- For more performance efficiency considerations for your solution, see Performance Efficiency principles in the Well-Architected Framework.
Deploy this scenario
The reference implementation of this architecture can be found in the Jumpstart ArcBox for IT Pros, included as part of the Azure Arc Jumpstart project. ArcBox is designed to be self-contained within a single Azure subscription and resource group. ArcBox makes it easy for a user to get hands-on experience with all available Azure Arc technology with nothing more than an available Azure subscription.
To deploy the reference implementation, select Jumpstart ArcBox for IT Pros and follow the steps in the GitHub repo.
Contributors
This article is maintained by Microsoft. It was originally written by the following contributors.
Principal author:
- Pieter de Bruin | Senior Program Manager
To see non-public LinkedIn profiles, sign in to LinkedIn.
Next steps
- Learn more about Azure Arc
- Learn more about Azure Arc-enabled servers
- Azure Arc learning path
- Review Azure Arc Jumpstart scenarios in the Azure Arc Jumpstart
- Review Azure Arc-enabled servers landing zone accelerator in the Cloud Adoption Framework
Related resources
Explore related architectures: