Upload usage data, metrics, and logs to Azure
Periodically, you can export out usage information for billing purposes, monitoring metrics, and logs and then upload it to Azure. The export and upload of any of these three types of data will also create and update the data controller, and SQL managed instance resources in Azure.
Before you can upload usage data, metrics, or logs you need to:
Examples in this article use angle brackets < ... > to identify values that you need to replace before you run the script. Replace the brackets and the values inside the brackets.
Install tools
The required tools include:
- Azure CLI (az)
arcdataextension
See Install tools.
Register the resource provider
Prior to uploading metrics or user data to Azure, you need to ensure that your Azure subscription has the Microsoft.AzureArcData resource provider registered.
To verify the resource provider, run the following command:
az provider show -n Microsoft.AzureArcData -o table
If the resource provider is not currently registered in your subscription, you can register it. To register it, run the following command. This command may take a minute or two to complete.
az provider register -n Microsoft.AzureArcData --wait
Create service principal
The service principal is used to upload usage and metrics data.
Follow these commands to create your metrics upload service principal:
Note
Creating a service principal requires certain permissions in Azure.
To create a service principal, update the following example. Replace <ServicePrincipalName>, SubscriptionId and resourcegroup with your values and run the command:
az ad sp create-for-rbac --name <ServicePrincipalName> --role Contributor --scopes /subscriptions/<SubscriptionId>/resourceGroups/<resourcegroup>
If you created the service principal earlier, and just need to get the current credentials, run the following command to reset the credential.
az ad sp credential reset --name <ServicePrincipalName>
For example, to create a service principal named azure-arc-metrics, run the following command
az ad sp create-for-rbac --name azure-arc-metrics --role Contributor --scopes /subscriptions/<SubscriptionId>/resourceGroups/myresourcegroup
Example output:
"appId": "<appId>",
"displayName": "azure-arc-metrics",
"name": "http://azure-arc-metrics",
"password": "<password>",
"tenant": "<tenant>"
Save the appId, password, and tenant values in an environment variable for use later. These values are in the form of globally unique identifier (GUID).
SET SPN_CLIENT_ID=<appId>
SET SPN_CLIENT_SECRET=<password>
SET SPN_TENANT_ID=<tenant>
After you have created the service principal, assign the service principal to the appropriate role.
Assign roles to the service principal
Run this command to assign the service principal to the Monitoring Metrics Publisher role on the subscription where your database instance resources are located:
Note
You need to use double quotes for role names when running from a Windows environment.
az role assignment create --assignee <appId> --role "Monitoring Metrics Publisher" --scope subscriptions/<SubscriptionID>/resourceGroups/<resourcegroup>
Example output:
{
"canDelegate": null,
"id": "/subscriptions/<Subscription ID>/providers/Microsoft.Authorization/roleAssignments/<globally unique identifier>",
"name": "<globally unique identifier>",
"principalId": "<principal id>",
"principalType": "ServicePrincipal",
"roleDefinitionId": "/subscriptions/<Subscription ID>/providers/Microsoft.Authorization/roleDefinitions/<globally unique identifier>",
"scope": "/subscriptions/<Subscription ID>",
"type": "Microsoft.Authorization/roleAssignments"
}
Verify service principal role
az role assignment list --scope subscriptions/<SubscriptionID>/resourceGroups/<resourcegroup> -o table
With the service principal assigned to the appropriate role, you can proceed to upload metrics, or user data.
Upload logs, metrics, or usage data
The specific steps for uploading logs, metrics, or usage data vary depending about the type of information you are uploading.
Upload metrics to Azure Monitor
General guidance on exporting and uploading usage, and metrics
Create, read, update, and delete (CRUD) operations on Azure Arc-enabled data services are logged for billing and monitoring purposes. There are background services that monitor for these CRUD operations and calculate the consumption appropriately. The actual calculation of usage or consumption happens on a scheduled basis and is done in the background.
Upload the usage only once per day. When usage information is exported and uploaded multiple times within the same 24 hour period, only the resource inventory is updated in Azure portal but not the resource usage.
Note
Note that usage data is automatically uploaded for Azure Arc data controller deployed in direct connected mode.
For uploading metrics, Azure monitor only accepts the last 30 minutes of data (Learn more). The guidance for uploading metrics is to upload the metrics immediately after creating the export file so you can view the entire data set in Azure portal. For instance, if you exported the metrics at 2:00 PM and ran the upload command at 2:50 PM. Since Azure Monitor only accepts data for the last 30 minutes, you may not see any data in the portal.
Related content
Learn about service principals
Upload billing data to Azure and view it in the Azure portal