synapse workspace publish templates, remove hardcoded resources

De Bondt, Hannes (Mechelen) 25

For deployment of our production synapse workspace, we have a dev workspace synced to a Git repo with a publish branch and a release pipeline to publish the template to our production workspace.

In this template (TemplatesForWorkspace.json) some resources seem to be hard coded. In particular the spark pool compute resource is specified to be the nonprod resources.

Obviously we want the prod resources to be different (because there is no option to use the nonprod resources from the prod workspace). What are the steps needed to generate the template with the resources to be parameterized or defined in the template instead of linked to the non-prod resources?

phemanth 11,470 Reputation points Microsoft Vendor

2024-11-22T16:42:55.4233333+00:00
@De Bondt, Hannes (Mechelen)

Thanks for using Microsoft Q&A forum and posting your query.

To modify your Synapse workspace template (TemplatesForWorkspace.json) and remove hardcoded resources, particularly for the Spark pool,

Steps to Parameterize Resources in Synapse Workspace Template

Identify Hardcoded Resources:

Open your TemplatesForWorkspace.json file and locate the sections where resources are hardcoded, such as the Spark pool configuration.

Define Parameters:

At the beginning of your JSON template, define parameters for the resources you want to make configurable. For example:JSON
"parameters": { "sparkPoolName": { "type": "string", "defaultValue": "your-default-spark-pool-name", "metadata": { "description": "Name of the Spark pool to use." } }, "sparkPoolSize": { "type": "string", "defaultValue": "Small", "metadata": { "description": "Size of the Spark pool." } } }

Update Resource Definitions:

Replace the hardcoded values in the resource definitions with the parameters you just created. For example:JSON
{ "type": "Microsoft.Synapse/workspaces/sparkPools", "name": "[parameters('sparkPoolName')]", "properties": { "nodeCount": "[parameters('sparkPoolSize')]" } }

Use Parameter Values in Deployment:

When deploying the template, ensure that you pass the appropriate values for these parameters based on the environment (production vs. non-production). This can be done in your CI/CD pipeline or deployment scripts.

Test the Template:

Deploy the modified template to a test environment to ensure that the parameterization works as expected. Verify that the correct resources are created based on the parameters provided.

Update CI/CD Pipeline:

If you have a release pipeline set up, ensure that it is configured to pass the correct parameter values for production deployments.

refer:https://learn.microsoft.com/en-us/azure/synapse-analytics/cicd/source-control

Continuous integration & delivery in Azure Synapse Analytics - Azure Synapse Analytics | Microsoft Learn

https://techcommunity.microsoft.com/blog/azuresynapseanalyticsblog/automating-the-publishing-of-workspace-artifacts-in-synapse-cicd/3603042

Hope this helps. Do let us know if you any further queries.
phemanth 11,470 Reputation points Microsoft Vendor

2024-11-25T06:18:37.2266667+00:00

@De Bondt, Hannes (Mechelen) We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Share via

synapse workspace publish templates, remove hardcoded resources

Your answer