How can i migrate jobs from one workspace to other workspace (Dev to QA)

Question

How can i migrate jobs from one workspace to other workspace (Dev to QA) .

Answer 1

@Lakshmi Narayana Sarma Bhamidipati

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Issue:
How can i migrate jobs from one workspace to other workspace (Dev to QA) .

Solution:

Team, couldnt complete this as time didnt permit and used CI/CD pipeline to move them. But anyways thanks for the help provided.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

Hope this helps. Do let us know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Hello Lakshmi, Welcome to MS Q&A

To migrate jobs from one Azure Databricks workspace to another (e.g., from Development to QA), you can use the Databricks CLI to automate the migration process. Here are the steps:

1. Install the Databricks CLI: Ensure that the Databricks CLI is installed on your local machine or virtual machine.
2. Configure Profiles: Set up two profiles in the Databricks CLI, one for the source workspace (Development) and another for the target workspace (QA).
3. Migrate Job Configurations: You can migrate job configurations using the Databricks CLI. This can be done automatically for all jobs, or you can opt for selective migration if needed.
4. Handle Cluster Configurations: If your jobs are configured to use specific clusters, ensure that the cluster configurations are also migrated. You may need to replace old cluster IDs with new ones if the clusters differ between workspaces.
5. Reapply Access Control: After migrating jobs, manually reapply any necessary access controls to the resources in the new workspace.

Make sure to test the migrated jobs to confirm they function correctly in the new environment.

References:

• Regional disaster recovery for Azure Databricks

Please let us know if any further question , you can check above doc for more ref.

Kindly accept answer if it helps

Thanks

Deepanshu