Dataverse to Azure Blob Storage

Question

Hello,

I am currently using Microsoft Dynamics 365, and all my documents are stored in Dataverse. I would like to migrate all existing documents to Azure Blob Storage (or Azure File Storage, based on recommendations) and continue using this solution for storing new documents.

My questions:

1. Is it possible to migrate existing documents from Dataverse to Azure Blob Storage while maintaining links to entities and records in Dynamics 365?
2. Are there any guides or recommended tools to assist with the migration process?
3. What are the best practices for properly configuring the integration between Dynamics 365 and Azure Blob Storage?
4. If there are any limitations when using Blob Storage, could you please outline them?

Technical background:

• We are using the cloud version of Dynamics 365.
• The current document size is approximately 128GB.
• It is essential for us to retain system integration and ensure no data or process disruptions.

I would also appreciate additional recommendations and detailed information about the required configuration steps.

Thank you very much for your assistance!Hello,

I am currently using Microsoft Dynamics 365, and all my documents are stored in Dataverse. I would like to migrate all existing documents to Azure Blob Storage (or Azure File Storage, based on recommendations) and continue using this solution for storing new documents.

My questions:

1. Is it possible to migrate existing documents from Dataverse to Azure Blob Storage while maintaining links to entities and records in Dynamics 365?
2. Are there any guides or recommended tools to assist with the migration process?
3. What are the best practices for properly configuring the integration between Dynamics 365 and Azure Blob Storage?
4. If there are any limitations when using Blob Storage, could you please outline them?

Technical background:

• We are using the cloud version of Dynamics 365.
• The current document size is approximately 128GB.
• It is essential for us to retain system integration and ensure no data or process disruptions.

I would also appreciate additional recommendations and detailed information about the required configuration steps.

Thank you very much for your assistance!

Answer 1

Hello קרן קמינסקי,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that you would like to migrate all existing documents to Azure Blob Storage (or Azure File Storage, based on recommendations) and continue using this solution for storing new documents.

I will provide you with best practices and leveraging the appropriate tools, you can ensure a seamless migration from Dataverse to Azure Blob Storage while preserving system integrity and enhancing functionality by answering your questions.

Regarding your question if migration is possible and how to maintain links?

Yes, it is feasible to migrate documents from Dataverse to Azure Blob Storage while preserving links to Dynamics 365 entities. To achieve this:

• Use tools like Power Automate or Azure Data Factory to extract documents and metadata (e.g., record IDs).
• Save relevant metadata alongside the files in Blob Storage. Metadata fields like record IDs can help maintain relationships.
• Implement plugins or workflows in Dynamics 365 to dynamically retrieve links from Azure Blob Storage when accessing documents.
• For more details use this links: https://learn.microsoft.com/en-us/power-platform/power-automate and https://learn.microsoft.com/en-us/azure/data-factory/copy-data-tool

The recommended Tools and Guides:

• Azure Data Factory (ADF) is the best for large-scale migrations due to its ability to handle batch operations and maintain relationships between entities. Using Azure Data Factory with Dataverse - https://learn.microsoft.com/en-us/azure/data-factory/connector-dynamics-ax
• Power Automate is suitable for smaller datasets or scenarios where custom triggers are required. - https://learn.microsoft.com/en-us/power-automate/dynamics-365
• Azure Synapse Link for Dataverse is use for real-time data replication to Azure Data Lake, which can connect to Blob Storage. - https://learn.microsoft.com/en-us/dynamics365/customer-insights/data/export-azure-blob-storage

Best Practices:

1. Authorization: Use Azure Active Directory (Azure AD) for secure and seamless authentication between Dynamics 365 and Azure Blob Storage. - https://learn.microsoft.com/en-us/azure/active-directory
2. Automated Workflows: Leverage Power Automate to automate document storage and retrieval workflows to reduce manual effort.
3. Testing Environment: Perform comprehensive testing in a sandbox environment before applying changes to production to ensure data integrity and system functionality.
4. Security Configuration: Use Azure Blob Storage encryption options and define robust access policies to secure sensitive documents. - https://learn.microsoft.com/en-us/azure/storage/blobs/security-features

Azure Blob Storage vs. Azure File Storage:

• Blob Storage: Best suited for storing large volumes of unstructured data, offering scalability and lower cost. - https://learn.microsoft.com/en-us/azure/storage/blobs
• File Storage: Designed for file-sharing scenarios requiring a hierarchical directory structure. - https://learn.microsoft.com/en-us/azure/storage/files/

Azure Blob Storage is the ideal choice for Dynamics 365 integration due to its cost-efficiency and native support for unstructured data.

Limitations of Azure Blob Storage

• A single blob can store up to 190.7 TiB, but larger datasets may require partitioning or performance tuning.
• Blob Storage supports up to 5 stored access policies per container, which may need adjustment based on your authentication model.
• Blob operations may experience latency depending on network conditions and request concurrency.

https://learn.microsoft.com/en-us/azure/storage/blobs/scalability-targets

This is an additional consideration:

1. Use the Azure Pricing Calculator - https://azure.microsoft.com/en-us/pricing/calculator/) to estimate storage and transaction costs for your solution.
2. Review and adhere to regional compliance requirements for data residency and privacy. - https://learn.microsoft.com/en-us/compliance
3. Set up Azure Monitor to track performance and detect anomalies during and after migration. - https://learn.microsoft.com/en-us/azure/storage/blobs/monitor-blob-storage

Most of the links provided will give you more guides and details on each point.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.