Transformations and validations scenarios
This unit presents transformation and validation scenarios.
Validate data alignment and application stability
The business relies on external applications for payroll and tax-related features, which involves inbound and outbound business information. It is important that the data required for integration with the external applications is available and accurate so that the integrations can continue to function after the cutover.
Patterns
You’ll need to validate that the data that’s being migrated aligns with the target database schema and format. Additionally, you need to verify that the application is stable post migration. Consider the following best practices (patterns):
- Identify the dataset(s) that needs to be tested.
- Verify the field values are correct and mandatory fields that are required by the external application are not missing data.
- Perform reconciliation on the data, including such activities as confirming the record count, and if possible comparing report outputs.
- Validate data integrity by testing critical business scenarios.
Anti-patterns
Avoid the following bad practices (anti-patterns) when you’re validating data alignment and application stability:
- Starting data migration without a documented migration or environment strategy in place.
- Lack business user involvement in validation and testing.
Perform data transformation before loading data into Dynamics 365
The business uses different systems across its multiple geographical locations. Additionally, the company has been maintaining their customers in a disparate manner across the locales, partly due to the mergers and acquisitions that they have experienced in recent years. However, having customer information in different systems has made getting an overall picture of the business impossible. To gain better insight and control of the information, the company has decided to create a global customer base.
Patterns
The business should complete the following best practices when performing data transformation before loading data in Dynamics 365:
- Clean up data in legacy systems (source) as much as possible.
- Perform data transformation outside of Dynamics 365.
- Automate data transformation because data migration will be run multiple times during validation, and dry runs to avoid manual mistakes.
- Rectify the errors that are related to data quality to reduce the code churn for X++ validations and error processing for optimal performance.
- Define data dependencies (entity sequence, dependencies on configuration or customizations, and mapping between existing legacy systems and the target system).
- Identify redundant and obsolete data in the source system(s).
- Conduct data profiling to determine if the data needs to reside in a data warehouse.
- Confirm that this data is needed by the integrating systems for regular operations and not only for reporting purposes.
- Confirm that the data is not present elsewhere.
- Analyze business and compliance risks.
- Identify subject matter experts (SMEs) who will be responsible for the validation and documentation.
- Ensure that Azure DevOps work items are set up for tracking and documenting the data changes and for defect triage.
Anti-patterns
The business should consider the following anti-patterns when performing data transformation before loading data in Dynamics 365:
- Using X++ for translations or conversions
- Implementing X++ logic for validations
- Copying legacy data structures into Dynamics 365