There is no single technique in managing data which can be leveraged all the time in a typical Dynamics 365 implementation. Discipline, ownership, and a process for master data governance are critical success factors for the sustainability of a system. Data management is not a one-time affair and, hence, should always be closely monitored, optimized, and executed as per plan. Extract, Transform, and Load (ETL) is amongst the most common approaches in data migration planning, and you will be using it in one way or the other, no matter which solution/application is in focus.

The following steps are involved in ETL technique:

• Identify all the source systems as per the data migration requirements
• Build data templates to extract information from the source system:
  • When volumes are high, you can leverage an SQL database as a common repository to extract the information
  • For smaller data and configurations, you can directly leverage Excel as the mechanism
  • For larger data sets, you should leverage the data entity framework
• Prepare for data export from the source system into the staging places
• Perform data cleansing and validation activities:
  • System validations and automation should be leveraged wherever you can generalize a rule for validation and cleansing, and use it to make the staging data in a format that can be imported to the target system
  • When some human decision is involved, then introduce manual checkpoints for data validations in the staging system, for example, mandatory data, data types, data length, and so on.
  • Leverage the tools available in the Dynamics 365 solution to import data

Let's now learn the select data mapping and transformation considerations:

• Cleanest data: If the data is stored at multiple places in a legacy system, you should pick the cleanest one to extract a copy from. Consider the update timings in the source and add dependencies in the go-live plan to get the source data updated, prior to starting the extraction.
• Business rules in transformation: Define and validate the field mapping between the legacy systems and Dynamics 365 for Finance and Operations, Enterprise edition (AX) along with any transformations that need to happen between the extraction and the import process. Define rules in the target system or in the source systems (for example, bad addresses, phone numbers, and so on) to enable automation and transformation as much as possible.

Leveraging the aforementioned techniques, let's now evaluate the various tools and see how we can benefit from them.