Rightsizing the scope is amongst the first steps towards a successful initiative. The scope must be well-documented and agreed upon by all the relevant stakeholders to ensure smooth and accurate data migration.

Often, implementation teams either do not consider data migration at all or have unreasonable expectations regarding the data migration requirements.

Even if the original sales proposal has explicit data migration requirements that have been identified, many of the project team members and stakeholders may not be aware of what was specified or may not agree to the mentioned scope.

We recommend that you always have a dedicated data scoping activity planned, which should give an opportunity to both the sides to be on the same page and leave very little room for any assumptions. This scoping activity can be achieved by checking on the following set of questions:

• What all is needed to keep the business running efficiently?

Define the business goals with this question in mind, and then approach the issue of what information needs to be migrated to meet these goals or what solutions can be provided to meet the goals without migrating the data. For example, a customer may say, I need to be able to collect my receivables and run aging for customers—this is the business goal. This means that you only need to migrate Open AR for the customers along with the due date.

• Is there an alternative to bringing the existing data over? Reporting out-of-legacy systems or data warehouses and defining a manual process, if it is going to be used only for inquiries over a short period of time, are some potential alternatives.
• How much of the data present in the legacy systems is up to date, validated, and good for future use?

Do you want the new system to have the same issues that you are trying to solve in the current system?

• How many records are involved or what is the volume of data migration to accurately pinpoint the tool and technique for loading?

Ensure that the ballpark numbers of record counts are defined for each area during scoping (for example, 4 million products, 200,000 customers, 2000 open orders, and so on). This will help you select the right tools.

• How often will you be asked to retrieve this data?

Usually, the activity of configuration and data migration is an iterative exercise, and hence, the need to leverage/build reusable capabilities to address this iterative nature.

Identify and document the business needs clearly and accurately with examples where possible. You can avoid the cascading effect and carve out the critical pieces of data that you need frequently in order to limit the scope. For example, just migrating the open balance of each customer invoice rather than trying to bring the complete line item detail requires less effort. If a customer service needs the invoice line detail to research a customer issue that happens once a month on an average, the detail generally would not be worth the effort of trying to migrate it.

Part of data migration planning process involves educating business stakeholders about cost of migration and focus on migrating information which would enable better business decisions, better servicing of customers and information insight. In principle, one should avoid migrating historical transactions, such as posted sales invoices, posted purchase orders, individual general ledger transactions, inventory transactions history, and so on.

The effort to clean and transform data is a humongous task and an expensive proposition. Certainly, historical transactions are needed for various purposes such as compliance, regulatory requirements, business analysis, customer support, and so on. However, there are other solutions available as alternatives for migrating all the legacy data. These solutions/tools can be selected based on the size of the data set, transformation requirements, and storage and access needs. Here are some common tools that we have used for our customers for historical transaction insights:

• Leverage an existing or new data warehouse to meet the reporting/analysis requirements, and extract and store the historical information from the legacy system into Cubes or SQL tabular format.
• Storing data on the cloud, such as Azure SQL, and then showing them in reports (SSRS).
• A shared folder or a SharePoint site to store the extracted files from the source system in various formats, such as Excel, CSV, PDF, and so on.
• Set the security of the legacy system to read-only and do historical lookups there. Make sure that support contracts and an exit strategy are part of any discussion regarding this option so that the customer is not paying for multiple systems indefinitely. This is a good option for a stable legacy system where support is still available (without paying a hefty annual support price) and also helps ease the transaction for the legacy system support vendor.

If none of the preceding fits into your solution design and customer requirements, then you can consider creating a new SQL database as an exact replica of the legacy data tables and pull in the data without having to do a mapping or cleansing process. As part of the business intelligence and analytics planning, you should factor in this new SQL database along with your new ERP database to combine and deliver reports involving historical transactions.