Chapter 11: The Real Migration Experience

This is the final chapter of the book. Let's put the technical details, methods, and setups aside and just talk about the real usage of the Cloud Migration tool. I'm proud that I was one of the first users of this tool to implement it in a real project. I want to tell you my story. I hope that you will find it interesting and it will inspire you to not be afraid of new technologies, even if they seem quite raw.

In this chapter, we are going to cover the following main topics:  

• Cloud Migration – early beginnings and evolution
• Issues in the course of implementation

Cloud Migration – early beginnings and evolution

The first time I heard about Cloud Migration was after the Business Central 2018 October release. It was referred to by the name Intelligent Edge. As I understood from different sources, the tool allowed you to replicate your data from NAV 2018, Business Central on-premises, and Dynamics GP to Business Central SaaS, but the table schema strictly had to be the same. Migration failed if the tables had even one different field. This was not very interesting to me, because each project I worked on had some customizations, which made such migrations impossible. And so I forgot about it (for several months at least).

Reference

You can see the old presentation here: https://docs.microsoft.com/en-us/business-applications-release-notes/october18/dynamics365-business-central/dynamics-intelligent-edge.

Before the Business Central 2019 April release was presented, Microsoft said that Intelligent Edge had been upgraded and the strict schema requirement was no longer the case. This was a big step forward, but without a real usage scenario, it could have ended up being nothing more than a demo for me.

In April 2019, just after the new Business Central release, I came into contact with a new customer. They had headquarters in Europe where they used Business Central SaaS and a branch office in Russia where they used the Business Central 2018 October release on-premises. The task was: we need a copy of all main tables that are on-premises in the cloud environment, because we want to consolidate data there.

Which tables were the main ones? How often would we need to replicate data? And how many APIs would I need for that? Stunning. This task looked like a big project, until I remembered Intelligent Edge.

Our solution was based on a non-standard tool usage scenario for two reasons:

• We had planned to use scheduled replication each day instead of one-time migration.
• We had planned to migrate to different localizations, which was not officially supported.

We offered this tool to the customer with the following pros and cons:

• Pros: Implementation time would be very fast and costs would be much lower than migrating table by table.
• Cons: It is a black box. There is a lack of information about real usage and the tool is quite new. There could be some issues during the implementation.

So, we decided to run a demo environment and test it. Intelligent Edge had been renamed Intelligent Cloud. We installed the Business Central 2018 October release on-premises with Russian localization and replicated it without errors to the Business Central SaaS 2019 April release with Great Britain localization. This was amazing because the tables' schemas were different.

This is a screenshot from the old setup. As you can see, it has a common version of Business Central and a separate Dynamics NAV 2018 product:

Figure 11.1 – Intelligent Cloud setup

Around that time, I wrote my first blog post about Intelligent Cloud setup. It seems to hold up, despite all the changes in the Cloud Migration tool. If you are interested, you can find it here: https://community.dynamics.com/business/b/andreysnavblog/posts/intelligent-cloud-setup-for-business-central-spring-release.

After the successful demo, we presented the solution to the customer. At this stage, we were interested to see how different localizations could be replicated. However, this was no miracle tool, and all tables and fields specialized for Russian localization had been missed. This was expected and the customer said that these tables were not the main ones and other concerns were about per-database tables, which could be overwritten, but here we did not meet any issues. In addition, we had a possibility to migrate data to the sandbox always, because it was planned to use it only for analysis as a read-only company.

We decided to accept all the risks and start a project with Intelligent Cloud.

Issues in the course of implementation

I created a copy of the customer's production database and started replication.

First issue

The first issue I met was that on-premises databases were in Azure SQL. From the migration admin's point of view, it looked perfect. You don't need to install Integration Runtime and setup runs faster. We were happy until the migration finished and we opened the migrated company in the cloud. The source database had a Cyrillic collation and the destination database had Latin. All Cyrillic text transformed into ??? characters, and after communicating with different people, I realized that there is no way to migrate different collations this way. We had to migrate to the on-premises SQL Server first. I know that this problem still exists because I've heard the same problem raised by colleagues at conferences.

After the migration to SQL Server on-premises, replication finished successfully and no more issues occurred. We decided to start our go-live.

Second issue

The second issue we met was after our SaaS environment got the 2019 October release update. The customer wrote that replication stopped and wouldn't run again. I decided to run the setup again. In fact, this is a good first-aid tip when something suddenly goes wrong with the replication process.

A surprise was waiting for me when I tried to select a product to migrate from. There were no earlier Dynamics 365 Business Central versions, just the current one. I tried to find the app in the application market but did not succeed. It supported only NAV 2018 and the current Business Central version. When I tried to use the current version, I got this message:

Figure 11.2 – Migration warning

I raised a support ticket and got an answer that earlier versions would be back after some fixes. In 1 week, we were able to migrate again. After that, I was very careful with major updates. This situation also repeated with 2020 release wave 1, but I postponed the SaaS upgrade until Intelligent Cloud was upgraded (which happened 2 weeks later).

Third issue

After some minor updates, we ran into a third issue – scheduled replication stopped just after the update. Microsoft explained that an upgrade with working replication could damage data and replication must be stopped before the upgrade. This rule works for now. If you schedule an environment upgrade and do not disable Cloud Migration, the upgrade will fail and re-schedule.

This did not look like a serious issue in the beginning, because running the migration setup when you have everything installed is quite fast. However, to reduce the amount of migrating companies before the upgrade, Microsoft removed the Scheduled run feature from Cloud Migration. This was a game-changer for us. I wrote a ticket but got the answer that it is disabled almost forever.

These are screenshots from the past when it was possible to create scheduled migration runs:

Figure 11.3 – Setting up a migration schedule

Figure 11.4 – Next scheduled run on the Migration Information tab

Figure 11.5 – Microsoft Docs tip about scheduled migration

After a discussion with the customer, we decided to run the migration manually, before they needed fresh numbers in the cloud. The database was not very big and the migration time was about 15 minutes, so it was not a problem.

Since 2022 release wave 1, it is possible to run the migration through an API call, so you can create scheduled runs again by yourself.

Extra issue

After hearing the experience of my colleagues, I want to share with you one interesting issue. When you create records with Excel import, such as customers or vendors, in an on-premises environment, you can get non-printing characters in the code key fields. They will migrate to the SaaS and you can get some issues when using the Record.Get() function there.

To prevent this situation, run the following command in the Business Central administration shell:

$result = Invoke-NAVSanitizeField -ServerInstance YOURSERVERINSTANCE -Tenant YOURTENANT

This command removes all non-printing characters from all fields of the code type in the tenant database and returns a list of modified records and tables.

Figure 11.6 – SanitizeField command

You can find more details here: https://demiliani.com/2021/12/14/dynamics-365-business-central-are-you-doing-cloud-data-migration-please-sanitize-your-database-before-doing-it.

Summary

To conclude, I can say that the project was definitely worth it. We dived deep into a new technology, which worked as expected, despite issues that occurred during the exploitation. The customer got exactly what they wanted and saved money. We got a huge amount of experience in using Business Central without writing a single line of code.

I hope that this story inspired you to use non-standard ways of task resolution. Business Central developers must not be limited to using only Business Central. They must know how to use as many Microsoft products as they can. Azure, Dataverse, Cloud Migration, Power BI – they all help you to complete some tasks much faster than you would be able to by writing a hundred lines of AL code.

In this book, I tried to collect everything that a Business Central specialist needs for the administration or migration of an environment to the cloud. I will be happy if someday I see this book on somebody's table. Thanks for having me! Good luck, colleagues, and I hope that this book is useful to you.