As described earlier, upgrades require detailed planning and consideration—it is just like executing an implementation project. The following diagram represents the typical upgrade project phases:
Once you decide to upgrade, it's important to do detailed planning and analysis, just like what is done for the implementation phase before starting the project. An upgrade is like moving. You don't know how much stuff you have accumulated until you are moving. Similarly, an upgrade is when the people start realizing how many customizations they have made in Dynamics AX and in the external applications that are built around Dynamics AX, including reporting.
The following sections define the key areas to be considered when doing an upgrade project.
Most of the upgrade projects that I have reviewed had a common theme. The new version was implemented in the old way, that is, all the customizations from the previous version were ported on to the new version—as is.
As part of the analysis/planning, you need to spend a good amount of time finding a match for the existing custom features, and deprecate the custom features. Most likely, Microsoft may have developed a feature that you had to customize years ago. This is your opportunity to unlock the power of a new version, and maximize your investment in the Dynamics AX platform by tearing off customizations. While it may sound like a no brainer, many projects fail to do so as it needs additional work to migrate the existing data from custom tables to the standard ones. A few minor customizations that are in place may be needed on top of a standard feature if the standard one does not completely replace what you have. Hence, shortcuts are taken and all of the old custom code is ported into the newer version.
For every customization you are porting to the new version, you need to think it through as if you were going to build that feature from scratch. Assess whether it is worth customization, or is there any feature in Dynamics AX that can be leveraged to meet your needs. Otherwise, you will end up bringing over all the customizations from the previous version to the new version and ultimately running the new version in the backward compatibility mode. The business will not get much value out of such an upgrade.
Also, there are customizations that may not be used anymore. Those need to be identified and removed/left behind as part of the upgrade project. Plan what customization can be replaced or reimplemented with new version. You should do a detailed Fit/Gap analysis for new features and see how these features can benefit the business, and if there are any gaps. Plan if they can be addressed during or after the upgrade. Fit/Gap analysis is also relevant for custom features in the old system—compare these features with any equivalent features in the new version. Identify the gaps and plan to address them as part of the upgrade. Sometimes, such a list of features can be overwhelming and may add significant scope to the upgrade project. Read the following section for ideas and experiences on managing the scope.
What functionality is available in the version you are upgrading to versus what is in the scope for the upgrade project is the issue that needs to be decided.
In my experience, delivering the current functionality available to the business is the first step towards the new platform. No one would like to go backwards on the features they already had. However, you should negotiate for not implementing brand new features as part of the upgrade project itself. These new features can come as a next step after the upgrade, even if these features are available out of the box in the new version of Dynamics AX. You need to consider the time needed for the implementation of such features and the impact to the overall timeline of the upgrade. I would rather run parallel projects to implement several new features once I have moved to the latest version, than delaying the upgrade itself. This approach allows you to divide the scope into smaller projects for the new features and is easier to manage as well.
At the same time, you may consider enhancement to the existing features to be included as part of the upgrade project. To take an example, on one of the upgrade projects from AX 4.0 to AX 2009, the client wanted to add another financial dimension. This was a very complex Dynamics AX environment, where the sales orders were received from many sources, several applications were integrated with Dynamics AX, and very few people knew the end-to-end processes. Due to the overall complexity of the environment and lack of visibility to the impact, the team did not want to make that change as part of the upgrade project.
We were engaged by the finance team as this was a critical item for them. Upon 4-5 weeks of analysis of all the sales order transactions from several sources, we came up with a list of changes that had to be made in Dynamics AX and other applications/integrations. It ended by taking up only 40 hours of development team's effort. Doing it with the upgrade was a big win for finance as they did not have to retest the entire application (the changes were tested along with rest of the upgrade testing). Another business challenge was addressed with the upgrade without much scope creep. It also helped in getting the finance team actively involved on the project (until then, they did not have much incentive to participate and looked at it as a technical upgrade). The bottom line is, you need to have a thorough analysis done for deciding on what to tackle with the upgrade, and it can help you address key strategic initiatives.
Many times, the users want everything that they are using currently to be included as-it-is. However, somebody needs to take stock of all the areas that are being utilized and put together the use cases of functionality that are being used. Otherwise, you would have a testing nightmare and also run the risk of porting over the old functionality, as noted earlier.
Evaluate the data in terms of quality and volume. Clean the data in the source system if possible. Dynamics AX 2012 provides an upgrade-readiness check tool that can evaluate your data and suggest clean-up activities that you should undertake before upgrading to the new version.
Consider purging or archiving the source data from the production dynamics AX environment to minimize the data upgrade time of the production environment. The Microsoft IDMF (Intelligent Data Management Framework) tool can be utilized to purge or archive data.
This would help reduce the time needed for each data upgrade iteration and the downtime needed for performing a production upgrade apart from improving the data quality in the next version.
Many times, upgrade projects are branded as a technology upgrade. For the very same reason, it's hard to get the business engaged in the upgrade projects. That might be true to some extent when you are doing a minor upgrade such as upgrading from AX 2012 R3 CU8 to AX 2012 R3 CU9. For major version upgrades like AX 2009 to AX 2012, business engagement is critical. You will need business agreement on various decisions, such as identifying unused features or customization, Fit/Gap analysis of custom features and the new features, defining the scope, data archival and purging, training, UAT, and so on.
You may have a lot of other applications in your ecosystem that are integrated with Microsoft Dynamics AX. An upgrade can impact those integrations for either the underlying technology changes or schema changes. For example, Microsoft has removed the business connector from AX 7. So if you are upgrading to AX 2012, you should redo your integration using services or other integration framework available. Any integrations built using AIF may also have to be changed if the underlying schema is changed.
How does the upgrade impact your reporting and BI solution? Do you have replicated data or data marts for ad hoc and external reporting? New versions may have significant schema change, which may require significant changes in all the reports and BI solution. Dynamics AX 2012 had huge changes in the data schema as compared to AX 2009; the tables were highly normalized, and several standard features were reimplemented from scratch.
Even small changes in the schema can significantly impact the reporting solutions. For example, one of the customers upgrading from AX 4.0 to AX 2009 had to update hundreds of reports written out of a replicated data source due to changes in the CreatedDateTime and ModifiedDateTime fields in AX 2009. Analyze all your reports and the ad hoc SQL queries that the business team uses to gather data, and plan to upgrade them. Another impact you may have to consider is the replicated data itself. If you are replicating tables from the Dynamics AX database, these tables may have been changed (new fields may have been added or deleted), or sometimes, the table itself may have been replaced with a new set of tables. These replicated data sources need to be rebuilt to support the new schema. One of my customers was feeding LedgerTrans data into their corporate data warehouse. Many processes and KPIs were dependent on this general ledger data. Removal of the LedgerTrans table in AX 2012 added a good amount of rework to the ETL process feeding the data warehouse.
Upgrade projects can take several months to year(s) for customers having many customizations and /or integrations. The customers also have continuous improvements projects and several business initiatives that require changes in their current Dynamics AX application. It is important to understand that these other projects may have to stop for a period of time for a smooth code upgrade, testing, data upgrade process, and avoiding rework in both the projects. It would also help in getting all the IT and business resources aligned for delivery of the upgrade project.
You may be able to make continuous improvements during the upgrade planning and analysis phase, but once the code upgrade activity is started, you should freeze the code of your current AX environment. Set the expectations with your business team for the code freeze start date and its impact on the existing projects that are in-flight, or have an approved budget. Prioritize the critical issues and fixes which need to be completed in the current AX implementation before starting the code upgrade project.
If any critical issues surface due to business priorities that cannot wait or for regulatory reasons, ensure that the coding changes are deployed in both environments (the current Dynamics AX Production and the code upgrade environment for the Dynamics AX upgrade project).
You may not realize it, but upgrade projects will often also require new infrastructure, unless it is a small implementation with a fairly small database size or if you are planning for an in-place upgrade. You should be able repurpose some of your old infrastructure once the upgrade project is complete. You will need additional hardware for the following purposes:
- The development environment: You will need a new development environment for code upgrade and other development activities. Most of the time, it is possible to install and run different versions of Dynamics AX in a single box; but that can also create performance issues or other inefficiencies, and it is advisable to have a separate environment.
- Test environments: You must not disrupt your current test environment to test any critical production issues and hotfixes when the upgrade project is on. Hence, you will need a separate test environment to test the code upgrades.
- Test data upgrade: You will need a production-like environment to test your test data upgrade process. Many customers utilize their future production environment for the purpose of a test data upgrade.
- The production environment: In typical upgrade projects, your old system is used until the final cutover. It is similar to an implementation project where you are replacing your old legacy system. All the deployment activities will move to the new infrastructure without disrupting your old system. You may also want to keep your old system intact as part of the rollback plan, so that the business can continue using the old system if anything goes wrong with deployment. You will also need additional storage and horsepower requirement for upgrading the data during the upgrade period.
Microsoft provides an upgrade analysis tool through Lifecycle Services to help the customers and vendors in planning an upgrade to Dynamics AX 2012. Upgrade analysis uses the Rapid Data Collector (RDC) tool to analyze information about the existing environment and helps in estimating the scale of the upgrade project. The following chart illustrates how the service works for both full-version upgrades and in-place upgrades:
The upgrade analysis tool analyses the source version code artefacts (AOD files or model store files) and key metadata information, such as count of records in the tables to create reports to identify the scale of the code upgrade.
Upgrade analysis creates an overview report in HTML, and a detailed report as a Microsoft Excel file that you can download and review. The following are the tabs in the Excel reports:
|
Tabs |
Description |
Useful for |
|---|---|---|
|
Upgrade summary |
The summary list is a list of objects impacted in each module | |
|
Global tables |
List of shared tables |
Data upgrade, code upgrade |
|
Customization statistics |
List of all customizations |
Code upgrade |
|
Customization view |
Number of customized tables and classes |
Data upgrade |
|
Modified objects |
List of modified objects |
Code upgrade |
|
Modified object details |
List of what is modified in each object |
Code upgrade |
|
Domain information |
Lists companies, domains, and related users |
Security upgrade |
|
Table statistics |
Lists table size, properties, and counts of rows and columns |
Data upgrade, minimizing downtime |
|
Parameters |
Data upgrade, code upgrade | |
|
Tables without |
List of tables that do not have a |
Data upgrade |
|
|
Lists the usage patterns of MSDAX objects |
Code upgrade |
The code upgrade in Dynamics AX is the process used to port customizations from the source environment to the target environment. The following are some key considerations to be kept in mind during the code upgrade process:
Code upgrades are different in different projects and largely depend on the level of customization and the changes that Microsoft made between the source and target version. Depending on your project, you should consider the following when doing a code upgrade.
Often, we find customizations in Dynamics AX which are not used. There can be various reasons for not using the customizations, such as change in the business process, development of a new advanced feature while the old version still existed, and so on. An upgrade project is a good time to clean up such customizations. Identify all such customizations and plan to remove them. Questions may be raised on the feasibility of investing time and money in removing the customization: what benefit do we gain by this clean up? Firstly, if you don't clean up old customizations, you might have to upgrade them as per the new version. This may introduce bugs and you will have to upgrade them again with the future upgrades; the obsolete code adds to the cost of each new upgrade. Having less code can also reduce your application-compile time and other application maintenance features.
In some cases, you might have code which can be replaced partially or completely with the new version code or features. You should identify such features and utilize the new features or code instead of upgrading the old ones.
Sometimes, a customer or partner might have added standalone code like creating new forms, classes, tables, and reports. Test these to make sure that the code compiles on the new version. It is also recommended to upgrade the UI and code patterns to utilize the latest features of the new version.
In some cases where Microsoft has changed the features, and the changes impact your customization, you will probably get compilation or runtime errors in your customization. You have to identify such areas and refactor your code to utilize the new code pattern.
The following diagram illustrates the typical code upgrade process:
The very first step of the code upgrade process is to create a baseline database to store a read-only copy of the Microsoft Dynamics AX 4.0 or Microsoft Dynamics AX 2009 code. The baseline database is used for reference purposes during your code upgrade.
The next step of the code upgrade process is selecting the appropriate code upgrade checklist. To get the code upgrade checklist, you must select Register database for Upgrade Mode, when you install Dynamics AX. When you open the Dynamics AX client the first time for the upgrade, you will be presented with some options, as shown in the following screenshot:
The next step is to import the license file and all the AOD or model files in the baseline database, as shown in the following screenshot:
You should import layers from the lowest to the highest. For example, SYS first, then SYP, and so on, until you reach the last Microsoft AOD file.
The final step of the code upgrade process is to complete a code upgrade checklist for each of the layers that you are upgrading. Dynamics AX provides you with multiple layers to build a custom code. For example, CUS, BUS, USR, and VAR are generally used for customizations and ISV solutions. For more information on layers, refer to Chapter 9, Building Customizations. It is important that you start at the lowest layer (for example, ISV). After the lowest layer is complete, start on the next layer up. Perform this task sequentially, until all the layers are upgraded.
The main goal of the code upgrade checklist is to detect code conflicts and to resolve those conflicts.
The following screenshot shows the code upgrade checklist for AOD files:
Microsoft Dynamics AX 2012 includes a tool to detect code conflicts. This tool analyses your customizations and creates a project (a project is a placeholder in AOT to group multiple objects, related to a specific functionality, together) that contains the application objects with conflicts. This tool is used when you upgrade from Microsoft Dynamics AX 4.0 or from Microsoft Dynamics AX 2009.
Conflict objects are code objects that have been both changed in the new release and customized in your application:
In some cases, especially on the forms, if the customizations are large, you may need to manually apply changes to the form to upgrade it.
The other alternatives for identifying code conflicts during the code upgrade project are the following:
- The project filter tool
- The code compare tool
The project filter tool can be used by a developer to create a project based on the criteria supplied in the query form. Such criteria could all be objects from a relevant layer or all objects that have a specific prefix, for example. Using the project filter, a developer can identify and group all the customized projects into a single project. The developer can later use the code compare tool to identify the code difference between the various layers and resolve them manually.
In some cases where you are planning to replace the customization with standard AX features or refactoring a customization to work with a new version, you may have to write a data upgrade script to support the changes.
For more information, refer to the Microsoft Dynamics AX 2012 white paper—How to Write Data Upgrade Scripts for Microsoft Dynamics AX 2012—at http://www.microsoft.com/en-us/download/details.aspx?id=16375.
If you are upgrading from AX 2009 or older versions to Dynamics AX 2012, you may need the following type of upgrade script for your custom features:
- Readiness checks: It is very important to write a data-readiness check script for AX 2012 for the custom tables where ledger account/ financial dimensions, address, or inventory dimensions are used. The readiness-check validation script checks if all the related data exists in the source system.
- Preprocessing and delta scripts: The Dynamics AX 2012 upgrade provides the ability to preprocess the application data in the source system using preprocessing and delta scripts. Shadow tables are created to map any new fields and assign key relations to the standard tables.
- Single-user steps and all target-side operations: The target-side operation is the final step of the data upgrade process. Data upgrade scripts on the target side must include all the tables and field mapping information.
The security framework in Dynamics AX 2012 has changed entirely since AX 4.0 and AX 2009. There is no automatic path to the upgrade of security configurations from the earlier versions to Dynamics AX 2012. However, Microsoft provides a security upgrade advisor tool, which can be used to help simplify the process of upgrading the security settings from the earlier versions to Microsoft Dynamics AX 2012.
The security upgrade advisor tool compares the user group access rights and roles in the current system to privilege mapping in AX 2012, and generates a list of matching privileges that can be used for a particular role. The following diagram shows the steps to upgrade security settings:
It is important to note that this tool is designed to help in upgrading security and developers are advised to review each suggestion carefully before making the final changes. For more details on the security upgrade tool, follow the Microsoft TechNet article at https://technet.microsoft.com/en-us/library/hh394895.aspx.
Data upgrade is the most important process of an upgrade project. As described earlier, the data upgrade process can differ, depending on the source version of Dynamics AX. Microsoft provides various upgrade checklists and scripts to convert table data from the source system to the target system. Data upgrade is, basically, executing the required checklists and the scripts necessary to transform and move data from the source system to the target system.
Testing the data upgrade is, basically, running the data upgrade processes on the copy of production data in a test environment. The key to having an optimal data upgrade experience is to plan well in advance, run multiple test cycles building on the lessons learned, and then plan the live data upgrade in complete detail, building in time for unexpected, last minute issues.
The following diagram shows a typical data test data upgrade process:
The following are the key objectives of the test data upgrade:
- Testing of the data upgrade scripts
- Identifying potential issues/bottlenecks in the data upgrade scripts
- Ensuring data integrity and completeness of the upgraded data
- Identifying the time required for each activity, and calculating the final downtime
- Preparing the data for system and regression testing
- Planning for the final data upgrade in the production environment
The following are the key objectives to plan the test data upgrade:
- Make sure that the code upgrade and upgrade scripts are completed before you start your test data upgrade.
- Identify an acceptable downtime window of the production data upgrade. Talk to the business team and understand for how long the production system can be down, without impacting the operations significantly.
- Plan multiple rounds of the data upgrade cycle to test all the data upgrade steps consistently and to identify and tune the approximate downtime needed for the upgrade process in a live environment.
- Plan for an evaluation and optimization window with each test data upgrade. The objective is to reach an acceptable downtime window, agreed upon with the business team. More effort will be required to reduce the downtime window.
- Proper database sizing for the target database and the
TempDBdatabase is also extremely important to avoid any possibility of database resizing during the bulk-copy phase. Determine the correct sizing for the target Microsoft Dynamics AX 2012 database, and set it before you begin to upgrade. A rough estimate to use as a starting point is 30 percent larger than the expanded size of the source database. A rough estimate for sizing yourTempDBdatabase is 20–25 percent of the expanded Microsoft Dynamics AX 2012 database. Optimal performance may require splitting theTempDBdatabase into separate files. - Plan to use the production environment or hardware specification close to the production environment.
- Use a copy of production data for testing the data upgrade process. Try to use the latest copy of production with each round of the data upgrade testing.
The following are the key objectives to execute the test data upgrade:
- Follow the Microsoft recommendations for database and AOS configuration for the upgrade. There are many configurations which need to be set specifically during the data upgrade process. Follow the Microsoft white paper—Data Upgrade Best Practices—which can be found at http://www.microsoft.com/en-us/download/confirmation.aspx?id=28701.
- Take backups or database snapshots before each step of the upgrade process. Backups and snapshots can improve efficiency in case of failure during a particular step, so you don't have start the process from the beginning. It is also recommended to take a backup or snapshot during the final, live upgrade, in case of unexpected errors (out of disk space, network failures, and so on).
- Monitor the performance during the data upgrade process by using performance tools such as DynamicsPerf and the Windows Performance Monitor. Evaluate long-running queries and bad execution plans during batch processes and identify the areas where index tuning or code changes are needed. In some cases, an index created on the fly, within the test environment, can provide immediate benefit to a long-running process.
- Set the database recovery mode to simple during the data upgrade. At the conclusion of all upgrade activity, set the recovery mode to full.
- Increase the max degree of parallelism setting for the target upgrade processing (in the single-user mode, the bulk-copy process gains a significant performance benefit with parallel query processing). After target upgrade processing, return max degree of parallelism to the default setting.
- If the source database and target database are on separate servers, network latency and performance are also important factors that should be monitored. In some tests, multi-server environments with no network latency issues have shown a 30 percent slowdown over a single-server environment. Extreme slowdowns are possible if there are latency issues as well to deal with.
The following is the outcome of the test data upgrade:
- Document the activities performed during the test upgrade process and record the execution time for each step.
- Identify and document the performance issues encountered and solutions for tuning.
- Validate the outcome of the data upgrade—validate completeness and data consistency after each data upgrade round. The upgraded data can be used for system and regression testing and training phases.
Like any other project, testing is important for an upgrade project. However, the nature of the testing will be different in an upgrade project. The following sections define the key areas to test in an upgrade project.
When upgrading using the source-to-target upgrade model, it is very important to validate the upgraded data for completeness and data integrity. SQL scripts can be used to compare the number of rows between the source and target databases. Assemble a test team to plan high-level data validation for configuration as well as transaction data between the source and target systems.
It's important to perform, system and regression testing after the code and data upgrade to identify the issues, if any. It is recommended to use the upgraded data during the test upgrade cycle for system and regression testing.
There may be features or customizations which are re-implemented or upgraded to the target version. Proper system and regression testing should be done to ensure that the features are working as expected.
Test that the security roles implemented in the target system are working correctly. Users in Microsoft Dynamics AX 2012 should have access to the data they had access to on the source system, and they should be able to perform all the functions which they were able to perform in the source system.
Test whether the upgraded and standard reports are working as expected in the new system.
As described in the earlier sections, your upgrade project may result in changes in integration, reporting, and so on. Testing of the upgrade must cover any such areas which are impacted. Depending on the impacted applications and changes, you may need to involve other departments in your organization or third-party application teams to test end-to-end scenarios.
Dynamics AX 2012 has a significantly different user interface than the previous versions, Dynamics AX 4.0 and AX 2009. It is important to train the end users in the new system for the new user interface as well as the new and changed features. The following are the key considerations for end-user training.
- In the upgrade planning phase, make sure that training is provided often, both, early in the process and after the upgrade
- Provide early access to key business users after the code and data upgrade to familiarize them with new system
Just like an implementation project, it is very important to plan the final upgrade on the live system. All activities which need to be performed during the go-live should be well documented, including the test data upgrade steps.
Final deployment will include several activities such as stopping transaction processing in the old system, stopping any integration points to the old system, deploying the code, final data upgrade, data validation, and so on. The following table shows a sample plan for the final upgrade process:
|
Step |
Description |
Type |
Owner |
Estimated time |
Actual start time |
Actual end time |
|---|---|---|---|---|---|---|
|
Target system | ||||||
|
1 |
Build target system – create database, validate storage settings, and so on. |
Pre-release |
Steve | |||
|
2 |
Deploy the latest code build on the target system |
Pre-release |
Steve | |||
|
Source system | ||||||
|
3 |
Preprocessing start |
Communication | ||||
|
4 |
Run preprocessing jobs in the source system |
Release |
Steve | |||
|
5 |
Validate preprocessing |
Validation |
Steve | |||
|
6 |
Run final preprocessing |
Release |
Steve | |||
|
7 |
Send final communication for system downtime |
Communication |
James | |||
|
8 |
Shut down user access, stop all the batch processes, stop AOS |
Release |
George | |||
|
9 |
Take database backup |
Release |
Peter | |||
|
10 |
Set to single-user mode |
Release |
George | |||
|
Target system | ||||||
|
11 |
Connect to the source database |
Release |
George | |||
|
12 |
Presynchronize |
Release |
George | |||
|
13 |
Release |
George | ||||
|
14 |
Generate table mappings |
Release |
George | |||
|
15 |
Release |
Peter | ||||
|
16 |
Generate upgrade task prioritization |
Release |
George | |||
|
17 |
Set batch threads to twice the number of cores. For example, with 8 cores, set the number of batch threads to 16. |
Release |
George | |||
|
18 |
Start the data upgrade |
Release |
George | |||
|
19 |
Communication – data upgrade completed |
Communication |
James | |||
|
20 |
Run data validation scripts |
Validation |
Peter | |||
|
21 |
Go/no go |
Decision |
Steering committee | |||
|
22 |
Deploy reports |
Release |
John | |||
|
23 |
Deploy integration solution |
Release |
John | |||
|
24 |
Restart all AOS |
Release |
George | |||
|
25 |
Run IT validation |
Validation |
Joe | |||
|
26 |
Run business validation |
Validation |
Tina | |||
|
27 |
Go/no go |
Decision |
Steering committee | |||
|
28 |
Set Database recovery mode to full |
Post release |
Peter | |||
|
29 |
Set max degree of parallelism to 1 |
Post release |
Peter | |||
|
30 |
Communication-go live |
Communication |
James |
There may be many activities which need to be done after the upgrade. The following are a few activities which may be applicable:
- After the upgrade, the indexes in the Microsoft Dynamics AX 2012 database will be highly fragmented. Before you start with the normal processing activities, it is strongly recommended to rebuild the indexes.
- Validate and reset the database settings, such as the max degree of parallelism; set the database recovery mode to simple.
- Rebuild and process database replication and additional reporting and BI solutions.
- Plan and upgrade the DR application code and data, if applicable.
- Monitor applications for performance and take corrective actions.