This is quite a crucial phase, since once the model has been in development and its quality has been judged as satisfactory, the job is just half-done. This is because a good level of performance today does not necessarily imply a good level of performance in the days to come, and this can mainly come from two sources:

• Technical changes in the process-generating data employed from the model, with the following need to change the activity of data acquisition and data preparation due, for instance, to changes in the type of file provided or the frequency of value updates.
• Structural changes in the processes producing employed data, which leads to unreliable estimates produced from the model. A typical example is the introduction of a new law which states constraint on the use of cash, which will lead to a structural change in the values of cash withdrawn from ATMs, with a subsequent need to re-estimate our hypothetical fraud-detection model.

How do we overcome these potential problems? With a well-conceived maintenance plan. The plan will state which kind of ongoing analyses will be performed to monitor the level of performance of the model, and, depending on what kind of results appear, the model will be subject to reestimation or even redevelopment activities.

Reproduced previously is a typical flow chart of model maintenance activities:

• The process starts with initial activity of model estimation
• Following the estimation comes the exploitation of predefined ongoing monitoring analyses, which could be, for instance, a systematic evaluation of accuracy metrics
• Here comes a decision point—are the assessed performances satisfactory? 
• If yes, the cycle starts again from the ongoing monitoring analyses
• If no, a proper re-estimation or even re-development activity is needed, based on how bad the results were

Once the deployment plan is ready, as well as the model maintenance plan, all that will be left to do is to actually implement them and taste the feel of your data mining success.