Małgorzata Mitoraj-Jaroszek
This case was prepared to serve as a basis for discussion rather than an illustration of either effective or ineffective administrative and management practices. Names, dates, places, and data may have been disguised at the request of the author or organization.
Abstract
This case study is a description of an ROI project based on sales training targeted at retail sales assistants in a chain of shoe stores in Poland. A trend analysis was used as a method of isolating the effects of training. The sales training brought overall positive results.
McArthur Sp. z.o.o. is a medium-sized trading company, with a network of several shoe shops in large shopping centers in Poland. The senior management felt that the current level of sales was unsatisfactory. In addition to implementing a sales training program, the company conducted a series of mystery shopper surveys to assess the level of customer service and found the results to be very negative.
It was acknowledged that there was an obvious sales training need for store managers, assistant store managers, and sales assistants. The sales training was conducted in nine stores and included the following topic areas: customer service standards, making contact with prospective customers, diagnosing customer needs, offering a solution (i.e., making a shoe presentation), presenting benefits, handling objectives, and closing the sale. In addition, to support and encourage the development of selling skills, managers and assistant store managers took part in tailor-made coaching training. In all, there were 45 participants in the training. Fortnightly, managers were asked to complete a competency assessment sheet for each of their staff members, and send it to the head office for verification by the sales and marketing director. After completing the observation sheet, the managers were asked to conduct coaching meetings with each of their staff members. The whole coaching process took three months.
The guidelines from the senior management were very clear, the aim was to raise the level of sales. To measure the effectiveness of the challenge, the ROI Methodology from the ROI Institute was applied. The process consisted of six areas of measurement: reaction and planned action, learning, application and implementation, business impact, ROI, and intangible measures.
The process includes a comprehensive plan for data collection and analysis, as shown in Figure 12-1. It contains steps leading up to the calculation of ROI, starting from evaluation planning and including types of information needed at each level of evaluation. To complete the process at Level 4, an analysis of intangible benefits is also carried out.
One of the important elements of the ROI evaluation is evaluation planning. Two key documents were used: the data collection plan and the ROI analysis plan.
Data Collection Plan
Figure 12-2 shows a completed data collection plan for this project, including all stages of the planning process and data collection. The objectives at Level 1 were identified as: participants indicate a positive response to the training program and complete an action plan at the end of training. The satisfaction level for this objective was a rating between 4.0 and 5.0 on a 5-point scale.
The objectives at Level 2 aimed to check the level of sales skills acquired during the training. At the end of the training, the trainer assessed all participants by a means of a pass or fail aptitude test focusing on the skills practiced. Over 80 percent of the participants passed the test.
At Level 3, two means of evaluation were implemented. The sales assistants were evaluated every two weeks by their managers or by the assistant store managers with a yes-or-no observation checklist for a period of three months. The satisfaction level was set at the implementation level of 80 percent of learned skills in the sales assistants’ daily work. In addition, information regarding the implementation of skills was also verified by a mystery shopper survey.
The business impact expected was sales growth, monitored throughout a period of three months after the training was completed. The results of the ROI calculation were expected to be 50 percent.
The data collection plan was a crucial part of the evaluation strategy. It allowed for a very clear outline in terms of the types of information needed, how the data should be collected, and when and by whom it should be collected.
ROI Analysis Plan
The ROI analysis plan was also important to the evaluation. As shown in Figure 12-3, the ROI analysis plan shows how to analyze data at Level 4 (business impact) and convert it to money so a credible ROI calculation can be achieved.
FIGURE 12-1. The ROI Methodology
FIGURE 12-2. Data Collection Plan
FIGURE 12-3. ROI Analysis Plan
All of the participants (100 percent) who took part in the training completed the evaluation questionnaire. The results are shown in Table 12-1. During the training, participants were evaluated by the trainer using a yes/no test. Eighty percent of the participants received a positive evaluation.
TABLE 12-1. Reaction Questionnaire Results
Participants’ Reactions | |
This program met the objectives. | 4,8 |
The program was relevant to my work. | 4,7 |
The program was important to my job success. | 4,6 |
The program provided me with new information. | 4,4 |
I will recommend the program to others. | 5,0 |
The program was good investment in me. | 4,9 |
The program met my needs. | 4,8 |
The program was a good use of my time. | 4,8 |
I intend to use the material. | 4,7 |
Overall result | 4,7 |
Note: Participants rated the following on a scale of 1-5.
Every two weeks from the end of the training, participants were assessed by their superiors. The process took three months. Managers evaluated the use of sales standards in the sales assistants’ daily work. As a result, the study found that 70 percent of the participants used all of the acquired sales standards. Thirty percent did not use one of the acquired standards, closing the sale. Also, 80 percent of store managers effectively conducted on-the-job coaching.
An additional source of information was a mystery client survey. The results after this period were satisfactory. In six of the stores, there was visible improvement, and in three of the stores no changes were observed. (Note: During the evaluation process, staff changes occurred in three of the stores—store managers and sales assistants were dismissed, or resigned on their own.)
Barriers to Success
After completing the sales training program, a focus group was conducted concentrating on the barriers linked to the implementation and application of sales standards. The results of the focus group brought out that the greatest barriers were:
Store management pointed out that this type of training should be organized more frequently, at the same time stressing that a crucial element of further trainings was to allow the participants to practice in non-standard situations in which they apply the acquired skills. Another crucial observation that was made was that the participants still faced problems with closing the sale. This was the first sales training carried out for McArthur sales assistants, therefore the level of motivation and commitment in this case was extremely high.
It was decided that the business analysis would only take into consideration the results from six of the shops, due to the fact that in three of the stores there was a high employee rotation during the project (this group was excluded from further analysis). Shoe sales were studied in six of the stores for a period of three months after completing the training.
Isolating the Effects of the Program
A key element of the ROI Methodology is isolating the effects of the program on the data. It was decided that in order to do this effectively, a sales trend analysis for a period of three months after the training would be used to compare the results with the same period from the previous year. Actual average unit sales taken starting the week the trainings commenced, during the three-month period was 4 928 pairs of shoes sold. Forecasted average unit sales taken starting the week the trainings commenced, during the three-month period was 3 610,25 pairs of shoes sold, as shown in Figure 12-4.
Average “real” – average “forecast” = 1 317,75
A group of experts most familiar with the situation indicated that no additional new influences entered during the post-evaluation period. They also indicated that the pre-program influences are still present in the post period.
In all of the stores surveyed, a positive difference was observed between the actual results and the forecasted results. Conducting the same analysis in all of the stores at the same time strengthened the credibility for the increased results.
FIGURE 12-4. Trend Analysis
Converting Data to a Monetary Value
To calculate the monetary value of the benefits, the following steps were taken, as shown in Table 12-2. The average price of shoes is 79 PLN, of which 40,29 PLN is the net profit (data supplied by the Sales and Marketing Director). The increase in sales within the three months of the evaluated period was 1 318 units, giving an increase of 439 units per month. Annually, this gives an increase of 5 268 units x the net sales profit of 40,29 PLN. Which gives an overall sum of 212 247,72 PLN.
TABLE 12-2. Steps for Converting Data to Monetary Value
Step 1: Sale value of 1 pair of shoes |
Step 2: V = Average price of 79 PLN for 1 pair = net profit 40,29 PLN |
Step 3: ∆P = 1 318 units / 3 = 439 units |
Step 4: Annual ∆P x12 = 439 x 12 = 5 268 units |
Step 5: A∆P x V = 5 268 x 40,29 PLN = 212 247, 72 PLN |
Table 12-3 outlines all program costs. The costs take into account the cost of training for nine stores, although the benefits are calculated only on the results of six of the stores.
TABLE 12-3. Project Costs
Item | Total |
Facilitators fees | 20 000 PLN |
Program materials | 300 PLN |
Meals and refreshments | 11 000 PLN |
Participant salaries and benefits | 18 700 PLN |
Cost coordination and evaluation | 5 000 PLN |
Mystery shopper | 5 000 PLN |
Total costs | 60 000 PLN |
The ROI and benefit-cost ratio (BCR) were calculated for the project as follows:
The benefit-cost ratio was very satisfying for the customer. ROI was calculated as follows:
The ROI interpretation is as follows: each zloty invested returned 2,5 zloty (PLN) in benefits. The training proved to be a good investment and worthy of continuing in the future in order to develop staff sales competence.
The whole process was consistent with the 12 Guiding Principles of the ROI Methodology:
An intangible benefit that emerged during the training was an increase in customer satisfaction in six of the shops. A customer satisfaction survey was conducted and mystery shoppers evaluated salespeople for the same criteria. Table 12-4 shows the criteria being examined within the sales staff.
In addition, during the store management training sum up, it was discovered that one of the additional advantages of this training was a much higher level of commitment on the part of the sales assistants. The training also helped to improve communication between the head office and individual stores.
TABLE 12-4. Mystery Shopper Salesperson Evaluation
1. | The sales assistant greeted me with a smile. |
2. | The sales assistant made eye contact with me. |
3. | The sales assistant offered help too soon (he was pushy). |
4. | The sales assistant said good morning. |
5. | The sales assistant offered to help me find the right shoes. |
6. | The sales assistant presented the benefits of the product. |
7. | The sales assistant handled the objections from my side concerning the product, offers, prices, etc. |
8. | The sales assistant comprehensively offered and presented alternative products. |
9. | The sales assistant suggested I purchase additional products. |
10. | The sales assistant informed me of the current promotion. |
11. | The sales assistant attempted to close the sale. |
12. | The sales assistant informed me about the possibility of refunds / exchanges. |
13. | The sales assistant encouraged me to re-visit the shop. |
14. | The sales assistant presented the companys loyalty program and encouraged me to take part in it. |
15. | The sales assistant was dressed neatly and aesthetically. |
The project results were presented at a meeting with the sales and marketing director and the chairman of the board. The six steps of assessment, the data collection plan, the method of isolating the effects of training, and proposals for the future were presented during the meeting. The overall results of the project were also presented to the participants of the project.
Conclusions which emerged during the discussion:
Małgorzata Mitoraj-Jaroszek PhD, CRP, has more than 15 years’ experience in the training industry. She is the author of various training programs on topics such as training for managers, social skills training, and sales training. Over the years she has worked on multiple consulting projects including the implementation of a formal job evaluation, establishment and implementation of company standards, sales and customer service standards, implementation of the open door policy, setting recruitment processes, and preparing specialists and managers for the role of company trainers. She is also the author of several articles printed in national journals such as “Management and Staff.” Her PhD thesis is devoted to management development in organizations, and was presented to the Management Institute at the Jagiellonian University.