5

CHAPTER FIVE

Implement Changes and Share Lessons Learned

Ultimately, evaluation findings need to be put to use, or else the resources and time that were invested in the evaluation were not used wisely. In some cases, it makes sense to use evaluation findings at the end of an initiative to guide future directions. In other cases, particularly in longer-term, more complex initiatives (for instance, leadership culture change), evaluation data can and should be used in every stage to test hypotheses about what is working to make progress toward desired outcomes and what is not.

One possible outcome of a leadership-development initiative is that it has the impact stakeholders and participants have expected and hoped for. In that case, no remedial action is necessary, but the organization might want to consider its next step in the development process and stakeholders may want to show off and celebrate the success of the initiative.

Another possible outcome is that the data show the initiative needs to be revised to achieve maximum impact. This is frequently the case for brand-new initiatives and should be viewed as part of the process rather than as a failure. In this case the specific areas and suggestions for revision, based on the evaluation findings, should be identified and prioritized by relevant stakeholder groups. For example, participants might gain insights but aren’t able to employ much of what they learn because of systemic obstacles in their work environment or a lack of understanding about how to transfer what they’ve learned. Evaluators can help identify barriers, but it is the responsibility of an organization to address those issues.

Depending on the nature of the obstacles, additional development interventions may be needed. Sometimes the evaluation uncovers things that weren’t previously known. For example, if the leadership-development initiative involved a culture or climate survey, the information gathered from the survey may indicate the work the organization needs to do. An organization may want to update reward systems, policies, or communication processes to better reflect the vision of the organization. Perhaps the leadership competencies being developed don’t align with a business strategy, a community’s values, or a foundation’s culture.

It’s best if stakeholders examine these issues before the leadership-development initiative, but sometimes that’s not possible. Changes in other contextual factors may occur during the initiative or may not be readily apparent before the initiative starts. It’s important to maintain attention on issues of alignment throughout the initiative to best ensure its success. Other experts, such as organizational-development professionals, may need to be consulted if there is a substantial incongruity among systems, processes, outcomes, and other factors.

Gather and Consult with Stakeholders

The most effective means for determining how to use what is learned from the evaluation is to bring together everyone with a stake in the evaluation and the initiative. This includes not just participants and stakeholders with an interest in the development initiative but also people who might be affected by changes made as a result of the findings and those individuals responsible for implementing the intended changes. The best time to do this is after you’ve reported evaluation results, which allows stakeholders some time to think about the results and correct any misunderstandings or errors in the report. Stakeholders should be encouraged to use data to respond to any perceived inaccuracies in the report because this creates a shared responsibility for report accuracy.

If your evaluation results illustrate the shortcomings of a specific group in particular (such as the need for participants’ managers to become better development coaches for their direct reports), it may be wise to allow that group time to devise a course of action before bringing in all the stakeholders. This time will allow the group to form a response and take ownership of the situation, instead of putting them in a defensive position.

Examine Possible Revisions to the Initiative and to Specific Areas That Strengthen Support Structures

Evaluation findings might indicate a need for minor or major revision to the development initiative, such as including additional learning needs for individuals or groups and improving specific elements to support learning more broadly, thereby increasing an intervention’s appeal or effect. Based on the evaluation findings, a group of stakeholders may assume the task of redesigning aspects of the initiative.

Develop an Action Plan

Once key stakeholders have discussed potential actions, the next step is for them to develop an action plan. As an evaluator, you may or may not be involved in or lead the action-planning process. An action plan is a tool for implementing the lessons that result from an evaluation study. Certainly, the lessons should point to areas of leadership-development initiatives that need revision, and in addition they can indicate areas (such as systems and processes) in the organization that aren’t supporting those initiatives.

You can determine which stakeholders need to be involved in the action planning by examining where action is needed and who has the responsibility and authority to enact change in that area. The action plan should outline specific tasks, identify individuals who are responsible for carrying them out, include necessary resources for implementing specific actions, and provide a timeline for carrying them out. As a guide to creating an action-plan document, consider these questions:

The action plan that results from this effort needs to be distributed to everyone involved with the proposed actions so that all are clear about their roles in implementing the plan.

Monitor the Action Plan’s Implementation

As an action plan is implemented, its progress will need to be monitored and communicated. Depending on your role, you may be the person tasked with this effort. Monitoring the action plan’s implementation means following up with individuals and groups responsible for specific actions, as indicated in the plan, to see how well they are progressing and what, if anything, needs to be modified. One way to monitor the plan is to periodically convene the group that developed the action plan and have each member describe how well the actions are being implemented and what help is needed to continue the process. Regardless of the method used, all of those involved should be aware of the extent to which the plan is being implemented and what impact the changes are having on individuals, groups, and the organization. The group might even decide that, as a result of making certain changes, additional evaluation needs have surfaced.

Use Evaluation to Create Broader Change and Learning

Evaluation measures more than impact. The process has the potential to create change in (and for) individuals, organizations, and communities. In addition to identifying gaps in, barriers to, and support for leadership development, evaluation can be used to help gather possible solutions and solicit ideas for improving a development effort. These changes can affect broad or small groups of people or individuals, depending on the type of change and the effect it has at various levels. These changes also provide an opportunity for additional evaluation and learning.

Important outcomes of evaluation include the individual and group learning that occurs as a result of participating in the evaluation and using its results. When individuals and groups reflect upon their experiences and share what they’ve learned with others in the organization, the organization as a whole learns. The challenge, however, is in finding ways to capture and share this learning.

One tactic is to develop a learning database that employees and other stakeholders can access on an as-needed basis. Another tactic is to create online discussion groups or communities of practice for groups within the organization. Yet another is to publish, in internal or external newsletters and bulletins, the lessons revealed through participation in the development initiative and in the evaluation’s results. Each of these methods is a vehicle for sharing information more broadly so that people and groups can learn from each other—a hallmark of learning. In creating an evaluation plan that fosters learning, it’s important to consider guidelines that make the process less susceptible to misuse. The collection, storage, and use of data representing individuals should adhere to appropriate and professional standards (Yarbrough, Shulha, Hopson, & Caruthers, 2011). Following these various standards helps reduce ethical problems that can arise during the evaluation process.

Three specific issues are especially relevant to the evaluation of leadership-development initiatives: maintaining confidentiality of individual data; evaluating the initiative, not the individuals; and linking the evaluation to broader goals and strategies. The evaluation data collected should be used to enhance the initiative itself, to foster changes in participants’ leadership knowledge and skills, to encourage relevant changes in the organization or community, and to illustrate relevant factors in the organization or community related to the success of the initiative. Evaluators should encourage organizations not to use the collected data for performance appraisals or for administrative decisions about individual performance, since the collection of that kind of information is held to a different set of legal and professional standards (Yarbrough et al., 2011). You can promote this value by aggregating any individual data collected during the evaluation to the group level.

Evaluation should be part of strategic intent. It should not be a valueless exercise that people contribute their time, energy, and resources to but has no productive purpose. Just as leadership development should be linked to organizational strategy, so should evaluation be considered strategically important and expected to contribute to the achievement of goals and learning agendas.