CHAPTER
9

Assessment and Information Gathering

DATA FROM ASSESSMENTS AND OTHER SOURCES ADDS TO THE DISCUSSION ABOUT CLIENT DEVELOPMENT AND IS USEFUL IN EXPANDING ON YOUR CLIENT’S SELF-INSIGHT AND YOUR OWN IMPRESSIONS. THIS NEW DATA includes the results of informational interviewing of your client’s colleagues as well as the results from standardized questionnaires or other assessment tools. These additional perspectives can help your client move from negotiated goals to designed objectives.

When approaching this aspect of the coaching process, consider the following points:

Decisions about what types of data to collect are shaped by multiple factors, such as the circumstances of the case, negotiated goals, the organization’s prior use of particular methods, your own preferences for certain assessments, your Personal Model, and your client’s interest in feedback.

Using several data-gathering methods can be especially helpful if the client has limited self-insight or is really unsure about what improvements or changes to work on.

Assessment methods differ with respect to how visible they are to the rest of the organization (e.g., assessment questionnaires versus 360-degree interviewing of colleagues), which may also be a consideration in selecting assessment methods.

Types of Data

There are two broad categories of data that can be collected for use in coaching: quantitative (i.e., scorable, standardized) and qualitative (i.e., subjective, descriptive). Data of both types can be very helpful in building your working relationship with clients and in suggesting insights. When you use multiple assessment methods, it is useful to interpret assessment results both separately and collectively. In other words, the total picture of a client using assessment results is often greater than the sum of the separate assessment parts.

Quantitative Data

Quantitative data come from standardized instruments that describe the client using various dimensional terms, often based on conceptualizations of personality, interpersonal dynamics, communication style, and leadership competencies. Quantitative data can be collected both through self-report assessments and multirater (360-degree) instruments.

Self-Report Assessments. These assessments are completed by the client, usually online, and then scored by the publisher of the questionnaire. Results, profiles, and narratives based on the client’s scores are sent to the coach for interpretation and for sharing with the client, almost always in conjunction with a feedback discussion. The most common categories of self-report questionnaires used in coaching include:

1. Personality Assessments (e.g., I6PF and California Psychological Inventory)

2. Motivational/Values Instruments (e.g., Hogan Values Scale)

3. Communication Preference Indicators (e.g., MBTI and DiSC)

4. Interpersonal-Style Measures (e.g., FIRO-B)

5. Learning Preferences (e.g., Learning Styles Inventory)

Multirater Assessments. Multirater surveys, also called 360-degree questionnaires, capture perceptions of the client and colleagues on a wide range of skills and leadership dimensions. The colleagues typically include manager(s), peers, direct reports, and potentially other groups such as internal customers. These surveys became popular in the 1990s as a means of providing developmental feedback. (For some of the more widely used 360-degree instruments see Figure 9-1.) They have continued to grow in application and complexity and have become so widely used that it is not unusual to find a prospective coaching client with the results of such a survey on hand.

Figure 9-1. Popular 360-degree feedback instruments

The client and as many as eight to twelve colleagues, representing different perspectives, complete the questionnaire online. Peer, direct report, and other group data are averaged for anonymity. Many standardized 360-degree questionnaires include norms so that clients can benchmark their scores against an external reference point of how managers are typically rated. The profiled results are provided to you, the coach, in preparation for a feedback discussion. These surveys focus on soft-skills areas that have become recognized as keys to leadership success and derailment avoidance. Capturing the perceptions of colleagues on these skills areas is compelling input to a coaching process and a development plan.

Early innovators in using 360-degree questionnaires conducted research on managerial jobs to determine objectively the competencies required; those competencies were further divided into key behaviors that were used to create the items to be rated. Different classes of jobs have different competencies, and so corresponding questionnaires were created: for middle managers, for executives, for sales, for individual contributors, and so on. There are also 360-degree questionnaires linked to particular leadership concepts such as conflict resolution and emotional intelligence.

As the use of these questionnaires has expanded, they have become a standard part of both feedback and leadership development processes. Multirater assessments are also sometimes used to aid in performance ratings for compensation and in bonus determination, although their use for these applications may distort their developmental meaning. In other words, if you have access to 360-degree feedback for your client, ask why it was collected and how it is used in the organization because it may not have as much developmental value as one used exclusively for that purpose.

Most often, 360-degree questionnaires use a five- or seven-point scale of effectiveness or frequency on each of the skill questions, with higher scores better than lower ones. There may be anywhere from a few dozen questions to over a hundred, depending on the complexity of the underlying model. Most questionnaires also ask the rater to identify the most important skills for job success. Open-ended, write-in questions yielding qualitative feedback are often included, using prompts such as, “In your own words, list the manager’s key strengths and development needs.”

Coaches, trainers, facilitators, as well as management and executive development professionals, all need experience in using 360-degree feedback tools. Interpreting the results to get to a well-founded development action plan is a frequent activity of all these professionals. You don’t have to be an expert in the psychometrics of such tools in order to use them effectively. Some of these tools require training and certification, but supervised experience is always recommended before using a specific survey with clients.

Many coaches working at the executive level prefer informational interviewing to multirater surveys. (Informational interviewing is a qualitative method discussed further in the next section.) For short-program coaching or at middle management levels, however, 360-degree questionnaires are more consistently used because they are more efficient in terms of professional time and cost. In addition, they provide comprehensive skills coverage and quantitative ratings so that the relative size of a particular strength or gap is indicated. Also, it is easy for the sample of colleagues included to be larger than with time-intensive interviews.

With regard to quantitative methods overall, both self-report and multirater, you may want to identify questionnaires popular among the coaches and Human Resource professionals you know as the starting point for your own toolkit of measures. Certification courses, manuals, user guides, and guidance from other coaches are all helpful as you begin to use questionnaires with clients. Adding additional measures is a lifelong activity for coaches as new conceptualizations and questionnaires become of interest. Building a foundation of how to learn about and use assessment methods facilitates that growth as you encounter new tools and apply them to client challenges.

Qualitative Data

Qualitative data come from perceptions that others have about the client as well as from your own impressions. There are many types of qualitative data available to you in your coaching work. Your own observations of the client in sessions and in the office environment, along with descriptive documents such as performance appraisals, can provide helpful qualitative data. Your shadowing of the client in actual meetings, when properly set up, can also yield useful information about client behavior. In executive coaching, however, the most frequently used qualitative data-collection method is informational interviewing of the client’s colleagues. Figure 9-2 is an example of how a coach might initiate an informational interview with a client’s professional colleagues.

Figure 9-2. Introduction to the informational interview

In early interactions, clients will share their self-perceptions, and you will form impressions of them. Tapping into others’ perceptions of the client builds on that foundation and provides context that will be useful to you in development planning. The goal of informational interviewing is similar to other assessment methods: to gain a broader and deeper understanding of a client more quickly and efficiently than could be done just through dialogue with the client. Based on the ease of application and the usefulness of the data, informational interviewing has become a standard element in supporting movement from relationship building toward insight and designed objectives.

Like all aspects of the coaching process, the specifics of arranging and conducting interviews require transparency and planning, especially with the client. Here are the recommended steps in arranging informational interviews:

1. Contract, or confirm earlier contracting, about the specifics of the informational interviewing process.

2. Discuss whom to interview and have your client gain the sponsors’ support for the respondent list. Typically, this list includes the client’s sponsors, direct reports, a sample of peers, and other colleagues who work or have worked with the client for at least six months. The total is usually eight to twelve people, but the number could go higher to include key stakeholders.

3. Support the client in finding a comfortable way to inform respondents about the process, request their participation, and set expectations for you to contact each person. (This is often completed via e-mail.)

4. Collaborate with the client about what his or her topics of interest are, what questions you should use, and what you should listen for in the answers. Be sure to consider your own preferred questions as well.

5. Confirm with the client the confidentiality of respondent information. Generally, results are shared only with the client and only as themes. Manager information/feedback can be used more directly or even verbatim, although this should be confirmed with the manager.

6. Discuss client follow-up, such as thanking the participants or later sharing with them the development themes that become part of a development action plan.

7. Consider the logistics of the interviews (where, when, how long).

8. Establish a general but consistent structure for the interviews, considering what you will say about yourself and the process and the questions you will ask; budget appropriate time and make sure respondents know what to expect.

9. Consider asking sponsors special questions about the client’s career, advancement potential, goals of coaching, and similar topics that only they would have information about.

10. Conduct the interviews (by phone and/or in person).

11. Analyze the contents of your interview notes.

12. Distill key points about the client’s strengths and development areas, looking within groups (e.g., direct reports) and across groups, focusing on specific questions that were of interest to the client.

13. Prepare a written summary to be shared with the client highlighting strengths, development areas, and other useful information.

14. Plan and conduct your feedback conversation.

Becoming both familiar with, and facile in, using a range of qualitative and quantitative assessment methods is an important component of your growth as a coach. The qualitative methods especially will draw on your well-developed questioning and listening skills and will be useful in most engagements.

Supervisor’s Observations

Prestructured coaching processes reduce the need for contracting with clients and sponsors, but challenges remain. Coaches sometimes mistakenly assume that because the coaching protocol has been set up in advance, everyone is familiar with and comfortable about what will happen. In fact, clients can have many reactions, including trepidation, distrust, excitement, or even disinterest, that need to be heard and explored.

Henry wisely did not assume that Marty had clear information about the coaching process and made sure to have a thorough conversation with him about the data gathering that would be taking place. Doing so builds the connection between coach and client, even though process changes are unlikely. In this case, Henry did a good job both of explaining the data gathering and of differentiating it from similar, but nondevelopmental, processes that the organization used.

In addition, even in prestructured processes, special challenges can arise. Henry believed that merging Yolanda’s point of view in with the others would lose important information. Even though Yolanda may have expected that her information would be used more directly than that of others, Henry did not make that assumption. He wisely sought counsel from a trusted colleague about how to balance the general requirement for confidential feedback with the need to share Yolanda’s specific concerns. He also found his own way to use her information directly, choosing not to quote her even though she gave him permission to do that. He also used the fact that a three-way meeting was a standard part of the process to make sure that Marty and Yolanda would have an opportunity to explore her point of view along with the draft development plan.

Takeaways

Decisions about the types of information and the timing of data collection in coaching depend upon many factors tied to client, context, and your preferred coaching process.

The goals of data collection include expanding your understanding of the client, the client’s own desire for self-awareness, and the need to explore how the client is perceived and experienced in the work context.

Both qualitative and quantitative assessment results can also be used to pique clients’ interest in development by providing new perspectives that are seen as credible.

A consideration in collecting qualitative information from colleagues is how public the coaching should be, as determined by you, the client, and the sponsors.

Involving colleagues using a 360-degree survey or informational interviews adds complexity but can be very beneficial in terms of capturing perceptions of the client, surfacing action ideas for on-the-job development, and gaining wider support for the client’s growth.

It is important to select questions and focal topics for use in your informational interviewing and to include your client in identifying them.

Selecting respondents can be a milestone since it includes the client’s manager and can reveal your client’s comfort in seeking feedback.

Ideally, data enriches your discussion with clients and provides new ideas and perspectives while building your working relationship in the journey toward a well-founded and motivating development plan.