Choosing the Right DEI Culture and Climate Survey Instrument If you decide to use an existing DEI culture and climate survey from a vendor, it is of paramount importance to use a survey instrument that assesses organizational culture and climate specifically through the lens of DEI. Not all culture and climate surveys are created equal. For example, a growing number of employee engagement surveys can be augmented with a few DEI questions. At BCT, we have generally found them to be insufficient for a thorough and comprehensive organizational DEI assessment. Make certain that your survey not only has questions that are specific to DEI but that it was also designed with DEI front and center.

Designing Your Own DEI Culture and Climate Survey Instrument While it can be tempting to design your own DEI culture and climate survey, you should strongly consider using a standardized instrument for the entire survey or a standardized set of questions for a given category. This is particularly important if you desire to benchmark your results now or in the future. Using standardized instruments or questions will allow you to draw comparisons to other organizations, of course, if you have (and/or the organization you are working with to conduct your organizational DEI assessment has) access to comparison data for benchmarking.

However, if you choose to design your own DEI culture and climate survey instrument, you should begin by identifying the major categories of interest, such as leadership commitment, inclusion and belonging, equitable outcomes, and so forth, and then identify a few questions in each category.

You should combine mostly closed‐ended questions (i.e., quantitative data) with a few open‐ended questions (i.e., qualitative data). Closed‐ended questions should be phrased as statements where respondents are asked to choose from a list of preselected options such as a Likert rating scale (i.e., strongly disagree, disagree, neutral, agree, and strongly agree) relative to the statements. They are excellent in providing quantifiable data and for categorizing respondents. The responses represent very structured data that can be analyzed using quantitative data analysis techniques such as statistics (discussed later in this step) and data science techniques such as machine learning (ML), natural language processing (NLP), and natural language understanding (NLU) (discussed in the conclusion). The following are examples of closed‐ended questions:

• Commitment: Leadership is committed to diversity, equity, and inclusion.
• Differences: Our organization values differences.
• Diversity: Leadership reflects the diversity of the organization.
• Equity: Promotions and compensation are given fairly at our organization.
• Inclusion: I feel empowered to contribute to our organization.
• Belonging: I feel like I belong at our organization.
• Supervisor: My supervisor handles diversity, equity, and inclusion issues effectively.
• Retention: I am considering leaving the organization.
• Respect: People treat each other with respect.
• Psychological Safety: I am comfortable speaking up in our organization.

Table 1.7 provides a list of DEI and other related survey instruments with standardized question sets that you can use. For example, if one of the categories you wanted to measure was the psychological climate of your organization, then you could use some or all of the 31 questions from the “Psychological Climate and Effort Measures (PC&EM).”

Open‐ended questions should be phrased as questions that require a longer response in the respondent's own words. They are excellent for providing context and soliciting opinions. The responses represent unstructured data that can be analyzed using qualitative data analysis techniques such as coding (discussed later in this step) and data science techniques such as natural language processing (NLP) (discussed in the conclusion). The following are examples of open‐ended questions:

• Culture: How would you describe our organization's culture?
• Strengths: What is working well related to diversity, equity, and inclusion?
• Limitations: What is not working well or could work better related to diversity, equity, and inclusion?
• Environment: How can our organization create an environment where everyone feels valued for their contribution? As individuals? Feels that they belong?
• Recommendations: What would improve DEI at our organization?

In her blog post, “7 Tips for Conducting Your Next DEI Survey,” Lyssa Test of Lattice, a people‐success platform, offers the following advice from Ashley Schwedt, Leadership Trainer & DEI Lead at LifeLabs, a New York‐based consultancy firm specializing in leadership training for managers and executives: “Every question on your survey should have a clear, specific purpose. Only ask questions you can do something about. When designing your survey, for each question, ask [yourself], ‘What will I do with this information?’,” advised Schwedt. “If the answer is ‘Nothing,’ it isn't a good question to ask. The responses should drive [organizational] strategy by highlighting current gaps and helping the leadership team determine where to focus resources.”¹ A good rule of thumb is that it should take no more than 20 to 30 minutes for respondents to complete the survey.

Example of an Organizational DEI Culture and Climate Survey: BCT's DEI Workforce and Workplace Assessment BCT has created a statistically validated and reliable DEI Workforce and Workplace Assessment™ (DWWA™) instrument. This instrument assesses organizational culture and climate through the lens of DEI in the 10 categories shown in Figure 1.21. Designed by BCT's senior director of health equity, David Hunt, JD, a former employment lawyer and civil rights attorney and nationally recognized DEI subject matter expert, BCT has successfully used this tool with many leading corporations, nonprofit organizations; educational, academic, and faith‐based institutions; hospital and health care systems; foundations and philanthropic organizations; and federal, state, and local government agencies.

The DWWA™ measures organizational DEI culture and climate along the following lines:

• Perceived areas of strengths and areas for improvement relative to your organization's culture
• Factors that characterize culture (i.e., improved performance, leadership qualities, organization, learning team members, learning environment, individuals empowered, individual values, departmental values, and teamwork)
• Needs and concerns of different stakeholder groups
• Social perceptions that might uncover any major biases that impact the culture and climate
• Cultural perceptions of your organization's manner and approach to diversity, equity, and inclusion associated with the above
• Legal and risk management issues related to uncivil or disrespectful treatment, DEI‐related incidents of disrespect, workplace bullying, and discrimination

The DWWA™ is an online survey that saves time and collects valuable data to inform DEI program design and implementation. It is often combined with in‐depth interviews (IDIs) and focus groups to gather anecdotal information and deeper insights from people.

Select the Survey Tool There is a plethora of low‐cost or no‐cost survey tools such as Microsoft Forms, Zoho Survey, Mentimeter, Typeform, Pollfish, Google Forms, QuestionPro, Alchemer (formerly SurveyGizmo), and SurveyMonkey, as well as enterprise platforms such as Ambivista, AskNicely, Emprising™, and Qualtrics. When selecting a survey tool, in addition to cost, you should also consider the question formats that are supported (e.g., check boxes and ratings for closed‐ended questions, text fields for open‐ended questions, and more), the survey logic that is supported (e.g., “skip logic” to skip certain questions for certain people, and “display logic” to display certain questions only for certain people), multi‐lingual support, customer support, the level of sophistication for analysis, and the level of detail for reports.

Launch the Survey Prior to launching the survey, it is always a good practice to pilot it with 5 to 10 test users and solicit their feedback. This will not only ensure the questions are clear and can be answered effectively and in a reasonable amount of time, but also that the survey tool and logic are configured and functioning properly. Once again, a good rule of thumb is that it should take no more than 20 to 30 minutes for respondents to complete the survey.

You should also develop an outreach and communication plan to maximize the response rate. The plan should include a message from executive leadership, human resources, and/or the chief DEI officer about the importance of the survey, equip supervisors with talking points to carry the message further, maximize the use of available channels (e.g., e‐mail, newsletters, intranets, town halls, etc.), establish a response deadline (typically two weeks from launch), send weekly reminders and advisory reminders as the deadline approaches, and deliver messaging that addresses the key questions of who, what, where, when, and why:

• Who will have access to the data?
• What is the purpose of the survey?
• Where will the results be shared?
• When will the survey close? When will results be shared?
• Why should the survey be important to people? To the organization?

Messaging should also be tailored to specific segments to motivate their participation. Three segments require distinctive messaging for a DEI survey.

The first segment, DEI supporters and champions, are passionate about DEI and connect to these issues through their “heart.” This is perhaps the easiest segment to motivate. A potential message to this segment is “your input matters because the results of this survey will shape the development of our DEI strategic plan and new initiatives.”

The second segment, DEI naysayers and detractors, are critical of, or resistant to, DEI and may only connect to these issues through their “head.” Naturally, this is a difficult segment to motivate. Potential messages to this segment are, “we welcome all voices as we endeavor to create a more welcoming workplace for everyone,” and “the organization is committed to this effort, so we invite everyone to participate,” and “research has shown that DEI leads to improved organizational performance, so your response will help us improve our bottom line.”

The third segment is DEI disadvantaged groups. DEI surveys are often the victim of the very issues they seek to address. In other words, if the lack of attentiveness to DEI matters has led to certain groups feeling isolated, excluded, targeted, or disadvantaged, it may inhibit them from responding to the survey truthfully, if at all, based on skepticism, distrust, and fear of reprisal. As mentioned previously, we recommend using a neutral, third party to administer the survey and the messaging to this segment (and all segments) is, “the survey is being administered by a neutral, third party with X years of experience in DEI. All responses will be anonymous, the data will remain private and confidential, and no data will be disclosed that can compromise a person's identity.”

The execution of the outreach and communication plan will signal to prospective respondents that the survey is now live and will motivate their participation.

Monitor Survey Responses You should actively monitor the survey response rate so you can adjust and adapt your outreach and communications plan accordingly. If there are specific subgroups with low response rates, you should consider extending the deadline another one to two weeks and tailoring your outreach and communication to those subgroups and, once again, enlist the voices of leadership, supervisors, human resources, and DEI. Our experience with DEI surveys suggests that 40% is considered a good overall response rate for relatively large organizations and 80% is a good overall response rate for relatively small organizations. However, there is no “one size fits all” when it comes to response rates. What is far more important than the overall response rate is the specific response rate for underrepresented subgroups and marginalized voices.

To avoid a double‐edged sword, you should pay particular attention to the response rates of subgroups that are underrepresented or marginalized. The fact that these subgroups are underrepresented or marginalized already poses a challenge to ensuring that their voices are adequately captured in the survey sample. A low response rate has the potential to further disempower their voices unless deliberate steps are taken to boost their participation (and/or factor the response rate into the analysis, which is discussed later). Referring back to our prior conversation on data sampling, the inclusion of underrepresented subgroups and marginalized voices should be considered of paramount importance.

To increase the response rate with underrepresented subgroups, enlist the help of ambassadors that identify and/or have credibility with those subgroups, and partner with organizations that represent those subgroups, such as employee resource groups (ERGs), in designing and executing the outreach and communication plan.

Share the Survey Findings as Broadly as Possible Later in this step, we will discuss sharing survey findings in the section “Phase VII: Deliver DEI Assessment Report” in the context of sharing the entire organizational DEI assessment results including quantitative and qualitative findings.

Phase II‐A: Perform DEI Human Resources (HR) Policies and Practices Assessment

A DEI HR policies and practices assessment, when compared to a traditional HR policies and practices assessment, is conducted specifically through the lens of DEI. While organizations are comprised of a multitude of organizational policies and management practices, the following is a representative but not exhaustive list of HR policies and practices that could be evaluated from a DEI perspective:

• Recruitment and Hiring
• Learning and Development
• Advancement and Promotions
• Performance Management and Evaluation
• Succession Planning
• Compensation and Rewards Management
• Discipline, Terminations, and Exit Policy
• Retention, Turnover, and Support
• Discrimination, Harassment, and Bullying
• Affirmative Action and/or Office of Federal Contract Compliance Programs (OFCCP) Regulations
• Benefits and Assistance
• Family Medical Leave Act (FMLA) and Americans with Disabilities Act (ADA)
• Employee Attire and Grooming Standards
• Protected and Unprotected Speech, Religious and Political Expression

Based on the collection of all documents pertaining to a policy or practice, the assessment can be broadly performed as a review or audit. The distinction between the two is summarized in Table 1.8. The audit process can also be combined with in‐depth interviews (IDIs) and/or focus groups (described later) to provide additional context and insight.

Conducting a DEI HR assessment yields the following benefits that are specific to DEI:

• Embrace diversity and unleash creativity within your workforce, spurring innovation
• Create an inclusive and equitable workplace, improving staff retention and helping to attract diverse talent
• Ensure compliance with applicable equal employment opportunity (EEO) and affirmative action laws
• Craft better HR and DEI strategies and organizational decisions, enhancing organizational performance

The deliverable is a final report that summarizes findings (review), recommendations (audit), and next steps regarding equity challenges and opportunities within each policy and practice examined.

Phase II‐B: Perform DEI Benchmarking or Ranking

Another useful DEI assessment tool is DEI benchmarking, which is an evaluation of your organization against best practices and/or other organizations. The benefits of DEI benchmarking are as follows:

• Establish a reference point relative to other organizations
• Compare your organization to others within your industry, sector, and/or geographic region including your peers and competitors
• Identify specific organizational areas of strengths and areas for improvement relative to others
• Drill down to specific divisions, departments, and other areas of your organization
• Focus on specific strategies and actions for organizational improvement

One approach to benchmarking is to assess your organization's capacity, maturity, ability, or “muscle strength” in specific categories that relate to DEI such as organizational structure, leadership involvement, approach, staffing and resources, measurement, and the like. These benchmarks are often based on a DEI maturity model, that is, a series of levels reflecting your DEI capacity and maturity to improve DEI (e.g., Level 1: Basic, Level 2: Intermediate, and Level 3: Advanced). The benchmarking process involves data collection, typically via a survey of organizational stakeholders, and/or submission of DEI data or organizational policy and practice documents. This data is reviewed and analyzed to establish a series of scores reflecting your level of maturity in each category. Once you've completed a DEI benchmarking, you essentially have a scorecard by which you can measure capacity and a reference point to assess maturity. We will revisit scorecards (and dashboards) in Step 4: DEI Initiatives.

Another approach to benchmarking is to engage with one of several entities that have established DEI and DEI‐related rankings such as DiversityInc's Top Companies for Diversity®, Black Enterprise's Best Companies for Diversity, DiversityMBA's Inclusive Leadership Index, Bloomberg's Gender‐Equality Index, Careers & the disABLED Top Employers, Disability:IN's Disability Equality Index, Hispanic Association on Corporate Responsibility Corporate Inclusion Index, DEI Disability Equity Index™ of the Best Place to Work for Disability Inclusion, Human Rights Campaign Corporate Equality Index™ of the Best Places to Work for LGBTQ Equality, Working Mother Best Companies for Working Mothers and Best Companies for Dads, Military Friendly, MilitaryTimes Best for Vets Employers, to name a few. These rankings are compiled based on the submission of a standardized set of data that is then analyzed and compared to the submissions of other organizations. For example, some rankings can tell you exactly how you compare to peers in your industry and/or geography such as an exact number (e.g., 5th out of 1,000 organizations) and/or a percentile (e.g., 95th percentile). Rankings are different from benchmarking in that they tell how well you are doing but may not necessarily go deeper to provide insights to recommended strategies for your organization.

Example of DEI Benchmarking: The Global Diversity, Equity & Inclusion Benchmarks (GDEIB) One of our favorite benchmarking tools at BCT is the Global Diversity, Equity & Inclusion Benchmarks (GDEIB): Standards for Organizations Around the World, which is published by the Centre for Global Inclusion.² Authored by Nene Molefi, Julie O'Mara, and Alan Richter, GDEIB is a free, downloadable, international instrument, developed and refined by 112 expert panelists, that contains 266 benchmarks in four primary processes (groups) and 15 concrete actions (categories), which cover the important elements that need to be addressed to create a world‐class DEI initiative (see Figure 1.22 and Table 1.9).

Among the characteristics of the GDEIB are:

• The well‐researched, definitive picture of quality DEI work.
• Use it to determine the level of your organization and set achievement goals.
• Use it to measure progress. Compare your organization to proven practices and outcomes.
• Agreed‐upon definitions of diversity, equity, and inclusion.
• Content crosses cultures and continents.
• Both global and local in scope; it is useable anywhere.
• Useable for all sectors, industries, types, and sizes of organizations.
• Useable for various approaches to DEI: Competence, Compliance, Dignity, Organization Development, and Social Justice.
• Focus on two ultimate goals of DEI: Creating a Better World and Improving Organization Performance.
• Can be customized.
• In English and Portuguese. Spanish and French coming soon.

Each of the 15 categories is rated according to five levels, summarized in Table 1.10, with the benchmarks at Level 5 considered best practice.

THE FOUNDATION GROUP: Drive the Strategy Foundational elements that are necessary to build a strong DEI initiative and for the effective operation of all other categories Category 1: Vision, Strategy, and Business Impact Category 2: Leadership and Accountability Category 3: DEI Structure and Implementation	THE BRIDGING GROUP: Align & Connect Provide critical linkages that bridge foundational work with the internal and external focus of DEI in the organization Category 8: Assessment, Measurement, and Research Category 9: DEI Communications Category 10: DEI Learning and Development Category 11: Connecting DEI and Sustainability
THE INTERNAL GROUP: Attract & Retain People Focus primarily on strengthening policies, systems, and processes to advance DEI Category 4: Recruitment Category 5: Advancement and Retention Category 6: Job Design, Classification, and Compensation Category 7: Work, Life Integration, Flexibility, and Benefits	THE EXTERNAL GROUP: Listen To & Serve Society Relate to how the organization offers its services and products to and interacts with its customers, clients, communities, and other stakeholders Category 12: Community, Government Relations, and Philanthropy Category 13: Services and Products Development Category 14: Marketing and Customer Service Category 15: Responsible Sourcing

Level	Global Benchmark	Description
Level 1	Inactive	No DEI work has begun; diversity, equity, and inclusion are not part of organizational goals.
Level 2	Reactive	A compliance‐only mindset; actions are taken primarily to comply with relevant laws and social pressures. Doing the bare minimum.
Level 3	Proactive	A clear awareness of the value of DEI; starting to implement DEI systemically. This is what is required and expected of all organizations.
Level 4	Progressive	Implementing DEI systemically and showing improved results and outcomes beyond what is required or expected.
Level 5	Best Practice	Demonstrating current global best practices in DEI; exemplary.

At BCT, we have operationalized the GDEIB into an online assessment survey that can be easily completed by anyone who manages people and is, therefore, familiar with and responsible for policies and practices—such as managers, supervisors, and executives—to quickly evaluate your organization's capability or maturity to improve DEI. We determine an organizational score (i.e., from Level 1 through Level 5) within each of the 15 categories, four groups, and overall. This gives organizations the ability to not only know how their organization ranks against global best practices but also to discover the strength of their capacity and maturity to improve DEI. We interpret the results as both a way to compare organizations to others, including within their sector/field/industry, as well as to right‐size their DEI program to their DEI capacity, so they are not undertaking more than they can reasonably accomplish. The GDEIB will reappear several times throughout the book, including Step 3: DEI Insights, as a “What Works” model for DEI benchmarking.

Phase III: Analyze DEI Quantitative Data

The next phase of the organizational DEI assessment process is analyzing the DEI quantitative data. The following three tips and techniques will aid you in your quantitative data analysis:

1. Compare current data with past data to identify trends—Longitudinal data tracks the same data at multiple points in time. When longitudinal data is available, a fairly straightforward way to analyze DEI quantitative data is to compare current data with past data over multiple points in time (e.g., months, quarters, years, etc.). Is the data increasing, decreasing, or remaining flat? Is the data changing at a rapid pace, a slow pace, or not at all? The answers to these simple questions can help illuminate key trends and identify important developments.
2. Use cross‐tabulations to better understand different subgroups—Cross‐tabulations (“crosstabs” or “contingency tables”) are data tables that can be used to analyze the correlation between two or more variables. A cross‐tabulation table groups variables together to better understand the relationship between the variables by producing various breakdowns according to your chosen subgroups from the internal, external, and organizational dimensions of Four Layers of Diversity (previously shown in Figure 1.23). These breakdowns can be extraordinarily helpful to understanding perceptions, perspectives, and lived experiences of different subgroups. For example, Table 1.11 shows a cross‐tabulation between race (internal dimension) and lived experiences with different forms of incivility rank ordered by disrespectful behavior. The table not only clearly shows the relationship between race and multiple variables but also that people of color are more likely to experience disrespect, bullying, and discrimination. As another example, Table 1.12 shows a cross‐tabulation between management status (organizational dimension) and responses to a statement about the organization benefitting from increased diversity using a five‐point Likert scale (i.e., strongly disagree, disagree, neutral, agree, and strongly agree) rank ordered by level of management status. The table suggests there is a high level of commitment to diversity among executives that drops off precipitously among directors, managers, supervisors, and team leaders.

   

3. Identify statistically significant differences to illuminate possible challenges and opportunities of different subgroups—As mentioned earlier in the context of data sampling, when conducting a DEI culture and climate survey it is of paramount importance that you have enough respondents from each subgroup to make valid and statistically significant inferences about that subgroup because representation matters. Moreover, it is of paramount importance when analyzing the quantitative data from your DEI culture and climate survey that you identify where there are statistically significant differences between the perceptions, perspectives, and lived experiences of different subgroups. A statistically significant difference can suggest that a subgroup is experiencing your organization's culture and climate in a markedly different way—positively or negatively—when compared to others. This can be indicative of deeper, underlying issues, strengths, and areas for improvement. The quantitative research (i.e., DEI culture and climate survey) can help describe “what” challenges and opportunities exist and be suggestive of the need for further qualitative research (i.e., in‐depth interviews and focus groups) to understand “why” they exist. For example, if the employees at a specific work location (organizational dimension) are found to have a statistically significant higher score on the Intrinsic Inclusion Inventory™ for Organizations, then there may be something very special happening there to foster an environment of inclusion and belonging that your organization would benefit from a deeper understanding and to potentially adopt more broadly. By comparison, if the employees at that work location are found to have a statistically significant lower score on the Intrinsic Inclusion Inventory™ for Organizations, then issues may be unfolding that warrant deeper understanding to address them.

   Please indicate which ethnicity(ies) best describe you:	N %	Disrespectful Behavior	Diversity‐Related Disrespect	Workplace Bullying	Discrimination
   African American or Black	867	64%*** ↑	43%** ↑	27%	22%
   	13.4%
   American Indian or Alaska Native	84	47%	19%	26%	10%
   	1.3%
   Asian	382	42%	20%	20%	11%
   	5.9%
   Hispanic, Latino, or Spanish Origin	1197	49%	20%	27%	11%
   	18.5%
   Middle‐Eastern or North African	58	41%	12%	18%	4%
   	0.9%
   Native Hawaiian or Other Pacific Islander	32	46%	18%	24%	9%
   	0.5%
   Some other race	97	41%	11%* ↓	18%	3%* ↓
   	1.5%
   Two or more races	136	48%	19%	26%	10%
   	2.1%
   White, Not Hispanic or Latino	3617	40%	11%* ↓	17%	3%* ↓
   	55.9%
   TOTAL	6471	44%	19%	23%	12%
   	100.0%

   ^*** Statistically significant at the p < 0.001 level

   ^** Statistically significant at the p < 0.01 level

   ^* Statistically significant at the p < 0.05 level

   • To determine statistical significance, you will need to determine the p‐value (or probability value) for each data point, which indicates the likelihood that your data would have occurred by random chance or the likelihood that the “null hypothesis” of there being no correlation or relationship between the two variables being studied is true (i.e., one variable does not affect the other), and establish a threshold or significance cutoff also known as an alpha value. An alpha value of 0.05—a p‐value of less than 0.05 (p ≤ 0.05)—is generally recognized as statistically significant. Here are some good rules of thumb to establish alpha values and interpret p‐values:

     		My organization would benefit from increased workforce diversity.
     Management Status	N %	Strongly Agree or Agree	Neutral	Disagree or Strongly Disagree
     Executive/Vice President/Associate Vice President	98	100%	0%	0%
     	1.5%
     Location, Unit, or Division Director	212	62%	32%	6%
     	3.3%
     Manager/Supervisor	568	45%	39%	16%
     	8.8%
     Team Leader	1524	40%	42%	18%
     	23.6%
     Staff	4069	65%	30%	5%
     	62.9%
     TOTAL	6471	62%	29%	9%
     	100.0%

     • An alpha value of 0.05—that is, p‐value less than 0.05 (p ≤ 0.05)—represents moderate statistical significance (1 in 20 chance of being random)
     • An alpha value of 0.05—p‐value less than 0.01 (p ≤ 0.01)—represents high statistical significance (1 in 100 chance of being random)
     • An alpha value of 0.05—p‐value less than 0.001 (p ≤ 0.001)—represents very high statistical significance (1 in 1,000 chance of being random)

     As you can see, the lower your p‐value threshold, the higher the likelihood that your results are not random. Looking back at Table 1.11, you can see the results of statistical significance tests at p ≤ 0.05, p ≤ 0.01, and p ≤ 0.001. The arrows point up for results that are significantly higher and down for significantly lower. These results suggest that the experiences of African American or Black employees with disrespectful behavior and diversity‐related disrespect reflect very high statistical significance that warrant further investigation and likely action.

There are several software applications that can automate comparing current data with future data, generating cross‐tabulations, and performing tests of statistical significance such as SAS, SPSS, Stata, SUDAAN, and WesVar. Moreover, if you leverage a standardized DEI culture and climate survey from a vendor, it is very likely that they have also automated these analyses into their standard reporting. Later in this step, in the section “Phase VII: Deliver DEI Assessment Report,” we will explore organizational DEI assessment reporting in the context of sharing both the quantitative and qualitative findings together. A detailed discussion of how to visualize quantitative data can be found in Step 5: DEI Impact.

Phase IV: Conduct DEI Interviews and Focus Groups (Qualitative Data Collection)

While quantitative data is effective in understanding what people think/feel about a DEI topic, qualitative data is helpful in understanding why people think/feel a certain way about that topic, such as their experiences or personal observations and feelings of inclusion and belonging (or exclusion and marginalization). Qualitative data is an essential part of a DEI assessment because it provides a different and valuable lens to the lived experiences of others when compared to quantitative data.

Qualitative data captures the opinions, experiences, perceptions, and perspectives of people. It can be collected in a variety of ways including interviews, observation, focus groups (or listening sessions), and written documents. Here, I will focus on the two most prevalent ways to collect DEI qualitative data: in‐depth interviews (IDIs) and focus groups, while drawing directly from Focus Groups: A Practical Guide for Applied Research, fifth edition, by Richard A. Krueger and Mary Ann Casey.³

IDIs are one‐on‐one sessions between a moderator and a person and focus groups are one‐to‐many sessions between a moderator and several people. The moderator creates a safe psychological space that encourages participants to share perceptions and points of view.

IDIs and focus groups have four characteristics:

1. They involve people who possess certain characteristics
2. They take place in a physical or virtual setting
3. They engender focused discussion
4. They product qualitative data

The purpose of an IDI or focus group for an organizational DEI assessment is to better understand how people feel or think about issues relating to DEI. The intent of IDIs and focus groups is not to infer but to understand; not to generalize but to determine the range of perspectives; and not to make statements about the population but to provide insights about how people in the groups perceive an issue. IDIs and focus groups work when participants feel comfortable, respected, and free to give their opinions without being judged.

Ideally, the IDIs and focus groups are conducted several times with similar types of participants, so the moderator can identify trends and patterns in perceptions. Then, careful and systematic analysis of the discussion provides clues and insights as to how a topic is perceived by individuals and members of the group.

We will now walk you through the steps of conducting IDIs and focus groups for an organizational DEI assessment including determining the purpose, developing questions, establishing timing, selecting the number of participants, selecting and sampling the type of participants, moderating and debriefing, and coding and analyzing data. Because an IDI is essentially a focus group with one participant, hereafter I will only refer to focus groups but please be aware that each step equally applies to IDIs. The step will then conclude with how to report integrated findings from an organizational DEI assessment that combines quantitative and qualitative data.

Phase V: Analyze DEI Qualitative Data

When all qualitative data have been coded and categorized, the qualitative data analysis team must get a sense of the frequency of themes and findings, but the analytic process is more than arriving at the number of times a comment was said. At this stage, the qualitative data analysis team should give thought to a cluster of concepts:

• Frequency—How often was a concept mentioned?
• Extensiveness—How many different people mentioned the concept?
• Intensity—How much passion or force was behind the comments?
• Specificity—How much detail was provided by respondents?
• Internal Consistency—Did individual participants remain consistent in their views?
• Participant Perception of Importance—Did participants cite this as an important concept?

To more easily arrive at integrated and consolidated answers to these questions, give thought to various analytic frameworks. These frameworks allow you to examine the problem from different vantage points and have the potential for bringing focus to your qualitative data analysis. Base your choice of framework on the purpose of the DEI focus groups and what you are seeking to discover. There is no right or wrong framework.

The examples of analytic frameworks in Table 1.17 are not intended to be exhaustive but rather to point out how your qualitative data collection strategies and process of qualitative data analysis will differ depending on the purposes of your DEI focus groups. For example, if you are intent on uncovering factors that might influence change, then you will need more note‐taking strategies that allow you to identify the participant who makes each comment. And then you will need to trace that individual's comments throughout the discussion to identify indications of consistency or change. Moreover, keep in mind that your quantitative data can help you better understand your qualitative data. For example, if 70% of Latina women have expressed a desire to leave the organization, much information like this could help guide the focus group discussion, it can also help focus your analysis on comments that help explain why.

There are several software applications that can aid and automate qualitative data analysis including NVivo, Atlas.ti, and MAXQDA. As it specifically relates to DEI, co:census (cocensus.io) is an all‐in‐one text analysis and data visualization studio powered by patent‐pending natural language processing (NLP), aimed to reduce biases and prioritize ethical analysis. It offers templates, tailored for different use cases, and multiple integrations (e.g., SurveyMonkey, Twitter, etc.) to connect feedback from surveys, public comments, audio transcriptions, and social media and produce streamlined analytics and real‐time insights. In the conclusion of this book, I discuss a genre of tools that leverage Big Data and NLP to offer the next generation of qualitative data analysis that will offer unparalleled analysis and insights.

Here is some final advice for analyzing DEI focus group data:

• Much analysis is based on pattern identification.
• Sometimes pattern identification might not be appropriate. In certain situations, such as critical incidents, an event might occur only once and yet be of great consequence.
• Beware of personal bias and preexisting opinions about the topic.
• Remember that you are the voice of the participants!

A detailed discussion of how to visualize qualitative data can be found in Step 5: DEI Impact. We will explore how to report qualitative data later in the section “Phase VII: Deliver DEI Assessment Report” in the context of sharing the entire organizational DEI assessment results including quantitative and qualitative findings.

Phase VI: Analyze DEI Mixed‐Methods Integrated Data

If you have undertaken some or all preceding phases, then you are now sitting on a lot of mixed‐methods (quantitative and qualitative) data that forms the basis for your organizational DEI assessment according to the “4 P's” framework including:

• People: Personal Preferences and Competences
  • Preferences and Competences Data—You have quantitative data from the personal and team DEI inventories that characterize the preferences and/or competences of the people and teams that comprise your organization. This offers a lens to individual and organizational blind spots (via preferences) and areas of strength and improvement (via preferences and competences).
• People: Personal Behaviors and Experiences (Organizational Culture and Climate)
  • Survey Data—You have quantitative data from your DEI culture and climate survey that characterize what people are experiencing with your organizational culture and climate.
  • Interview and Focus Group Data—You have qualitative data from your IDIs and focus groups that characterize why people throughout your organization think and feel a certain way about how they experience your organizational culture and climate.
• Management Practices and Organizational Policies
  • Review or Audit Data—You have qualitative data from your DEI HR policies and practices review or audit including fact‐based findings and insight (review) or expert conclusions and recommendations (audit) relative to your HR functions as they relate to DEI.
• Performance
  • Benchmarking Data—You may have data that assesses your organization's capacity, maturity, ability, or “muscle strength” in specific categories that relate to DEI such as organizational structure, leadership involvement, approach, staffing and resources, measurement, and the like.
  • Ranking Data—You may have ranking data, based on a standardized submission, that compares your organization's DEI performance to other organizations or peers in your industry—regionally, nationally, and/or internationally.

Naturally, this can be a lot of data to process and interpret! As shown in Table 1.18, we use a simple mixed‐methods integrated data analysis framework at BCT—Findings, Evidence, Implications, and Recommendations—to translate what can be a multitude of data into actionable recommendations.

Leveraging this framework necessitates a close review of the reports and findings across all the organizational DEI assessments to identify the consistent themes and insights. We have found the four groups and the 15 categories of the GDEIB to be extremely helpful in organizing the analysis into useful categories. In other words, the GDEIB is not just a useful tool for benchmarking purposes, it also provides useful categories for analyzing and reporting assessment findings (and, as you will see in Step 2: DEI Imperatives and Step 4: DEI Initiatives, these categories are also useful for developing a DEI strategic plan). We have also found it helpful to engage a DEI Council—a diverse and inclusive body representing different functions, levels, roles, identities, and responsibilities throughout the organization—to aid in reviewing and interpreting the results.

For illustrative purposes, Table 1.19 provides an example of a mixed‐methods integrated data analysis based on the four groups of the GDEIB. However, do note that a comprehensive mixed‐methods integrated data analysis might necessitate a further breakdown into some or all of the 15 categories of the GDEIB.

Please note that the findings, evidence, implications, and recommendations generated from the organizational DEI assessment you have performed here in Step 1: DEI Inventory are critical to the proceeding steps of the Data‐Driven DEI cycle. During Step 2: DEI Imperatives, the findings will help you establish clear DEI objectives; the evidence, data, and results will establish a profile and baseline from which you can set specific DEI goals and determine impact; and the recommendations will inform which strategies you pursue to achieve your DEI objectives and goals. However, that is only after you complete Step 3: DEI Insights, which will identify promising and proven strategies that have worked for others. It is not until Step 4: DEI Initiatives that you will make a final determination of your DEI strategies and the measures to gauge progress.

Phase VII: Deliver DEI Assessment Report

The final phase is to deliver an integrated DEI assessment report that combines the quantitative and qualitative results and shares the findings as broadly as possible and as transparently as possible. This process may begin with a briefing of executive leadership, HR, and general counsel to mitigate any organizational or legal risks and then more broadly throughout the entire organization. Lyssa Test further advises, “Next, you'll need to share your analysis and action plan with your employees. Oftentimes, Human Resources teams or a [senior executive leader] will present … results during an [all‐hands] meeting, so they can walk employees through the findings and add context. These presentations should include an overview of participation rates, survey scores, organizational strengths, opportunities for growth, key findings, and historical or benchmark comparisons. You should end the meeting by reviewing your detailed action plan, showing employees exactly what your [organization] will be doing to address any issues raised by the survey.”⁵ Very few organizations emerge from an organizational DEI assessment unscathed but do not bury the truth. The more transparent your organization is with the results, the more you will engender trust, uplift the very tenets of DEI, and set the tone for the DEI journey that lies ahead.

An outline of a sample organizational DEI assessment report is shown in Figure 1.25.

For personal DEI assessments that leverage the kinds of tools highlighted in Step 1: DEI Inventory for People, in almost all cases, reporting work will be done for you and delivered in the form of an electronic or downloadable assessment report. For Step 1: DEI Inventory for Organizations, the form and fashion of organizational DEI assessment reports can vary. Organizations that specialize in conducting organizational DEI assessments will very likely have processes that semi‐automate or fully automate the data analysis and data reporting processes.

Once again, I cannot emphasize enough the critical relationship between data collection, data analysis, and data reporting. As mentioned earlier, by establishing your range of subgroups during data collection, you set the stage for data disaggregation—stratifying the data by the subgroups you selected, and data analysis and reporting—identifying and disclosing statistically significant differences between subgroups (e.g., women vs. men vs. nonbinary; married vs. single; people who are vision impaired vs. people who are hearing impaired, etc.) and between intersections of subgroups (e.g., married men who are vision impaired vs. single women who are hearing impaired). In other words, your data analysis and reporting can only be as detailed as the subgroups you have selected.

If disaggregated data may reveal a respondent's identity, it must be further aggregated to protect their privacy, confidentiality, and anonymity. For example, if your organization only has one female Hispanic employee, then results should not be disaggregated by ethnicity and gender because doing so will reveal the employee's responses simply by the process of elimination. Additionally, if the pool of Hispanic employees or female employees are sufficiently large to maintain privacy, confidentiality, and anonymity as separate pools, then the results can be stratified and disaggregated by ethnicity or gender but not by ethnicity and gender together. At BCT, our rule of thumb is that the number of respondents in a disaggregated group must be greater than 5 to 10 people to justify disaggregation.