7
CROSSING PARTY LINES
Incorporating Measures of Individual
Differences in Groups
LONDON BUSINESS SCHOOL
When asked to write a chapter on measuring personality and other individual differences in the context of groups research, I immediately said “yes” because I have been doing it for20 years since I was in graduate school at Berkeley (see Tetlock, Peterson, & Berry, 1993). Then I recalled all of the problems, questions, difficulties, and challenges in publishing individual difference papers that cross the strong party line in our profession between individual and group levels of analysis and thought that perhaps I had agreedtoo quickly. However, I have also always believed that real progress ismade by those who focus on interesting problems, regardless of, and in many cases especially because they focus on those problems that cross artificial professional boundaries. So, if I can help and encourage moreresearchers to span the divide and address the very important problems of how individuals interact to create group-level phenomenon, it will have been worth the time to write this chapter. In doing so, I have tried to outline my thinking through a series of questions that someone ne to the profession might have. First, why are these questions of individual differences in the study of groups interesting? Second, what are the challenges of studying individual differences in groups? Third, how do I work through some of those challenges and the specific problems of publishing group-level papers with individual difference measures?
Why Are Individual Difference Variables Interesting?
The first question any reasonable person interestedin studying groups should ask is why individual differences are interesting and important questions. If one looks at any one issue of journalslike Small Group Research or Group Processes and Intergroup Relations, one will see a multitude of important group-level variables out there (e.g., polarization, conflict management, virtual teams, cohesion, group development, etc.). So if one is interested in groups, why address questions by moving down a level of analysis? This question is, I suspect, behind many colleagues who have over the years shrugged their shoulders at my work on personality and individual differences. For many of the problems in our field, moving to individual differences might well be reductionist and unnecessary. However, many of the key questions about groups are actually questions about either individuals, or about how individuals come together to create group phenomenon. My own interest in individual differences came from being interested in leadership within groups. Does group leadership matter? Are some individuals better group leaders than others? What do effective leaders do that is different from those who are ineffective? Each of these questions, and others like them, require the researcher to lookat both the individual and group levels of analysis simultaneously.
Many other and less obviously individual-level questions being asked in the field of groups research also require a careful look at the individual level of analysis and an understanding of the interplay between individuals and groups. For example, one question that has interested many scholars in recent years is why two groups that report similar levels of conflict report very different levels of cohesion, satisfaction, and even levels of performance (e.g., Jehn, Rispens, & Thatcher, 2010). One explanation scholars have investigated looks at other related variables rather than individual differences: issuessuch as conflict resolution. Perhaps level of conflict experienced doesnot really predict these dependent measures reliably. Rather, what matters is not level of conflict, but how it is resolved. If it is resolvedamicably, then the group will have a positive trajectory and, if not, then there will be a downward trajectory (e.g., Behfar, Peterson, Mannix, & Trochim, 2008). However, there is a long history of group-level research on conflict in groups and the data would seem to suggest that levels of conflict experienced does matter often. So, the questions have come back to looking at the individuals in the team.
In response, scholars have begun to investigate the individuals within the group. Perhaps there is an asymmetry of conflict assessments and this is what might explain the differences between groups. For example, a group of four people could assess the level of task conflict or debate at a moderate level within the group in one of two very different ways. One group could individually assess the level on a five-point scale as 3, 3, 3, and 3 while a second group could assess itas 1, 1, 5, and 5. The first group has uniformly experienced moderate task conflict, believed by most scholars in the area as generally healthy with enough conflict to bring out differences, but not too much to bring out the risk of task conflict mutating into relationship conflict and damaging group satisfaction; see Simons & Peterson (2000) formore detail. The second group has half of its members who have experienced a potentially debilitating level of debate that makes it difficult for them to work with the group, thus damaging group satisfaction and performance.
To the extent that individuals within the group assess the level of conflict within the group differently, this invites a whole series of questions about individual differences and what is happening within the group between individuals. (1) Is it the case that the two who score the group more highly are clashing between themselves within the meeting and take it more seriously than theother two? (2) Is it the case that the two who score the group more highly are arguing outside of the official meetings so that the other two do not see it? Or (3) might it be the case that the two who score the group more highly are more sensitive to expressions of conflict and see things that the other two are not noticing? To address the phenomenon, one needs to consider all of these questions, and then the relevant follow-on questions. For example, if the answer is (3) that some individuals notice more nuance of conflict behavior than others, one needs to understand whether this is an individual personality difference (e.g., high agreeableness people are more attuned to, and place a higher priority on, how people are feeling as they interact), or potentially a cultural difference (e.g., individuals from high context and collectivist national cultures are more likely to notice and prioritize conflict behaviors because overt conflict is generally socially unacceptable). Whatever the answer, one is now firmly into the issues of group composition and reliable individual differences.
My example started with a simple group-level questionabout why some groups perform better than others, despite reporting similar levels of conflict. Our research questions lead us to a series ofindividual difference questions, that when answered will allow us to work back up to the group level to address the original question. There are many such issues and questions within the field of groups. Sometimeswe can get lucky and answer our group-level questions with exclusively group-level research. However, to answer many of the most interesting and practical questions about groups, we simply need to delve into individual-level research. That means that group researchers need to be prepared to measure individual differences within the groups that we study.
What Are the Challenges of Studying
Individual Differences?
So, if studying individual differences in the context of studying groups is so obvious, why do so relatively few people actually publish this kind of work? For many years I thought that the answers were simple–scholars just did not want to move away from what they know, there were not many scholars doing this kind of work so it might be perceived as risky, etc. In more recent years I have to cometo understand that the problem is much more complex. The story as I nowunderstand it is that personality and social psychology (i.e., the historical disciplines for where individual difference and group research began) both went through a crisis period in the late 1960s and early 1970s. Social psychology was chastised for recording the history of how university undergraduates responded to questionnaires rather than being about science and identifying universal situationalresponses (e.g., Gergen, 1973). Social psychologists, including most group researchers, responded by conducting controlled experiments, thus ensuring scientifically valid conclusions. There was also a secondary emphasis on ensuring that ideas were relevant to the world rather than being only about students. The result of which is that social psychologists, including the vast majority of scholars conducting groups research at the time, came to put special emphasis on ideas to be tested in a “critical experiment” pitting one explanation against another.
As the mainstream study of groups broadened to include business schools, communication departments, etc., the emphasis on experimentation has faded away and a much broader array of methods has become acceptable (see the range within this volume, for example). Thus, the long-term impact on the study of groups has been a focus on theoretical ideas to be tested, and a re-invigoration of the idea of the importance of relevance in the study of groups. Ironically, the latter of which was where groups research began with the Lewin studies of the 1930s and 1940s.
At about the same time as social psychology was in crisis, the study of individual differences was also under attack by one of its own, suggesting that consistency across situations was not actually very helpful or likely (see Mischel, 1968, 1969). The challenge wasnever about the relevance of the work to the world, but whether it was a fruitful path for personality psychologists to be looking for consistency across situations. Personality psychologists responded to this critique by focusing on measurement and methods to demonstrate how, when, and where there are consistencies in behavior over time, and thus to show which individual differences matter over time and situation. It took 25 years to establish that there are five large cross-situational individual differences that account for significant variance, the aptly named “Big Five” personality dimensions, including neuroticism/emotional stability, extraversion/surgency, openness to experience/intellectance, agreeableness, and conscientiousness (see McAdams, 1997). Thus, the legacy of the crisis in the study of individual differencesandpersonality is a special emphasis on methods and measurement and a fascination with the five key personality factors.
The problem for us as scholars who might be interested in studying groups by occasionally looking at some individual difference measures, is that the reviewers we are likely to encounter will be a mix of scholars who come from these two very different traditions. Any manuscript looking at both group and individual differences variablesis likely to get individual reviewers who are strongly focused on very different aspects of the manuscript. The reviewers who come more from the group-level tradition will be focused on the quality of the ideas (i.e., with a premium on whether it is new to the literature) and the reviewers who come from the individual differences tradition are more likely to be focused on precise and careful measurement. That is not to saythat someone from the individual differences tradition will not be focused on theory and ideas or that someone from a groups tradition will not be focused on methods and measurement. However, every research study requires trade-offs between rigor and relevance, theory and practical, and the newness of an idea versus how well measured it is. If the reviewers come from very different places and would be willing to make very different trade-offs, this constrains the choices one can make (more later on that problem) and makes navigating the gauntlet of journal revisions extremely difficult indeed!
This problem of pleasing two very different sets of reviewers is what I mean by the title of this chapter “Crossing Party Lines.” The reference here is to political parties. Political parties are defined not so much by specific policies: if they work, it is easy for both parties to join together and agree how to move forward. It is where there is no right answer, where it is a matter of taste or inclination, or more importantly guesswork as to what might succeed. Different political parties will start in different places when searching for answers to social problems. Similarly, individual difference and groups scholars will tend to start in different places when lookingfor answers and when needing to make judgment calls about how to address a problem. Neither is right all of the time, or probably even more ofthe time (i.e., do you not have findings because the idea is bad, or because it is measured poorly?).
On the other hand, where they both agree there is a problem (i.e., your “new” construct looks like and is measured similarly to an existing construct already in the literature), then as with political parties, the direction to be taken is probably fairly clear. The problem is that, as in politics, it is likely to be a relatively rare event. My experience in publishing papers with individual difference and group-level variables is that they tend to follow a hardroad in the publishing process, with many papers taking many years fromconception to publication; for instance, Peterson, Smith, Martorana, and Owens (2003) took 13 years from start to publication, most of which was spent in trying to appease these two very different sets of reviewers.
Before I go any further I do want to say categorically that it is possible to get individual difference and group-level measures published together, and in a reasonable time frame, if you are fully prepared to deal with the challenges. The third section of this chapter is specifically about those challenges, and includes my best advicefor how to avoid making some of the big mistakes, where to stretch versus where to be more conservative, and how to approach the use of individual difference and composition variables in the study of groups.
How Do I Work Through The Specific Challenges?
I would suggest that there are four key sets of interrelated decisions you will be making that will affect your experiencein trying to have the research published that includes both individual difference and group-level variables. They are: (1) writing multilevel theory; (b) selecting an individual difference measure; (c) deciding how to aggregate the individual difference measure to the group level; and (d) choosing a statistical approach to analyzingdata. I will address each of these questions in turn, drawing on my analysis of the challenges of publishing individual-difference variables in the context of groups research above.
Writing multilevel theory
Writing theory and developing novel hypotheses is never easy. When one combines it with the complications of working across levels of analysis, it becomes all the much harder. The key problem to avoid sounds simple enough, but is very difficult to avoid in practice. That is, assuming that how something works at the individual level is how it will work at the group level–just “add-up” the levels. For example, we know that extroverts talk more than introverts as individuals generally, but will a group where there is a mix of introverts and extroverts talk less than a group of all extroverts? It would seem obvious at one level to say “yes.” However, that would not necessarily be a good assumption to make. All it takes is one extrovert in a group to talk and fill the air space. Alternatively, looking at individuals who score highly on neuroticism (i.e., defined by the experience of negative emotions such as stress, anxiety, worry, etc.), does a group with two individuals who score highly on neuroticism worry twice as much as a group with one? Similarly to extroversion, it takes only one person who scores highly on neuroticism at the individual level to create the demand characteristic for the whole group to operate at a higher level of response to their concerns (see Peterson, Davidson, & Moynihan, 2007). In both of these two cases, how the individual difference variable operates at the individual level is decidedly not how it operates at the group level. All it takes is one high scorer on extroversion to fill the airspace in a meeting, or one high scorer on neuroticism to make the entire group respond to their worries.
The net effect of trying to avoid the problem of assuming individual processes translate directly to group processes is the need to identify when and under what conditions individual differences will manifest themselves. Will this difference affect all behavior across all time? Or is it contingent on other processes, situational circumstances, or organizational culture? Or might the effects of personalitydepend on the configuration of people in the group such that the group will look like the average of the individuals in the group, the most extreme scorer in the group (i.e., as with neuroticism), or the dispersion in the group (i.e., heterogeneity versus homogeneity on an individualdifference is key, e.g., homogeneity is helpful with agreeableness). This is what Lisa Moynihan and I called the contingent configurational approach to theory about the role of personality in groups (Moynihan & Peterson, 2001). When theorizing across individual and group level, one needs to be clear, careful, and consistent in explaining how eachof the levels operates, and specifically how individual behavior will interact to create group-level behavior.
Selecting an individual difference measure
The first instinct for many groups scholars trained in the social psychology tradition (i.e., whether they are in a communication department, business school, etc.) is to invent or modify an existing scale to accommodate the theoretical innovation or, even more acceptably, to take a scale someone else has published with once, since if it is published that means it must be alright. This is standard practice for those most who engage in groups research, but would be a highlyhazardous step in measuring personality or indeed most any individual difference variable. Remember from our discussion above about the importance of measures and methods within the individual differences tradition, reviewers are likely to want reassurance that your measure is both valid and reliable. So unless you want to engage in detailed scale construction yourself, which is highly technical and incredibly time consuming and data intensive (e.g., DeVellis, 2003; Rea & Parker, 1997), then one is best picking something “off the shelf.” That is not to say one can never innovate, but it will not be as easy as it is when one is dealing only with group-level constructs.
The best guideline is to ask yourself whether you candefend the reliability and validity of the scale applying the exact same wording or use of your individual difference scale. If the answer is anything other than a strong “yes,” then it is best to look further for other measures. There are literally hundreds of measuresthat have already been fully validated to the satisfaction of the profession. The most efficient way I have found to identify validated scalesis to go to the International Personality Item Pool (IPIP) website (http://www.ipip.ori.org/; see also Goldberg, 1999; Goldberg, et al.,2006). This website shares hundreds of scales, along with their full validity information and cross-correlations with other scales. It provides invaluable assistance in helping to select individual difference variables that are appropriate for particular questions.
One thought about the timing of when you might collect your individual difference variable, based on which variable(s) you choose: if your individual variable is personality, which is conceptualized as stable across time, then in principle the timing of your data collection should not matter. However, reviewers from the group traditionwill be uncomfortable with both personality and group-level data being collected on the same survey at the same time. They worry that the personality measures might be somehow tainted by the group experience and the questionnaires or both. So, ideally you will measure personality before the group interaction so that it is clearly unaffected by the groupexperience. If for some reason you can only administer it during or after the group experience, you will need to make the argument in the paper and to reviewers–another reason to stick with well-establishedscales. If your individual difference variable might be affected by situation (e.g., Integrative Complexity has been shown to be partly personality and partly situational; see Tetlock et al.,1993; Tetlock & Suedfeld, 1988), then the timing of when it is measured becomes critical and should be made in accordance with the predictions of your study.
Although personality questionnaires are standard practice in the profession and are able to handle most situations in which you might wish to measure personality, you may occasionally find yourself in a situation in which you cannot ask your participants to completea personality questionnaire. I found myself in this situation when I wanted to study Chief Executive Officers. Hard as I have tried, they generally do not answer researcher personality questionnaires! Or perhaps more realistically, if you have conducted a study of groups, having interviewed group members individually, and a reviewer starts asking some difficult questions about personality or group composition. It is possible to study individual differences and derive measures of personality from archival material, provided one has data describing individual behavior (see Peterson, Smith et al., 2003). There is a fairly wide range of variables that can be coded from archival materials, ranging from theBig Five personality variables to measures of Integrative Complexity and counts of specific individual behaviors. Here you will need to engagemultiple coders with a q-sort or other semi-projective measure of personality (see Meyers & Seibold, this volume, on engaging coders). Luckily, the IPIP does also have some help for you on these types of measures.
Deciding how to aggregate the individual difference
measure to the group level
Building on the points above about how to write good multilevel theory, the next key decision is about how to aggregate your individual difference scores to the group level. In the old days (i.e., ten years ago or more) the answer was simple: create a mean score to represent the group-level construct. The simple days are, alas, long gone, and definitely for the better. This crude approach to creating group-level variables that could then be run in a simple regression or ANOVA obscured many important effects and implied what we now know are incorrect assumptions about how individual level variables combine to create group-level variables. If you have built your group-level theory carefully (as explained above), it should tell you how to aggregate individual variables to create/predict group-level effects. Much of the timethis leads to straightforward answers. For example, if one is measuringthe effects of neuroticism, based on the argument that the group operates at the level of the most neurotic individual (see earlier) the bestway to represent this is by using the maximum score within the group, regardless of how the other members of the group score. Your group-level variable will be very different from that found if you had used the meanscore or standard deviation for the group. And if your theory is correct, you are much more likely to find significant results. Sometimes the answers are not always obvious, however. For help when the aggregation questions are much more complicated, see Harrison and Klein (2007) and Klein, Dansereau, and Hall (1994) for detailed help.
One additional complication has arisen in more recent years: group researchers are meant to provide justification for aggregating individual scores or individual assessments of group processes tothe group level. Particularly if a mean-level score is used, the reviewers want to know that the variable does have meaning at the group level. The key question is whether there is more variance within or between groups in the sample. If the variable in question has group-level meaning, then one expects much less variation within groups than between groups (i.e., with people who have not interacted). Typically, this is established by providing a set of Intra-Class Correlation or ICC scores (see James, Demaree, & Wolf, 1984). However, this is not always relevant for individual difference variables (like personality) that are not expected to be affected by group interaction. It could be a useful measure of attraction, selection, and attrition in naturalistic groups, demonstrating how those processes have narrowed the range of personalities within a particular group or organizational culture. However, reviewers may become confused and expect measures of justification for aggregation for representing group-level personality effects with groups that have been assigned (i.e., where each group would start and end with afull range of personality scores). Do not get confused that some of your measures may need justification (i.e., those that are directly affected by group interaction) and others may not!
Choosing a statistical approach to analyzing data
Once you have written good multilevel theory, carefully selected an individual difference measure that is appropriate to your theory, and then thought carefully about how you will operationalize that individual difference variable at the group level, it is now time to think about the statistical and methodological approach to your data. Are your hypotheses all at the group level? If they are, then you can choose a relatively simple approach to analyzing your data: an ANOVAif you have done a controlled experiment, or a simple regression if youare in the field (see those chapters in this volume as well). However, if you have hypotheses at both the individual and group levels, you ar going to need to engage in the newer multilevel methods (see Kashy & Hagiwara, this volume; Kozlowski, this volume; Klein & Kozlowski, 2000). These methods are much more complicated than the traditional ones you will see in older published work in the profession, but the rewards are in greater sensitivity to the data and the ability to hypothesize across levels of analysis.
A Few Final Thoughts
Having been trained as social psychologist to do groups research, rather than as a personality psychologist or individual differences person, my mistakes, and this chapter, have focused more on methodological issues than on issues of theory andidea development. If that is also your perspective, and for most readers I expect it will be since this is in a book about research methods for group research, I hope this chapter has highlighted the absolute necessity of getting better with methods, data, and levels of analysis issues if you are to incorporate individual difference measures successfully into your research. Fast. If you are already a methods and data guru, however, and you would like more on theory development in the area, I would recommend Kilduff (2006) for a punchy summary of what makes for good theory. There are many more resources out there to help you, but that summary will tell you succinctly what the idea-oriented people in theprofession will be looking for.
Finally, I wanted to share one final thought about the party lines analogy. As frustrating as it can be to have reviewers giving you divergent advice on how to develop your research, the rewards of pleasing both sides are in creating exceptional scholarship that is recognized across a very wide array of social scientists. Some of the very best work in the study of groups today is happening in this area. This work is fundamental to understanding the nature of groups and groupinteraction. If we can fully get to grips with how individuals come together to create team-level outcomes, we will have unlocked the black box that holds many of the remaining answers about why groups are at the core of every human society. And like a great democracy that is open tomany points of view, the study of individual differences in groups is frustrating and messy but somehow it works to create an excellent outcome. I encourage you to join into this fascinating research discussion.
References
Behfar, K. J., Peterson, R. S., Mannix, E. A., & Trochim, W. M. K. (2008). The critical role of conflict resolution in teams: A close look at the links between conflict type, conflict management strategies, and team outcomes. Journal of Applied Psychology, 93, 170–188.
DeVellis, R. F. (2003). Scale development: theory and applications (2nd ed.). London : Sage.
Gergen, K. J. (1973). Social psychologyas history. Journal of Personality and Social Psychology, 26(2),309–320.
Goldberg, L. R. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe (Vol. 7, pp. 7–28). Tilburg : Tilburg University Press.
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. C. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality,40, 84–96.
Harrison, D. A., & Klein, K. J. (2007). What's the difference? Diversity constructs as separation, variety, or disparity in organizations. Academy of Management Review, 32(4), 1199–1228.
International Personality Item Pool : AScientific Collaboratory for the Development of Advanced Measures of Personality Traits and Other Individual Differences (http://ipip.ori.org/), Internet website.
James, L., Demaree, R., & Wolf, G. (1984). Estimating within-group inter-rater reliability with and without response bias. Journal of Applied Psychology, 69(1), 85–98.
Jehn, K., Rispens, S. & Thatcher, S. M. B. (2010). The effects of conflict asymmetry on workgroup and individuals outcomes. Academy of Management Journal, 53(3), 596–616.
Kilduff, M. (2006). Editors comments: Publishing theory. Academy of Management Review, 31(2), 252–255.
Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data-collection, and analysis. Academy of Management Review, 19(2), 195–229.
Klein, K. J. & Kozlowski, S. (2000). Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp.512–553) . San Francisco, CA: Jossey Bass.
McAdams, D. P. (1997). A conceptual history of personality psychology. In R. Hogan, J. Johnson, & S. Briggs (Eds.), Handbook of personality psychology (pp. 4–29). San Diego, CA: Academic Press.
Mischel, W. (1968). Personality and assessment. New York, NY: Wiley.
Mischel, W. (1969). Continuity and change in personality. The American Psychologist, 24 (11), 1012, 8.
Moynihan, L. M., & Peterson, R. S. (2001). A contingent configuration approach to understanding the role of personality in organizational groups. Research in Organizational Behavior, 23, 327–378.
Peterson, R. S., Davidson, J., &Moynihan, L. M. (2007). Does one rotten apple spoil the barrel? Using aconfiguration approach to assess the conflict-inducing effects of a high neuroticism team member. In K. J. Behfar and L. L. Thompson (Eds.), Conflict in Organizational Groups (pp.93–112) . Evanston, IL : Northwestern University Press.
Peterson, R. S., Smith, D. B., Martorana, P. V., & Owens, P. D. (2003). The impact of chief executive officer personality on top management team dynamics: One mechanism by which leadership affects organizational performance. Journal of Applied Psychology, 88, 795–808.
Rea, L. M., & Parker, R. A. (1997). Designing and conducting survey research: A comprehensive guide.San Francisco, CA: Jossey-Bass Publishers.
Simons, T. L., & Peterson, R. S.(2000). Task conflict and relationship conflict in top management teams: The pivotal role of intragroup trust. Journal of Applied Psychology, 85, 102–111.
Tetlock, P. E., Peterson, R. S., & Berry, J. M. (1993). Flattering and unflattering personality Portraits of integratively simple and complex managers. Journal of Personality and Social Psychology, 64, 500–511.
Tetlock, P. E., & Suedfeld, P. (1988). Integrative complexity coding of verbal behavior. In C. Antaki (Ed.), Lay explanation. Beverly Hills, CA: Sage.