3

Cooperation and reputation in dynamic networks*

3.1 Introduction

Chapter 2 introduced a simulation study on coordination problems in dynamic social networks. This chapter introduces another simulation model of social dilemmas in dynamic networks but this time with a different game as the social dilemma— instead of coordination, we now look at cooperation. Dyadic cooperation is one of the building blocks of human societies. Whether this cooperation be colleagues working on a project together, friends providing social support, neighbors providing practical support, or firms involved in R&D collaboration, people interact to produce something they could not have produced alone. In many cases, both parties can benefit from mutual cooperation, but both are also tempted to take advantage of the other party for an even larger benefit. In such cases, cooperation takes the form of the well-known two-person Prisoner's Dilemma (see Figure 3.1). The standard game-theoretic prediction is that actors will not cooperate in such a dilemma. Consequently, a large body of literature has evolved to address the question—under what conditions is cooperation in the Prisoner's Dilemma possible? One major finding is that cooperation is possible (but not guaranteed) when an interaction is repeated. Another mechanism that is often claimed to promote cooperation is embeddedness in social networks. In this chapter, as before, we study what happens if we do not assume that these networks are exogenously fixed but instead assume that they co-evolve with behavior in Prisoner's Dilemmas.

Figure 3.1 The Prisoner's Dilemma, in general form and with numerical payoffs, with T > R > P > S.

The chapter is structured as follows. In the remainder of this section, we discuss the theoretical motivation for this study, introduce learning effects in networks, and discuss the related theoretical literature. In the next section, we introduce a theoretical model for cooperation in dynamic networks that is based on learning heuristics. In Section 3.3, we mathematically analyze the model, resulting in a general characterization of stable states of the process. However, as in Chapter 2, these characterizations still leave room for many different stable states. Therefore we set up a simulation model in Section 3.4 with the aim of examining more closely what kind of stable states are likely to emerge under given initial and structural conditions. Section 3.5 presents the results of the simulation, and Section 3.6 concludes with a general discussion.

3.1.1 Cooperation and network effects

A classic sociological argument holds that embeddedness in social networks can promote cooperation in situations involving social dilemmas (e.g., Coleman 1990; Granovetter 1985; Homans 1951). Casual observation suggests that cooperation is more frequent in close-knit groups than in anonymous interactions, and indeed, there are many empirical examples in the literature of the association between network cohesion and cooperation. For instance, Uzzi (1996, 1997) shows that small firms are more likely to cooperate if their interactions are embedded in a dense network.

There are a number of mechanisms that have been proposed to produce higher rates of cooperation in dense networks. Buskens and Raub (2002) distinguish between control and learning effects in social networks. Control, also known as “social control” (e.g., Homans 1951), refers to the idea that actors are more inclined to cooperate if information about defection can spread through a given network in a way that leads to sanctions by third parties. Thus, actors cooperate because they are concerned with the future consequences of defection. Raub and Weesie (1990) develop a formal model that shows that network embeddedness can encourage cooperation in the Prisoner's Dilemma; Buskens (2002) similarly analyzes how network embeddedness can promote cooperation in one-sided Prisoner's Dilemmas or Trust Games. Empirical illustrations of control effects are described by Ellickson (1991), among others.

The second mechanism through which networks affect cooperation is learning. Here, it is information on past interactions that matters. An actor may be more inclined to cooperate with another if he or she learns from a third party that cooperation with this other actor may be more fruitful than defection, because, for example, this other actor appears to be playing according to a tit-for-tat strategy. Other useful information that actors may learn through a network concerns the payoffs of the other player (Buskens 2003; Kreps and Wilson 1982). While the focus in the literature is mainly on the positive effects of network density on cooperation, this effect may also be negative. Burt and Knez (1995), for instance, argue that higher network density tends to make actors more certain of their perceptions regarding interaction partners. If this perception happens to be negative, higher network density will only amplify the tendency to defect. Rapoport et al. (1995) similarly report negative effects of third-party information in an experimental study.

Both the control and the learning mechanisms rely on the transfer of information about the behavior of an actor to third parties. In the case of control, this information allows a third party to threaten an actor with sanctions if he or she does not cooperate. In the learning case, this information allows a third party to decide on the fruitfulness of cooperation. That is, both mechanisms are examples of reputation effects. Reputation is one of those concepts that is so widely used that its precise meaning has become obscured. Generally, it refers to an attribute of an actor ascribed to him by other actors (Raub and Weesie 1990; Wilson 1985). Raub and Weesie (1990) further distinguish between reputation in the narrow sense and reputation in the broad sense. Reputation in the narrow sense refers to situations in which the behavior of an actor influences his or her reputation in this situation, while reputation in the broad sense refers to situations in which behavior in one interaction influences other interactions. We are concerned with reputation in the broad sense.

Notice that the inclusion of reputation effects is one of the most important differences with the model of coordination problems discussed in Chapter 2 next to the difference in the underlying social dilemma. In the coordination model of Chapter 2 the network played no role in the dissemination of information. In the context of coordination problems there is also little for modeling information diffusion in a more complex way, as myopic best-reply strategies are typically already reasonable approximations of optimal behavior in such problems, and actors have little need for judging the reputation of other actors. In cooperation problems this is quite different—here, myopic best-reply behavior would immediately lead to a breakdown of cooperation, and we need to look at more complex mechanisms to explain the emergence of long-term cooperation.

3.1.2 The case for network dynamics

Summarizing the theoretical ideas described above, we can say that networks are presumed to affect cooperation because they facilitate reputation effects by allowing for the transfer of information. Theories on reputation effects share the assumption that social networks are determined exogenously. Generally, the social network is considered to be a stable social context that is imposed on actors and that provides the means for the spread of information through which the mechanisms of control and learning can work. In recent years this assumption has been challenged in network research. More and more, it is recognized that social networks themselves are also the result of interactions in which actors make conscious decisions about their social relations. The recognition of this fact has led to an explosion of interest in social network dynamics over the past several decades in the field of sociology (e.g., Doreian and Stokman 1997; Snijders 2001; Weesie and Flap 1990), economics (e.g., Dutta and Jackson 2003; Jackson and Wolinsky 1996), mathematics, and physics (e.g., Newman et al. 2006; Watts and Strogatz 1998).

There are at least two reasons to suspect that relaxing the assumption that networks are exogenously fixed has implications for the theoretical predictions about the effects of reputation. First, it is not self-evident that reputation is equally effective if the network through which information is transferred changes as a result of what occurs during actual interactions. For instance, if an actor experiences defection by an interaction partner, he or she might “spread the word” through his or her network connections such that the defector can be sanctioned. Alternatively, he or she might end the interaction with this partner altogether, because he or she prefers not to interact with partners who defect. In that case, he or she unintentionally also changes the possibilities for the spread of reputations for other actors in the network. Conversely, network decisions themselves may be affected by reputation effects; actors might be more willing to initiate interactions with potential partners who have a cooperative reputation and may be less willing to interact with actors who are known to defect. It is theoretically unclear whether and how reputation mechanisms function in a dynamic context. The main aim of this chapter is to shed some light on this question.

A second problem is that understanding networks as dynamic raises a possible problem of causal order. Traditionally, it is assumed that the network comes first in the relationship between network density and cooperation; cooperation is seen as the result of the network structure. If we assume that networks may be dynamic, however, we must also acknowledge that the causation could also operate in the other direction. If interactions are more likely between actors who tend to cooperate than between actors who defect (for instance, because mutually cooperative interactions are more rewarding than relationships of mutual defection), then high density would be the result of an already high level of cooperation rather than its cause. Because most empirical studies of network effects on cooperation are cross-sectional, they cannot distinguish between these two possible causal directions.

3.1.3 Learning in networks

In this chapter we limit the model of reputation effects to learning and neglect control effects. Thus far, combining control and learning effects in one model has proven to be difficult, even for fixed networks (Buskens and Raub 2002). For dynamic networks the complexity of these models would likely increase even further. We believe that more theoretical progress can be made at this point by studying the effects of network dynamics on one of the two mechanisms than by immediately trying to combine learning, control, and network dynamics into one model.

Aside from this somewhat pragmatic consideration, there are also substantive reasons to first focus on learning. Extending the cooperation problem with network dynamics increases complexity not only for the modeler, but also for the actor we are trying to model. He or she now finds himself or herself in a situation in which he or she has repeated interactions with multiple partners and in which an action may have complex externalities due to reputation effects. Models that assume that actors are perfectly rational (as game-theoretic models of control effects typically do) assume that actors can perfectly anticipate all future moves of their interaction partners, and thereby already make quite strong demands on the cognitive capabilities of the actors. If the network is also dynamic, this means that actors are required to anticipate the consequences of their actions by taking into account that the information structure itself may also change as a result of their actions. This makes it increasingly unlikely, in our opinion, that actors will display the rational forward-looking behavior assumed by game-theoretic models of control. Learning models, in contrast, rely on a backward-looking logic and, as a consequence, must make much less demanding assumptions regarding the rationality of actors (Macy and Flache 2002). For this reason we feel that a learning approach would be most useful for studying our problem.

There are different ways of modeling learning in social dilemmas. One broad class of models concerns reinforcement learning (e.g., Camerer and Ho 1999; Erev and Roth 1998). According to this method actors learn to choose actions that result in high payoffs based on previous experiences. These experiences may include their own experiences as well as the experiences of others. In the latter case actors might imitate others who are successful by adopting their strategies. This model is often used as a social analogue of evolutionary reproduction dynamics (see Cohen et al. 2001 and Ohtsuki et al. 2006 for models incorporating networks). Such models often make the rather strong assumption that actors are informed regarding both the relative success of other actors and the strategies used by these actors. We feel that these assumptions are too strong for most social interactions (cf. Weesie 1996).

A second class of models comprises belief-based learning (Camerer and Ho 1999). In these models actors form beliefs about the behavior of their interaction partners and then choose the best response against this behavior. Beliefs can be based on an actor's own past experience, but information can also come from other sources. Given our conception of reputation, belief-based learning provides a natural approach to study the effects of reputation in networks. Reputation in a belief-based learning context enters the decision-making of actors as information learned from third parties.

3.1.4 Related theoretical literature

Studies that model repeated Prisoner's Dilemmas (RPDs) using some sort of partner choice can be roughly distinguished into two categories. On the one hand there are models in which actors play dyadic games, as in our model. This means that actors can discriminate between different interaction partners through their behavior in the game. On the other hand there are models in which actors choose one action for all interaction partners and thus cannot discriminate. This difference changes the strategic situation considerably; if one cannot specify different actions for different partners, it is impossible to sanction or reward specific partners because every choice affects all partners. Models of cooperation in dynamic networks without discrimination are studied in different setups by Eguíluz et al. (2005), Ule (2005), and Biely et al. (2007). Of these, Ule (2005) includes a reputation mechanism for control, yet in this model reputation does not depend on the network. Biely et al. (2007) assume the development of some type of reputation via learning but do not explicitly model its underlying mechanism.

Models that study dyadic RPDs with partner choice can be traced back to Schuessler (1989), who studies the effects of an “exit-option” in a computational tournament using a method similar to that of Axelrod (1984). Vanberg and Congleton (1992), Stanley et al. (1994), Yamagishi et al. (1994), Weesie (1996), the EdK-Group (2000), and Hauk (2001) all conduct similar analyses using various setups.

None of these studies, however, includes a reputation system. In effect, interaction still takes place in independent dyads, and although in principle we can speak of a network of interactions, there are no effects of network structure. The only study that we are aware of that does discuss a network-based reputation mechanism is Vega-Redondo (2006), who presents a game-theoretic analysis of the RPD in an endogenous network. Reputation (albeit not by that name) enters the analysis as players punish interaction partners when they learn of defection by a partner via the network. This is indeed reputation in the sense of control, which is different from the reputation included in our approach. Moreover, reputation only plays a limited role in network formation. The analysis of this model focuses mainly on the effects of the volatility of the environment and shows that more volatility in the long-run leads to lower network density as well as to lower average distance in the network.

Finally, Pujol et al. (2005) study dyadic support games in a dynamic network setting with reputation effects. Although reputation is network dependent in their model, its role is limited. Third-party information comes only from direct neighbors and, moreover, is only used when first-hand experience is not available. A second drawback of their model for our purposes is that free-riding is excluded by assumption; actors are modeled as “benevolent” (Pujol et al. 2005, Section 2.20). This partly assumes away the very problem we want to address, namely, the emergence of cooperation among egoistic actors.

In the following section we develop a model for cooperation in dynamic networks with reputation effects based on belief-based learning. We then present some analytical results on stable states for different levels of reputation effects in both fixed and dynamic networks. Because the resulting theorems characterize stable states in rather general terms and say nothing about the generating process, we extend these formal analyses via a computer simulation of our model.

3.2 The model

3.2.1 Formalization of the problem

The stage game

The basic interaction is modeled as an infinitely repeated two-person Prisoner's Dilemma. Thus, in every stage of the process, actors play a game, as illustrated in Figure 3.1. In this game actors can collectively benefit from mutual cooperation, but they also have an individual incentive to free-ride on the efforts of the other player. Future payoffs are discounted by a discount parameter w.

Games in networks

There is a finite set of actors N = {1,2,3, …, n}. Actors play two-person RPDs and can be involved in multiple games at the same time. They can also behave differently with different partners. For a population of actors the collection of dyadic relationships results in a network of relations. A network of n actors can be represented by the n × n adjacency matrix g, where g(i, j) = 1 if there is a link from i to j, and g(i, j) = 0 otherwise. By assumption, reflexive links are ruled out such that g(i, i) = 0 for all i. The network of interactions is undirected by nature, and therefore, g(i, j) = g(j, i) for all i and j. The set of actors with whom actor i has a link is formally denoted as , and these actors will be referred to as the neighbors of i.

Information

We make restrictive assumptions regarding the information available to the actors. Actors are informed of the actions chosen by their interaction partners, but they are not informed of the strategies according to which their partners are playing the game. Moreover, actors are not aware of the structure of the network beyond their own connections. This also implies that actors are not informed about the payoffs their partners receive.

However, while information available to individual actors is thus rather limited, the network structure allows for diffusion of information among actors, which makes the emergence of reputations possible. However, the diffusion of information is assumed to have its limits. It does not flow effortlessly through the network but rather decreases in reliability with each “step” through the network. We model this process explicitly when we discuss how actors use information.

Network dynamics

A key property of our model is that the interaction network is not exogenously imposed but can rather be changed by the actors. A simple way to build this into the model is to consider a number η of randomly drawn pairs of actors who can decide whether they want to change their relation, that is, create a new tie between them or delete an existing tie. To have a network tie means to interact, and so it seems natural to allow these ties to be created with mutual consent, though they can be deleted unilaterally.

Another important assumption is that maintaining network ties is costly. The underlying reasoning for this assumption is that the maintenance of social interactions requires some effort, and therefore, actors may want to end relationships that are less profitable. Assigning a fixed cost to every interaction is a convenient way to model this (cf. Jackson and Watts 2002).

Formally, the total cost k for maintaining z ties for actor i in each round of play is given by the simple linear function

(3.1)

in which zi denotes the number of ties actor i is involved in (i.e., zi = |Ni(g)|) and α ≥ 0.

The dynamic process

The various components of the model are combined into a repeated game in the following way. Each period of the process consists of three phases:

After phase 3, we return to phase 1, and the process repeats. In phases 1 and 2, actors rely exclusively on the information they received in phase 3, from their own experiences, or through reputation. Thus, after new ties are formed in phase 1, there is no additional spread of information, and decisions during the game are based on the same information as the decisions made during the network formation phase.

3.2.2 Individual strategies

The description of the situation above shows that the decision problem for the individual actor is rather complex. How does one decide which strategy to choose in this repeated game? In principle, the number of possible strategies from which to choose is infinite. This is also true for the possible strategies of the opponent, which makes the situation even more complicated. Moreover, our network setting in which information can spread as well as the choice of relationships may introduce further externalities with regard to this spread of information, thus leading to an extremely complex decision situation with a very large range of action alternatives and possible outcomes. This situation would demand such extreme requirements in terms of information processing and computation that we think it is unlikely that human actors would be capable of acting completely rationally in such a context (cf. Conlisk 1996).

Instead, we prefer to model actors as boundedly rational in the sense that actors make a number of simplifying assumptions about the world around them as well as make use of information in a possibly suboptimal manner (Rubinstein 1998; Simon 1956).

More specifically, we assume that actors consider only “t − 1 matching strategies” by the opponent (Downing 1975). That is, actors assume that their opponent's action in the game is a response to their own action in the previous period. A simple and famous example of such a t − 1 matching strategy is the tit-for-tat strategy in which an actor always copies the last move of his or her opponent, though many other strategies can be expressed in this fashion. Generally, a t − 1 matching strategy of actor j against i can be described in terms of two probabilities:

: The probability that j will cooperate at time t after i cooperated at time t − 1;

: The probability that j will cooperate at time t after i defected at time t − 1.

We propose that actors try to maximize utility in the repeated game by assuming they interact with an opponent who uses a t − 1 matching strategy with unknown response probabilities.1 As the precise strategy of the opponent is not revealed, maximizing utility involves making an assessment of the most likely values of and with regard to the strategy of the opponent. In effect, this means that actors assume they are playing against a probabilistic automaton that is driven by two conditional probabilities. This leaves the actor with two tasks—first, to figure out the probabilities according to which the opponent is playing and, subsequently, to determine the optimal response given these probabilities.

To obtain an assessment of the behavior of the opponent in terms of and , actors simply use the frequency distribution of the opponent's responses in the game so far. That is,

(3.2)

in which is the total number of times that i played action a (C or D) against j until time t, and is the total number of times that j reacted with cooperation to action a by i until time t. We assume that at t = 0 actors have a prior belief about the behavior of each opponent in terms of a “fictional” and .

The next task is to determine the optimal response when playing against an automaton with two given response probabilities, and . It can be shown that to maximize utility in such a situation it is sufficient to consider only three different courses of action, which we call sub-strategies for convenience. These are

For , which we assume throughout for simplicity, the expected payoff πij for actor i interacting with actor j for each of the three courses of action can be written as

(3.3)

(3.4)

(3.5)

The actor's behavior as described so far is equivalent to the strategy known as the “DOWNING” strategy, which was originally proposed by Downing (1975) and was subsequently the name of one of the competitors in the famous computer tournament conducted by Axelrod (1984) From here on, we use the label “DOWNING” to refer to the individual strategy used in this chapter.

3.2.3 Reputation

According to the original DOWNING strategy the actor bases his or her behavior on expectations he or she forms from his or her own experience with an opponent, but it is relatively straightforward to include reputation (i.e., third-party information) in the model. Experiences of other actors with whom an actor is directly or indirectly connected can be used to estimate the probabilities of the opponent's expected behavior as discussed above. Since it is likely that information from third parties is treated differently (Burt and Knez 1995; Granovetter 1985), the weight an actor assigns to third-party information, as opposed to information from his or her own experience, is a parameter of the model.2 Another parameter pertaining to reputation may be the extent to which information spreads through the network, for example, the maximum distance that information can travel.

In everyday language reputation is often conceived as an attribute of a single actor. For example, we say that a certain person “has a good reputation” or has “lost his good reputation.” In contrast, in our model, “the” reputation of an actor may consist of a set of reputations that are attributes of other actors. The reputation of actor i is the set of beliefs in the minds of other actors about the behavior of i learned from sources other than their own experience. These beliefs do not need to be the same; different actors can have different beliefs about the behavior of i.3

More specifically, the reputation of actor j with actor i (i.e., the information that i has about the behavior of j in interactions not with i) consists of

• The total number of times actors other than i played C to j to the knowledge of i;
• The total number of times j responded to C with C in interactions not with i to the knowledge of i;
• The total number of times that actors other than i played D to j to the knowledge of i;
• The total number of times, to the knowledge of i, that j responded to D with C in interactions not with i to the knowledge of i.

The phrase “to the knowledge of i” is meant to indicate that i does not necessarily have information on other interactions in which j is involved; that is, i has this information only if i is directly or indirectly connected to other actors interacting with j. Moreover, it is likely that information received by third parties has a smaller influence on i's decision than his or her own experience. For instance, i might have less confidence in third-party information, because it is more likely to be distorted the farther it travels through the network. To model such effects, we assume that the weight of the third-party information that i receives about j depends on the network distance δ through which this information is transferred. If i and j are connected, then the reputation of j with i consists of the expectations of j's neighbors k about j weighted by the network distance δik between i and every k. δik is defined as the shortest path in the network between i and k (see Wasserman and Faust 1994, p. 110). The information is weighted in the sense that the information obtained from k is subject to network decay, and it is considered less important to i the larger the shortest distance from k to j becomes. These quantities, in combination with those obtained from one's own experience, are used to compute the probabilities that the DOWNING uses to assess an opponent's future behavior. We modify Equation (3.2) in the following way—recall that Nj(g) denotes the set of j's neighbors. Let ω () denote the extent of network decay of information. Then, the probability that j will cooperate at time t after i played action a is computed as

(3.6)

with . In the numerator of this equation, represents i's own experiences with j, while the term on right-hand side denotes the combined experiences of j's other neighbors; the terms in the denominator are likewise interpreted. The experiences of j's neighbors are weighted by ωδ. Thus, if , information learned via reputation is discounted with a factor with every step it travels through the network. When ω = 0, reputation does not play any role; when ω = 1, information from any source is given the same weight.

A few additional remarks are needed before we proceed. First, note that the reliability of different kinds of information is modeled by counting events. If information from third parties is based on more observations (e.g., from sources with a long interaction history, a large number of sources, or both), then this information might have more influence on than information from one's own experience (given ω). In this sense, this reputation system is similar to the systems that are implemented by online auction websites such as eBay in which a potential buyer can verify the reputation of a seller in terms of both the number of positive ratings by previous buyers and the total number of transactions in which the seller has been involved (Snijders and Zijdeman 2004).

Second, it is important to realize that information does not “accumulate” when it travels through the network. That is, when an actor reports to another actor about the behavior of a third actor, he or she reveals only information from his or her own experience and does not include the information he or she has obtained from his or her own sources. Again, compare this to eBay's system. There, buyers are supposed to base their feedback only on their own transactions and not to include previous ratings by other buyers of the same seller in their own ratings.

3.2.4 Network decisions

How do actors decide on the creation of a new tie or the deletion of an existing tie? Basically, a tie is only created or maintained if the expected payoff from the interaction exceeds the cost of maintaining the tie. The expected payoff is the payoff the actor would obtain if he or she interacted with the potential partner under consideration. Thus, when deciding on whether to change the status of a tie, an actor i

1. Computes and as if he or she were interacting with j using available information;

2. Decides which sub-strategy he or she would use if he or she were interacting with j (Equations (3.3)–(3.5));

3. Compares the expected payoff from using this sub-strategy (which is normalized to one period) with the cost of maintaining the relation.

If the result of this evaluation is nonnegative for both actors, the tie is created or kept; if the result is negative for one or both of the actors, it is either not created or is deleted.

Because we focus this chapter on reputation effects in dynamic networks, we abstract here from strategic considerations concerning the transmission of information. That is, we assume that information flows through the network without modeling the decisions of actors to actually pass information. Lippert and Spagnolo (2005) discuss this issue but abstract from network formation (see also Buskens and Raub 2002 and Buskens 2002, Chapter 7). Similarly, we assume that actors do not purposefully try to obtain strategic information.4 In some applications, it is likely that actors maintain certain relationships regardless of the payoffs, only because a relationship is a source of important information that can be used in other interactions. However, for now, we consider information transmission to be an unintended consequence of network formation. The situation in which actors form links with the explicit aim of obtaining information is captured in general terms by the “connections model” presented in Bala and Goyal (2000).

3.2.5 Convergence

We define the dynamic process as converged when two conditions are met. First, there is no pair of actors willing to create a new tie, and there is no single actor willing to remove a tie. This criterion conforms to the notion of pairwise stability of networks, as formulated by Jackson and Wolinsky (1996). Second, the beliefs of actors are stable in the sense that they converge. The convergence of beliefs implies that sub-strategies are also stable. Looking at behavior in the game directly (as is done in, e.g., Jackson and Watts 2002 and Buskens et al. 2008) is not a sufficient criterion for stability of the process under study, because alternation is one of the possible sub-strategies, and moreover, stability in behavior for some period of time does not imply stability of beliefs.

To conclude the presentation of the model, it is useful to once again make a comparison between the model of cooperation in the current chapter and the model of coordination presented in Chapter 2. We already mentioned the different role of information—in the coordination model actors are not interested in how actors interacted with third parties, while in the cooperation model reputations play a central role. Another difference is that in the cooperation model actors play dyadic games, in the sense that they choose actions in interactions with each of their partners separately. In the coordination model actors choose only one action for the interactions with all their partners. This has implications for how networks come into play in the respective models. In the coordination model network effects emerge because interdependencies between actors overlap—if actors i and j are both neighbors of actor k, k's actions will be influenced by those of i, and because k can only choose one action against all his or her neighbors, j is also affected by these actions of k. Consequently, i and j are also interdependent because of the network structure even if they are not connected to each other.

In the cooperation model network interdependence comes from an entirely different source. Here, because actors play separate games with each of their neighbors, there is no “overlap of interdependence” as in the coordination model. Instead, network effects come from reputation mechanisms, which are here modeled as by the flow of information through the network. In the absence of such effects (which is a baseline case that we will examine shortly), the network structure plays no role at all, and the process reduces to a collection of two-person Prisoner's Dilemmas, in which just some actors happen to play more games at a time than others. Only as we increase the importance of reputation effects the network structure starts to be consequential.

3.3 Analysis of the model

In this section we derive some analytical results for the model described above. The main focus is on stable states of the co-evolution process, that is, converged states of the process. First, however, we briefly discuss the behavior of the DOWNING strategy in the two-person case, as this will be helpful in the subsequent analyses of fixed and dynamic networks.

3.3.1 Dynamics of behavior with two actors

In this section we analyze the dynamics of play when two players using the DOWNING strategy are interacting and when neither player can end the interaction. Examining this relatively simple case provides a useful basis from which to move forward to the more complex case involving reputation effects and network dynamics. The main result for the two-player case can be summarized as follows.

Theorem 3.1 If two actors are both using the DOWNING strategy in a two-person RPD, the only stable sub-strategy-combinations are (ALLC,ALLC) and (ALLD,ALLD).

We omit the complete proof here, instead providing a sketch of the argument. Generally, the actor playing the DOWNING strategy cooperates if he or she finds that the opponent is sufficiently reactive in the sense that the opponent is sufficiently more likely to cooperate after cooperation than after defection. This implies that must be sufficiently larger than for i to play ALLC. If, however, and are equal, the actor playing DOWNING concludes that the opponent is not reactive at all. That is, the behavior of the opponent does not depend upon what the actor does. In that case, it is more lucrative to play ALLD, because each period can then be considered effectively independent. Thus, in a sense, one could say that the actor playing DOWNING defects when he or she can but cooperates when he or she has to.

More precisely, from Equations (3.4) and (3.5), we can derive that an actor i playing DOWNING will switch from ALLD to ALT if π (ALT)ij > π (ALLD)ij, that is, if

(3.7)

Similarly, actor i switches from ALT to ALLC if

(3.8)

This implies that the conditions under which DOWNING plays ALLC are rather limited. Regarding the numerical example in Figure 3.1, this can be illustrated by plotting as a function of according to Equations (3.7) and (3.8) (see Figure 3.2).

Figure 3.2 Conditions for switching between sub-strategies, with T = 5, R = 3, P = 1, S = 0.

From Figure 3.2, it can also be inferred that in the absence of reputation effects, the only combinations of sub-strategies that are stable are (ALLC,ALLC) and (ALLD,ALLD). Table 3.1 summarizes the sequences of play for the different combinations of sub-strategies. Suppose that two actors are both using ALLC, which implies that they are both in the region above the solid line in Figure 3.2. In that case, will converge to 1 for both as t goes to infinity, while will not change, because D is never played. This implies that both will remain in the ALLC region and continue playing ALLC. Therefore, this combination is stable. If both are playing ALLD, both and will converge to 0. In Figure 3.2, this means that the beliefs of both actors would move leftwards parallel to the horizontal axis, resulting in the continued play of ALLD or a change to ALT by one or both actors as soon as the dotted line is crossed. However, from Table 3.1, it can be verified that no combination of sub-strategies that involves ALT can be stable; in all cases, the resulting beliefs converge to a combination that does not support ALT. Therefore, (ALLC,ALLC) and (ALLD,ALLD) are the only two stable combinations.

Table 3.1 Sequences of play for different combinations of sub-strategies of DOWNING and convergence of actor beliefs for .

Table 3.2 Parameter values of the simulation.

Parameter	Description	Values
t	Temptation payoff	5
R	Reward payoff	3
P	Punishment payoff	1
S	Sucker payoff	0
ω	Extent of reputation diffusion	0,
η	Speed of network formation	0, 25, 435
n	Number of actors	30
α	Linear tie costs	0.9, 1.9, 2.9

3.3.2 Stable states in fixed networks

We now extend the analysis to a “network” setting in which actors play with multiple partners, but we limit ourselves to fixed networks (η = 0). The crucial difference between the network setting and the two-player setting analyzed in the previous section is that actors can share information, and reputations can thus emerge. In the model the extent to which reputation plays a role is determined by the parameter ω. We first look at two extreme cases, namely, ω = 0 (no diffusion of reputations) and ω = 1 (perfect diffusion of reputations). Let denote the sub-strategy used by i against a neighbor j; may be ALLC, ALLD, or ALT.

Theorem 3.2 

(i) If ω = 0 and η = 0, then the only stable sub-strategy-combinations are (ALLC, ALLC) and (ALLD, ALLD).

(ii) If ω = 1 and η = 0, then in stable networks. That is, all sub-strategies chosen against a given actor j are identical.

Proof. Case (i) of Theorem 3.2 is simply a reiteration of Theorem 3.1. In the absence of reputation effects (ω = 0), the network setting is equivalent to the two-person setting. If ω = 1 (Case (ii)), then Equation (3.6) reduces to

(3.9)

(omitting the subscript t; , ) such that for all i, j, k who are directly or indirectly connected. This implies that if two actors are (directly or indirectly) connected, they are acting on the same information. Because the choice of a sub-strategy depends exclusively on the conditional probabilities of cooperation, it follows that all i and k will choose the same sub-strategy against j.

Theorem 3.2 states that in the absence of reputation effects, interaction in fixed networks does not differ from interaction in isolated dyads, and behavior converges to mutual ALLC or mutual ALLD. In the presence of reputation effects, however, behavior within dyads is no longer necessarily symmetric, because observations of the behavior of a partner can be “compensated” by observations of his or her behavior in interactions with other actors. In the extreme case of perfect information transmission (Case (ii)), all partners of a given actor will choose the same strategy against this actor, but this does not imply that all actors choose the same behavior in interactions with all their partners. Examples can be constructed in which a group of actors cooperate with each other but defect against a single actor, who, in turn, cooperates with all of them. In this case the defections of these actors are “offset” by the observation that they cooperate in all other relations.

3.3.3 Stable states in dynamic networks

Finally, we turn to the situation in which games are played in dynamic networks such that actors can choose not only their behavior in the games, but also with whom they play. The important difference as compared with the fixed network case is that certain types of interactions can be discontinued if the expected reward from the interaction is less than the cost of its maintenance. We distinguish between two different scenarios with regard to the cost of network formation: the case in which T > R > α > P > S and the case in which T > R > P > α > S, where α is the “maintenance cost” of a tie. In the first case only interactions in which cooperation takes place are attractive. In the second case relationships with mutual defection are also attractive, but actors will prefer isolation over exploitation (α > S). Let denote a component of network g, that is, a subnetwork of g consisting of a maximal subset of nodes and links such that all nodes are directly or indirectly connected.

Theorem 3.3 

(i) If ω = 0 and R > α > P, then in stable networks. Any network configuration is possible.

(ii) If ω = 0 and P > α > S, then the network is complete and in stable networks.

(iii) If ω = 1 and R > α > P, then the network consists of one or more complete components , while actors may also be isolated. .

iv If ω = 1 and P > α > S, all links are present in the network, and .

Proof. Case (i) is a reiteration of Case (i) of Theorem 3.2 with the addition that interactions in which both actors use ALLD are no longer stable, as α > P (note that the expected payoff per round in that case converges to P). Case (ii) differs from Case (i) in that interactions in which both actors use ALLD are also stable, as the expected payoff in these interactions converges to P and P > α. Moreover, since P is the minimal expected payoff, all links are created. Case (iii) partly relies on the same argumentation as Case (ii) of Theorem 3.2. All actors who are directly or indirectly connected will base their actions regarding a given actor i on the same information. Therefore, within a component all links must be present, because if it is profitable for any actor i to connect to j, it must be profitable for all actors who are directly or indirectly connected to i to connect to j. For the same reason all neighbors of j use the same sub-strategy in interactions with j. It is not possible that for all . In that case would tend toward zero for all , and the expected payoff of all interactions would tend toward P, which is lower than α. However, it is possible to construct examples such that some of the actors in a component play ALLD and others play ALLC. Case (iv) differs from Case (iii) in that all links must be created such that any stable network must be complete, because α < P and P is the lowest possible payoff. This implies that the situation in which for all is also stable.

In brief, Theorem 3.3 states that if the cost of tie maintenance is low enough (Cases (ii) and (iv)), the complete network will form. If the cost of tie maintenance is high, either any network configuration is possible (Case (i)) or the network will consist of fully-connected components (Case (iii)). Perfect transfer of information (Cases (iii) and (iv)) has the effect that components must be fully connected. Next, let us compare the situation without reputation (ω = 0) with the situation with full reputation (ω = 1). When costs are low (Cases (ii) and (iv)), the complete network will form, and there are no clear effects of reputation. When costs are high (Cases (i) and (iii)), however, we see that the presence of reputation effects clearly has an effect on the possible stable distribution of behavior. Without reputation (Case (i)), only mutually cooperative ties are stable. With reputation (Case (iii)), the constellations in which some actors defect are also stable, as long as it is not the case that all actors in a component play ALLD against all their neighbors. Thus, we see that when there is a high cost of relations, reputation opens the door for defection. However, this does not necessarily mean that reputation always leads to lower overall cooperation. While Case (i) states that only cooperative interactions are stable, the emerging network of interactions may be very sparse or even empty, yielding a very low level of cooperation. The question whether processes with and without reputation effects lead to different levels of overall cooperation will be addressed in the next section through the use of computer simulations.

Finally, we compare our formal results regarding the effect of reputation on cooperation between fixed and dynamic networks. In fixed networks we did not observe a clear effect of reputation on the level of cooperation, although we did find an effect on the combinations of sub-strategies that can be stable. In dynamic networks we found similar effects on possible combinations of sub-strategies, but we also found that in dynamic networks reputation increases the potential for defection.

The theorems characterize stable networks for the extreme cases in which ω = 0 or ω = 1, but they do not characterize such networks for intermediate values of ω. In many cases the characterizations are rather general and allow for many combinations of network configurations and behaviors to be stable. Moreover, the theorems do not provide any insight into which of the possible stable states are more or less likely to emerge given some present state of the process. That is, the analyses say nothing about the dynamic process which brings about stable states. In the following sections we conduct computer simulations to address these issues.

3.4 Setup of the simulation

In this section, we describe the setup of the simulation (or “computational experiment”) that we use to explore the properties of our model with regard to the questions formulated in Section 3.1. In the simulation, we vary the parameters of the model in a systematic way to assess their effects on the outcomes of the process.

The simulation input consists of a set of parameters and a set of initial conditions. The parameters determine how the process works and are not changed in the process. These include the payoffs of the game, the strength of the reputation effect (ω), the speed of network adaptation (η), and the cost of maintaining network ties (α).

The initial conditions constitute the state in which the process starts. They include the initial network structure and the initial distribution of the beliefs of the actors. From a theoretical point of view, these parameters are somewhat less interesting, because they change as the process proceeds and can therefore not be used to make general claims about the typical outcomes of the process. However, their role is not merely instrumental. It is not uncommon for complex dynamic systems, such as the one under study here, to have a wide range of stable states, even for one set of exogenous parameter values. In such cases, studying which equilibria are more likely given certain initial conditions increases the empirical applicability of the model (cf. Buskens et al. 2008; Jackson and Watts 2002).

Before we turn to the actual values of exogenous and endogenous parameters, however, we discuss the outcome variables in the simulation.

3.4.1 Dependent variables

There are basically two types of outcomes that are of interest for our inquiry: those related to cooperation and those related to the emerging network. To express the amount of cooperation in the process we define two different measures.

We use the proportion of cooperation per interaction (or cooperation per tie) to study emerging cooperation in fixed networks. To look at total cooperation (i.e., the second measure) in fixed networks would not make much sense, as this measure is restricted by the density of the network. Networks with higher density show more cooperation, simply because there are more interactions. Conversely, for dynamic networks total cooperation is a more suitable measure than cooperation per interaction, because the number of interactions is itself an outcome of the process.

Although these two measures may look very different at first sight, note that when one is applied to fixed networks and the other to dynamic networks, they both measure the proportion of maximally attainable cooperation. Thus, we believe it is justified to use these two measures to compare cooperation between fixed and dynamic networks.

3.4.2 Parameters of the simulation

We make a distinction between the parameters of the simulation, which govern the mechanisms of the process, and the initial conditions, which comprise the input for the process. The parameters are the payoffs of the game, that is, ω, η, α, and the number of actors in the network. We ran simulations with 30 actors. For the payoffs of the game, we chose T = 5, R = 3, P = 1, and S = 0. We varied the extent to which reputation traveled through the network ω between 0 and 1. The case in which ω = 0 represents the situation in which there are no reputation effects and interactions are completely independent across dyads. We varied the speed of network formation η to be zero (η = 0; the network is fixed), relatively slow (η = 30, the number of actors), and relatively fast (η = 435, the number of dyads). η = 0 refers to the situation in which networks are fixed. The linear cost of ties α is chosen such that S < α < P, P < α < (T + S)/2, or (T + S)/2 < α < R. These values are chosen such that, in the first case, the cost of tie formation allows relationships with mutual defection, mutual alternation, and mutual cooperation. In the second case, the cost of ties allows only mutual alternation and mutual cooperation, and in the third case, it allows only mutual cooperation.

3.4.3 Initial conditions of the simulation

The initial conditions of the simulation consist of the initial beliefs of the actors and the initial network structure. As for beliefs, recall from Section 3.2.2 that actors base their beliefs on the future behavior of other actors using four quantities. We obtain initial values for these quantities in the following way. For each directed belief (i, j), we start with a random “assumed history” in which i has cooperated with j times and defected times, where and are random integers between 0 and − tmax. In a sense, − tmax can be interpreted as the number of periods the process had been running before we began observing it. We have set this value to − tmax = 5. The remaining two quantities and for t = 0 are chosen as random proportions of and . In effect, this means that we choose and randomly for t0. We introduce one extra parameter, λ, to govern the relative values of and . We choose and such that

(3.10)

and

(3.11)

where xij and yij are random variables between 0 and 1 and λ > 0. The higher λ is, the larger is relative to and the higher is the average expectation that opponents will be reactive. Therefore, a higher λ is associated with a higher overall tendency toward cooperation.

For the initial network structure we draw from a set of artificially generated network structures. To construct these networks, we use various well-known network models, including the random graph model, the small-world model by Watts and Strogatz (1998), and the preferential attachment model by Barabási and Albert (1999). These models have been shown to reproduce some key characteristics of empirical networks, and the resulting networks therefore provide a reasonably plausible set of initial networks for the simulation.5 We vary the parameters of the algorithms in such a way as to obtain a reasonable variance in network density and network centralization. Table 3.3 summarizes the initial conditions used in the simulation.

Table 3.3 Initial conditions in the simulation.

3.4.4 Convergence of the simulation

As we described in Section 3.2.5, the process is theoretically stable when the beliefs of the actors are stable. In a numerical simulation, however, such convergence might take a very long time, and it is also possible that beliefs will never become stable at all, even though they do converge to a certain value. As an example, suppose that j defected once after i cooperated, but thereafter, j always cooperated. In that case, − 1, and as t tends toward infinity, the fraction approaches 1, even though it never becomes perfectly stable. For this reason, we introduce a pragmatic convergence criterion, which states that the process is assumed to be converged when the largest change in the beliefs of all actors is smaller than 5%.

3.5 Simulation results

The results reported here are based on a total of 7200 simulation runs. In the fixed-network runs (η = 0) the process always converged within 1000 rounds. In the dynamic network (η > 0) 99.8% converged within 1000 rounds. In the results that follow we include only runs that converged within 1000 rounds.

3.5.1 Results for fixed networks

Before we turn to the results for dynamic networks we discuss some of the results for fixed networks (η = 0). These results constitute the “baseline” to which we compare our results for dynamic networks. Figure 3.3 shows how cooperation (here represented by the average of cooperation per tie; see Section 3.4.1) depends on the density of the initial network for different combinations of λ (“optimism”) and ω (“reputation diffusion”). To indicate the general trends median splines where added to the scatter plots.

Figure 3.3 Average cooperation per interaction by ω in fixed networks with median splines added—graphs by λ (rows) and density (columns); density rounded to the nearest multiple of 0.25.

First, cooperation depends heavily on λ. If λ = 1 (which means that all beliefs are initially completely random), virtually no cooperation occurs, while if λ = 4, cooperation levels are rather high. This result indicates that the initial conditions of the process have a strong impact on the outcomes. Nevertheless, we also see effects of reputation diffusion on the outcomes. The simulation results are in line with the analytical result that the presence of reputation effects generally increases the range of stable states. While the outcomes lie close together when reputation effects are absent (ω = 0), the stable states tend to “spread out” when ω increases. This is most visible when λ = 3. This implies that the presence of reputation effects allows for more extreme levels of cooperation, both high and low.

The exception to this trend is the case in which λ = 4, which is the most favorable condition for cooperation. Here, strong reputation effects decrease the variance in cooperation outcomes, especially in dense networks. If density is smaller than 0.75, a small reputation effect (ω = 0.25) allows for lower levels of cooperation. Generally, however, it seems safe to say that the spread of reputations helps to maintain high levels when λ = 4.

3.5.2 Results for dynamic networks

Table 3.4 shows average levels of cooperation and network density for different combinations of values of α (cost of ties) and ω (reputation). Cooperation at the macro-level is measured in two different ways (see Section 3.4.1): the average level of cooperation per dyad and the average level of cooperation per actual interaction. For α = 0.9 all stable networks are complete, because the cost of maintaining a tie is lower than P, the minimally expected payoff (cf. Theorem 3.3, Cases (ii) and (iv)). The net effects of reputation on cooperation are marginal at this cost level.

Table 3.4 Average density and cooperation in stable networks by α and ω.

If α = 1.9, the cost of maintaining a tie is higher than P, which makes mutually defective relationships unstable in the absence of reputation effects (cf. Theorem 3.3, Case (i)). Indeed, cooperation per interaction is almost 100% if ω = 0.6 The cooperation per dyad, however, is comparable to the lower cost regime. When ω > 0, we find that the density of stable networks increases, while the level of cooperation per dyad remains more or less constant. There are more interactions, but these are not cooperative interactions. Thus, in line with the analytical results (Theorem 3.3), when reputations are allowed to spread through the network, this allows for defection to continue, even if the cost of maintaining a relationship is higher than the expected payoff from a mutually defective relationship. As a consequence, the level of cooperation across all interactions decreases if ω > 0. These patterns are, however, not linear in ω; density jumps when ω increases from 0 to 0.25 but then decreases. Similarly, the drop in cooperation per interaction is largest between ω = 0 and ω = 0.25 and much smaller between higher levels of ω. Finally, if the cost of a tie is only slightly lower than the expected payoff from a mutually cooperative relationship (α = 2.9), we see that in the absence of reputation effects (ω = 0), cooperation is maximal in all existing relationships, but density is much lower than when the cost is lower. With ω > 0, we again see that cooperation per interaction decreases. At the same time, density decreases with ω until almost 0 when ω = 1. In this case, cooperation per interaction is still high, but there are very few interactions so that the network is extremely sparse.

The averages in Table 3.4 are informative, but given the strong effects of the initial tendency for cooperation that we found for fixed networks, we should also compare the outcomes for different values of λ. Figure 3.4 shows how the average cooperation per dyad depends on the strength of the reputation mechanism (ω) for different values of α and λ. This figure can be compared to Figure 3.3, but instead of initial density (which is not so informative for dynamic networks), we now look at different values of α. As compared to the case of fixed networks, we generally find a somewhat stronger effect of ω. The direction of this effect depends heavily on α and λ. For the two lower values of α, the effect of ω is negative for lower values of λ and positive for higher values of λ. Thus, on average, the spread of reputation “catalyzes” the tendency the process already had at the start. This does not mean, however, that a strong reputation effect leads to high cooperation when there is a high λ and low cooperation when there is a low λ. As in fixed networks the range of stable outcomes also increases with ω. For high costs (α = 2.9) the result is different in a number of respects. First, we see that the effect of ω on cooperation is nearly always negative or 0. This is even the case for the highest value of λ, which mostly leads to full cooperation with lower costs. Second, we see an interesting divergence of outcomes when λ = 4 and ω = 0.25. Here, a number of simulation runs converged on an average lower level of cooperation compared with the situation where ω = 0, while another group of runs converged on significantly higher levels of cooperation. Closer analyses of these latter cases reveal that they are characterized by a relatively lower initial network density. Among all the runs with λ = 4, α = 2.9, and ω = 0.25, the correlation between initial density and cooperation is 0.59. An explanation of these results might be that if the network is initially sparse, limited diffusion of reputations helps to form a network of cooperative relations. If the network is dense from the start, in contrast, the diffusion of reputation mostly serves to “spread bad news,” which prevents the further build-up of a cooperative network.

Figure 3.4 Average cooperation per dyad by ω in dynamic networks with median splines added—graphs by λ (rows) and α (columns).

To conclude the discussion of simulation results we look at the effects of initial density in dynamic networks. For this purpose we again draw scatter plots of average cooperation per dyad by ω, λ, and α, as depicted in Figure 3.4, but we make a further distinction between different initial network densities. To reduce the number of graphs we round density by multiples of 0.25 and show only values of λ for which we find interesting differences. The result is shown in Figure 3.5. This figure can be read as an extension of Figure 3.4 in which different values of initial density are separated in the rows, while different values of cost α have different sub-figures. For the two lower values of α, we only show the results for λ = 2 and λ = 3, while for the highest value of α, we only show results for λ = 3 and λ = 4. For the remaining values of λ in each case, results do not differ for the different values of initial density.

Figure 3.5 Effects of reputation on cooperation by initial density, λ and α.

Figure 3.5 shows an interesting interaction effect between initial density and ω, which is the weight of third-party information. For lower values of α, the effect of ω is clearly stronger for lower initial density. Moreover, we see that the variance of stable states increases more strongly with ω, especially when λ = 3. This means that when the initial density is higher, higher levels of cooperation can be reached with lower levels of ω.

For the highest value of α, the effects are less clear. When λ = 3, the effect of ω becomes stronger for higher initial density, while for λ = 4, there is no clear interaction effect. We can, however, identify “special cases” as mentioned in the previous section in which an exceptionally high level of cooperation is reached when .

The implication of these results is that the combination of a high initial density and the presence of reputation effects is not the best recipe for cooperation if the network is dynamic. On the contrary, if there is a positive effect of reputation, this effect is most pronounced when the initial density is low.

3.6 Conclusions and discussion

In this chapter we developed a model to study the co-evolution of cooperation in dyadic relationships and social networks. Given that the spread of reputations is generally thought to improve the chances of cooperation if networks are given exogenously, we were specifically interested in studying how reputation effects change if the network co-evolves with behavior in strategic interactions.

For this purpose we constructed a model in which boundedly rational actors play two-person Prisoner's Dilemmas and, at the same time, choose their relationships. Through the network actors learn about the past behavior of other actors and adjust their behavior accordingly. We derived a number of formal propositions regarding theoretically stable states in this model. In addition we applied computer simulation techniques to examine which stable states are more or less likely, depending on the parameters of the model and the initial conditions of the process. The most important model parameter was the strength of the reputation effect, that is, the extent to which information on the past behavior of others can diffuse through the network.

The overall results can be summarized as follows. First, we find that if networks are exogenously determined, the range of possible stable states increases with the extent of reputation diffusion and the density of the network. States with higher overall cooperation levels emerge as compared with situations with less reputation diffusion but also states with lower cooperation rates. Thus, we do not find that reputation effects always lead to more cooperation, as is most commonly assumed in the sociological literature. Rather, we find that relatively higher cooperation is a possible consequence of reputation effects but so is lower cooperation. These findings are in line with previously formulated arguments, such as Burt and Knez (1995), who argue that reputation effects generally lead to more extreme outcomes. While Burt and Knez (1995) rely on psychological mechanisms to explain this phenomenon, we have shown that it can also emerge from a simple learning model.

Second, we find that if the network is dynamic, the spread of reputation tends to “catalyze” the initial tendency of the process toward higher or lower levels of cooperation. Moreover, we find strong interaction effects with regard to the cost of maintaining ties and reputation effects in dynamic networks. When the cost of maintaining a tie becomes very high, maintaining a network of cooperative relationships becomes difficult; the addition of reputation effects makes this even worse, leading to empty networks in many cases.

Third, we find no indications that in a context in which the network structure is endogenous, high cooperation levels are likely to be the result of reputation effects in an initially dense network. Instead, we find that in dynamic networks the effect of the spread of reputations tends to be stronger if the network is less dense. That is, the diffusion of reputations is most likely to lead to high cooperation rates if initial beliefs are “optimistic” (i.e., λ is high) and the network is sparse. As a result of high cooperation a dense network emerges. An interpretation of this effect could be that if the network is initially sparse, actors have the opportunity to initiate interactions only with those partners whom they expect to act cooperatively. The diffusion of reputation then helps in the further build-up of this “cooperative network.” If, in contrast, the network is dense from the beginning, there will also be some relationships in which actors do not behave cooperatively. In this case, the diffusion of reputations only helps to spread to “bad news,” which in turn hinders the development of cooperation.

While we believe that our analysis adds new insights to the study of cooperation in networks, the model also has some limitations. First, we modeled reputation only as learning, and we did not take into account that actors may care about their future reputation. Adding such “control” mechanisms to the model would not only make the model much more complex to analyze, but it would also put considerably higher demands on the rationality of actors.

Second, we did not model actor expectations with regard to the linking behavior of their interaction partners. In effect, actors in our model assume that their opponents will never unilaterally end the interaction. We decided not to do so because we wanted to focus strictly on reputation effects, but in principle, the model could be extended in this direction by including expectations about the opponent's linking behavior as conditional probabilities assigned to the beliefs of the actor.

Third, we did not consider information diffusion as a strategic choice. For simplicity, we assumed that the transfer of information is automatic and that actors do not make an explicit decision to pass along specific information. In many empirical applications, this assumption is unrealistic. As we can learn from studying gossip (Burt 2001; Gambetta 1994), people often have reasons to think strategically about what to tell to whom. For instance, actors may selectively relay information about their own interactions (only passing positive information about themselves) or even strategically spread negative information about others in order to sanction them (Ellickson 1991).

Fourth, and related to the previous point, we treated the weight of reputation as a property of the process rather than as a property of individual strategies. In most applications, it would probably be more appropriate to assume that actors decide to what extent they want to make use of available third-party information. Considering this, one interpretation of our current model could be that all actors choose the same strategy with regard to the use of information. An alternative interpretation could be that all actors by default make maximal use of available information but that the availability of this information is determined by macro-level circumstances, such as the ease of communication (i.e., the value of ω in our model). A relatively straightforward way to check the robustness of the results to relaxation of this assumption would be to let ω vary between actors and thereby make actors heterogeneous in their susceptibility to third-party information. Alternatively, one might try to model the circumstances under which actors will take third-party information into account.

Fifth, the learning model applied here is rather simple. Actors have very simplified expectations about their opponent's behavior and update those beliefs using very basic methods. More complex learning models are conceivable in which actors, for example, assume that the behavior of their partners is conditional on a longer history than the previous round or take the reliability of their own estimations into account.

Finally, it would be interesting to look at the stochastic stability of the process. At present our model is basically deterministic; the only stochastic element is the order in which actors update their behavior. Adding some “noise” to the process—in the sense that actors can make small “mistakes” every now and then—could have two advantages. First, it might help to resolve the problem that the model predicts many different stable states for any given set of parameters by showing which stable states recur in a perturbed process. This approach is adopted by, for example, Jackson and Watts (2002). Second, a “noisy” model would also be of substantive interest to investigate how reputation affects cooperation and network formation in a volatile environment. First intuition is that the introduction of noise would make cooperation even more fragile, but the spread of reputations may help to “protect” group cohesion and cooperation from small mistakes, as they can be compensated for by the good reputation of an actor.

Broadly speaking, we see two ways to develop the model further. The first is to address some of the theoretical issues mentioned above as extensions of the model. Another question, of course, concerns the empirical validity of the model. Eventually, we aim to explain and predict empirical processes related to cooperation and network formation with our model. In the present state, however, with so many theoretical issues still unresolved, directly testing predictions from the model with real-world data is likely to be problematic. For instance, if discrepancies between predictions and the data are found, it will be difficult to determine whether the discrepancy is caused by overly simplified assumptions about actor decision-making or by a misspecification of the underlying game. A more fruitful approach to test this model empirically would involve conducting controlled experiments. Using the methods developed in experimental economics and social psychology, one can study human behavior in complex strategic interactions “at close range” while also controlling properties of the larger environment. Such an approach would be most useful to assess the extent to which the model's assumptions about actor decision-making are sufficient or are in need of modification.

References

Axelrod R (1984) The Evolution of Cooperation. Basic Books, New York.

Bala V and Goyal S (2000) A noncooperative model of network formation. Econometrica 68, 1181–1229.

Barabási AL and Albert R (1999) Emergence of scaling in random networks. Science 286(5439), 509–512.

Biely C, Dragostis K, and Thurner S (2007) The prisoner's dilemma on co-evolving networks under perfect rationality. Physica D 228, 40–48.

Burt RS (2001) Bandwidth and echo: trust, information, and gossip in social networks. In Networks and Markets: Contributions from Economics and Sociology (eds Casella A and Rauch JE). Russell Sage Foundation, New York, pp. 30–74.

Burt RS and Knez M (1995) Kinds of third-party effects on trust. Rationality and Society 7(3), 255–292.

Buskens V (2002) Social Networks and Trust. Kluwer Academic Publishers, Boston, MA.

Buskens V (2003) Trust in triads: effects of exit, control, and learning. Games and Economic Behavior 42, 235–252.

Buskens V and Raub W (2002) Embedded trust: control and learning. In Group Cohesion, Trust and Solidarity (eds Thye SR and Lawler EJ), Vol. 19: Advances in Group Processes. Elsevier Science, Amsterdam, The Netherlands, pp. 167–202.

Buskens V, Corten R, and Weesie J (2008) Consent or conflict: coevolution of coordination and networks. Journal of Peace Research 45(2), 205–222.

Camerer CF and Ho TH (1999) Experience-weighted attraction learning in games: estimates from weak-link games. In Games and Human Behavior (eds Budescu DW, Erev I, and Zwick R). Lawrence Erlbaum, Mahwah, NJ, pp. 31–51.

Cohen MD, Riolo RL, and Axelrod R (2001) The role of social structure in the maintenance of cooperative regimes. Rationality and Society 13(1), 5–32.

Coleman JS (1990) Foundations of Social Theory. Belknap, Cambridge, MA.

Conlisk J (1996) Why bounded rationality? Journal of Economic Literature 34(2), 669–700.

Corten R and Cook KS (2009) Cooperation and reputation in dynamic networks. Proceedings of the First International Conference on Reputation: Theory and Technology—ICORE 09 (ed. Paolucci M) Gargonza, Italy, pp. 20–34.

Doreian P and Stokman FN (1997) The dynamics and evolution of social networks. In Evolution of Social Networks (eds Doreian P and Stokman FN). Gordon and Breach Publishers, Amsterdam, The Netherlands, pp. 1–17.

Downing L (1975) The prisoner's dilemma game as a problem-solving phenomenon. Simulation & Games 6(4), 366–391.

Dutta B and Jackson MO (eds) (2003) Networks and Groups. Models of Strategic Formation. Springer-Verlag, Heidelberg, Germany.

EdK-Group (2000) Exit, anonymity and the chances of egoistical cooperation. Analyse und Kritik 22, 114–129.

Eguíluz VM, Zimmerman MG, and Cela-Conde CJ (2005) Cooperation and the emergence of role differentiation in the dynamics of social networks. American Journal of Sociology 110(4), 977–1008.

Ellickson RC (1991) Order Without Law. How Neighbors Settle Disputes. Harvard University Press, Cambridge, MA.

Erev I and Roth AE (1998) Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. American Economic Review 88, 848–881.

Fudenberg D and Levine DK (1998) The Theory of Learning in Games. MIT Press, Cambridge, MA.

Gambetta D (1994) Godfather's gossip. Archives Européennes de Sociologie 35(2), 199–223.

Granovetter MS (1985) Economic action and social structure: the problem of embeddedness. American Journal of Sociology 91, 481–510.

Hauk E (2001) Leaving the prison: permitting partner choice and refusal in prisoner's dilemma games. Computational Economics 18, 65–87.

Homans GC (1951) The Human Group. Routledge, London.

Jackson MO and Watts A (2002) On the formation of interaction networks in social coordination games. Games and Economic Behavior 41(2), 265–291.

Jackson MO and Wolinsky A (1996) A strategic model of social and economic networks. Journal of Economic Theory 71(1), 44–74.

Kreps DM and Wilson R (1982) Reputation and imperfect information. Journal of Economic Theory 27(1982), 253–279.

Lippert S and Spagnolo G (2005) Networks of relations and social capital. SSE/EFI Working Paper Series in Economics and Finance, Stockholm School of Economics.

Macy MW and Flache A (2002) Learning dynamics in social dilemmas. Proceedings of the National Academy of Sciences of the United States of America 99(3), 7229–7236.

Newman M, Barabási AL, and Watts D (eds) (2006) The Structure and Dynamics of Networks. Princeton University Press, Princeton, NJ.

Ohtsuki H, Hauert C, Lieberman E, and Nowak MA (2006) A simple rule for the evolution of cooperation on graphs and social networks. Nature 441, 502–505.

Pujol JM, Flache A, Delgado J, and Sangüesa R (2005) How can social networks ever become complex? Modelling the emergence of complex networks from local social exchanges. Journal of Artificial Societies and Social Simulation 8.

Rapoport A, Diekmann A, and Franzen A (1995) Experiments with social traps IV: reputation effects in the evolution of cooperation. Rationality and Society 7(4), 431–441.

Raub W and Weesie J (1990) Reputation and efficiency in social interactions: an example of network effects. American Journal of Sociology 96, 626–654.

Rubinstein A (1998) Modeling Bounded Rationality. MIT Press, Cambridge, MA.

Schuessler R (1989) Exit threats and cooperation under anonymity. Journal of Conflict Resolution 33(4), 728–749.

Simon HA (1956) Rational choice and the structure of the environment. Psychological Review 63(2), 129–138.

Snijders TAB (2001) The statistical evaluation of social network dynamics. In Sociological Methodology (eds Sobel ME and Becker MP), vol. 31. Blackwell, Boston, MA, pp. 361–395.

Snijders C and Zijdeman R (2004) Reputation and internet auctions: ebay and beyond. Analyse und Kritik 26, 158–184.

Stanley EA, Ashlock D, and Tesfatsion L (1994) Iterated prisoner's dilemma with choice and refusal of partners. In Artificial Life III (ed. Langton CG), Vol. XVII: Santa Fe Institute Studies in the Sciences of Complexity. Addison-Wesley, Reading, MA.

Ule A (2005) Exclusion and Cooperation in Networks. Thela Thesis, Amsterdam, The Netherlands.

Uzzi B (1996) The sources and consequences of embeddedness for the economic performance of organizations: the network effect. American Sociological Review 61(4), 674–698.

Uzzi B (1997) Social structure and competition in interfirm networks: the paradox of embeddedness. Administrative Science Quarterly 42, 35–67.

Vanberg VJ and Congleton RD (1992) Rationality, morality, and exit. American Political Science Review 86(2), 418–431.

Vega-Redondo F (2006) Building up social capital in a changing world. Journal of Economic Dynamics and Control 30, 2305–2338.

Wasserman S and Faust K (1994) Social Network Analysis: Methods and Applications. Cambridge University Press, Cambridge.

Watts DJ and Strogatz SH (1998) Collective dynamics of ‘small world’ networks. Nature 393, 440–442.

Weesie J (1996) Disciplining via exit and voice. ISCORE Paper, Utrecht University.

Weesie J and Flap H (eds) (1990) Social Networks Through Time. ISOR, Utrecht.

Wilson R (1985) Reputations in games and markets. In Game-Theoretic Models of Bargaining (ed. Roth AE). Cambridge University Press, Cambridge, pp. 27–62.

Yamagishi T, Hayashi N, and Jin N (1994) Prisoner's dilemma networks: selection strategy versus action strategy. In Social Dilemmas and Cooperation (eds Schulz U, Albers W, and Müller U). Springer-Verlag, Berlin, pp. 233–250.

*. *This chapter was written in collaboration with Karen Cook. A shortened version appeared in the Proceedings of the First International Conference on Reputation: Theory and Technology – ICORE 09 (Corten and Cook 2009).

1. This approach is closely related to fictitious play (Fudenberg and Levine 1998).

2. Alternatively, one might also think of this parameter as being part of a strategy, that is, as a conscious choice by the actor to use third-party information or not. While that would be an interesting extension of the current model, we treat the weight of third-party information as an environmental variable here.

3. The fact that in ordinary language, reputation is usually thought of as an attribute of a single “target” actor suggests that in reality, beliefs about the target actor tend to be—or at least seem to be—so similar that it becomes justifiable to speak of “the” reputation of the target actor. As we will see, this also occurs in our model under certain conditions.

4. We hope to relax this assumption in future developments of this model.

5. The results do not differ between different network-generating algorithms, which we take as an indication that the precise method used does not matter much and that studying additional methods is not likely to yield new insights.

6. The proportion of cooperation is not exactly 1 because in the simulation, convergence may be imperfect (see Section 3.4).