Introduction

Although the foundations of game theory were formally laid as early as 1944 in economics (Von Neumann & Morgenstern 1944) and 1984 in life sciences (Axelrod 1984), biologists have only recently begun to establish a solid theoretical framework within which to probe the properties of many natural cooperative systems (Nowak 2006). Prisoner's dilemma has been a dominant metaphor for the evolution of cooperation (Nowak & Sigmund 1993) as it presents a genuine and mathematically interesting puzzle as to whether to cooperate or defect in repetitive games between the same individuals with an unknown number of repetitions (Axelrod & Hamilton 1981, Axelrod 1984). Real natural systems, however, have more degrees of freedom in that various factors (i.e. such as the level of supply and demand for particular commodities) influence the payoff for one of the two possible actions, namely to cooperate or to defect. To make any evolutionary explanation of cooperation more convincing, one would have to simulate stochastic processes by taking into account the characteristics of different individuals and of additional variables such as resource availability and access. Our initial study (Johnson et al. 2002) found that only small variations in payoffs for one or both players can be enough to disrupt the payoff rankings of a true Prisoner's dilemma and generate different games instead. However, that study only identified how frequently the Prisoner's dilemma fails as a model. It did not assess whether cooperation was more or less likely when that occurs. This study fills that gap.

In nature, individuals are caught repeatedly in situations that are subject to cooperative games, but where payoffs for certain acts vary according to the levels of supply and demand for various commodities — e.g. the need for cooperation or help (Noë et al. 1991, Dunbar 1992, Barrett et al. 1999, Stopka & Macdonald 1999). As a consequence of this, the payoff structure that characterises each game for a given pair of interacting individuals changes over time and, therefore, tendencies to perform certain acts change accordingly (Dunbar 1992) see Fig. 1. For example, once the benefits of leaving (minus the costs of leaving) exceed the value of the benefits of staying (minus the costs of staying) there is no longer any advantage to cooperate with a particular individual, so the only rational decision would be defection as in the Prisoner's dilemma (i.e. T > R > P > S and R > (T + S)/2). Here T is the temptation to defect, R is the reward for mutual cooperation, S is the sucker's payoff for cooperating unilaterally and P is the punishment for mutual defection. In the opposite scenario (i.e. net benefits of staying > net benefits of leaving) the only rational decision would be to cooperate and the underlying game would be close to the definition of Mutualism (R > T ≥ S > P) or Cruel Bind (T > R > S > P). In Cruel Bind (Trivers 1972), even if one of the two individuals consecutively defects it is still advantageous for the other one to continue to cooperate, because the payoff for unilateral cooperation is higher than that for mutual defection (i.e. S > P). Higher benefits of staying for a cooperating individual are, therefore, maintained as long as S > P.

But where in nature might such a situation arise? And how common is it? The following is a good example of the Prisoner's dilemma, and its limitations, in applying game-theoretical solutions to a specific social scenario. Some authors have suggested that reciprocity of allogrooming between impalas, Aepyceros melampus — where males and females engage in a type of allogrooming in which partners alternately deliver bouts of oral grooming to the head and neck — may be a candidate for a strategy called Tit-for-Tat (Hart & Hart 1992), a solution to the iterated Prisoner's dilemma (Axelrod & Hamilton 1981). However, others (Connor 1995) argue that the strategy being played is one that minimises potential losses in what is given and received, not Tit-for-Tat, though it mimics it, but a strategy termed ‘parcelling’. Parcelling (Connor 1995) is a concerted action of two individuals where during mutual cooperation each individual incrementally decreases the time invested into each bout by which method, as we show here, both individuals maintain equilibrium in the need for help. If one individual stops allogrooming the other one also terminates, pausing the interaction until one of them resumes cooperation. Whether these two strategies may be functionally homologous to each other can only be determined on the basis of the benefits and costs of staying or leaving — conforming to either Mutualism or Prisoner's dilemma games.

For example, for parcelling to occur it must be true that, contrary to Tit-for-Tat, at each decision point the net benefits of staying must exceed the net benefits of leaving (Connor 1995). Therefore, parcelling is a strategy to maximise the reward for cooperation R by maintaining R > T but which, like Tit-for-Tat, may converge to a simultaneous reciprocal action. Tit-for-Tat is a strategy which minimizes potential losses when the condition T > R, S < P and R > (T + S)/2 applies. Therefore, these two strategies are not completely homologous because the different payoff structures represent qualitatively different constraints on potential responses over the course of an interaction. To a human observer, however, they may look similar, leading to the erroneous application of certain games as models. In previous work (Johnson et al. 2004), we suggested that group living animals may get around the Prisoner's dilemma if they are cooperating to get rid of something, not to gain something (as is more usually assumed to be the case). The presence of ectoparasites such as fleas means all individuals have a stake in reducing the parasite burden of the group as a whole. Thus an individual X allogrooming another individual Y not only reduces Y's parasite load, but also the total parasite population in the group or den site that may potentially affect X (Johnson et al. 2004). In this paper, we argue that the logic of probing cooperative systems by applying Prisoner's dilemma without appropriate evaluation of the costs and benefits involved may cause an error in uncovering the real causes of the evolution of cooperative behaviours. Here, we introduce a model which suggests a potential solution to probing the properties of cooperation in animal societies using stochastic simulations based on a detailed knowledge of behaviour of the coevolving agents involved, and using cooperative allogrooming as a test case. A prime assumption of the model is that having ectoparasites incurs a stochastic cost. This means that the higher the chance of having ectoparasites, the higher the chance that some of them will be infected with a fitness-diminishing pathogen which may be transmitted to a susceptible host. The model is based solely on knowledge about host-parasite interactions and attempts to further the understanding of the logic and evolution of these complex adaptive systems from the perspective of variations in the payoff structure.

Methods

The simulation model

The main purpose of this model was to determine the relative payoff structures individuals actually experienced to ascertain the game(s) which best describe motivations to perform particular actions. Our simulation model begins with 100 social groups each including 50 individuals, where they meet randomly and mutually exchange two bouts of allogrooming per single interaction (out of 100 in total). Allogrooming events are not synchronised so individuals do not switch at the same time thus allowing individual variability. We used a simple swap algorithm in our stochastic simulation to demonstrate how mobile ectoparasites such as fleas can move between bodies of interacting individuals (Cox et al. 1999) in order to approach an Ideal Free Distribution (Fretwell & Lucas 1970) — a distribution which maximises mean per capita food intake (Sutherland 1996). At time t₀ each individual (i) is assigned a parasite load Γ_i,t drawn from exponentially distributed random numbers to represent a process of group emergence, or in other cases, normally distributed parasite loads to represent preexisting groups. In a series of interactions, individuals of the group meet random partners for bouts of allogrooming until time t₁. Fleas are allowed to migrate on the basis of a fraction rule, i.e. a fraction (β = 1/N_P) of fleas migrate from the body of one individual to another and vice versa. Of course, there is no means by which fleas can predict in advance how many fleas are on an opponent's body. Nevertheless, they proceed to disperse in this way until a ‘high-quality’ patch (a host) is discovered, at which point they stay, temporarily. This leads to a diffusion of ectoparasites among members of the group, represented by a diminishing standard deviation whilst the mean is decreased by allogrooming (i.e. compensating for an increase due to parasite reproduction).

Fig. 1.

A basic model of animal cooperation (adapted from Clements & Stephens 1995) in which the sum of payoffs equals nine in all four singular games. T is the temptation to defect, R is the reward for mutual cooperation, S is the sucker's payoff for cooperating unilaterally and P is the punishment for mutual defection.

Fig. 2.

Results of the Individual based model following the game structure depicted in Fig. 1: grooming efficiency α explained most of the variance in P/S (P < 0.001) whilst the mobility of parasites (varied in A, B and C) was only marginally important (P = 0.077). Grooming efficiency was also responsible for a linear increase in mean P/S (P < 0.001) revealing that low grooming efficiency causes P to approach S. Note that out of all four singular games presented in Fig. 1., only the Cruel Bind and Mutualism games are explanatory when grooming is efficient. Grooming efficiencies α are scaled at 1/5 (o), 1/10 (+) and 1/5000 (□).

Once assigned a partner, every individual either cooperates or defects where the decision for any action (C or D) is randomly selected with a probability of 0.5. The payoff to each is then calculated simply as the increase or decrease in parasite load, thus contributing to the individual's fitness value. Our simulation searched for the best responses to the actions of any other individual in the social group, judged by the maximisation of fitness. Two main parameters were changed in consecutive simulations: the mobility of parasites β and the efficiency of allogrooming α. The overall reproductive rate of parasites compensated for the grooming efficiency of the hosts, meeting the assumption of a closed system, so the total parasite pool did not change but individual parasite loads fluctuated in time according to the particular action taken.

The equation given below summarises the relationship between parasite loads Γ, mobility of parasites β and the grooming efficiency α in our algorithm defining one interaction, hence assuming that if parasite loads are equal, reciprocity in terms of invested effort holds.

(where τ denotes a tendency (willingness) with which the grooming efficiency α is utilised, here assigned a value τ = 1).

Results and Discussion

In our model, hosts are trapped in the constantly changing dynamics of mobile parasites approaching Ideal Free Distribution. Therefore, our model of dilution of ectoparasites takes three parameters: migration rate of parasites β, their reproductive rate and the efficiency of allogrooming α, given by the number of parasites removed per time unit. Any individual involved in allogrooming takes the risk that an opponent individual may have a higher parasite burden, in which case some fraction will transfer to him. Therefore, the cost of being infested is higher if allogrooming is not reciprocated, because a fraction of fleas will always move to the more parasitefree individual anyway, regardless of whether the parasite-laden individual cooperates or not. Thus, the cooperating individual (formerly with the higher load) will actually remove some of his own parasites from the defecting partner. Because of the particular payoff structure (i.e. R > T), however, the resulting game that would characterize this scenario is Mutualism. But when two individuals with different parasite loads meet and interact, the need for allogrooming differs (due to differential payoffs). An individual with a relatively higher parasite load (i.e. large need for help) should cooperate, even if the other one defects. An initiation of allogrooming by such an individual can therefore be considered as an honest signalling of need. This situation conforms to the Cruel Bind game (Trivers 1972, Clements & Stephens 1995). Even if the distribution of parasites is initially uniform, disequilibria emerge after several simulation steps due to individuals interacting asynchronously and randomly with each other.

Our simulations reveal that reciprocity in providing allogrooming does not necessarily hold under certain conditions of a host-parasite relationship. Moreover, they reveal that non-reciprocal allogrooming may simply be a result of the particular game that emerges due to the differential needs for allogrooming between interacting individuals (note that the structure of Fig. 2 corresponds to the game structure in Fig. 1). As R > T in at least half of the cases, individuals would have to parcel in providing bouts to maintain the equilibrium in need for help/allogrooming. As R > T in such a scenario, parcelling, a strategy by which individuals optimise the outcome by manipulating their partner's decision, is not Tit-for-Tat. When R = T parcelling maximises R relative to S and as such resembles Titfor-Tat to some extent, but since P < S, this is not a solution to Prisoner's dilemma. Furthermore, as a potential constraint P/S increased linearly with R/T when grooming efficiency was ≥ 0.1 (P < 0.0001) in our simulation (Fig. 2) conforming to the idiom — the more you gamble (i.e. the less you parcel), the more you may gain or loose. Another interesting aspect of our model is its versatility with various known outcomes. For example, when an opponent's parasite load is unknown between two ‘cooperators’, the emerged pattern of non-reciprocal allogrooming due to Cruel Bind may be avoided by premature termination of the current interaction whilst moving to a new interaction with a similar partner until reciprocity is established. This happens when both partners are equally in need of cooperation (i.e. their parasite loads are similar) and such a phenomenon may converge to symmetrybased reciprocity (de Waal 2000).

Traditionally popularised, Prisoner's dilemma is only one of four singular games (Prisoner's dilemma, Mutualism, Cruel Bind and Synergism) that are potentially applicable to many natural situations (Clements & Stephens 1995) such as, for example, cooperative allogrooming. Nevertheless, it is the Prisoner's dilemma which is most used for explaining the evolution of cooperation and of other social phenomena, such as intra-specific competition or the logic of predator inspection in shoaling stickleback, Gasterosteus aculeatus (Milinski 1987; for a wide review of Prisoner's dilemma, see Poundstone 1992). However, evidence of the Prisoner's dilemma or of its solutions such as Tit-for-Tat like strategies have been largely insufficient (Clements & Stephens 1995), suggesting that this paradigm may be rare in nature. Furthermore, reciprocal altruism (the behaviour expected to evolve in response) has been argued to be beyond the cognitive capacities of many or even most non-human animals (Stevens & Hauser 2004). It is, therefore, important to establish whether stochastic models reject the least likely scenarios in favour of more appropriate ones by analysing common parameters of all possible games: the payoff variables R, T, P, S. The payoffs that are governed by a payoff matrix can be interpreted as a contribution to a basal Darwinian fitness value.

This paper demonstrates a crucial paradox in analysing cooperation in natural systems: as shown in our simulations, behavioural responses may appear to conform to at least four different games. Even empirical evidence can therefore lead to the application of inappropriate models. When the system is analysed in its complexity (for example, when parasites and incurred costs are taken into account) it becomes obvious that a combination of specific state-dependent responses conforms to a mixture of two games, neither of which is Prisoner's dilemma. Paradoxically, although the fitness-maximising behavioural responses may seem similar to a human observer, when new information — essential to the system — is considered, inference about the logic of such systems are qualitatively changed.