Advances in Dynamic Game Theory

We explore the persistence of corn oil sensitivity in a population of the flour beetle Tribolium castaneum using evolutionary game methods that model population dynamics and changes in the mean strategy of a population over time. The strategy in an evolutionary game is a trait that affects the fitness of the organisms. Corn oil sensitivity represents such a strategy in the flour beetle. We adapt an existing model of the ecological dynamics of T. castaneum into an evolutionary game framework to explore the persistence of corn oil sensitivity in the population. The equilibrium allele frequencies resulting from the evolutionary game are evolutionarily stable strategies and compare favorably with those obtained from the experimental data.


Introduction
An evolutionarily stable strategy (ESS) is the endpoint of natural selection and is resistant to changes in the frequencies of alleles that affect fitness [8]. When a population reaches a local fitness maximum in an adaptive landscape determined by allele frequencies (adaptive peak), natural selection will preserve the allele frequencies associated with that adaptive peak, as any change in those allele frequencies will result in a lower population mean fitness. Classical population genetics techniques can be used to predict equilibrium allele frequencies in an ESS, but they do not model the population dynamics.
Evolutionary game theory (EGT) can model both genetic and population dynamics simultaneously, allowing us to study an ESS in systems as they are influenced by changing population sizes. The three main elements of an evolutionary game are players, strategies, and payoffs. The players are the individual organisms, the strategies are heritable phenotypes, and the payoffs are relative fitness gains or losses, usually expressed as net reproductive rates of the individuals. Although EGT is usually used to model dynamics in a Darwinian sense, where strategies are phenotypes, we use it here to estimate allele frequencies with genotypic strategies. In this case, the ESSs are allele frequencies-genetic "strategies"-that maximize population mean fitness. Thus, EGT provides a quantitative prediction of an ESS in addition to the population size when this strategy has been reached.
The evolutionary game has two parts. One part involves ecological dynamics as influenced by the strategy that governs fitness; the other part involves the evolutionary process resulting in changes in strategy frequency [8]. The parts are coupled; the ecological dynamics influence the change in strategy frequencies, and changes in strategy frequencies influence payoffs to the players. The ecological dynamics of the flour beetle Tribolium castaneum (Herbst) have been extensively studied experimentally and theoretically, and well-established models supported by experimental data have been developed [2][4][5] [6]. The strategy dynamics portion of the game is provided by EGT, which models changes in strategy frequencies over time as they are influenced by the ecological dynamics.
In this study, the strategy is a genetically determined phenotype in T. castaneum that results from variation in a single gene that controls sensitivity to corn oil. There are two forms of the gene (alleles) that control corn oil sensitivity. Individuals homozygous for the cos allele are unable to properly digest the unsaturated fatty acid present in corn oil [1]. This adaptive trait therefore imparts variable rates of fecundity and mortality among flour beetles cultured on a corn oil substrate [3][4] [5]. Individuals may have one of three possible genotypes: cos/cos, cos/+, or +/+, where + is the wild type allele. We define strategy as a numerical representation of genotype where cos/cos individuals have strategy 0, cos/+ individuals strategy 0.5, and +/+ individuals strategy 1. This allows us to interpret the mean strategy of the population as the frequency of the + allele. A difference equation model modified to reflect the influence of corn oil sensitivity on fitness is used for the population dynamics of T. castaneum in this study [6]. We use this model in an evolutionary game setting to explore the persistence of corn oil sensitivity in a flour beetle population and compare the results of this game to experimental data.

Model and Data
The experimental data we use in this study are from a demographic and genetic experiment conducted by Desharnais and Costantino [5]. In that study, twenty-two cultures of T. castaneum homozygous for the cos allele were initiated with identical demographic distributions of 70 small larvae, 20 large larvae, 16 pupae, and 64 adults. Each population was contained in a one-half pint milk bottle with 20 grams of corn oil medium (90% wheat flour, 5% dried brewers yeast, 5% liquid corn oil), and all were kept at constant environmental conditions. Populations were censused biweekly for 80 weeks. After ten weeks, three replicates were randomly assigned to each of three treatments that introduced the + allele into cos/cos homozygous populations. The three treatments included the addition of one female adult, three female adults, or three male and three female adults with the +/+ genotype. We refer to these as treatments 1, 2, and 3, respectively. In the control treatment, no manipulations were imposed. Gene frequency data were recorded every four weeks for 24 weeks after perturbation. We use gene frequency and population size data averaged over three replicates for each treatment in which + alleles were added, and averaged over four replicates for the control treatment.
The flour beetle goes through three distinct life history stages upon emerging from an egg: larva, pupa, and adult. Individuals spend approximately two weeks in both the larval and pupal stages. An important biological characteristic of these organisms is cannibalism. Adults eat eggs and pupae, and larvae consume eggs [2]. This characteristic imparts density-dependent regulation on the population size [2]. Several fixed-strategy models exist that do a good job of predicting population sizes in this organism, including models that incorporate multiple life history stages Dennis et al. [4].
The model we used as the basis for the first part of the evolutionary game is a single-stage version of the LPA (larvae-pupae-adult) model developed by Dennis et al. [4]. This is given by In this model, x is the number of adults at time t, b is the rate of recruitment into the adult class (or the probability of an individual surviving to adulthood), c is the cannibalism rate, and µ is the adult death rate. The term exp(−cx) is the probability that a potential recruit into the adult stage is not cannibalized between times t and t + 1. We used parameter values derived from experimental data. One time step represents two weeks in the model. To cast this system in the standard form for an evolutionary game, we first restate the ecological dynamics equation in terms of a fitness function [8]. This part of the evolutionary game is expressed as where H i (u, x) is the fitness function for population i, which represents the per capita change in population density from one time step to the next [8]. In this equation, x denotes the population vector, and u denotes the strategy vector, so that the mean strategy of population i is u i . The fitness generating function, or G-function provides a convenient way of expressing all of the H i functions. The G-function is defined as follows [8]: The variable v is a "virtual" variable. When v is replaced by some u i , the G-function gives the fitness function for population i. The model becomes We evaluated the evolution of a single population dependent on a scalar strategy, and thus used the following equation for the ecological dynamics: where the G-function, as obtained from equation (1), is given by We assume that the rate of cannibalism is unaffected by the strategy of the beetle and use the constant value c = 0.028 [5]. Sensitivity to corn oil does, however, affect reproduction and mortality rates, so the model is modified to account for these effects by making the recruitment and death rates (b and µ) functions of strategy v. We note that the G-function depends on the strategy; however, the fitness of beetles using strategy u i is unaffected by strategies used by other beetles. The dependence of b and µ on the strategy was determined from experimental data. The average values of b and µ are known for each of the three possible genotypes [5] [6]. We fit a quadratic equation to the three data points for each parameter as shown in Figure 1 to determine the functions b(v) and µ(v). These are defined as These are interpreted as the average recruitment and death rates of a population with a + allele frequency of v. When modeled this way, a population with a + allele frequency of approximately 0.583 has the highest reproduction rate, and one with a frequency 0.65 has the lowest mortality.
Next we define the second part of the game, which produces the strategy dynamics. The standard form for the strategy dynamics according to EGT is given by [8]   This is derived under the assumption that the rate of change of a trait over time is proportional to the amount of variation present in the population. The strategy at time t is given by u. The parameter σ 2 represents a small variance in strategies from the mean. The data indicate that this parameter ranges from 0.02 to 0.16. We found that the model fit the data well using values of σ 2 in or near this range, including 0.14, 0.2, and 0.3. These are the values we used in the model for comparing to treatments 1, 2, and 3, respectively. Combining the ecological dynamics (2) with the strategy dynamics (5) gives the complete evolutionary game model: In this model, the mean strategy of a population may be interpreted as the frequency of the + allele. Running the complete evolutionary game (6) yields changing population sizes and fitnesses as they are influencing one another. Using this model, the population evolves toward an ESS that in this case maximizes fitness.

Results
The EGT model yields results that are reasonably consistent with experimental data. Although the model tended to predict that the population size and strategy reach equilibrium slightly faster than the treatments, they both reached values that are reasonably close to the data. Figures 2 and 3 show comparisons of model results to experimental data for each of the three treatments. The treatments varied by initial allele frequencies. treatment. Figure 3 shows the model results for allele frequency (strategy) along with the mean data over the three treatments. Each data point in these figures is an average over three replicates of the given treatment with the exception of the control treatment data, which are averages over four replicates. The model is run using the initial values for population size and + allele frequency corresponding to those of the treatments upon addition of + alleles into the population. These values along with those we used for the variance parameter σ 2 are given in Table 1. In addition to the values used for comparing model results to each of the treatments, Table 1 includes the values we used in the model run that we compare to the mean of treatments 1, 2, and 3, and the values we used in the model run for comparison to the control treatment, in which gene frequencies do not change.
For the three genetically segregating populations with initial + allele frequencies of 0.02, 0.04, and 0.08, the model predicts an equilibrium allele frequency of approximately 0.61 with an equilibrium adult population size of approximately 179. In the model, the population size had reached equilibrium at time 70. The homozygous (cos/cos) control treatment remains genetically unchanged and is forecast by the model to have an equilibrium adult population size of 128. Although all populations had reached equilibrium in the model at time 70, the gene frequencies had not. The population sizes and gene frequencies from the model at the final times for which we have data are listed in Table 2.

Discussion
Evolutionary game theory can be applied to the maintenance of polymorphisms within populations [8]. While this is typically done with strategies representing phenotypes, our results show that evolutionary games are potentially useful tools for predicting long-term allele frequencies as well. Using genotype as the adaptive trait, the EGT model produced allele frequencies and population sizes similar to those found in experimental data. These data suggest that there is a stable polymorphism in the population; that is, multiple alleles persist due to a fitness advantage conferred by the heterozygous (cos/+) genotype. The population in the evolutionary game reaches an equilibrium due to the population density regulation imposed by the cannibalism in the model [2]. This results in an ESS that in this model also represents the strategy of highest population fitness. This agrees with the experimental data, which support the theoretical prediction that selection is expected to move an ecologically stable system toward an ESS resulting in the maximization of adult numbers [5] [6]. This prediction assumes that selection is not frequency dependent. Thus, our model provides a quantitative prediction of an ESS in addition to the population size when the population has reached this strategy. Furthermore, since the + allele frequency at the ESS is less than one, the cos allele is still present in the population, which supports the existence of a stable polymorphism at this locus. Our model produces higher equilibrium population sizes when + alleles are introduced than when no manipulations are imposed, as supported by the data [5] [6]. This is an important pattern to note, as it supports the theory that populations have the ability to reach a larger size if they have a higher mean fitness [5]. Because the ESS in this case is the strategy of highest fitness, a population at this ESS reaches the largest population possible as determined by this maximum fitness. The individual fitness conferred by the strategy (sensitivity to corn oil) is not influenced by the strategy of others. For this reason, the G-function we use is solely density dependent.
Genetic variation in a population directly affects the rate of evolution, or change in mean strategy of a population [8]. Since higher amounts of variation in a population are associated with faster evolution, it is expected that introducing different amounts of the + alleles would produce different rates of evolution. The rate of evolution in the model is captured by the parameter σ 2 , as strategy dynamics scale with variance [8]. However, there is some difficulty in estimating the variance parameter σ 2 directly from the data. The values of σ 2 that provided good fits to the frequency data were at the upper end of the range of variance in the data and slightly higher. However, values that provided good fits to the data agreed with the initial pattern of variance introduced. That is, treatment 1 had the lowest initial variance, followed by treatment 2, then treatment 3. The values of σ 2 in the model that produced good fits to the data followed this same pattern.
In this model, the mean strategy corresponds to the frequency of the + allele in the population with our numerical convention for representing genotypes as strategies. This is illustrated as follows. If we let P 1 , P 2 , and P 3 denote the number of individuals with the cos/cos, cos/+, and +/+ genotypes, respectively, then the frequency of the + allele (p) is given by [7] This is also the mean strategy of the population given that the strategies of the above genotypes are 0, 0.5, and 1, respectively. This convention therefore allows us to apply EGT in order to determine long-term gene frequencies.
We expect that replacing the model of population dynamics used in the evolutionary game with a multi-stage model will provide a more realistic evolutionary game. We will use the LPA model for T. castaneum, which is a difference equation model that includes ecological dynamics equations for the larval and pupal stages of the flour beetle in addition to the adult stage [4]. By incorporating this model into the evolutionary game, demographically detailed population sizes can be predicted, and the allele frequency dynamics will reflect the influence of all life history stages. The game theory model could then be extended further to incorporate multiple interacting flour beetle populations.
While classical genetic models may predict equilibrium allele frequencies based on fitness, the evolutionary game gives a more detailed picture of allele frequency dynamics by incorporating population dynamics. These represent two characteristics of populations that significantly influence one another, ultimately playing a role in determining the ESS. In this study, the theoretical evolutionary game model is connected to experimental data, and we obtain a quantitative estimate for an ESS along with an equilibrium population size for T. castaneum. This study shows the utility of EGT models for predicting ESS using genetic information in the modeling of population dynamics.