31:T5a9,md`### What are you seeing here? This notebook builds off a basic Prisoner's Dilemma model I built that you can find [here](https://observablehq.com/d/68247923b0ebd148). It provides a brief introduction of what the Prisoner's Dilemma is and why it's interesting. In this notebook, I show a simple but fascinating extension to the model that evolutionary biologists have extensively explored. It's called the [Green Beard Effect](https://en.wikipedia.org/wiki/Green-beard_effect). The idea is that if an animal has an unusual inherited physical characteristic, such as a green beard, it can help lead to the rise of altruism. If related animals in the population share the same trait and can recognize it in others, it gives them a clue about who it's safe to cooperate with. If I have a green beard, and I see another person with a green beard, it's safe to help them because the chance they'll cheat me is much lower than for the average stranger. I can feel confident they'll have the same behaviors as me because they're probably my cousin. In this sim, agents can adopt behaviors and markers from game partners if *pairwise interaction* is active, so there's no difference between inheriting traits and learning them from others. Try picking some positions for the sliders under the display and seeing what kinds of behaviors dominate the population. I recommend comparing the results to the original notebook so you can see the difference. `32:T569,{ const context = DOM.context2d(width, height); const colWidth = width / agentCount; let maxVal = initFitness; while (true) { context.clearRect(0, 0, width, height); if (agentList.length > 0) { let topAgent = agentList[agentCount - 1]; maxVal = topAgent.fitness; for (let i = 0; i < agentCount; i++) { let x = i * colWidth; let h = (agentList[i].fitness * height) / maxVal; let y = (height - h) / 2; context.beginPath(); if (agentList[i].strategy.other == 0) { context.fillStyle = "black"; } else { context.fillStyle = "steelblue"; } context.rect(x, y, colWidth-1, h/4); context.fill(); context.beginPath(); if (agentList[i].strategy.kin == 0) { context.fillStyle = "black"; } else { context.fillStyle = "firebrick"; } context.rect(x, y + (h/4), colWidth-1, h/4); context.fill(); context.beginPath(); let hue = Math.floor(agentList[i].color * 360 / nMarkers); context.fillStyle = `hsl(${hue},60%,40%)`; context.rect(x, y + (h/2), colWidth-1, h/2); context.fill(); } } context.canvas.onmousedown = event => { mutable paused = !paused; }; yield context.canvas; } }33:Tb15,{ const context = DOM.context2d(width, height); while (true) { context.clearRect(0, 0, width, height); if (history.length >= 1) { let segWidth = Math.floor(width / history.length); let startIndex = 0; let segments = history.length; if (history.length > width) { segWidth = 1; startIndex = history.length - width; segments = width; } let barGap = 1; if (segments > 1500) { barGap = 1000; } else if (segments > 150) { barGap = 100; } else if (segments > 15) { barGap = 10; } else { barGap = 1; } for (let i = 1; i < segments; i++) { let xStart = (i-1) * segWidth; let xEnd = i * segWidth; let iStart = startIndex + (i-1); let iEnd = iStart + 1; let yStart = height - (history[iStart].value.none) * height; let yEnd = height - (history[iEnd].value.none) * height; context.beginPath(); context.moveTo(xStart, yStart); context.lineTo(xEnd, yEnd); context.strokeStyle = "black"; context.stroke(); yStart = height - (history[iStart].value.kin) * height; yEnd = height - (history[iEnd].value.kin) * height; context.beginPath(); context.moveTo(xStart, yStart); context.lineTo(xEnd, yEnd); context.strokeStyle = "firebrick"; context.stroke(); yStart = height - (history[iStart].value.other) * height; yEnd = height - (history[iEnd].value.other) * height; context.beginPath(); context.moveTo(xStart, yStart); context.lineTo(xEnd, yEnd); context.strokeStyle = "steelblue"; context.stroke(); yStart = height - (history[iStart].value.all) * height; yEnd = height - (history[iEnd].value.all) * height; context.beginPath(); context.moveTo(xStart, yStart); context.lineTo(xEnd, yEnd); context.strokeStyle = "darkorchid"; context.stroke(); if (iEnd % barGap == 0) { let v = iEnd * speed; context.beginPath(); context.moveTo(xEnd, 0); context.lineTo(xEnd, height); context.strokeStyle = "gray"; context.stroke(); context.fillText(`${v}`, xEnd + 2, height-10); } } } for (let i = 0; i < 3; i++) { let f = (i+1) / 4; context.beginPath(); context.moveTo(0, (1-f) * height); context.lineTo(width, (1-f) * height); context.strokeStyle = "gray"; context.stroke(); context.fillText(`${f}`, 0, -2 + ((1-f) * height)); } context.canvas.onmousedown = event => { mutable paused = !paused; }; yield context.canvas; } }34:T802,md` ### Explanation In this version of the Prisoner's Dilemma game, agents have not one behavior, but two. There's the behavior they apply when they meet a game partner with whom they share a marker and the behavior they use for everyone else. Unlike in real life, agents can always perfectly identify kin. Biologists are interested in the Green Beard Effect because exactly how altruistic behavior shows up in nature really isn't obvious. Some creatures, like humans and ants, show extraordinary teamwork skills. Most species show very litle. Some species, like chimpanzees, show tight cooperation within small bands of related individuals and a murderous dislike for others of their kind. For many years, the Green Beard Effect was the best explanation we had for how teamwork might evolve, and it's still a touchstone of evolutionary theory research. ### Things to look for Adding just a little more complexity to the original Prisoner's Dilemma model creates a much broader landscape of behaviors to explore. I recomemnd trying to figure out what range of behaviors support kin cooperation. Then see if you can find *any* conditions under which universal cooperation arises. Being able to identify potential kin (or former collaborators who've learned how to collaborate) makes it easier for cooperative behavior to dominate in the population, but it also means that universal cooperation with non-kin almost never shows up. Furthermore, the higher the number of possible markers, the more specific they become. Thus, having more markers makes the green beard effect stronger. Sometimes, you'll see the strategy of *cooperate only with non-kin*, which shows up as blue, appearing in the simulation when almost all the agents have become related. These agents are hardly ever meeting non-kin, so it doesn't matter that they notionally cooperate. They're just differently colored defectors. But their dominance isn't stable. If another strain of agent arises that defects against them but cooperates with its own kin, they're in trouble. `35:T464,//This function gets called each time-step, and updates the state of the population. function update() { /* NOTE: When referencing simulation variables directly, we *always* use the keyword mutable, otherwise the sim won't evaluate as expected. */ if (mutable steps % speed == 0) { captureState(mutable agentList, mutable history, mutable steps); } mutable agentList.sort((a, b) => a.fitness - b.fitness); if (Math.random() < pRepeat) { //Agent pair doesn't update } else { // Select a pair of agents to interact const indexA = Math.floor(Math.random() * agentCount); let indexB = indexA; while (indexB == indexA) { indexB = Math.floor(Math.random() * agentCount); } mutable agentA = agentList[indexA]; mutable agentB = agentList[indexB]; } applyCosts(mutable agentA, mutable agentB); // Trigger recycling with some probability if (Math.random() < pChange) { if (changeType == 'Pairwise') { makePairwiseChange(mutable agentA, mutable agentB); } else { makeAnonymousChange(mutable agentA, mutable agentB); } } }36:T841,function makeAnonymousChange(agentA, agentB) { /* The aim here is to have selection that's general and mixed for both strong and weak agents. I perform roulette selection both times. When selecting a weak agent, I measure maxFitness-fitnessA for each agent A instead of just fitnessA. */ // Sum fitness of population let totalFitness = 0; for (const agent of agentList) { totalFitness += agent.fitness; } // Make agent selection from fitness range const hiSelection = Math.random() * totalFitness; // Iterate through agents looking for selection let parent = null; let cumulFit = 0; for (const agent of agentList) { cumulFit += agent.fitness; if (hiSelection < cumulFit) { parent = agent; break; } } // Find agent with highest fitness let maxFit = 0; for (const agent of agentList) { maxFit = Math.max(maxFit, agent.fitness); } //Add one to the max so that the highest scoring agent still has a gap of one maxFit++; //Find the sum of the deltas from each fitness value to the current max const totalDelta = (maxFit * agentCount) - totalFitness; // Make agent selection from total delta range const loSelection = Math.random() * totalDelta; // Iterate through agents looking for selection let child = null; let cumulDelta = 0; for (const agent of agentList) { cumulDelta += (maxFit - agent.fitness); if (loSelection < cumulDelta) { child = agent; break; } } if (Math.random() < pMutation) { //There's a small chance in every round that an agent will pick a random strategy child.strategy = { kin: Math.floor(Math.random() * 2), other: Math.floor(Math.random() * 2) } } else { // Replace the low scoring agent strategy with the high scoring strategy child.strategy = Object.assign({}, parent.strategy); } if (Math.random() < pMutation) { //There's a small chance in every round that an agent will pick a random color child.color = Math.floor(Math.random() * nMarkers); } else { child.color = parent.color; } }37:[] 2f:{"title":"Prisoner's Dilemma with Kin Cues","cells":[{"id":0,"value":"md`# Prisoner's Dilemma with Kin Cues`","pinned":false,"hidden":false,"mode":"ojs"},{"id":400,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":389,"value":"md`### Agents ranked by fitness\nEach bar represents an agent, and their height represents their 'fitness'. (Taller bars means higher fitness.) A red mark on a bar indicates that this agent cooperates with kin. Blue marks indicate cooperation with non-kin. Black marks indicate agents with the 'defect' strategy (self-interest). The lower half of each bar shows the particular kin marker that the agent displays. \n(Click on the plot to start or pause the simulation.) \n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":166,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":394,"value":"md`### Strategy frequency over time\nThis plot shows the relative fraction in the population of the four possible strategies. \n* Red = cooperate only with kin\n* Blue = cooperate only with strangers\n* Purple = cooperate with everyone\n* Black = cooperate with no one\n\n(Click on the plot to start or pause the simulation.)`","pinned":false,"hidden":false,"mode":"ojs"},{"id":175,"value":"$33","pinned":false,"hidden":false,"mode":"ojs"},{"id":163,"value":"// viewof bReset = button({value: \"Reset\", description: \"Click here to return the simulation to its initial conditions\"})\nviewof bReset = md``","pinned":false,"hidden":false,"mode":"ojs"},{"id":108,"value":"md`#### pRepeat\nThe probability that two agents interacting will meet each other again in the next simulation step.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":83,"value":"viewof slideRepeat = html``","pinned":false,"hidden":false,"mode":"ojs"},{"id":71,"value":"pRepeat = slideRepeat / 100;","pinned":false,"hidden":false,"mode":"ojs"},{"id":110,"value":"md`#### pChange\nThe probability that an agent will change their strategy if they interact with another that has higher fitness.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":86,"value":"viewof slideChange = html``","pinned":false,"hidden":false,"mode":"ojs"},{"id":15,"value":"pChange = slideChange / 100;","pinned":false,"hidden":false,"mode":"ojs"},{"id":558,"value":"md`#### nMarkers\nThe number of kin markers that the population can take on. More markers means greater specificity of kin detection. NOTE: If you alter the number of markers, you'll have to reset the simulation for the change to take effect.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":553,"value":"viewof slideMarkers = html``","pinned":false,"hidden":false,"mode":"ojs"},{"id":559,"value":"nMarkers = slideMarkers;","pinned":false,"hidden":false,"mode":"ojs"},{"id":453,"value":"md`### Strategy change method\nWhether agents get new strategies from the partners they interact with, or the population at large. As in the previous sim, pairwise update makes it easier for cooperation to dominate because exploitative cooperator-defector pairings aren't stable.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":448,"value":"viewof changeType = html``","pinned":false,"hidden":false,"mode":"ojs"},{"id":404,"value":"$34","pinned":false,"hidden":false,"mode":"ojs"},{"id":120,"value":"md`### Simulation Functions`","pinned":false,"hidden":false,"mode":"ojs"},{"id":137,"value":"//This gets called at the start of the simulation. It sets up our initial conditions. \nfunction init() {\n /*\n NOTE:\n When referencing simulation variables directly, we *always* use the keyword mutable, \n otherwise the sim won't evaluate as expected.\n */\n \n mutable agentList = [];\n for (let i = 0; i < agentCount; i++) {\n let agent = {};\n agent.id = i;\n agent.fitness = initFitness;\n agent.strategy = {\n kin: Math.floor(Math.random() * 2),\n other: Math.floor(Math.random() * 2)\n };\n agent.color = Math.floor(Math.random() * nMarkers);\n mutable agentList.push(agent);\n }\n \n const indexA = Math.floor(Math.random() * agentCount);\n let indexB = indexA;\n while (indexB == indexA) {\n indexB = Math.floor(Math.random() * agentCount);\n } \n mutable agentA = agentList[indexA];\n mutable agentB = agentList[indexB];\n\n mutable steps = 0;\n mutable initialized = true;\n mutable history = [];\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":140,"value":"$35","pinned":true,"hidden":false,"mode":"ojs"},{"id":142,"value":"function applyCosts(agentA, agentB) {\n\n let aStrat = agentA.strategy.other;\n let bStrat = agentB.strategy.other;\n \n if (agentA.color == agentB.color) {\n aStrat = agentA.strategy.kin;\n bStrat = agentB.strategy.kin;\n }\n \n // Allocate costs to agents based on strategy\n if (aStrat == bStrat) {\n if (aStrat == 0) {\n // Both players defect, no change in fitness\n } else {\n // Both players cooperate, both get b-c\n agentA.fitness += benefit - cost;\n agentB.fitness += benefit - cost;\n }\n } else {\n if (aStrat > bStrat && agentA.fitness > 0) {\n agentA.fitness += -cost;\n agentB.fitness += benefit;\n } else if (agentB.fitness > 0) {\n agentA.fitness += benefit;\n agentB.fitness += -cost;\n }\n }\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":147,"value":"// Occasional peerwise strategy contagion based on absolute fitness\nfunction makePairwiseChange(agentA, agentB) {\n \n let loAgent = agentA;\n let hiAgent = agentB;\n if (loAgent.fitness > hiAgent.fitness) {\n loAgent = agentB;\n hiAgent = agentA;\n }\n \n //There's a small chance in every round that an agent will pick a random strategy\n if (Math.random() < pMutation) {\n loAgent.strategy = {\n kin: Math.floor(Math.random() * 2),\n other: Math.floor(Math.random() * 2)\n }\n } else {\n loAgent.strategy = Object.assign({}, hiAgent.strategy);\n }\n \n //There's a small chance in every round that an agent will pick a random color\n if (Math.random() < pMutation) {\n loAgent.color = Math.floor(Math.random() * nMarkers);\n } else {\n loAgent.color = hiAgent.color;\n }\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":447,"value":"$36","pinned":true,"hidden":false,"mode":"ojs"},{"id":158,"value":"//Function to store the outcome of a round for display in the history timeline\nfunction captureState(agentList, history, steps) {\n \n let coopAll = 0;\n let coopKin = 0;\n let coopOther = 0;\n let coopNone = 0;\n for (let agent of agentList) {\n if (agent.strategy.kin == 0 && agent.strategy.other == 0) {\n coopNone++;\n } else if (agent.strategy.kin == 0 && agent.strategy.other == 1) {\n coopOther++;\n } else if (agent.strategy.kin == 1 && agent.strategy.other == 0) {\n coopKin++;\n } else if (agent.strategy.kin == 1 && agent.strategy.other == 1) {\n coopAll++;\n }\n }\n let coopFracs = {\n none: coopNone / agentCount,\n other: coopOther / agentCount,\n kin: coopKin / agentCount,\n all: coopAll / agentCount\n }\n \n let moment = {}\n moment.time = steps;\n moment.value = coopFracs;\n history.push(moment);\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":29,"value":"md`### Simulation Constants`","pinned":true,"hidden":false,"mode":"ojs"},{"id":5,"value":"agentCount = 100","pinned":true,"hidden":false,"mode":"ojs"},{"id":9,"value":"initFitness = 100","pinned":true,"hidden":false,"mode":"ojs"},{"id":11,"value":"height = 200","pinned":true,"hidden":false,"mode":"ojs"},{"id":13,"value":"speed = 100","pinned":true,"hidden":false,"mode":"ojs"},{"id":19,"value":"cost = 2","pinned":true,"hidden":false,"mode":"ojs"},{"id":34,"value":"benefit = 5","pinned":true,"hidden":false,"mode":"ojs"},{"id":17,"value":"pMutation = 0.05","pinned":true,"hidden":false,"mode":"ojs"},{"id":61,"value":"md`### Simulation Variables`","pinned":true,"hidden":false,"mode":"ojs"},{"id":51,"value":"mutable agentList = [];","pinned":true,"hidden":false,"mode":"ojs"},{"id":63,"value":"mutable agentA = null;","pinned":true,"hidden":false,"mode":"ojs"},{"id":65,"value":"mutable agentB = null;","pinned":true,"hidden":false,"mode":"ojs"},{"id":125,"value":"md`### Framework Functions\nWhere possible, I've split away the functionality that relates to the simulation running framework from the code that runs the simulation itself. The hope here is that the same framework could be used for a large number of possible sims.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":127,"value":"{\n while (true) {\n yield Promises.delay(tickRate, tickFunc());\n }\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":132,"value":"function tickFunc() { \n if (!initialized) {\n reset();\n }\n if (!paused) {\n update();\n mutable steps++;\n }\n return 0;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":135,"value":"function reset() {\n mutable steps = 0;\n mutable initialized = true;\n init();\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":180,"value":"{\n //Enables the reset button to work\n bReset;\n mutable initialized = false;\n}","pinned":true,"hidden":false,"mode":"ojs"},{"id":22,"value":"md`### Framework Constants`","pinned":false,"hidden":false,"mode":"ojs"},{"id":30,"value":"tickRate = 1;","pinned":false,"hidden":false,"mode":"ojs"},{"id":39,"value":"md`### Framework Variables`","pinned":false,"hidden":false,"mode":"ojs"},{"id":42,"value":"mutable initialized = false;","pinned":false,"hidden":false,"mode":"ojs"},{"id":43,"value":"mutable steps = 0;","pinned":false,"hidden":false,"mode":"ojs"},{"id":46,"value":"mutable history = [];","pinned":false,"hidden":false,"mode":"ojs"},{"id":48,"value":"mutable paused = true;","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W37","stdlib":"1","titling":"auto"}