29:null 2a:["$","title",null,{"children":"Urn question | huw | Observable"}] 2f:[] 2d:{"id":"c659f572b2430bd6","slug":null,"title":"Urn question","thumbnail":"6be9685e1ba5e2f9fefa95d537ed7026f1052692bf9114825870ee78fde137a3","access_level":"public","create_time":"$D2024-01-29T03:00:52.573Z","update_time":"$D2024-01-29T11:27:01.198Z","view_time":null,"first_public_time":"$D2024-01-29T03:00:52.573Z","fork":{"id":"8467e6cb450fd43e","slug":"urn-question","title":"Urn question","owner":{"id":"cc48459e4293f942","login":"mjbo","name":"Max Bo"}},"stars":0,"starred":false,"owner":{"type":"team","id":"fd09b4e2aa9630ce","login":"huw","name":"huw","avatar_url":"https://avatars-next.observableusercontent.com/d4b15139cfa29ef742c90b040fbd38621c517cac8270b3ec32211ee0669b0920","tier":"starter_2024"},"version":616,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"16379e9551bd850d","login":"huw","name":"huw","avatar_url":"https://avatars-next.observableusercontent.com/d4b15139cfa29ef742c90b040fbd38621c517cac8270b3ec32211ee0669b0920","tier":null}],"members":[{"type":"individual","id":"16379e9551bd850d","login":"huw","name":"huw","avatar_url":"https://avatars-next.observableusercontent.com/d4b15139cfa29ef742c90b040fbd38621c517cac8270b3ec32211ee0669b0920","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"title":"Urn question","cells":[{"id":17,"value":"# Urn question","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":614,"value":"## Lol, I got this totally wrong, but fun to watch me have a stab at it and also hopefully a not-awful explanation of the difference between frequentist and Bayesian approaches?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":26,"value":"> You are given an urn containing 100 balls; ${tex`n`} of them are red, and ${tex`100-n`} are green, where n is chosen uniformly at random in ${tex`[0, 100]`}. You take a random ball out of the urn — it’s red — and discard it. The next ball you pick (out of the 99 remaining) is:\n\n> a) More likely to be red\n\n> b) More likely to be green\n\n> c) Equally likely","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":331,"value":"# ‘Frequentist’ approach\nMathematically, we can write out ${tex`P(\\textcolor{red}{R})`} and ${tex`P(\\textcolor{red}{RR}|\\textcolor{red}{R})`} in terms of ${tex`n`}:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":335,"value":"\\begin{aligned}\n P(\\textcolor{red}{R}) &= \\frac{n}{100} \\\\\n P(\\textcolor{red}{RR}|\\textcolor{red}{R}) &= \\frac{n - 1}{99}\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":346,"value":"If we’re more likely to draw a red the second time around, then:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":350,"value":"\\begin{aligned}\n P(\\textcolor{red}{R}) &< P(\\textcolor{red}{RR}|\\textcolor{red}{R}) \\\\\n \\frac{n}{100} &< \\frac{n - 1}{99} \\left( = \\frac{n}{99} - \\frac{1}{99} \\right) \\\\\n \\frac{n}{100} - \\frac{n}{99} &< -\\frac{1}{99} \\\\\n \\frac{100n - 99n}{9900} &> \\frac{1}{99} \\\\\n n &> \\frac{9900}{99} = 100\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":359,"value":"We're only more likely to draw a red the second time around if ${tex`n > 100`}, which is untrue for all ${tex`n \\in [0, 100]`}. We should answer (b), although in the perverse case where ${tex`n = 100`}, we're equally likely.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":363,"value":"The intuition behind this answer is clear. Imagine we fix ${tex`n`} at 50 red balls, giving me a 50/50 chance. If I draw a red ball, now there are 49 red balls and 50 green balls, giving me a lower, 49/50 chance. You can repeat this for any choice of ${tex`n < 100`}.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":423,"value":"I’ll let you pick ${tex`n`}, and we’ll run 100,000 simulations of the urn game. If we draw red, we’ll count that toward ${tex`P(\\textcolor{red}{R})`}, and if we draw 2 reds, we’ll count that toward ${tex`P(\\textcolor{red}{RR}|\\textcolor{red}{R})`} (remembering to divide by the number of reds to get that true probability).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":372,"value":"viewof n = Inputs.range([1, 100], { step: 1, label: tex`n`, value: 50 })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":380,"value":"urn = {\n const red = Array.from({ length: n }, () => \"red\");\n const green = Array.from({ length: 100 - n }, () => \"green\");\n return red.concat(green);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":408,"value":"P = {\n let countRed = 0;\n let countRedRedGivenRed = 0;\n const N = 100000;\n\n for (let i = 0; i < N; i++) {\n const urnCopy = _.shuffle([...urn]);\n\n if (urnCopy[0] === \"red\") {\n countRed++;\n\n if (urnCopy[1] === \"red\") {\n countRedRedGivenRed++;\n }\n }\n }\n\n return [countRed / N, countRedRedGivenRed / countRed];\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":403,"value":"In this case, ${tex`P(\\textcolor{red}{R})`} ${P[0] > P[1] ? \">\" : P[0] < P[1] ? \"<\" : \"=\"} ${tex`P(\\textcolor{red}{RR}|\\textcolor{red}{R})`}. I encourage you to slide ${tex`n`} around a bit. You’ll find that occasionally, the predicted relationship doesn’t hold true, but in most practical simulations, it does!","pinned":true,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":439,"value":"# ‘Bayesian’ approach\nThis is harder to reason about because there is a lot of arbitrariness involved. However.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":442,"value":"Imagine that ${tex`n`} isn’t a ‘true property about the world that actually exists in some sense’, but a random variable, of the kind that doesn’t take an actual value until it’s observed. From the first-person perspective, this is kind of intuitively true—you don’t know ${tex`n`}, it could be anything!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":444,"value":"Without knowing anything about ${tex`n`}, we’ve been assured that it’s a random number between 0 and 100 (and specifically, uniformly distributed). That is:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":449,"value":"n \\sim U(0, 100)","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":451,"value":"Now imagine there are hundreds of universes out there, each with their own ${tex`n`}. Some of these worlds will have ${tex`n = 20`}, others ${tex`n = 71`}, and so on. Let’s chart that our empirically:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":469,"value":"viewof rerun = Inputs.button(\"Re-run\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":454,"value":"allPossibleNs = d3.range(0, 100 + 1)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":458,"value":"aSampleOfNs = {\n rerun;\n const sample = [];\n for (let i = 0; i < 100000; i++) {\n sample[i] = _.sample(allPossibleNs);\n }\n return sample;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":462,"value":"The average ${tex`n`} is ${d3.mean(aSampleOfNs)}. Here’s what it all looks like on a graph:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":464,"value":"Plot.barY(aSampleOfNs, Plot.groupX()).plot({ width, height: 200 })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":474,"value":"We know we live in one of these universes, but we don’t know which one we live in. How could we figure it out? By drawing balls, of course!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":489,"value":"Our chance of drawing a red ball right now is ${tex`P(\\textcolor{red}{R}) = \\frac{n_1}{100}`}. Our best guess at ${tex`n`}, knowing nothing else, is that it’s probably around the average value for ${tex`n`}. We can call this our _prior_ for ${tex`n`} (which we’ll label as ${tex`n_1`}). The real definition is a little more complex, but the important thing right now is to remember that our prior could be different, and in fact, as Bayesians, we can pick any prior we want.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":533,"value":"n_1 = d3.mean(aSampleOfNs)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":499,"value":"(This is a _good_ prior because it makes good use of the available information and doesn’t bias upward or downward for no reason)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":497,"value":"Given that we don’t know what universe we’re in, we can compute ${tex`P(\\textcolor{red}{R}) = \\frac{${d3.mean(aSampleOfNs)}}{100} = ${d3.mean(aSampleOfNs) / 100}`}.","pinned":true,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":478,"value":"Now we go and draw a red ball. This means we cannot possibly live in the ${tex`n = 0`} universe, with 0 red balls. So let’s rule it out:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":479,"value":"newPossibleUniverses = aSampleOfNs.filter((n) => n !== 0)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":484,"value":"Plot.barY(newPossibleUniverses, Plot.groupX()).plot({\n width,\n height: 200,\n x: { domain: d3.range(0, 100 + 1) }\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":531,"value":"We call the method of updating ${tex`n`} based on our evidence ‘updating our priors’. While we previously had a prior of ${tex`n_1 = ${n_1.toFixed(4)}`}, our new estimate for ${tex`n`} is ${tex`n_2 = ${n_2.toFixed(4)}`}, which is ${n_2 > n_1 ? \"larger\" : n_2 < n_1 ? \"smaller\" : \"the same\"} than ${tex`n_1`}. I encourage you to ${Inputs.bind(htl.html``, viewof rerun)} the simulation a few times to how often this relationship holds.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":535,"value":"n_2 = d3.mean(newPossibleUniverses)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":487,"value":"Since we know we can’t live in the ${tex`n = 0`} universe, we can compute ${tex`P(\\textcolor{red}{RR|R}) = \\frac{n_2 - 1}{99} = \\frac{${n_2.toFixed(4) - 1}}{99} = ${((n_2 - 1) / 99).toFixed(4)}`}.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":551,"value":"Here, ${tex`P(\\textcolor{red}{R})`} ${n_1 / 100 > (n_2 - 1) / 99 ? \">\" : n_1 / 100 < (n_2 - 1) / 99 ? \"<\" : \"=\"} ${tex`P(\\textcolor{red}{RR}|\\textcolor{red}{R})`}. However, I encourage you to ${Inputs.bind(htl.html``, viewof rerun)} a few times to see how often _this_ relationship holds.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":519,"value":"You’ll notice the sign flips a lot more often. This is because we have a slightly different set of equations:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":525,"value":"\\begin{aligned}\n P(\\textcolor{red}{R}) &= \\frac{n_1}{100} \\\\\n P(\\textcolor{red}{RR}|\\textcolor{red}{R}) &= \\frac{n_2 - 1}{99}\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":558,"value":"And we can derive:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":560,"value":"\\begin{aligned}\n P(\\textcolor{red}{R}) &< P(\\textcolor{red}{RR}|\\textcolor{red}{R}) \\\\\n \\frac{n_1}{100} &< \\frac{n_2 - 1}{99} \\left( = \\frac{n_2}{99} - \\frac{1}{99} \\right) \\\\\n n_1 &< \\frac{100}{99}n_2 - \\frac{100}{99}\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":566,"value":"Mathemtically, how do ${tex`n_2`} and ${tex`n_1`} relate? Well, you can choose your priors however you want in practice. But under our priors, we’ve updated ${tex`(n_1 \\to n_2) \\sim \\left(U(0, 100) \\to U(1, 100)\\right)`}, i.e. we keep ${tex`n`} distributed the same, but we just rule out the lower bound of that distribution being 0. This is exactly what we did in our simulation above. Therefore:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":568,"value":"\\begin{aligned}\n n_1 &= \\mathbb{E}[n] &= \\mathbb{E}[U(0, 100)] \\\\\n n_2 &= \\mathbb{E}[n|\\textcolor{red}{R}] &= \\mathbb{E}[U(1, 100)]\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":583,"value":"Substituting:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":587,"value":"\\begin{aligned}\n n_1 &= \\frac{a_1 + b_1}{2} &= \\frac{100}{2} &= 50 \\\\\n n_2 &= \\frac{a_2 + b_2}{2} &= \\frac{101}{2} &= 50.5\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":590,"value":"Compare these with the simulated values ${tex`n_1 = ${n_1.toFixed(4)}`} and ${tex`n_2 = ${n_2.toFixed(4)}`}. Substituting these equations back into our inequality:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":595,"value":"\\begin{aligned}\nn_1 &< \\frac{100}{99}n_2 - \\frac{100}{99} \\\\\n&< \\frac{100}{99} \\times \\frac{101}{2} - \\frac{100}{99} \\\\\n&= 50\n\\end{aligned}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined"},{"id":598,"value":"Now ${Inputs.bind(htl.html``, viewof rerun)}, and observe ${tex`P(\\textcolor{red}{R})`} ${n_1 / 100 > (n_2 - 1) / 99 ? \">\" : n_1 / 100 < (n_2 - 1) / 99 ? \"<\" : \"=\"} ${tex`P(\\textcolor{red}{RR}|\\textcolor{red}{R})`}. What happens when ${tex`n_1 < 50`}? And what happens when it doesn’t?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":600,"value":"To answer the original question, the Bayesian must assume that ${tex`n_1`} maintains its expected value of 50. In this case, then, ${tex`P(\\textcolor{red}{R}) = P(\\textcolor{red}{RR}|\\textcolor{red}{R})`}.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":608,"value":"Consider our intuitions. It is true that drawing a red ball increases the likely number of red balls in the urn, since we’ve ruled out a whole class of universes where there were no red balls. However, the increase in likelihood is perfectly counteracted by the fact that we just drew a red ball out of the urn! (If we put the red ball _back into_ the urn, however, then the probability would be higher).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"}],"files":"$W2f","stdlib":"1","titling":"auto"}} 2e:"$2d:body"