2a:null 2b:["$","title",null,{"children":"Bayesian inference for Gaussian iid data with known variance | Mattias Villani | Observable"}] 31:T480,problemdescription = md`### Internet connection speed data The maximum internet connection speed downstream in my home is 50 Mbit/sec. This maximum will typically never be reached, but my internet service provider (ISP) claims that the average speed is at least 20Mbit/sec. To test this, I collect a total of five measurements, ${tex`x = (15.77, 20.5, 8.26, 14.37, 21.09)`}, over the course of five consecutive days using a speed testing internet service. We will here analyze these data using the iid Gaussian model ${tex`X_1,\ldots,X_n \mid \theta, \sigma^2 \sim \operatorname{N}(\theta,\sigma^2)`}, where ${tex`\theta`} is the average speed; we ignore for simplicity that the measurements cannot be negative. We will use a Gaussian prior for the mean ${tex`\theta`}. The measurements are reported to have a standard deviation of ${tex`\sigma = 5`} by the speed testing service, which will take as known. I will use a prior centered on the average claimed by the ISP, ${tex`\mu_0 = 20`}, with a prior standard deviation of ${tex`\tau_0 = 5`}. My prior beliefs are therefore that ${tex`\theta \in [10, 30]`} with approximately 95% probability.` 32:[] 2e:{"id":"58afe28db9cfc1ee","slug":"bayes-iid-gaussian-known-var","title":"Bayesian inference for Gaussian iid data with known variance","thumbnail":"1fce3fbc6b4c468c9b1ee9e45c0b25545cf19c2ad456c734618254e72efa3a29","access_level":"public","create_time":"$D2023-02-09T13:00:38.135Z","update_time":"$D2025-02-10T10:41:28.007Z","view_time":null,"first_public_time":"$D2023-02-09T13:00:38.135Z","fork":{"id":"32c30160ed8888fa","slug":"normalfordelningen","title":"Normalfördelningen","owner":{"id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani"}},"stars":1,"starred":false,"owner":{"type":"team","id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":"pro_2024"},"version":1399,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"56499be8cf43f0ef","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":null}],"members":[{"type":"individual","id":"56499be8cf43f0ef","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"subscribed":null,"forks":2,"collection":{"id":"9fdb7c9ba7be6ba8","slug":"bayesian-learning","title":"Bayesian Learning","owner":{"id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani"}},"type":"notebook","body":{"title":"Bayesian inference for Gaussian iid data with known variance","cells":[{"id":38,"value":"# Bayesian inference for Gaussian iid data with known variance","pinned":false,"hidden":false,"mode":"md"},{"id":1198,"value":"toc()","pinned":false,"hidden":false,"mode":"ojs"},{"id":700,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":719,"value":"titletext1 = md`### Prior-to-Posterior updating`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1036,"value":"resultstext = html`

\n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n\n

Model:	${tex`X_1,\\ldots,X_n \\mid \\theta, \\sigma^2 \\sim \\operatorname{N}(\\theta,\\sigma^2)`} with ${tex`\\sigma^2`} known.
Prior:	${tex`\\theta \\sim N(\\mu_0, \\tau_0^2)`}
Posterior:	${tex`\\theta \\vert \\boldsymbol{x} \\sim N(\\mu_n, \\tau_n^2)`}
Posterior precision:	${tex`\\frac{1}{\\tau_n^2} =\\frac{n}{\\sigma^2} + \\frac{1}{\\tau_0^2}`} = ${(1/(d.taun**2)).toPrecision(3)}
Posterior mean:	${tex`\\mu_n = w\\bar x + (1-w)\\mu_0`} = ${d.mun.toPrecision(3)}
Weight on data:	${tex`w = \\frac{\\frac{n}{\\sigma^2}}{\\frac{n}{\\sigma^2} + \\frac{1}{\\tau_0^2}}`} = ${d.w.toPrecision(3)}

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1265,"value":"**Data**:","pinned":false,"hidden":false,"mode":"md"},{"id":118,"value":"viewof datasettings = Inputs.form([\n Inputs.range([1, 5], {value: 1, step: 1, label: tex`n`}),\n Inputs.range([1, 20], {value: 5, step: 1, label: tex`\\sigma`}),\n ])","pinned":false,"hidden":false,"mode":"ojs"},{"id":1295,"value":"dataused = md`\nObservations used: ${tex`\\boldsymbol{x}=(`} ${datastring}${tex`)`}\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1268,"value":"**Prior**:","pinned":false,"hidden":false,"mode":"md"},{"id":1258,"value":"viewof priorsettings = Inputs.form([\n Inputs.range([0, 100], {value: 20, step: 1, label: tex`\\mu_0`}),\n Inputs.range([0.5, 100], {value: 5, step: 0.5, label: tex`\\tau_0`})\n ])","pinned":false,"hidden":false,"mode":"ojs"},{"id":1368,"value":"viewof quantile_input = Inputs.form([\n Inputs.range([5, 35], {value: 20, step: 1, label: \"quantile posterior\"}),\n Inputs.toggle({label: \"plot quantile prob\", value: false})\n])","pinned":false,"hidden":false,"mode":"ojs"},{"id":305,"value":"plt_pdf = Plot.plot({\n color: {\n legend: true,\n domain: plot_quantile ? [\"prior\",\"likelihood\",\"posterior\", \"posterior quantile\"] : [\"prior\",\"likelihood\",\"posterior\"],\n range: [\"#D6B656\", \"#6C8EBF\", \"#B85450\", d3.color(\"rgba(184, 84, 80, 0.2)\")]\n },\n x: {\n label: \"theta\",\n axis: true\n },\n y: {\n axis: false,\n domain: [0,1.02*maxpdf]\n },\n caption: \"Prior-to-Posterior mapping. The likelihood is normalized.\",\n marks: [\n Plot.ruleY([0]),\n Plot.line(densdata, {x: \"theta\", y: \"pdf\", stroke: \"type\", strokeWidth: 2.5}),\n Plot.areaY(densdata, {filter: d => d.theta >= post_quantile && d.type == \"posterior\" && plot_quantile, x: \"theta\", y: \"pdf\", \n fill: \"#B85450\", opacity: 0.2})\n ]\n })","pinned":false,"hidden":false,"mode":"ojs"},{"id":614,"value":"postresultstext = html`

\n\n \n \n \n \n

Posterior probability:

${tex`P( \\theta\\geq ${post_quantile} \\vert \\boldsymbol{x}) =${(1-postcdf).toPrecision(3)}`}

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1061,"value":"
\n### The sequential nature of Bayesian updating","pinned":false,"hidden":false,"mode":"md"},{"id":1131,"value":"Data observations can be processed sequentially in Bayesian learning in a so called **online** fashion. One way to phrase this is that that \"yesterday’s posterior becomes today’s prior\". The figure below illustrates this.","pinned":false,"hidden":false,"mode":"md"},{"id":1103,"value":"plt_seq = Plot.plot({\n color: {\n legend: \"ramp\", \n label: \"number of observations\",\n type: \"ordinal\",\n range: [\"#D6B656\",\"#D0A255\",\"#CA8F54\",\"#C47B52\",\"#BE6851\",\"#B85450\"]\n },\n x: {\n label: \"theta\",\n axis: true\n },\n y: {\n axis: false,\n //domain: [0,1.02*maxpdf]\n },\n caption: \"Bayesian updating as more data arrives. Note that n=0 is the original prior without any data.\",\n marks: [\n Plot.ruleY([0]),\n Plot.line(seqdensdata, {x: \"theta\", y: \"pdf\", stroke: \"nobs\", strokeWidth: 2.5}),\n Plot.areaY(seqdensdata, {filter: d => d.theta >= post_quantile && d.nobs == \"n=5\", x: \"theta\", y: \"pdf\", \n fill: \"#B85450\", opacity: 0.2})\n ]\n })","pinned":false,"hidden":false,"mode":"ojs"},{"id":1043,"value":"predtext = md`
\n### Predictive distribution`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1046,"value":"My streaming service becomes unreliable and buffers at speeds below 5Mbit/sec. I am therefore particularly interested in this ’catastrophic’ event happening tonight while watching my favourite\nmovie. Finding the probability of a single measurement lower than\n5MBit/sec is an exercise in prediction.","pinned":false,"hidden":false,"mode":"md"},{"id":1275,"value":"**Data**:","pinned":false,"hidden":false,"mode":"md"},{"id":1273,"value":"Inputs.bind(\n Inputs.form([\n Inputs.range([1, 5], {value: 1, step: 1, label: tex`n`}),\n Inputs.range([0.1, 50], {value: 5, step: 0.1, label: tex`\\sigma`}),\n ])\n , viewof datasettings)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1332,"value":"datausedpred = md`\nObservations used: ${tex`\\boldsymbol{x}=(`} ${datastring}${tex`)`}\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1281,"value":"**Prior**:","pinned":false,"hidden":false,"mode":"md"},{"id":1283,"value":"Inputs.bind(\n Inputs.form([\n Inputs.range([0, 100], {value: 20, step: 1, label: tex`\\mu_0`}),\n Inputs.range([0.1, 100], {value: 5, step: 0.1, label: tex`\\tau_0`})\n ])\n , viewof priorsettings)","pinned":false,"hidden":false,"mode":"ojs"},{"id":988,"value":"viewof pred_quantile = Inputs.range([0, 30], {value: 5, step: 1, label: tex`\\text{quantile prediction}`})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1002,"value":"predsresultstext = html`

\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n\n

Predictive density:	${tex`\\tilde x \\vert \\boldsymbol{x} \\sim N(\\mu_n, \\tau_n^2 + \\sigma^2)`}
	${tex`\\tilde x \\vert \\boldsymbol{x} \\sim N(${d.mun.toPrecision(4)}, ${(Math.sqrt(d.taun2 \n + d.sigma2)).toPrecision(4)}^2)`}
Predictive quantile:	${tex`P( \\theta\\leq ${pred_quantile} \\vert \\boldsymbol{x}) =${predcdf.toPrecision(3)}`}

\n\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":978,"value":"plt_pred = Plot.plot({\n x: {\n label: \"x_tilde\",\n axis: true\n },\n y: {\n axis: false,\n domain: [0,1.03*maxpredpdf]\n },\n caption: \"Predictive density for tomorrow's download speed.\",\n marks: [\n Plot.ruleY([0]),\n Plot.line(preddata, {x: \"xtilde\", y: \"pdf\", stroke: \"#006d77\", strokeWidth: 2.5}),\n Plot.areaY(preddata, {filter: d => d.xtilde <= pred_quantile, x: \"xtilde\", y: \"pdf\", \n fill: \"#006d77\", opacity: 0.2})\n ]\n })","pinned":false,"hidden":false,"mode":"ojs"},{"id":109,"value":"
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n---\n#### 🙈 Appendix 🙈\n\nHere is just some data, imports and function defintions that you probably don't want to read.","pinned":false,"hidden":false,"mode":"md"},{"id":221,"value":"#### Imports","pinned":false,"hidden":false,"mode":"md"},{"id":26,"value":"jstat = require('jstat')","pinned":false,"hidden":false,"mode":"ojs"},{"id":1195,"value":"function toc(selector = \"h2,h3\", heading = \"Table of Contents\") {\n return Generators.observe(notify => {\n let headings = [];\n\n function observed() {\n const h = Array.from(document.querySelectorAll(selector));\n if (h.length !== headings.length || h.some((h, i) => headings[i] !== h)) {\n notify(html`${heading}

${DOM.text(h.textContent)}`,\n {onclick: e => (e.preventDefault(), h.scrollIntoView())}\n );\n })}`);\n }\n }\n\n const observer = new MutationObserver(observed);\n observer.observe(document.body, {childList: true, subtree: true});\n observed();\n return () => observer.disconnect();\n });\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":223,"value":"#### Data","pinned":false,"hidden":false,"mode":"md"},{"id":1383,"value":"post_quantile = quantile_input[0]","pinned":false,"hidden":false,"mode":"ojs"},{"id":1380,"value":"plot_quantile = quantile_input[1]","pinned":false,"hidden":false,"mode":"ojs"},{"id":940,"value":"data = [15.77, 20.5, 8.26, 14.37, 21.0];","pinned":false,"hidden":false,"mode":"ojs"},{"id":777,"value":"d = \n{\n const datasel = d3.range(datasettings[0]).map(i => data[i]);\n const n = datasettings[0]\n const sigma = datasettings[1]\n const mu0 = priorsettings[0]\n const tau0 = priorsettings[1]\n const xbar = d3.mean(datasel)\n const taun = Math.sqrt(1/( n/sigma**2 + 1/tau0**2 ));\n const w = (n/sigma**2)/( n/sigma**2 + 1/tau0**2 )\n const mun = w*xbar + (1-w)*mu0\n return {datasel: datasel, n:n, xbar: xbar, sigma: sigma, mu0: mu0, tau0: tau0, taun: taun, w: w, mun: mun}\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":17,"value":"densdata = {\n const thetas = d3.range(0, 50, 0.1);\n const priorpdf = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.normal.pdf(theta, d.mu0, d.tau0),\n type: \"prior\"\n }\n ));\n const likepdf = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.normal.pdf(theta, d.xbar, d.sigma/Math.sqrt(d.n)),\n type: \"likelihood\"\n }\n ));\n const postpdf = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.normal.pdf(theta, d.mun, d.taun),\n type: \"posterior\"\n }\n ));\n return priorpdf.concat(likepdf).concat(postpdf)\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":1066,"value":"seqdensdata = {\n const thetas = d3.range(0, 50, 0.1);\n var pdfs = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.normal.pdf(theta, d.mu0, d.tau0),\n nobs: \"n=0\"\n }));\n const sigma = datasettings[1]\n const mu0 = priorsettings[0]\n const tau0 = priorsettings[1]\n for (let i = 1; i<= 5; i++){\n const datasel = d3.range(i).map(i => data[i]);\n const n = i\n const xbar = d3.mean(datasel)\n const taun = Math.sqrt(1/( n/sigma**2 + 1/tau0**2 ));\n const w = (n/sigma**2)/( n/sigma**2 + 1/tau0**2 )\n const mun = w*xbar + (1-w)*mu0\n \n pdfs = pdfs.concat(\n thetas.map(theta => ({theta: theta, pdf: jstat.normal.pdf(theta, mun, taun), nobs: \"n=\"+i}))\n )\n }\n return pdfs\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":972,"value":"preddata = {\n const xtilde = d3.range(0, 50, 0.1);\n const predpdf = xtilde.map(xtilde => (\n {\n xtilde: xtilde, \n pdf: jstat.normal.pdf(xtilde, d.mun, Math.sqrt(d.taun**2 + d.sigma**2))\n }\n ));\n return predpdf\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":19,"value":"priorcdf = jstat.normal.cdf(post_quantile, d.mu0, d.tau0);","pinned":false,"hidden":false,"mode":"ojs"},{"id":793,"value":"postcdf = jstat.normal.cdf(post_quantile, d.mun, d.taun);","pinned":false,"hidden":false,"mode":"ojs"},{"id":1016,"value":"predcdf = jstat.normal.cdf(pred_quantile, d.mun, Math.sqrt(d.taun**2 + d.sigma**2));","pinned":false,"hidden":false,"mode":"ojs"},{"id":621,"value":"maxpdf = Math.max(jstat.normal.pdf(d.mu0, d.mu0, d.tau0), jstat.normal.pdf(d.xbar, d.xbar, d.sigma/Math.sqrt(d.n)), jstat.normal.pdf(d.mun, d.mun, d.taun))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1024,"value":"maxpredpdf = jstat.normal.pdf(d.mun, d.mun, Math.sqrt(d.taun**2 + d.sigma**2))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1308,"value":"datastring = {\n let text = \"\";\n for (let i = 0; i<= datasettings[0]-1;i++){\n if (i < datasettings[0]-1){\n text = text.concat(data[i]+\",\")\n }else{\n text = text.concat(data[i])\n }\n }\n return text;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":229,"value":"#### Style","pinned":false,"hidden":false,"mode":"md"},{"id":202,"value":"","pinned":false,"hidden":false,"mode":"html"},{"id":58,"value":"viewof md_colour = Inputs.color({label: \"Highlight color cells\", value: \"#F5F5DC\"})","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W32","stdlib":"1","titling":"auto"}} 2f:"$2e:body"