2a:null 2b:["$","title",null,{"children":"Bayesian inference for Bernoulli iid data | Mattias Villani | Observable"}] 31:T415,titletext1 = md`### Posterior The **likelihood** from the Bernoulli model is ${tex` p(x_1,\ldots,x_n \vert \theta) = \prod_{i=1}^n \theta^{x_i} (1-\theta)^{1-x_i} = \theta^{s} (1-\theta)^{f}, `}
where ${tex`s = \sum_{i=1}^n x_i`} is the number of successes, ${tex`f=n-s`} is the number of failures.
The **prior** distribution is the ${tex`\mathrm{Beta}(\alpha,\beta)`} density ${tex` p(\theta) = \frac{1}{\mathrm{B}(\alpha,\beta)} \theta^{\alpha-1} (1-\theta)^{\beta-1} \propto \theta^{\alpha-1} (1-\theta)^{\beta-1}. `}
By Bayes' theorem we therefore have the **posterior**
${tex` \begin{aligned} p(\theta \vert x_1,\ldots,x_n) &\propto p(x_1,\ldots,x_n \vert \theta)p(\theta) \\ &\propto \theta^{s} (1-\theta)^{f}\theta^{\alpha-1} (1-\theta)^{\beta-1} \\ &= \theta^{\alpha + s - 1} (1-\theta)^{\beta + f -1} \end{aligned} `}
which is proportional to the ${tex`\mathrm{Beta}(\alpha+s,\beta+f)`} density.
The prior to posterior updating is therefore: `32:[] 2e:{"id":"35d7cf4c080b7826","slug":"bayesian-inference-for-bernoulli-iid-data","title":"Bayesian inference for Bernoulli iid data","thumbnail":"54e775ad9faf60b305b030660bfa28e4500b085c7e349778e877dada05a9fc0a","access_level":"public","create_time":"$D2023-11-18T12:24:56.906Z","update_time":"$D2025-02-10T19:50:38.961Z","view_time":null,"first_public_time":"$D2023-11-18T13:04:11.274Z","fork":{"id":"dde99ac1c5a54388","slug":"bayesian-inference-for-exponential-iid-data","title":"Bayesian inference for Exponential iid data","owner":{"id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani"}},"stars":0,"starred":false,"owner":{"type":"team","id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":"pro_2024"},"version":1642,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"56499be8cf43f0ef","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":null}],"members":[{"type":"individual","id":"56499be8cf43f0ef","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":0,"collection":{"id":"636adfa34cc7a030","slug":"sda3","title":"SDA3","owner":{"id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani"}},"type":"notebook","body":{"title":"Bayesian inference for Bernoulli iid data","cells":[{"id":38,"value":"# Bayesian inference for Bernoulli iid data","pinned":false,"hidden":false,"mode":"md"},{"id":700,"value":"model = md`### Model\n\nAssume data from the iid [Bernoulli](https://observablehq.com/@mattiasvillani/bernoulli-distribution) model \n\n${tex`X_1,\\ldots,X_n \\mid \\theta \\overset{\\mathrm{iid}}{\\sim} \\operatorname{Bern}(\\theta)`}

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1345,"value":"prior = md`\n### Conjugate prior\n\nThe conjugate prior for ${tex`\\theta`} in the iid Bernoulli model is the [Beta distribution](https://observablehq.com/@mattiasvillani/beta-distribution)\n\n${tex`\\theta \\sim \\operatorname{Beta}(\\alpha, \\beta)`}.\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":719,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":1036,"value":"resultstext = html`

\n\n \n \n \n \n\n \n \n \n \n\n \n \n \n \n

Model:	${tex`X_1,\\ldots,X_n \\mid \\theta \\sim \\operatorname{Bern}(\\theta)`}.
Prior:	${tex`\\theta \\sim \\operatorname{Beta}(\\alpha,\\beta)`}
Posterior:	${tex`\\theta \\mid x_1,\\ldots,x_n \\sim \\operatorname{Beta}(\\alpha+s, \\beta+f)`}

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1493,"value":"The interactive widget below lets you experiment with different data, summarized by the number of successes ${tex`s`} in ${tex`n`} trials, and with different values for the prior hyperparameters ${tex`\\alpha`} and ${tex`\\beta`}.","pinned":false,"hidden":false,"mode":"md"},{"id":1265,"value":"datatext = md`**Data**`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1589,"value":"viewof n = Inputs.range([1, 100], {value: 5, step: 1, label: tex`n`})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1591,"value":"viewof s = Inputs.range([0, n], {value: 2, step: 1, label: tex`s`})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1268,"value":"**Prior**:","pinned":false,"hidden":false,"mode":"md"},{"id":1258,"value":"viewof priorsettings = Inputs.form([\n Inputs.range([0.5, 10], {value: 3, step: 0.1, label: tex`\\alpha`}),\n Inputs.range([0.5, 10], {value: 2, step: 0.1, label: tex`\\beta`})\n ])","pinned":false,"hidden":false,"mode":"ojs"},{"id":614,"value":"postresultstext = html`

\n\n \n \n \n \n \n \n \n \n \n \n \n \n

Prior mean:	${tex`\\mathbb{E}( \\theta) = ${priormean.toPrecision(3)}`}	Posterior mean:	${tex`\\mathbb{E}( \\theta \\vert \\boldsymbol{x}) = ${postmean.toPrecision(3)}`}
Prior st.dev:	${tex`\\mathbb{S}( \\theta) = ${Math.sqrt(priorvariance).toPrecision(3)}`}	Posterior st.dev:	${tex`\\mathbb{S}( \\theta \\vert \\boldsymbol{x}) = ${Math.sqrt(postvariance).toPrecision(3)}`}

\n`","pinned":false,"hidden":false,"mode":"ojs"},{"id":305,"value":"plt_pdf = Plot.plot({\n color: {\n legend: true,\n domain: [\"prior\",\"likelihood\",\"posterior\"],\n range: [\"#D6B656\", \"#6C8EBF\", \"#B85450\"]\n },\n x: {\n label: \"theta\",\n axis: true\n },\n y: {\n axis: false,\n domain: [0,1.02*maxpdf]\n },\n caption: \"Prior-to-Posterior mapping. The likelihood is normalized.\",\n marks: [\n Plot.ruleY([0]),\n Plot.line(densdata, {x: \"theta\", y: \"pdf\", stroke: \"type\", strokeWidth: 2.5}),\n //Plot.areaY(densdata, {filter: d => d.theta <= post_quantile && d.type == \"posterior\", x: \"theta\", y: \"pdf\", \n // fill: \"orange\", opacity: 0.2})\n ]\n })","pinned":false,"hidden":false,"mode":"ojs"},{"id":109,"value":"
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n\n---\n#### 🙈 Appendix 🙈\n\nHere is just some data, imports and function defintions that you probably don't want to read.","pinned":false,"hidden":false,"mode":"md"},{"id":221,"value":"#### Imports","pinned":false,"hidden":false,"mode":"md"},{"id":26,"value":"jstat = require('jstat')","pinned":false,"hidden":false,"mode":"ojs"},{"id":223,"value":"#### Data","pinned":false,"hidden":false,"mode":"md"},{"id":1447,"value":"alpha = priorsettings[0]","pinned":false,"hidden":false,"mode":"ojs"},{"id":1450,"value":"beta = priorsettings[1]","pinned":false,"hidden":false,"mode":"ojs"},{"id":777,"value":"d = \n{\n const alpha = priorsettings[0]\n const beta = priorsettings[1]\n return {n:n, s: s, alpha: alpha, beta: beta, alphapost: alpha + s, betapost: beta + n - s}\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":17,"value":"densdata = {\n const lowprob = 0.005\n const highprob = 1 - lowprob\n const xlimlow = d3.min([\n jstat.beta.inv(lowprob, d.alpha, d.beta), \n jstat.beta.inv(lowprob, d.s, d.n - d.s),\n jstat.beta.inv(lowprob, d.alpha + d.s, d.beta + d.n - d.s) \n ])\n const xlimhigh = d3.max([\n jstat.beta.inv(highprob, d.alpha, d.beta), \n jstat.beta.inv(highprob, d.s, d.n - d.s),\n jstat.beta.inv(highprob, d.alpha + d.s, d.beta + d.n - d.s) \n ])\n const thetas = d3.range(xlimlow, xlimhigh, (xlimhigh-xlimlow)/200 );\n const priorpdf = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.beta.pdf(theta, d.alpha , d.beta),\n type: \"prior\"\n }\n ));\n const likepdf = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.beta.pdf(theta, d.s, d.n - d.s),\n type: \"likelihood\"\n }\n ));\n const postpdf = thetas.map(theta => (\n {\n theta: theta, \n pdf: jstat.beta.pdf(theta, d.alpha + d.s, d.beta + d.n - d.s),\n type: \"posterior\"\n }\n ));\n return priorpdf.concat(likepdf).concat(postpdf)\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":621,"value":"maxpdf = d3.max(densdata.map(d => d.pdf))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1604,"value":"priormean = d.alpha/(d.alpha + d.beta)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1606,"value":"postmean = (d.alphapost)/(d.alphapost + d.betapost)","pinned":true,"hidden":false,"mode":"ojs"},{"id":1611,"value":"priorvariance = (d.alpha*d.beta)/((d.alpha + d.beta)**2 * (d.alpha + d.beta + 1))","pinned":false,"hidden":false,"mode":"ojs"},{"id":1614,"value":"postvariance = (d.alphapost*d.betapost)/((d.alphapost + d.betapost)**2 * (d.alphapost + d.betapost + 1))","pinned":false,"hidden":false,"mode":"ojs"},{"id":229,"value":"#### Style","pinned":false,"hidden":false,"mode":"md"},{"id":1526,"value":"colors = [\"#6C8EBF\", \"#D6B656\", \"#B85450\", \"#006d77\", \n \"#bf9d6c\", \"#55a12a\", \"#a6bbd8\",\"#e6d399\", \"#d49896\", \"#66a7ad\", \"#bf8d6c\", \"#407d24\"]","pinned":true,"hidden":false,"mode":"ojs"},{"id":202,"value":"","pinned":false,"hidden":false,"mode":"html"},{"id":58,"value":"viewof md_colour = Inputs.color({label: \"Highlight color cells\", value: \"#F0F0F0\"})","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W32","stdlib":"1","titling":"auto"}} 2f:"$2e:body"