3e:null 3f:["$","title",null,{"children":"Forecasting Election Results: An Interactive Tutorial | Nicolas Grislain | Observable"}] 45:T78d,md`# Forecasting Election Results: An Interactive Tutorial Recently, as I was following the 🇺🇸 US elections 🇺🇸, I got puzzled by the fact that *days after the elections happened some betting odds of Biden winning the elections were still below 90%*, while [FiveThirtyEight](https://fivethirtyeight.com/) published a probability of 90% *before* the election actually happened, when way less information was known. I know that many technical factors, like risk aversion, liquidity, platform fees or constraints to enter a betting platform can explain strange price pattern. Nevertheless this observation triggered a heated discussion with a friend working in finance about the [Silver vs Taleb controversy](https://towardsdatascience.com/nate-silver-v-nassim-nicholas-taleb-and-isaac-faber-888e56e2acac) and, what in my opinion is the deepest [argument](https://arxiv.org/pdf/1703.06351.pdf) of Taleb: *a proper forecast shall be a [martingale](https://en.wikipedia.org/wiki/Martingale_%28probability_theory%29)*. My observation was that Taleb's argument should apply even more than the polls are expected to give a [poor signal](https://www.wired.com/story/how-wrong-were-polls-election-2020-trump-biden/) of actual voting intentions. Namely, when there is still a lot of uncertainty to uncover the day of the election, any forecast prior to the election should be very prudent and closer to 50% chances. This is the case, for instance, when there is a large unpredictable bias in the way people answer the polls. *Any forecast swinging between 0 and 1 long before the election and also closer to the election if we assume a lot of uncertainty is to be uncovered, likely violates the martingale property*. I realized that this point was not so easly understood even among *data-litterate* people. So I decided to spend my Sunday of COVID lockdown making this point in a funnier and, hopefully, more compelling way. `46:T543,md`## What a martingale is, and why any prediction is always one In short, a martingale is a stochastic process, namely a sequence of random values: ${tex`M_t`}, verifying: ${tex.block`\forall tT`}. We are interested in determining the expected value ${tex`B_t = \mathbb{E}\left[B_U|\mathcal{F}_t\right]`} of candidate ${tex`C`} winning the election. Let ${tex`M_t`} be the *intended vote margin process*, namely the difference between voting intentions for ${tex`C`} and against ${tex`C`} at time ${tex`t\in [0,T]`}. We assume ${tex`M_t`} follows the process: ${tex.block`M_t = M_0 + \int_0^t \tau\mathbf{1}_{[0,T]}(s) dW_M(s)`} The *intended vote margin* ${tex`M_t`} is constant and equal to the final decision after election day when everyone has voted. We assume there is a discrepancy betweeen the intended votes and the declared votes. Let ${tex`D_t`} be the *declared vote margin process*, namely the difference between how many would declare they vote for ${tex`C`} and against ${tex`C`} at time ${tex`t\in [0,U]`}. We assume: ${tex.block`D_t = M_t + \int_t^U \rho\mathbf{1}_{[T,U]}(s) dW_D(s)`} Note that the discrepancy ${tex`D_t - M_t`} between declared and intended margin is constant, equal to ${tex`\rho \left(W_D(U)-W_D(T)\right)`}, until election day. Then, after election day, as the votes are counted, declared votes are gradually replaced by actual votes ${tex`\rho \left(W_D(U)-W_D(t)\right)`}, letting the uncertainty on the election result vanish. The result of the election is: ${tex.block`B_U = \mathbf{1}_{\mathbb{R}^+}\left(M_T\right) = \mathbf{1}_{\mathbb{R}^+}\left(D_U\right)`} `48:Tb48,{ // Main parameters const height = 400; const padding = {left:60, right:30, bottom:30, top:10}; let min_values = Math.min(d3.min(intended), d3.min(declared)); let max_values = Math.max(d3.max(intended), d3.max(declared)); let delta_values = max_values-min_values; min_values -= 0.1*delta_values; max_values += 0.1*delta_values; const svg = d3.create('svg') .attr('viewBox', [0, 0, width, height]) .attr('font-family', 'sans-serif') .attr('font-size', 12) .attr('stroke-miterlimit', 1) .attr('stroke-linejoin', 'round') .attr('stroke-linecap', 'round'); let xScale = d3.scaleLinear() .domain(d3.extent(time)) .range([padding.left, width - padding.right]); let yScale = d3.scaleLinear() .domain([min_values, max_values]) .range([height - padding.bottom, padding.top]); // Clip path let clipPath = svg.append('clipPath') .attr('id', 'chart-area') .append('rect') .attr('x', padding.left) .attr('y', padding.top) .attr('width', width - padding.left - padding.right) .attr('height', height - padding.top - padding.bottom); let line = d3.line() .x(d => xScale(d[0])) .y(d => yScale(d[1])); svg.append('path') .datum(d3.zip(time, intended, declared).slice(0,U)) .attr('fill', colors.secondary) .attr('fill-opacity', 0.05) .attr('d', d3.area() .x(d => xScale(d[0])) .y0(d => yScale(d[1])) .y1(d => yScale(d[2]))); svg.append('path') .datum(d3.zip(time, declared).slice(0,U)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.secondary) .attr('stroke-width', 0.5) .attr('d', line); svg.append('path') .datum(d3.zip(time, intended).slice(0,U)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.secondary) .attr('stroke-width', 2) .attr('d', line); svg.append('text') .attr('x', width - padding.right - 24) .attr('y', yScale(parameters.terminal_intended) + 24) .attr('text-anchor', 'end') .attr('fill', colors.secondary) .text(`C ${parameters.terminal_intended > 0?'won':'lost'} by ${Math.abs(parameters.terminal_intended)} votes`); let xAxis = d3.axisBottom(xScale); let yAxis = d3.axisLeft(yScale); svg.append('g') .attr('transform', `translate(0,${height-padding.bottom})`) .attr('transform', `translate(0,${height-padding.bottom})`) .call(xAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Time')); svg.append('g') .attr('transform', `translate(${padding.left},0)`) .call(yAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Votes')); return svg.node() }49:Tdbf,{ // Main parameters const height = 400; const padding = {left:60, right:30, bottom:30, top:10}; let min_values = Math.min(d3.min(intended), d3.min(declared)); let max_values = Math.max(d3.max(intended), d3.max(declared)); let delta_values = max_values-min_values; min_values -= 0.1*delta_values; max_values += 0.1*delta_values; const svg = d3.create('svg') .attr('viewBox', [0, 0, width, height]) .attr('font-family', 'sans-serif') .attr('font-size', 12) .attr('stroke-miterlimit', 1) .attr('stroke-linejoin', 'round') .attr('stroke-linecap', 'round'); let xScale = d3.scaleLinear() .domain(d3.extent(time)) .range([padding.left, width - padding.right]); let yScale = d3.scaleLinear() .domain([min_values, max_values]) .range([height - padding.bottom, padding.top]); // Clip path let clipPath = svg.append('clipPath') .attr('id', 'chart-area') .append('rect') .attr('x', padding.left) .attr('y', padding.top) .attr('width', width - padding.left - padding.right) .attr('height', height - padding.top - padding.bottom); let line = d3.line() .x(d => xScale(d[0])) .y(d => yScale(d[1])); svg.append('path') .datum(d3.zip(time, intended, declared).slice(0,U)) .attr('fill', colors.secondary) .attr('fill-opacity', 0.05) .attr('d', d3.area() .x(d => xScale(d[0])) .y0(d => yScale(d[1])) .y1(d => yScale(d[2]))); svg.append('path') .datum(d3.zip(time, declared).slice(0,parameters.T)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.secondary) .attr('stroke-width', 0.5) .attr('d', line); svg.append('path') .datum(d3.zip(time, declared).slice(parameters.T,U)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.main) .attr('stroke-width', 2) .attr('d', line); svg.append('path') .datum(d3.zip(time, intended).slice(0,U)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.secondary) .attr('stroke-width', 2) .attr('d', line); svg.append('g') .attr('fill', colors.main) .attr('stroke', 'none') .attr('stroke-width', 1) .attr('clip-path', 'url(#chart-area)') .selectAll('circle') .data(d3.zip(time, poll).filter((d,i) => is_poll[i]).slice(0,parameters.T)) .join('circle') .attr('cx', d => xScale(d[0])) .attr('cy', d => yScale(d[1])) .attr('r', 2); svg.append('text') .attr('x', width - padding.right - 24) .attr('y', yScale(parameters.terminal_intended) + 24) .attr('text-anchor', 'end') .attr('fill', colors.secondary) .text(`C ${parameters.terminal_intended > 0?'won':'lost'} by ${Math.abs(parameters.terminal_intended)} votes`); let xAxis = d3.axisBottom(xScale); let yAxis = d3.axisLeft(yScale); svg.append('g') .attr('transform', `translate(0,${height-padding.bottom})`) .attr('transform', `translate(0,${height-padding.bottom})`) .call(xAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Time')); svg.append('g') .attr('transform', `translate(${padding.left},0)`) .call(yAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Votes')); return svg.node() }4a:T4b4,md` ## Evolution of the distribution of belief We have a gaussian bayesian *belief* on ${tex`D_j`} for every time ${tex`j`} given all the available information up to time ${tex`i \leq j`}. We note ${tex`\mu_{j|i}`} its mean and ${tex`\sigma_{j|i}^2`} its variance. We note ${tex`\mu_i = \mu_{i|i}`} and ${tex`\sigma_i = \sigma_{i|i}`}. ${tex.block`\begin{cases} \mu_0 = 0 \\ \sigma_0^2 = \tau^2T + \rho^2\left(U-T\right) \end{cases}`} This belief is updated by observing the sequence of polls: ${tex`\pi_i = M_i + \eta_i\epsilon_{i}`}. Using the bayes rule with prior ${tex`\mu_{i|i-1}`}, ${tex`\sigma_{i|i-1}`}, we can estimate: ${tex.block`\forall 1\leq i\leq U \; \begin{cases} \mu_i = \mu_{i|i-1} + \frac{\sigma_{i|i-1}^2}{\sigma_{i|i-1}^2 + \eta_i^2}\left(\pi_i - \mu_{i|i-1}\right) \\ \sigma_i^2 = \frac{\sigma_{i|i-1}^2\eta_i^2}{\sigma_{i|i-1}^2 + \eta_i^2} \end{cases}`} ## Forecasting equations Given all the information today, one can forecast the future value of ${tex`D_i`}. ${tex.block`\forall i \leq j \begin{cases} \mu_{j|i} = \mu_i \\ \sigma_{j|i}^2 = \sigma_i^2 + \sum_{k=i+1}^j\left[\tau^2\mathbf{1}_{[1,T]}(k) + \rho^2\mathbf{1}_{[T+1,U]}(k)\right] \end{cases}`} `4b:Tec9,{ // Main parameters const height = 400; const padding = {left:60, right:30, bottom:30, top:10}; let min_values = Math.min(d3.min(intended), d3.min(declared)); let max_values = Math.max(d3.max(intended), d3.max(declared)); let delta_values = max_values-min_values; min_values -= 0.1*delta_values; max_values += 0.1*delta_values; const svg = d3.create('svg') .attr('viewBox', [0, 0, width, height]) .attr('font-family', 'sans-serif') .attr('font-size', 12) .attr('stroke-miterlimit', 1) .attr('stroke-linejoin', 'round') .attr('stroke-linecap', 'round'); let xScale = d3.scaleLinear() .domain(d3.extent(time)) .range([padding.left, width - padding.right]); let yScale = d3.scaleLinear() .domain([min_values, max_values]) .range([height - padding.bottom, padding.top]); // Clip path let clipPath = svg.append('clipPath') .attr('id', 'chart-area') .append('rect') .attr('x', padding.left) .attr('y', padding.top) .attr('width', width - padding.left - padding.right) .attr('height', height - padding.top - padding.bottom); let line = d3.line() .x(d => xScale(d[0])) .y(d => yScale(d[1])); svg.append('path') .datum(d3.zip(time, intended, declared).slice(0,U)) .attr('fill', colors.secondary) .attr('fill-opacity', 0.05) .attr('d', d3.area() .x(d => xScale(d[0])) .y0(d => yScale(d[1])) .y1(d => yScale(d[2]))); svg.append('path') .datum(d3.zip(time, declared).slice(0,parameters.T)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.secondary) .attr('stroke-width', 0.5) .attr('d', line); svg.append('path') .datum(d3.zip(time, belief.mu, belief.sigma_2)) .attr('fill', colors.main) .attr('fill-opacity', 0.1) .attr('d', d3.area() .x(d => xScale(d[0])) .y0(d => yScale(d[1]-2*Math.sqrt(d[2]))) .y1(d => yScale(d[1]+2*Math.sqrt(d[2])))); svg.append('path') .datum(d3.zip(time, belief.mu)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.main) .attr('stroke-width', 2) .attr('d', line); svg.append('path') .datum(d3.zip(time, intended).slice(0,U)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.secondary) .attr('stroke-width', 2) .attr('d', line); svg.append('g') .attr('fill', colors.main) .attr('stroke', 'none') .attr('stroke-width', 1) .attr('clip-path', 'url(#chart-area)') .selectAll('circle') .data(d3.zip(time, poll).filter((d,i) => is_poll[i]).slice(0,parameters.T)) .join('circle') .attr('cx', d => xScale(d[0])) .attr('cy', d => yScale(d[1])) .attr('r', 2); svg.append('text') .attr('x', width - padding.right - 24) .attr('y', yScale(parameters.terminal_intended) + 24) .attr('text-anchor', 'end') .attr('fill', colors.secondary) .text(`C ${parameters.terminal_intended > 0?'won':'lost'} by ${Math.abs(parameters.terminal_intended)} votes`); let xAxis = d3.axisBottom(xScale); let yAxis = d3.axisLeft(yScale); svg.append('g') .attr('transform', `translate(0,${height-padding.bottom})`) .attr('transform', `translate(0,${height-padding.bottom})`) .call(xAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Time')); svg.append('g') .attr('transform', `translate(${padding.left},0)`) .call(yAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Votes')); return svg.node() }4c:T83d,{ // Main parameters const height = 400; const padding = {left:60, right:30, bottom:30, top:10}; let min_values = 0; let max_values = 1; const svg = d3.create('svg') .attr('viewBox', [0, 0, width, height]) .attr('font-family', 'sans-serif') .attr('font-size', 12) .attr('stroke-miterlimit', 1) .attr('stroke-linejoin', 'round') .attr('stroke-linecap', 'round'); let xScale = d3.scaleLinear() .domain(d3.extent(time)) .range([padding.left, width - padding.right]); let yScale = d3.scaleLinear() .domain([min_values, max_values]) .range([height - padding.bottom, padding.top]); // Clip path let clipPath = svg.append('clipPath') .attr('id', 'chart-area') .append('rect') .attr('x', padding.left) .attr('y', padding.top) .attr('width', width - padding.left - padding.right) .attr('height', height - padding.top - padding.bottom); let line = d3.line() .x(d => xScale(d[0])) .y(d => yScale(d[1])); svg.append('path') .datum(d3.zip(time, forecast)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.main) .attr('stroke-width', 2) .attr('d', line); svg.append('path') .datum(d3.zip(time, nowcast).slice(0,parameters.T)) .attr('clip-path', 'url(#chart-area)') .attr('fill', 'none') .attr('stroke', colors.main) .attr('stroke-width', 0.5) .attr('d', line); let xAxis = d3.axisBottom(xScale); let yAxis = d3.axisLeft(yScale); svg.append('g') .attr('transform', `translate(0,${height-padding.bottom})`) .attr('transform', `translate(0,${height-padding.bottom})`) .call(xAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Time')); svg.append('g') .attr('transform', `translate(${padding.left},0)`) .call(yAxis) .call(g => g.select(".tick:last-of-type text").clone() .attr('y', -12) .attr('text-anchor', 'end') .text('Votes')); return svg.node() }4d:T62a,{ let inputs = inputsGroup( [[slider({min: 1, max: U-1, step: 10, value: 1000, description: "Time of election", format: v => `${v} steps`}), slider({min: -10000, max: 10000, step: 100, value: 0, description: "Terminal vote difference", format: v => `${v} votes`})], [slider({min: -10000, max: 10000, step: 100, value: 0, description: "Initial intended vote margin", format: v => `${v} votes`}), slider({min: 0, max: 1000, step: 10, value: 100, description: "Intended vote margin variation standard error", format: v => `${v} votes`})], [slider({min: -10000, max: 10000, step: 100, value: 1000, description: "Initial discrepancy between declared and intended", format: v => `${v} votes`}), slider({min: 0, max: 1000, step: 10, value: 100, description: "Discrepancies discovery standard error", format: v => `${v} votes`})]], [['T', 'terminal_intended'], ['initial_intended', 'tau'], ['initial_discrepancy', 'rho']], form => { form.views.forEach((row,i) => row.forEach((input,j) => viewof parameters.views[i][j].value = input.value)); // viewof parameters.views.forEach(row => row.forEach(input => input.dispatchEvent(new CustomEvent('input')))); viewof parameters.dispatchEvent(new CustomEvent('input')); }); // Listen to parameters const update = () => inputs.views.forEach((row,i) => row.forEach((input,j) => input.value = viewof parameters.views[i][j].value)); viewof parameters.addEventListener('input', update); invalidation.then(() => viewof parameters.removeEventListener('input', update)); return update(), inputs; }4e:[] 43:{"id":"537f7c6146a18eb6","slug":"forecasting-election-results-an-interactive-tutorial","title":"Forecasting Election Results: An Interactive Tutorial","thumbnail":"5f6f3eaf763f920737c4eadb9c18adf7674f60359db362c560c44eec3befc3ce","access_level":"public","create_time":"$D2020-11-09T17:18:04.952Z","update_time":"$D2020-11-18T16:27:11.370Z","view_time":null,"first_public_time":"$D2020-11-17T22:56:08.649Z","has_unpublished_edits":null,"fork":false,"stars":7,"starred":false,"owner":{"type":"team","id":"abf6c0508f02b218","login":"ngrislain","name":"Nicolas Grislain","avatar_url":"https://avatars-next.observableusercontent.com/d9d690de51e271b26a32af3f96c642029a5a0636a610083e0ffe1049ef087a89","tier":"starter_2024","bio":""},"version":1818,"publish_version":1818,"role":null,"authored":false,"authors":[{"type":"individual","id":"e67862f7a487e446","login":"ngrislain","name":"Nicolas Grislain","avatar_url":"https://avatars-next.observableusercontent.com/d9d690de51e271b26a32af3f96c642029a5a0636a610083e0ffe1049ef087a89","tier":null,"bio":""}],"members":[{"type":"individual","id":"e67862f7a487e446","login":"ngrislain","name":"Nicolas Grislain","avatar_url":"https://avatars-next.observableusercontent.com/d9d690de51e271b26a32af3f96c642029a5a0636a610083e0ffe1049ef087a89","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Forecasting Election Results: An Interactive Tutorial","cells":[{"id":0,"value":"$45","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":78,"value":"$46","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":95,"value":"$47","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1384,"value":"md`**Figure 1. Intended and declared vote margin**`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":227,"value":"$48","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1311,"value":"viewof U = slider({min: 2, max: 2000, step: 1, value: 1200, description: \"Time of results\", format: v => `${v} steps`});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":146,"value":"viewof parameters = inputsGroup(\n [[slider({min: 1, max: U-1, step: 1, value: 1000, description: \"Time of election\", format: v => `${v} steps`}),\n slider({min: -10000, max: 10000, step: 100, value: 0, description: \"Terminal vote difference\", format: v => `${v} votes`})],\n [slider({min: -10000, max: 10000, step: 100, value: 0, description: \"Initial intended vote margin\", format: v => `${v} votes`}),\n slider({min: 0, max: 1000, step: 10, value: 100, description: \"Intended vote margin variation standard error\", format: v => `${v} votes`})],\n [slider({min: -10000, max: 10000, step: 100, value: 1000, description: \"Initial discrepancy between declared and intended\", format: v => `${v} votes`}),\n slider({min: 0, max: 1000, step: 10, value: 100, description: \"Discrepancies discovery standard error\", format: v => `${v} votes`})]],\n [['T', 'terminal_intended'],\n ['initial_intended', 'tau'],\n ['initial_discrepancy', 'rho']]);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1449,"value":"md`*The thick line represents the intended vote margin. It is constant after election day as voters made their mind. The thin line represents the declared vote margin. The discrepancy between both values is 0 in expected value, any predictable bias hasd been taken into account*`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":311,"value":"md`\n## A simple observation model\n\nTo simplify the math, we will assume the time is discrete.\nDeclared votes are observed at every time step: ${tex`i \\in \\{1,\\ldots,T,T+1,\\ldots,U\\}`}.\n\nBefore election day ${tex`t\\leq T`}, they are observed imperfectly thanks to polls:\n${tex.block`\\pi_i = D_i + \\eta_i\\epsilon_{i}`}\nThese polls ${tex`\\pi_i`} are estimates of ${tex`D_i`}.\nWe assume ${tex`\\epsilon_i \\sim \\mathcal{N}\\left(0,1\\right)`} are i.i.d. gaussian errors.\nThe absence of poll at time ${tex`i`} can be simulated by having ${tex`\\eta_i\\to +\\infty`}.\n\nAfter election day ${tex`t> T`} the declared votes are observed perfectly as they are gradually replaced by actual votes and converge to the final result. Again it is equivalent to say ${tex`\\eta_i\\to 0`}.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1453,"value":"md`**Figure 2. Observation before and after the election**`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":342,"value":"$49","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":872,"value":"viewof poll_parameters = inputsGroup([\n [slider({min: 0, max: 10000, step: 100, value: 1000, description: \"Poll standard error\", format: v => `${v} votes`}), slider({min: 1, max: 100, step: 1, value: 10, description: \"Poll periodicity\", format: v => `${v} steps`})]],[['eta', 'poll_period']]);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1457,"value":"md`*Polls results are represented as red dots. They are a noisy version of the declared vote margin. After the election, we assume the declared votes are observed perfectly as they converge to the final result.*`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":184,"value":"$4a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":300,"value":"md`## Probability of victory ${tex`B_U = 1 \\Leftrightarrow D_U > 0`}\n\nSo at time step ${tex`i`}, with all the available information the expected value ${tex`B_i`} is given by:\n\n${tex.block`B_i = \\Pr\\left[B_U=1|\\mathcal{F}_i\\right] = \\mathbb{E}\\left[B_U|\\mathcal{F}_i\\right] = \\mathbb{E}\\left[B_U|\\mu_{U|i}, \\sigma_{U|i}\\right]`}\n\nThe victory derives from the counted vote margin:\n${tex.block`B_U = \\mathbf{1}_{\\mathbb{R}^+}\\left(D_U\\right)`}\n\nSo, knowing ${tex`\\mu_{U|i}`} and ${tex`\\sigma_{U|i}`}:\n${tex.block`B_i = \\mathbb{E}\\left[\\mathbf{1}_{\\mathbb{R}^+}\\left(D_U\\right)|\\mu_{U|i}, \\sigma_{U|i}\\right]`}\n\nTherefore:\n${tex.block`B_i = \\Phi\\left(\\frac{\\mu_{U|i}}{\\sigma_{U|i}}\\right)`}\nWhere ${tex`\\Phi`} is the CDF of a standard gaussian random variable.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1502,"value":"md`## Full solution\n\nThe price dynamics can be written:\n${tex.block`\\forall \\; 1\\leq i\\leq T\\begin{cases}\n \\kappa_i = 0 & \\text{if there is no poll}\\\\\n \\kappa_i = \\frac{\\sigma_{i-1}^2+\\tau^2}{\\sigma_{i-1}^2+\\tau^2 + \\eta^2} & \\text{else}\\\\\n \\mu_i = \\mu_{i-1} + \\kappa_i\\left(\\pi_i - \\mu_{i-1}\\right) \\\\\n \\sigma_i^2 = \\left(\\sigma_{i-1}^2+\\tau^2\\right)\\left(1-\\kappa_i\\right) \\\\\n B_i = \\Phi\\left(\\frac{\\mu_i}{\\sqrt{\\sigma_i^2 + (T-i)\\tau^2} + (U-T)\\rho^2}\\right)\n\\end{cases}`}\n\n${tex.block`\\forall \\; T+1\\leq i\\leq U\\begin{cases}\n \\mu_i = D_i \\\\\n \\sigma_i^2 = 0 \\\\\n B_i = \\Phi\\left(\\frac{\\mu_i}{\\sqrt{\\sigma_i^2 + (U-i)\\rho^2}}\\right)\n\\end{cases}`}\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1798,"value":"md`\n# Conclusion\n\nBy playing with this interactive notebook, one can get some intuition of what kind of constraints, being a martingale imposes.\n\nYou can, for example experience that with a large uncertainty on the discrepancy between the intended vote and declared vote (*discrepancies discovery standard error*) the forecast is close to 50% even when the election is close.\n\nConversely, when the uncertainty is mainly uncovering before the election (large *intended vote margin variation standard error*) the prudence in forecasting should be large when the election is far ahead (a lot can happen in between) and much smaller close to the election day (forecast and nowcast get closer).\n\n❤️ If you like this, please share it! ❤️\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1568,"value":"md`**Figure 3. Beliefs before the elections**`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1549,"value":"$4b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1804,"value":"md`*By just observing the polls (red dots) one can make a prediction of the declared vote margin (red line).\nNote that between 2 polls no new information is available so __no change happens__* to the forecast, otherwise it would violate the martingale property.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1616,"value":"md`**Figure 4. Forecast vs Nowcast**`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1575,"value":"$4c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1809,"value":"md`*The forecast of the election result (thick red line) is closer to 50% than the forecast of the result of a fictitious election hapening the day of the forecast based on what is known of the declared intentions (the nowcast, the thin red line)*.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1643,"value":"$4d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1761,"value":"{\n let inputs = inputsGroup([[\n slider({min: 0, max: 10000, step: 100, value: 1000, description: \"Poll standard error\", format: v => `${v} votes`}), slider({min: 1, max: 100, step: 1, value: 10, description: \"Poll periodicity\", format: v => `${v} steps`})]],[['eta', 'poll_period']],\n form => {\n form.views.forEach((row,i) => row.forEach((input,j) => viewof poll_parameters.views[i][j].value = input.value));\n viewof poll_parameters.dispatchEvent(new CustomEvent('input'));\n });\n // Listen to parameters\n const update = () => inputs.views.forEach((row,i) => row.forEach((input,j) => input.value = viewof poll_parameters.views[i][j].value));\n viewof poll_parameters.addEventListener('input', update);\n invalidation.then(() => viewof poll_parameters.removeEventListener('input', update));\n return update(), inputs;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":125,"value":"md`# Parameters`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":133,"value":"// Time step is 1\ntime = d3.range(U);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":207,"value":"// Random number generator\nnormal = d3.randomNormal();","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1190,"value":"epsilon = d3.range(2*U).map(normal);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1220,"value":"epsilon_adjustment_pre = (parameters.terminal_intended - parameters.initial_intended - parameters.tau*d3.sum(epsilon.slice(0,parameters.T)))/(parameters.tau*parameters.T);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1261,"value":"epsilon_adjustment_post = (0 - parameters.initial_discrepancy - parameters.rho*d3.sum(epsilon.slice(parameters.T,U)))/(parameters.rho*(U-parameters.T));","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1219,"value":"adjusted_epsilon = epsilon.map((d,i) => i i==0 ? parameters.initial_intended + parameters.initial_discrepancy + parameters.tau*adjusted_epsilon[i] : i i declared[i] + poll_parameters.eta*adjusted_epsilon[U+i]);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":897,"value":"is_poll = d3.range(parameters.T).map(d => d%poll_parameters.poll_period==0);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1553,"value":"belief = {\n let mu = [0];\n let sigma_2 = [parameters.tau*parameters.tau*parameters.T + parameters.rho*parameters.rho*(U-parameters.T)];\n let kappa = [];\n for (let i=0; i jStat.normal.cdf(d[0]/Math.sqrt(d[1] + parameters.tau*parameters.tau*Math.max(0,parameters.T-1-i) + parameters.rho*parameters.rho*Math.min(U-parameters.T,U-1-i)),0,1));","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1603,"value":"nowcast = d3.zip(belief.mu, belief.sigma_2).map((d,i) => jStat.normal.cdf(d[0]/Math.sqrt(d[1]),0,1));","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":140,"value":"viewof colors = inputsGroup([[\n color({value: d3.rgb(250,100,50).formatHex(),title: \"Main color\"}),\n color({value: d3.rgb(0,100,200).formatHex(),title: \"Secondary color\"}),\n color({value: d3.rgb(90,90,100).formatHex(),title: \"Neutral color\"})]],\n [['main','secondary','neutral']]);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1710,"value":"function inputsGroup(views,names,oninput=f=>null){\n const form = html`

${\n views.map(row => html`

${\n row.map(input => html`

${input}

`)\n }

`;\n form.views = views;\n form.oninput = () => {\n form.value = views.map(row => row.map(input => input.value));\n if(names){\n names.forEach((row,i)=>row.forEach((c,j)=> form.value[i][j]!=null && (form.value[c]=form.value[i][j])))\n }\n oninput(form);\n };\n form.oninput();\n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":130,"value":"import {slider, color} from '@jashkenas/inputs'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1578,"value":"jStat = require('jStat');","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":128,"value":"d3 = require('d3');","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W4e","stdlib":"1"}} 44:"$43:body"