2c:null 2d:["$","title",null,{"children":"Maximum likelihood - Poisson regression | Mattias Villani | Observable"}] 32:T424,Each row in the dataset represents an auction with the following data: - **`BookVal`** is the coin's value according to a coin catalog - **`MinorBlem`** is 1 if the coin has a minor defect, otherwise 0 - **`MajorBlem`** is 1 if the coin has a major defect, otherwise 0 - **`PowerSeller`** is 1 if the seller is a power seller with large volumes, otherwise 0 - **`IDSeller`** is 1 if the seller is formally identified on eBay, otherwise 0 - **`Sealed`** is 1 if the coin's packing is sealed, otherwise 0 - **`NegFeedback`** is 1 if the seller has a large proportion of negative reviews from buyers, otherwise 0 - **`ReservePriceFrac`** is the seller's reserve price (the lowest accepted price) as a fraction of BookVal. - **`NBidders`** is the number of bidders in the auction. - **`FinalPrice`** is the final price that the winner pays according to eBay's so-called proxy system: - NA (not available) if no bids were placed - the reserve price if the auction only had one bidder - the second-highest bid if two or more bidders participated in the auction33:T5dc,Since `NBidders` is a count variable, a natural model to try is a Poisson model ${tex.block`X_1, X_2, \ldots, X_n \sim \text{Pois}(\lambda)`} independently, where ${tex`X_i`} is the number of bidders in auction *i*. The interactive graph to the left below shows how well a Poisson model with a certain ${tex`\lambda`} fits the observed data in the selected auctions with low reserve price (blue bars). The height of the bars are *proportions* of observations falling into that bin, so that they can be compared to the Poisson model's probabilities given by the dots on the orange line. A well fitting Poisson model should therefore have an orange line that is a close fit to the blue bars. With the initial choice of ${tex`\lambda=2`}, we see that the model fits the data poorly. For example, the model gives a probability for zero bidders of just over 0.13, but auctions without bidders were observed in less than 1% of the auctions. So the model with ${tex`\lambda=2`} grossly overestimates the probability of zero bidders. Change ${tex`\lambda`} until the model fits the data well. Click on *show ML fitting* to add an optimal model fit according to the maximum likelihood (ML) method. This method chooses the ${tex`\lambda`} that maximizes the average probability (likelihood) of the observed data. For the Poisson distribution, this optimal estimator of ${tex`\lambda`} is actually the mean ${tex`\bar X`} of the data. We prove this mathematically in an appendix at the end of the notebook.34:T42f,In fact, you could say that we have performed a simple variant of **Poisson regression** by dividing the data material into two parts: auctions with a low reserve price and auctions with a high reserve price. In what way is this a regression model? Well, regression is about modeling **conditional** expected values, which is exactly what we have done here: we model the number of bidders as being Poisson distributed with different expected values for auctions with a low versus high reserve price: ${tex`E(\text{number of bidders} \mid \text{low reservation price}) = \lambda_{\mathrm{low}}=${mle.toPrecision(4)}`}
${tex`E(\text{number of bidders} \mid \text{high reservation price}) = \lambda_{\mathrm{high}}=${mle_high.toPrecision(4)}`} It is therefore a rather special regression where the regression line is horizontal for ReservePriceFrac **lower** than the threshold value we have chosen, and then jumps down to a new level when ReservePriceFrac is **higher** than the threshold value:35:T8d4,text_poissonregression = md` So, dividing the auctions into low versus high reserve prices can be written as a regression model with a Poisson-distributed response variable. A natural extension is then to move away from the somewhat contrived hard division into low versus high reserve prices and instead let the expected value ${tex`\lambda`} be a smooth function of the variable ReservePriceFrac. That is, using ReservePriceFrac directly as an explanatory variable in a *Poisson regression*. We will then get a Poisson model for *every value* of the explanatory variable, i.e. the ${tex`\lambda`}-parameter will have different values for different observations, depending on ReservePriceFrac. Let ${tex`Y_i`} be the number of bidders in auction *i*, and let ${tex`x_i`} be the variable ReservePriceFrac in the same auction, which we usually assume to be non-random in regression contexts. The **Poisson regression** model is then: ${tex.block`Y_i \vert x_i \sim \mathrm{Pois}(\lambda_i)`} where ${tex`\lambda_i`} for the *i*:th observation is given by the function ${tex.block`\lambda_i = \exp(\beta_0 + \beta_1\cdot x_i)`} We use the exponential function ${tex`\exp(x)`} since it always returns positive values, which is precisely what is required in a Poisson model because ${tex`\lambda`} must be positive. We just can't model the expected value by ${tex`\lambda_i = \beta_0 + \beta_1\cdot x_i`} as in the usual linear regression since we then risk getting negative ${tex`\lambda_i`}!

Note that ${tex`\lambda_i`} varies between auctions because ${tex`x_i`} is different in different auctions. For example, suppose ${tex`\beta_0 = 2`} and ${tex`\beta_1 = -3`}. For the first auction in the dataset that has ReservePriceFrac = 0.369, we then get that ${tex.block`\lambda_1 = \exp(\beta_0 + \beta_1\cdot x_1) = \exp(2-3\cdot 0.369) = 2.442`} and therefore the number of bidders in that auction is ${tex`Y_1 \sim \mathrm{Pois}(\lambda_1 = 2.442)`}.
For the second auction in the dataset that has ReservePriceFrac = 0.230, we instead get ${tex.block`\lambda_2 = \exp(\beta_0 + \beta_1\cdot x_2) = \exp(2-3\cdot 0.230) = 3.706`} and therefore the number of bidders in that auction is ${tex`Y_2 \sim \mathrm{Pois}(\lambda_2 = 3.706)`}.
`36:T71a,textslh_reg = { if (!view_ml_reg){ if (beta[1]>0){ return html`Average probability for the observed data with the model Y | x ~ Pois(λ = exp(${beta[0]} + ${beta[1]}x)) is ${geomean_like_reg(regdata, beta).toPrecision(4)}`} else{ return html`Average probability for the observed data with the model Y | x ~ Pois(λ = exp(${beta[0]} ${beta[1]}x)) is ${geomean_like_reg(regdata, beta).toPrecision(4)}`} } else{ if (ml_beta[1]>0){ return html`Average probability for the observed data with the model Y | x ~ Pois(λ = exp(${beta[0]} + ${beta[1]}x)) is ${geomean_like_reg(regdata, beta).toPrecision(4)}
Average probability for the observed data with the ML estimate Y | x ~ Pois(λ = exp(${ml_beta[0].toPrecision(4)} + ${ml_beta[1].toPrecision(4)}x)) is ${geomean_like_reg(regdata, ml_beta).toPrecision(4)}`} else{ return html`Average probability for the observed data with the model Y | x ~ Pois(λ = exp(${beta[0]} ${beta[1]}x)) is ${geomean_like_reg(regdata, beta).toPrecision(4)}
Average probability for the observed data with the ML estimate Y | x ~ Pois(λ = exp(${ml_beta[0].toPrecision(4)} ${ml_beta[1].toPrecision(4)}x)) is ${geomean_like_reg(regdata, ml_beta).toPrecision(4)}`} } }37:Td3d,// Simple function to generate a contour plot using d3.contours. // Largley lifted from https://observablehq.com/@d3/contours. // Copied from @mcmcclur/contour-plots to be able to set xsteps and ysteps function contour_plot(f, [[a, b], [c, d]], thresholds, opts = {}) { let { xsteps = 50, ysteps = 50, w = width < 360 ? width : 360, h = w, pad = 30 } = opts; let dx = (b - a) / xsteps; let dy = (d - c) / ysteps; let grid = d3.range((xsteps + 1) * (ysteps + 1)).map(function (k) { let i = k % (xsteps + 1); let x = a + i * dx; let j = Math.floor(k / (xsteps + 1)); let y = d - j * dy; return [x, y]; }); grid.m = xsteps; grid.n = ysteps; grid.a = a; grid.b = b; grid.c = c; grid.d = d; let x_scale = d3 .scaleLinear() .domain([grid.a, grid.b]) .range([pad, w - pad]); let y_scale = d3 .scaleLinear() .domain([grid.c, grid.d]) .range([h - pad, pad]); let extent = d3.extent(thresholds); let zmin = extent[0]; let zmax = extent[1]; let color = d3 .scaleLinear() .domain([zmax, zmin]) // .interpolate((d) => d3.interpolateBlues); .interpolate((d) => (t) => d3.interpolateBlues(1 - t)); // .interpolate((d) => (t) => "white"); let contours = d3 .contours() .size([grid.m + 1, grid.n + 1]) .thresholds(thresholds)(grid.map(f)) .map(transform); const svg = d3 .create("svg") .attr("class", "contour_plot") .attr("width", w) .attr("height", h); svg .append("g") .attr("stroke", "#000") .attr("stroke-opacity", 0.8) .selectAll("path") .data(contours) .join("path") .attr("fill", (d) => color(d.value)) .attr("d", d3.geoPath()) .append("title") .text((d) => Math.abs(d.value) < 10 ** -5 ? 0 : d3.format("0.4")(d.value) ); svg .append("g") .attr("transform", `translate(0, ${h - pad})`) .call(d3.axisBottom(x_scale).ticks(4)); svg .append("g") .attr("transform", `translate(${pad})`) .call(d3.axisLeft(y_scale).ticks(4)); svg.append("text") .attr("class", "x label") .attr("text-anchor", "middle") .attr("x", w/2) .attr("y", h) .text("b0"); svg.append("text") .attr("class", "y label") .attr("text-anchor", "middle") .attr("x", pad/2) .attr("y", h/2) .text("b1"); svg .append('circle') .attr('cx', pad + ((beta[0]-a)/(b-a))*(w-2*pad)) .attr('cy', pad + (1-((beta[1]-c)/(d-c)))*(h-2*pad)) .attr('r', 3) .attr('stroke', 'orange') .attr('fill', 'orange'); if (view_ml_reg){ svg .append('circle') .attr('cx', pad + ((ml_beta[0]-a)/(b-a))*(w-2*pad)) .attr('cy', pad + (1-((ml_beta[1]-c)/(d-c)))*(h-2*pad)) .attr('r', 3) .attr('stroke', '#5F9EA0') .attr('fill', '#5F9EA0'); } return svg.node(); // Transform the contours by mapping from grid coordinates (indices) // to svg coordinates (px). function transform({ type, value, coordinates }) { let i_scale = d3 .scaleLinear() .domain([0, grid.m + 1]) .range([pad, w - pad]); let j_scale = d3 .scaleLinear() .domain([0, grid.n + 1]) .range([pad, h - pad]); return { type, value, coordinates: coordinates.map((rings) => { return rings.map((points) => { return points.map(([i, j]) => [i_scale(i), j_scale(j)]); }); }) }; } }38:[{"id":"47a98ee79de7b4b3716e827299fc52ebaed0d3bc6bc47841a6eb324fe4c9c19eda2424b7e2b38ecdc5e4c0c332a1112f9df83e2b55f4be3c95dd11f8b5c929e9","href":"https://static.observableusercontent.com/files/47a98ee79de7b4b3716e827299fc52ebaed0d3bc6bc47841a6eb324fe4c9c19eda2424b7e2b38ecdc5e4c0c332a1112f9df83e2b55f4be3c95dd11f8b5c929e9","name":"ebaydata@2.csv","type":"text/csv","lastModified":1699088108721.717,"size":47654}] 30:{"id":"630c0c263bea1ca0","slug":"maximum-likelihood-poisson-regression","title":"Maximum likelihood - Poisson regression","thumbnail":"d6ee2d482ce46c2bc9c616b2cfab72a8dddc1a7a940a367b2ad7e0541894c1d9","access_level":"public","create_time":"$D2023-11-04T08:55:08.721Z","update_time":"$D2025-05-18T11:18:22.642Z","view_time":null,"first_public_time":"$D2023-11-04T08:55:08.721Z","has_unpublished_edits":null,"fork":{"id":"6ad25aff8c6dc593","slug":"maximum-likelihood-poissonmodellen","title":"Maximum likelihood - Poissonregression","owner":{"id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani"}},"stars":0,"starred":false,"owner":{"type":"team","id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":"education_2026"},"version":2278,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"56499be8cf43f0ef","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":null}],"members":[{"type":"individual","id":"56499be8cf43f0ef","login":"mattiasvillani","name":"Mattias Villani","avatar_url":"https://avatars-next.observableusercontent.com/a440aaf39a3abefb86cbcc758fd7a5ab42555ae68d6825b52ec006f385a94d1d","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":5,"collection":{"id":"3247547e8812c274","slug":"statistics","title":"Basic Statistics","owner":{"id":"7592a7ab121f74a0","login":"mattiasvillani","name":"Mattias Villani"}},"type":"notebook","body":{"titling":"auto","title":"Maximum likelihood - Poisson regression","cells":[{"id":38,"value":"# Maximum likelihood - Poisson regression","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2260,"value":"The notebook [Maximum likelihood for iid Poisson data](maximum-likelihood-for-iid-poisson-data) analyzes the number of bidders in eBay coin auctions under the assumption that the number of bidders in each auction are independent and identically distributed (iid) Poisson distributed; that is all auctions are assumed to have the same mean number of bidders, ${tex`\\lambda`}. In order for that assumption to reasonable, we there had to restrict our analysis to the 550 auctions with lowest so called *reserve prices*.\n\nWe will here be more ambitious and find the maximum likelihood estimate in a Poisson *regression* model for the number of bidders using all 1000 auctions using the reserve prices as covariates.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":771,"value":"### Data from coin auctions on ebay\nThe whole eBay coin auction dataset is ^{[[1](#footnote1)]} is provided in the scrollable table below.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2069,"value":"ebaydata","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1832,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":778,"value":"### Separate Poisson models for auctions with different reserve prices\n\nWe will focus here on analyzing the variable `NBidders`, which is the number of unique bidders in each auction. Auctions sell coins of highly varying values and with varying reserve prices, which is the lowest price the seller is willing to accept for the coin. To obtain more homogeneous observations, we initially limit ourselves to auctions with the lowest reserve price in relation to the coin's book value. We analyze auctions where the `ReservePriceFrac` variable is less than 0.5, but you can change this threshold value with a slider below.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1158,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1175,"value":"As you can see by interacting with the graph, each value of ${tex`\\lambda`} provides an *average probability* for the data. We use a [geometric mean](https://en.wikipedia.org/wiki/Geometric_mean), which is more suitable for an average of probabilities. In the figure to the right, we plot all these values as a function of ${tex`\\lambda`}. The maximum likelihood estimate of ${tex`\\lambda`} is the value that maximizes this function. The orange dot in the graph indicates the value of ${tex`\\lambda`} that you have chosen with the slider. When it is at the peak of the function to the right, you have found the maximum likelihood estimate!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2145,"value":"modell = tex`\\text{Model for auctions with low reserve price: }X_1,X_2,\\ldots,X_{${y.length}} \\overset{\\mathrm{indep}}{\\sim}\\mathrm{Pois}(\\lambda)`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1913,"value":"viewof reserve_threshold = \n Inputs.range([\n d3.min(ebaydata.map(d => d.ReservePriceFrac)), d3.max(ebaydata.map(d => d.ReservePriceFrac))-0.001], \n {value: 0.5, label: \"threshold for ReservePriceFrac\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":118,"value":"viewof settings = Inputs.form([\n Inputs.range([0.1, 10], {value: 2, step: 0.1, label: tex`\\lambda:`}),\n Inputs.toggle({label: \"show ML fit\", value: false})\n ])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1121,"value":"textslh = {\n if (!settings[1]){\n return html`Average probability for observed data with the model \n Pois(λ = ${lambda}) is ${geomean_like(y,lambda).toPrecision(4)}`\n }\n else{\n return html`Average probability for observed data with the model \n Pois(λ = ${lambda}) is ${geomean_like(y,lambda).toPrecision(4)}
\n Average probability for observed data with the ML estimate \n Pois(λ = ${mle.toPrecision(4)}) is ${geomean_like(y,mle).toPrecision(4)}\n`\n }\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1218,"value":"bothplots = html`\n \n \n \n

\n ${pltdata}\n

\n ${pltlikelihood}\n

\n \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1891,"value":"### Poisson regression\n\nJust as we were able to calculate the ML estimate of ${tex`\\lambda`} for auctions with a low reserve price, we can also calculate the ML estimate of ${tex`\\lambda`} for auctions with a high reserve price.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1896,"value":"text_two_lambda = {\n return html`ML estimate of λ for ${y.length} auctions with low reservation price is λ = ${mle.toPrecision(4)} with an average probability ${geomean_like(y,mle).toPrecision(4)}
\n\n ML estimate of λ for ${y_high.length} auctions with high reservation price is λ = ${mle_high.toPrecision(4)} with an average probability ${geomean_like(y_high,mle_high).toPrecision(4)}\n`\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1909,"value":"Inputs.bind(\n Inputs.range([d3.min(ebaydata.map(d => d.ReservePriceFrac)), d3.max(ebaydata.map(d => d.ReservePriceFrac))], \n {value: 0.5, label: \"threshold for ReservePriceFrac\"})\n , viewof reserve_threshold)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1944,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1935,"value":"plt_scatter_two = Plot.plot({\n color: {\n legend: true,\n type: \"categorical\",\n },\n y: {label: \"NBidders\"},\n marks: [\n Plot.dot(ebaydata_reg, {x: \"ReservePriceFrac\", y: \"NBidders\", fill: \"lightgray\", r: 2 }),\n Plot.line(lambda_ml_grid_two, {x: \"ReservePriceFrac\", y: \"lambda\", stroke: \"reserveprice\", strokeWidth: 3}),\n Plot.ruleX([0.0]),\n Plot.ruleY([0.0])\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1978,"value":"The connection to regression becomes even more clear if we introduce a dummy variable `lowReservePrice` that is 1 for auctions where `ReservePriceFrac` is lower than the threshold and 0 for auctions where `ReservePriceFrac` is higher than the threshold. We can then write the expected value in the Poisson model as a function of `lowReservePrice`:\n\n${tex`E(\\textsf{NBidders} \\mid \\textsf{lowReservePrice}) = \\beta_0 + \\beta_1\\cdot \\textsf{lowReservePrice}`}\n\nwhere\n\n- ${tex`\\beta_0`} is the expected value ${tex`\\lambda_{\\mathrm{high}}`} for auctions with `lowReservePrice`=0\n- ${tex`\\beta_0 + \\beta_1`} is the expected value ${tex`\\lambda_{\\mathrm{low}}`} for auctions with `lowReservePrice`=1\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1404,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1455,"value":"text_poissonregression2 = md`OK, but how do we know which values for ${tex`\\beta_0`} and ${tex`\\beta_1`} fit the data? That is, how can we **estimate** ${tex`\\beta_0`} and ${tex`\\beta_1`} from our data material with 1000 auctions? Here we see the beauty and power of the maximum likelihood method: it it can always be used regardless of the model. We simply choose ${tex`\\beta_0`} and ${tex`\\beta_1`} so that the likelihood of observing NBidders in the 1000 auctions is as high as possible. There is no known formula for computing the maximum likelihood estimator of ${tex`\\beta_0`} and ${tex`\\beta_1`} in Poisson regression 😢. But we can let the computer do the work and calculate the maximum likelihood estimate of ${tex`\\beta_0`} and ${tex`\\beta_1`} with numerical maximization 🤓😍. We can do exactly the same if we have more than one explanatory variable, the only difference is that there are more ${tex`\\beta`} coefficients to maximize over.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1680,"value":"Try changing the values of ${tex`\\beta_0`} and ${tex`\\beta_1`} with the sliders below and see how the average likelihood of the data changes. In the figure to the left, you see a scatterplot of the data and the fitted regression curve ${tex`E(Y\\vert x)= \\lambda(x) = \\exp(\\beta_0 + \\beta_1\\cdot x)`} in orange. The figure to the right plots the average likelihood of the data plotted as a function of ${tex`\\beta_0`} and ${tex`\\beta_1`} in a so-called contour plot. The brighter the blue color, the higher the average likelihood of the data. The values of ${tex`\\beta_0`} and ${tex`\\beta_1`} chosen with the sliders are shown as an orange point.\n\nClick on *show ML fit* to see the fit with the ML estimate, which then appears as a green-blue point at the \"top of the hill\" in the figure to the right; the ML estimate is after all defined as the value of the parameters that maximizes the (average) likelihood of the observed data. The regression curve for the ML estimate appears in the figure to the left.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1464,"value":"viewof beta = Inputs.form([\n Inputs.range([-5, 5], {label: tex`\\beta_0`, value : 2, step: 0.1}),\n Inputs.range([-5, 5], {label: tex`\\beta_1`, value : -3, step: 0.1})\n])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1815,"value":"viewof view_ml_reg = Inputs.toggle({label: \"show ML fit\", value: false})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1544,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1586,"value":"bothplots_reg = html`\n \n \n \n

\n ${plt_scatter}\n

\n ${plt_contour}\n

^ ${text}

`;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1502,"value":"import {addColumn} from \"@observablehq/excel-add-columns\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1762,"value":"optimjs = require('optimization-js@1.5.0/dist/optimization.js')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1231,"value":"#### Dolda grafer som visas ovan","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":780,"value":"pltdata = Plot.plot({\n x: {label: \"x\"},\n marks: [\n Plot.barY(data, Plot.groupX({y: \"proportion\"}, {x: \"NBidders\", fill: \"steelblue\", opacity: 0.7})),\n Plot.line(poispdf_mle, {filter: d => settings[1], x: \"x\", y: \"pdf\", stroke: \"#5F9EA0\", strokeWidth: 3, \n marker: \"circle\", opacity: 1}),\n Plot.line(poispdf, {x: \"x\", y: \"pdf\", stroke: \"#F28C28\", strokeWidth: 3, marker: \"circle\"}),\n Plot.ruleY([0])\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1062,"value":"pltlikelihood = Plot.plot({\n y: {label: \"average probability for single observation\"},\n x: {label: \"lambda\"},\n marks: [\n Plot.ruleX([mle], {filter: d => settings[1], stroke: \"#5F9EA0\", strokeWidth: 2}),\n Plot.ruleY([0]),\n Plot.lineY(meanprob_grid, {x: \"lambda\", y: \"geomean\", stroke: \"steelblue\", strokeWidth: 3}),\n //Plot.line([{x: lambda-1, y: geomean_like(y,lambda)-0.02}, {x: lambda + 1, y: geomean_like(y,lambda)}],{x: \"x\", y: \"y\", stroke: \"green\"}),\n Plot.dot([{x: lambda, y: geomean_like(y, lambda)}], {x: \"x\", y: \"y\", fill: \"orange\", marker: \"circle\", r: 7})\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1482,"value":"plt_scatter = Plot.plot({\n y: {label: \"NBidders\"},\n marks: [\n Plot.dot(ebaydata_reg, {x: \"ReservePriceFrac\", y: \"NBidders\", fill: \"steelblue\", r: 2 }),\n Plot.line(lambda_grid, {x: \"ReservePriceFrac\", y: \"lambda\", stroke: \"orange\", strokeWidth: 3}),\n Plot.line(lambda_ml_grid, {filter: d => view_ml_reg, x: \"ReservePriceFrac\", y: \"lambda\", stroke: \"#5F9EA0\", strokeWidth: 3}),\n Plot.ruleX([0.0]),\n Plot.ruleY([0.0])\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1576,"value":"plt_contour = contour_plot(([b0, b1]) => geomean_like_reg(regdata, [b0, b1]), [[0,4], [-5,2]], d3.range(0, 0.2, 1/100))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":223,"value":"#### Data","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1881,"value":"lambda = settings[0]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":768,"value":"ebaydata = FileAttachment(\"ebaydata@2.csv\").csv({typed: true}); ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":948,"value":"data = ebaydata.filter(d => d.ReservePriceFrac <= reserve_threshold)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1900,"value":"data_high = ebaydata.filter(d => d.ReservePriceFrac > reserve_threshold)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":17,"value":"poispdf = {\n const x = d3.range(0, d3.max(ebaydata, d => d.NBidders) + 5, 1);\n const data = x.map(x => ({x: x, pdf: jstat.poisson.pdf(x, lambda)}));\n return data\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":978,"value":"poispdf_mle = {\n const x = d3.range(0, d3.max(y) + 5, 1);\n const data = x.map(x => ({x: x, pdf: jstat.poisson.pdf(x, mle)}));\n return data\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1081,"value":"y = Array.from(d3.map(data, d => d.NBidders))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1904,"value":"y_high = Array.from(d3.map(data_high, d => d.NBidders))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":981,"value":"mle = d3.mean(y)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1902,"value":"mle_high = d3.mean(y_high)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1327,"value":"meanprob_grid = {\n const lambdaGrid = d3.range(1, 10, 0.01);\n const data2 = lambdaGrid.map(lambda => ({lambda: lambda, geomean: geomean_like(y, lambda)}));\n return data2;\n}\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1548,"value":"regdata = ebaydata.map(d => ({y: d.NBidders, x: d.ReservePriceFrac}))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1510,"value":"ebaydata_reg = addColumn(ebaydata, 'lambda', row => Math.exp(beta[0] + beta[1]*row['ReservePriceFrac']))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1477,"value":"lambda_grid = d3.range(0, 2, 0.01).map(x => ({ReservePriceFrac: x, lambda: Math.exp(beta[0] + beta[1]*x)}))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1809,"value":"lambda_ml_grid = d3.range(0, 2, 0.01).map(x => ({ReservePriceFrac: x, lambda: Math.exp(ml_beta[0] + ml_beta[1]*x)}))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1937,"value":"lambda_ml_grid_two = {\n var lambda_ml_grid_two;\n lambda_ml_grid_two = d3.range(0, reserve_threshold, 0.01).map(x => ({ReservePriceFrac: x, lambda: mle, reserveprice: \"low\"}));\n return lambda_ml_grid_two.concat(d3.range(reserve_threshold + 0.01, 2, 0.01).map(x => ({ReservePriceFrac: x, lambda: mle_high, reserveprice: \"high\"})))\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1775,"value":"ml_pois_reg = function(data){\n var fnc = function(param){\n return -geomean_like_reg(data, param);\n }\n const optres = optimjs.minimize_Powell(fnc, [0,0]);\n return optres.argument\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1800,"value":"ml_beta = ml_pois_reg(regdata)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":961,"value":"#### Functions","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":849,"value":"function geomean_like(data, param){\n const probs = data.map(x => (Math.exp(-param)*param**x)/mathjs.factorial(x));\n const logprobs = probs.map(d => Math.log(d))\n return Math.exp(mathjs.mean(logprobs))\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1522,"value":"function geomean_like_reg(data, beta){\n const probs = data.map(d => (Math.exp(-Math.exp(beta[0] + beta[1]*d.x))*Math.exp(beta[0] + beta[1]*d.x)**d.y)/mathjs.factorial(d.y));\n const logprobs = probs.map(d => Math.log(d))\n return Math.exp(mathjs.mean(logprobs))\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1578,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":229,"value":"#### Style","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":202,"value":"","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":58,"value":"viewof md_colour = Inputs.color({label: \"Highlight color cells\", value: \"#F5F5DC\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W38","stdlib":"1"}} 31:"$30:body"