28:null 29:["$","title",null,{"children":"Predictor Distributions, Statistical Power, and Effect Size | Gary McClelland | Observable"}] 2e:T59d,The principle is illustrated in the graph below. When the page loads, it displays the optimal experimental design of half the observations at one extreme and the other half at the other extreme. The display reports three statistics: The factor that one would need to multiply the sample size by to have the same statistical power as the optimal design. That N factor of course equals 1.00 for the optimal design. The second statistic is V, the variance, where the variance for the optimal design has been normalized to be 1.0. Also displayed for illustrative purposes is a hypothetical effect size of ${tex`r^2 = 0.5`} for the optimal design. As you move the slider "weight" from left to right, the distribution of the variable moves towards a symmetric, normal-like distribution with a center peak. As you move the slider, observe the changes in N factor, the variance V, and the corresponding changes in the effect size ${tex`r^2`}. Surprisingly to many, the normal-like, center-peaked distirubtion for a predictor is effectively the worst-case scenario. The observed effect size is a function not only of the underlying effect but also of the distribution of the predictor variable. Hence, comparing the effect size from a lab experiment with an optimal design to a field study with a happenstance, likely normal or at least center-peaked distribution, confounds the effect of the different distributions in the two studies.2f:T4f5,chart2 = { const svg = d3.create("svg") .attr("viewBox", [0, 0, width, height]); svg.append("g") .attr("fill", color) .selectAll("rect") .data(makeQuad()) .join("rect") .attr("x", (d, i) => x(d[0])-barWidth/2) .attr("y", d => yq(d[1])) .attr("height", d => yq(0) - yq(d[1])) .attr("width", barWidth); svg.append("g") .call(xAxis3); svg.append("g") .call(yAxisq); svg.append('text') .attr('x', x(0)) .attr('y', margin.top) .style('font-size', '18px') .style('font-family', 'sans-serif') .style('text-anchor', 'middle') .text('Linear: N factor = ' + (1/variance(makeQuad())).toFixed(2)+ ', V = ' + variance(makeQuad()).toFixed(2) + ', r\xB2 = ' + (variance(makeQuad())/(1+variance(makeQuad()))).toFixed(2)); svg.append('text') .attr('x', x(0)) .attr('y', margin.top - 20) .style('font-size', '18px') .style('font-family', 'sans-serif') .style('text-anchor', 'middle') .text('Quadratic: N factor = ' + (0.25/quadVariance(makeQuad())).toFixed(2)+ ', V = ' + quadVariance(makeQuad()).toFixed(2) + ', r\xB2 = ' + (quadVariance(makeQuad())/(0.25+quadVariance(makeQuad()))).toFixed(2)); return svg.node(); }30:[] 2c:{"id":"9c4221663c256e5d","slug":null,"title":"Predictor Distributions, Statistical Power, and Effect Size","thumbnail":"0c4e97cad4de4e836673cec350ab9fc513e26a2e3a14ca058e03979d33c0e0f6","access_level":"public","create_time":"$D2021-05-11T23:31:59.351Z","update_time":"$D2023-01-22T20:45:39.734Z","view_time":null,"first_public_time":"$D2023-01-19T23:15:16.821Z","has_unpublished_edits":null,"fork":true,"stars":2,"starred":false,"owner":{"type":"team","id":"8e902c6c168a5327","login":"mcclella","name":"Gary McClelland","avatar_url":"https://avatars-next.observableusercontent.com/5fb0f9ce97ef2ff380c45d797b84b995cfeccb679fa1fcc25ae40f1a1b81414b","tier":"starter_2024"},"version":678,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"bb02701f3bca15dc","login":"mcclella","name":"Gary McClelland","avatar_url":"https://avatars-next.observableusercontent.com/5fb0f9ce97ef2ff380c45d797b84b995cfeccb679fa1fcc25ae40f1a1b81414b","tier":null}],"members":[{"type":"individual","id":"bb02701f3bca15dc","login":"mcclella","name":"Gary McClelland","avatar_url":"https://avatars-next.observableusercontent.com/5fb0f9ce97ef2ff380c45d797b84b995cfeccb679fa1fcc25ae40f1a1b81414b","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":3,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Predictor Distributions, Statistical Power, and Effect Size","cells":[{"id":1,"value":"md`# Predictor Distributions, Statistical Power, and Effect Size\nAlmost everyone learns in introductory statistics classes, that a reduction in the range of a variable reduces the size of its correlation coefficient with other variables. More precisely, the reduction in range of a variable reduces its variance and it is that reduction in variance that causes the reduction in the effect size (correlation coefficient) and necesarily a reduction in statistical power. Even when there is not a reduction in range of a variable, a reduction in its variance necessarily reduces its effect size when predicting other variables and the statistical power of detecting that effect. Formally, the expected ${tex`1 - \\alpha`} confidence interval for the slope coefficient in a linear model is given by:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":671,"value":"\\beta_X \\pm t_{n; \\alpha} \\sqrt{MSE\\over{n V_X}}","pinned":false,"hidden":false,"mode":"tex","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":675,"value":"Reducing the size of ${tex`V_X`} necessarily increases the width of the confidence interval and thereby reduces the effect size and statistical power. This set of notebooks explores the consequences of different distributions of the predictor variable X by examining its variance.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":655,"value":"$2e","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":380,"value":"viewof wt = Range([0,1], {label: \"Weight: \",step: .01, value: 0})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":398,"value":"chart = {\n const svg = d3.create(\"svg\")\n .attr(\"viewBox\", [0, 0, width, height]);\n \n\n svg.append(\"g\")\n .attr(\"fill\", color)\n .selectAll(\"rect\")\n .data(makeAverage())\n .join(\"rect\")\n .attr(\"x\", (d, i) => x(d[0])-barWidth/2)\n .attr(\"y\", d => y2(d[1]))\n .attr(\"height\", d => y2(0) - y2(d[1]))\n .attr(\"width\", barWidth);\n\n svg.append(\"g\")\n .call(xAxis3);\n\n svg.append(\"g\")\n .call(yAxis2);\n \n svg.append('text')\n .attr('x', x(0))\n .attr('y', margin.top)\n .style('font-size', '18px')\n .style('font-family', 'sans-serif')\n .style('text-anchor', 'middle')\n .text('N factor = ' + (1/variance(makeAverage())).toFixed(2)+ ', V = ' + variance(makeAverage()).toFixed(2)\n + ', r\\xB2 = ' +\n (variance(makeAverage())/(1+variance(makeAverage()))).toFixed(2));\n \n\n\n return svg.node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":573,"value":"md`# Linear vs. Quadratic Effects\nIt is important to note that the above demonstration applies only to linear effects. The ideal distribution for detecting a linear effect has half the observations at each extreme. In that case, there is no possibility of detecting a quadratic effect, which requires at least three levels. The optimal design for detecting a quadratic effect has half the observations at each extreme and the remaining half of the observations midway between those two extremes. In designing a study to detect both linear and quadratic effects, there must be a tradeoff between the two respective optimal designs. Use the slider in the graph below to change the proportion of the observations allocated to the middle category. The remaining observations are then split between the two extremes. `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":592,"value":"viewof wt2 = Inputs.range([0.01, 0.99], {value: 0.5, label: \"Middle %\", step: 0.01})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":585,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":637,"value":"For the quadratic, the optimal design is when the the middle proportion equals 0.5. For the linear, the optimal design is when the middle proportion is near 0.0. Find a design for which the N factor is equal for detecting to the two types of effects. We might consider this a \"hedge\" design requiring only one third more observations than the optimal linear design while still providing adequate power for detecting a quadratic effect if there is one. Compare this to a design that equal numbers of observations at all three levels. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":643,"value":"## References\nMcClelland, G.H. (1997). Optimal design in psychological research. *Psychological Methods, 2*, 3-19.\n\nMcClelland, G.H. (2000). Increasing statistical power without increasing sample size. *American Psychologist, 55*, 963-964.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":580,"value":"Next","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":411,"value":"md `## Appendix`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":293,"value":"variance = function(grp) {\n let mean = 0;\n let variance = 0;\n for(let i=0; i< grp.length; i++)\n mean = mean + grp[i][0]*grp[i][1];\n for(let i=0; i< grp.length; i++) \n variance = variance + Math.pow(grp[i][0]-mean,2)*grp[i][1]\n return variance;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":619,"value":"quadVariance = function(grp) {\n let mean = 0;\n let v = 0;\n for(let i=0; i g\n .attr(\"transform\", `translate(0,${height - margin.bottom})`)\n .call(d3.axisBottom(x).ticks(3))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":405,"value":"xAxis3 = g => g\n .attr(\"transform\", `translate(0,${height - margin.bottom})`)\n .call(d3.axisBottom(x).tickValues(tickVals[5-2]).tickFormat(x => x.toFixed(1)))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":110,"value":"yAxis = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .call(d3.axisLeft(y))\n ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":371,"value":"y2 = d3.scaleLinear()\n .domain([0,.5])\n .range([height - margin.bottom, margin.top])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":602,"value":"yq = d3.scaleLinear()\n .domain([0,1])\n .range([height - margin.bottom, margin.top])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":374,"value":"yAxis2 = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .call(d3.axisLeft(y2))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":605,"value":"yAxisq = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .call(d3.axisLeft(yq))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":255,"value":"color = \"steelblue\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":69,"value":"height = 500","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":74,"value":"margin = ({top: 60, right: 0, bottom: 30, left: 40})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"d3 = require(\"d3@6\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":286,"value":"import {Range} from \"@observablehq/inputs\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":578,"value":"","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"}],"files":"$W30","stdlib":"1"}} 2d:"$2c:body"