2f:Tdc5,md`# Statistical Difficulties of Detecting Interactions An interaction or moderator is when the relationship between one of the predictor variables and the outcome variable depends on the level of the other predictor variable. Researchers using laboratory studies frequently find interactions whereas field researchers either cannot find theoretically-expected interactions or lament the small effect sizes of the ones they do find. We will assume the following model with fixed coefficients applies throughout: ${tex.block` \large y_i = \beta_0 + \beta_x x_i + \beta_z z_i + \beta_{xz} x_i z_i + \epsilon_i `} Testing ${tex ` \beta_{xz}`} against zero tests the interaction represented by the product ${tex `x_i z_i`} of the two predictors. And the expected squared partial correlation measures the effect size of the interaction: ${tex.block` \large pr_{y,xz}^2 = [1 + \frac{\sigma_\epsilon^2}{ \beta_{xz}^2 \sigma_{xz.x,z}^2}]^{-1}`} where ${tex `\sigma_\epsilon^2`} is the error variance (estimated by the mean squared error) and ${tex `\sigma_{xz.x,z}^2`} represents the residaul variance of the product after controling for or partialling out the two predictors. The residual variance of the product is entirely a function of the research design while the other terms on the right pertain to the theoretical model. By assuming all terms on the right are constant except for the residual variance of the product we can isolate the effect of the design on effect size.. We can then compare the effects of different bivariate distributions by examining the residual variance of the product for those distributions. A useful representation of the expected value of the F-statistic for testing the interaction is ${tex.block `\large F = n \frac{pr_{y,xz}^2}{1 - pr_{y,xz}^2}`} The graph below is a bubble plot of the predictors' bivariate distribution across two 7-point scales. The slider changes the peakedness and makes the bivariate distribution more normal. The limits are approached slowly so at the ends there is a discontinuous jump to the limit. The far left end of the slider represents a planned experimental design where a quarter of the observations are located at each extreme corner. When the slider value equals 1, it represents a uniform distribution across all scale values. As the slider moves to the right, it represents the kind of distributions that might occur in a field study where there is no control over the levels. And higher values represent more observations in the center, close to the respective means. At the extreme far right, is the discontinuity of all the observations at in the center. Without loss of generality, we will assume that the predictor values are scaled so that for the optimal experimental four-corner design, the residual variance is 1. The graph displays the variance for the other distributions, which can be considered relative to the optimal design. For illustrative purposes, we assume the effect size (the squared partial correlation) is 0.25 for the interaction in the optimal experimental design. We can then compute the expected effect size for the other designs assuming all other model parameters remain the same. Finally, given the change in effect size, we can compute by what factor n would need to be increased to yield the same value for *F*. This serves an approximation of how much the sample size would need to be increased to have the same statistical power for detecting the interaction was in the optimal experimental design. `30:T416,chart = { const svg = d3.create("svg") .attr("viewBox", [0, 0, width, graphSize]); svg.append('g') .selectAll("dot") .data(makeArray()) .enter() .append("circle") .attr("cx", d => x(d.row+1) ) .attr("cy", d => y(d.col+1) ) .attr("r", d => z(d.prob) ) .style("fill", graphColors.orange) .attr("stroke", "none") svg.append('text') .attr('x', x(4)) .attr('y', y(7.1)) .style('text-anchor', 'middle') .style('font-family', 'sans-serif') .style('font-size', '16px') .text(' pr^2 = ' + effectSize().toFixed(3)) svg.append("g") .call(xAxis); svg.append('g') .call(yAxis); svg.append('text') .attr('x', x(7.2)) .attr('y', y(-.1)) .style('font-family', 'sans-serif') .style('font-size', '16px') .text('X') svg.append('text') .attr('x', x(0)) .attr('y', y(7.1)) .style('text-anchor', 'middle') .style('font-family', 'sans-serif') .style('font-size', '16px') .text('Z') return svg.node(); }31:T4d8,chart2 = { const svg = d3.create("svg") .attr("viewBox", [0, 0, width, graphSize]); let barOffset = .35; svg.selectAll('bars') .data(sums) .enter() .append('rect') .attr('x', d => xx(d[0]-barOffset)) .attr('y', d => yy(d[1])) .attr('width', xx(1 +barOffset) - xx(1 - barOffset)) .attr('height', d => yy(0) - yy(d[1])) .style('fill', graphColors.blue) svg.append("g") .call(xxAxis); svg.append('g') .call(yyAxis); svg.append('text') .attr('x', xx(10.2)) .attr('y',graphSize) .style('text-anchor', 'start') .style('font-family', 'sans-serif') .style('font-size', '16px') .text('XZ') svg.append('text') .attr('x', xx(-11)) .attr('y', 18) .style('text-anchor', 'start') .style('font-family', 'sans-serif') .style('font-size', '16px') .text('Probability') svg.append('text') .attr('x', xx(0)) .attr('y', yy(.9)) .style('text-anchor', 'middle') .style('font-family', 'sans-serif') .style('font-size', '16px') .text("Relative XZ Var = " + (prodVar()/81).toFixed(3)+ ' nFactor = ' + nFactor().toFixed(1)) return svg.node(); }32:T490,md` ## References Irwin, J.R., & McClelland, G.H. (2001). Misleading heuristics and moderated multiple regression models. *Journal of Marketing Research, 38*, 100-109. McClelland, G.H. (1997). Optimal design in psychological research, *Psychological Methods, 2(1)*, 3-19. McClelland, G.H., Irwin, J.R., Disatnik, D., & Sivan, L. (2017). Multicolinearity is a red herring in the search for moderator variables: A guide to interpreting moderated multiple regression models and a critique of Iacobucci, Schneider, Popovich, and Bakamitsos (2016). *Behavior Research Methods, 49(1)*, 394-402. McClelland,G.H., & Judd, C.M. (1993). Statistical difficulties of detecting interactions and moderator effects. *Psychological Bulletin, 114(2),* 376-390. Spiller, S.A., Fitzsimons, G.J., Lynch, J.G. Jr., & McClelland, G.H. (2013). Spotlights, floodlights, and the magic number zero: Simple effects tests in moderated regression. *Journal of Marketing Research, 50(2)*, 277-288. Whisman, M.A., & McClelland, G.H. (2005). Designing, testing, and interpreting interactions and moderator effects in family research. *Journal of Family Psychology, 19(1),* 111-120. `33:[] 2d:{"title":"Statistical Difficulties of Detecting Interactions","cells":[{"id":0,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs"},{"id":70,"value":"viewof ab = logSlider({\n min: 0.01,\n max: 60,\n step: 0.1,\n value: 0.01,\n description: 'Move the slider to change the design',\n title: 'Degree of Peakedness'\n})\n ","pinned":false,"hidden":false,"mode":"ojs"},{"id":33,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":323,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":438,"value":"chart3 = {\n const svg = d3.create(\"svg\")\n .attr(\"viewBox\", [0, 0, width, graphSize]);\n \n svg.append(\"g\")\n .call(xxxAxis);\n \n svg.append('g')\n .call(yyyAxis);\n \n svg.append('line')\n .attr('x1', xxx(-9))\n .attr('y1', yyy(slope(-9)))\n .attr('x2', xxx(9))\n .attr('y2', yyy(slope(9)))\n .style('stroke-width', 3)\n .style('stroke', graphColors.orange)\n \n svg.append('line')\n .attr('x1', xxx(-9))\n .attr('y1', yyy(0))\n .attr('x2', xxx(9))\n .attr('y2', yyy(0))\n .style('stroke-width', 1)\n .style('stroke', 'black')\n \n \n \n svg.append(\"path\")\n\t\t\t.attr(\"d\", ciBandLo(d3.range(-9,9,0.5)))\n\t\t\t.style(\"stroke\", graphColors.blue)\n\t\t\t.style(\"stroke-width\", 3)\n\t\t\t.style(\"fill\", 'none');\n \n svg.append(\"path\")\n\t\t\t.attr(\"d\", ciBandHi(d3.range(-9,9,.5)))\n\t\t\t.style(\"stroke\", graphColors.blue)\n\t\t\t.style(\"stroke-width\", 3)\n\t\t\t.style(\"fill\", 'none');\n \n return svg.node();\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":210,"value":"md` The strong implication is that as there is any movement away from the optimal experimental four-corners design to more observations in the center, the residual variance of the product plummets and consequently so does the expected effect size. Further, to obtain the same F value, the number of observations must be increased by an enormous factor. For example, suppose n = 120 (30/cell) provided adequate statistical power. Then when the slider in the graph equal about 5, a researcher obtaining such a distribution in a field study would need to have about 17 times as many observations, or about 2000 observations. It is therefore no surprise that interactions are so difficult to detect in field studies.`","pinned":false,"hidden":false,"mode":"ojs"},{"id":212,"value":"$32","pinned":false,"hidden":false,"mode":"ojs"},{"id":498,"value":"Next","pinned":false,"hidden":false,"mode":"html"},{"id":207,"value":"md` ## Appendix`","pinned":false,"hidden":false,"mode":"ojs"},{"id":390,"value":"abMax = 60;","pinned":false,"hidden":false,"mode":"ojs"},{"id":39,"value":"x = d3.scaleLinear()\n .domain([0, 7])\n .range([margin.left, graphSize - margin.right])","pinned":false,"hidden":false,"mode":"ojs"},{"id":312,"value":"xx = d3.scaleLinear()\n .domain([-10,10])\n .range([margin.left, graphSize-margin.right])","pinned":false,"hidden":false,"mode":"ojs"},{"id":415,"value":"xxx = d3.scaleLinear()\n .domain([-9,9])\n .range([margin.left, graphSize-margin.right])","pinned":false,"hidden":false,"mode":"ojs"},{"id":245,"value":"xAxis = g => g\n .attr(\"transform\", `translate(0,${graphSize - margin.bottom})`)\n .call(d3.axisBottom(x).ticks(7))","pinned":false,"hidden":false,"mode":"ojs"},{"id":318,"value":"xxAxis = g => g\n .attr(\"transform\", `translate(0,${graphSize - margin.bottom})`)\n .call(d3.axisBottom(xx))","pinned":false,"hidden":false,"mode":"ojs"},{"id":419,"value":"xxxAxis = g => g\n .attr(\"transform\", `translate(0,${graphSize - margin.bottom})`)\n .call(d3.axisBottom(xxx))","pinned":false,"hidden":false,"mode":"ojs"},{"id":47,"value":"y = d3.scaleLinear()\n .domain([0,7])\n .range([graphSize - margin.top, margin.bottom])","pinned":false,"hidden":false,"mode":"ojs"},{"id":332,"value":"yy = d3.scaleLinear()\n .domain([0,1])\n .range([graphSize - margin.top, margin.bottom])","pinned":false,"hidden":false,"mode":"ojs"},{"id":422,"value":"yyy = d3.scaleLinear()\n .domain([-4,6])\n .range([graphSize - margin.top, margin.bottom])\n","pinned":false,"hidden":false,"mode":"ojs"},{"id":251,"value":"yAxis = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .call(d3.axisLeft(y).ticks(7))","pinned":false,"hidden":false,"mode":"ojs"},{"id":334,"value":"yyAxis = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .call(d3.axisLeft(yy))","pinned":false,"hidden":false,"mode":"ojs"},{"id":426,"value":"yyyAxis = g => g\n .attr(\"transform\", `translate(${margin.left},0)`)\n .call(d3.axisLeft(yyy))","pinned":false,"hidden":false,"mode":"ojs"},{"id":49,"value":"z = d3.scaleLinear()\n .domain([0,.2])\n .range([0,24])","pinned":false,"hidden":false,"mode":"ojs"},{"id":34,"value":"graphSize = Math.min(400,width)","pinned":false,"hidden":false,"mode":"ojs"},{"id":428,"value":"function slope(x) {\n return 1 + .5*x\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":430,"value":"function ci(x) {\n let se = 7.9 * Math.sqrt(1/200 + (x*x)/(200 * prodVar()))\n return se * 2; //approximate t\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":441,"value":"slope(-9) + ci(-9)","pinned":false,"hidden":false,"mode":"ojs"},{"id":432,"value":"ciBandLo = d3.line()\n\t\t\t.x( function(d) {return xxx(d)})\n\t\t\t.y( function(d) {return yyy(slope(d) - ci(d ))});","pinned":false,"hidden":false,"mode":"ojs"},{"id":435,"value":"ciBandHi = d3.line()\n\t\t\t.x( function(d) {return xxx(d)})\n\t\t\t.y( function(d) {return yyy(slope(d) + ci(d ))});","pinned":false,"hidden":false,"mode":"ojs"},{"id":197,"value":"function effectSize() {\n if(ab <= 0.01) return 0.25;\n else return 1/(1 + 3/(prodVar()/81))\n \n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":200,"value":"function nFactor() {\n if(ab <= 0.01) return 1;\n else return (1/3)/(effectSize()/(1-effectSize()));\n\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":12,"value":"function betaBinomialpdf(k,n,a,b) {\n return (mathfn.gamma(n+1)/(mathfn.gamma(k+1)*mathfn.gamma(n-k+1))) *\n (mathfn.gamma(k+a)*mathfn.gamma(n-k+b)/mathfn.gamma(n+a+b)) * \n mathfn.gamma(a+b)/(mathfn.gamma(a)*mathfn.gamma(b))\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":83,"value":"function betaBinomialmean(n,a,b) {\n return n*a/(a+b)\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":21,"value":"function params() {\n return {\"a\": ab, \"b\": ab}\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":347,"value":"params()","pinned":false,"hidden":false,"mode":"ojs"},{"id":294,"value":"function computeProd() {\n let c = [];\n if(ab <= 0.01) {\n c.push({\"xz\": -9,\"prob\": 0.25})\n c.push({\"xz\": 9, \"prob\": 0.25})\n c.push({\"xz\": 9, \"col\": 0, \"prob\": 0.25})\n c.push({\"xz\": -9, \"col\": 6, \"prob\": 0.25})\n }\n else {\n for(let i=0; i<=6; i++)\n for(let j=0; j<=6; j++)\n c.push({\"xz\": (i-3)*(j-3), \"prob\": betaBinomialpdf(i,6,params().a,params().b) *\n betaBinomialpdf(j,6,params().a,params().b)})\n \n \n } \n \n\n \n return c;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":297,"value":"prod = computeProd()","pinned":true,"hidden":false,"mode":"ojs"},{"id":300,"value":"sums = d3.rollups(prod, v => d3.sum(v, d=> d.prob), d => d.xz)","pinned":true,"hidden":false,"mode":"ojs"},{"id":100,"value":"function prodVar() {\n let sum = 0;\n if(ab <= 0.01) return 81;\n else {\n for(let i=0; i<=6; i++) \n sum = sum + (i-3)*(i-3) * betaBinomialpdf(i,6,params().a,params().b)\n return sum*sum;\n }\n\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":17,"value":"function makeArray() {\n let c = [];\n if(ab <= 0.01) {\n c.push({\"row\": 0, \"col\": 0, \"prob\": 0.25})\n c.push({\"row\": 0, \"col\": 6, \"prob\": 0.25})\n c.push({\"row\": 6, \"col\": 0, \"prob\": 0.25})\n c.push({\"row\": 6, \"col\": 6, \"prob\": 0.25})\n }\n else {\n for(let i=0; i<=6; i++) \n for(let j=0; j<=6; j++)\n c.push({\"row\": i, \"col\": j, \n \"prob\": betaBinomialpdf(i,6,params().a,params().b)*betaBinomialpdf(j,6,params().a,params().b)})\n } \n \n \n return c;\n}","pinned":false,"hidden":false,"mode":"ojs"},{"id":44,"value":"margin = ({top: 30, right: 30, bottom: 30, left: 30})","pinned":false,"hidden":false,"mode":"ojs"},{"id":4,"value":"mathfn = require('https://bundle.run/mathfn@1.1.0')","pinned":false,"hidden":false,"mode":"ojs"},{"id":403,"value":"graphColors = ({blue: d3.hsl(223,1.,0.7), orange: d3.hsl(23,1.0,0.5), red: d3.hsl(331,0.73,0.51)})","pinned":true,"hidden":false,"mode":"ojs"},{"id":284,"value":"import {logSlider} from '@severo/log-slider'","pinned":true,"hidden":false,"mode":"ojs"},{"id":327,"value":"//viewof ab = logSlider({\n// min: 0.001,\n// max: 10.1,\n// step: .1,\n// value: 0.001,\n// title: 'Degree of Peakedness',\n// description: 'Move the slider to change the design'\n//})","pinned":true,"hidden":false,"mode":"ojs"},{"id":275,"value":"import {slider} from \"@severo/inputs-setter\"","pinned":false,"hidden":false,"mode":"ojs"},{"id":6,"value":"d3 = require('d3@6.2')","pinned":false,"hidden":false,"mode":"ojs"},{"id":467,"value":"","pinned":false,"hidden":false,"mode":"html"}],"files":"$W33","stdlib":"1","titling":"auto"}