2c:null 2d:["$","title",null,{"children":"An interactive explanation of the statistical t-test used to compare sample means | bluprince13 | Observable"}] 32:T58a,md`

xkcd.com

Say we want to compare the mileage between two different types of cars: A and B, we could measure it on a sample of cars from both populations. We can then calculate a sample mean and standard deviation for both samples. Wouldn't it be just fantastic if we could then 'infer' how the mean of the entire population of A compares to B, based on just the two small samples of data we have? A t-test is a statistical test used to determine if there is a significant difference between the means of two groups. It was introduced in 1908 by [William Sealy Gosset](https://en.wikipedia.org/wiki/William_Sealy_Gosset), a chemist working for the Guinness brewery in Ireland. He devised the t-test as a cheap way to monitor the quality of stout (an awful type of beer :P). He published the test in a journal (as scientists do), but was forced to use a pen name by his employer, who wanted to keep their use of statistics a secret from their competitors. The t-test he devised is known as the Student's t-test since he published it under the pseudonym Student.

William Sealy Gosset

`33:T878,{ const refresh = refresh_1; const n = 20; const mu = 0; const mu0 = mu0Default; const numberOfSamples = 1; const experiment1 = getExperiment({ mu, n, sigma: 10, numberOfSamples }); const experiment2 = getExperiment({ mu, n, sigma: 1, numberOfSamples }); const xaxis = "x1"; const yaxis = "y1"; const showlegend = true; const weak = "weak
σ = 10"; const strong = "strong
σ = 1"; const trace_x1 = { name: xText, x: Array(n).fill(weak), y: experiment1.flatData, type: "scatter", mode: "markers", marker: { color: "black" } }; const trace_x2 = { name: xText, x: Array(n).fill(strong), y: experiment2.flatData, type: "scatter", mode: "markers", marker: { color: "black" }, showlegend: false }; const trace_mean = { name: meanText, x: [weak, strong], y: [experiment1.means[0], experiment2.means[0]], type: "scatter", mode: "markers", marker: { color: "red" } }; const trace_mu = { name: muText, x: [weak, strong], y: [mu], type: "scatter", mode: "lines", marker: { color: "green" } }; const trace_mu0 = { name: mu0Text, x: [weak, strong], y: [mu0], type: "scatter", mode: "lines", marker: { color: "blue" } }; const data = [trace_x1, trace_x2, trace_mean, trace_mu, trace_mu0]; var layout = { title: "Statistical signficance", yaxis: { title: scoreText, range: range.slice() }, width: width, showlegend: true, shapes: [ { type: "line", x0: 0, y0: mu, x1: 1, y1: mu, line: { color: "green", width: 2 }, xref: "paper", yref: "y" }, { type: "line", x0: 0, y0: mu0, x1: 1, y1: mu0, line: { color: "blue", width: 2 }, xref: "paper", yref: "y" } ] }; console.log({ data, layout }); var div = DOM.element("div"); Plotly.newPlot(div, data, layout); return div; }34:T615,md`There are a few subtleties I'd like you to appreciate here. 1. t-values follow a distribution with μ = 0, and σ = 1 because the formula for the t-value 'standardises' the sample means. If ${tex`\mu_{0} = \mu`} the formula for the t-value becomes: ${tex.block` t=\frac{\overline{x}-\mu}{s / \sqrt{n}}`} A random variable is standardised by subtracting the mean of the distribution from the value being standardised, and then dividing by the standard deviation of the distribution. 2. t-values follow a t-distribution rather than a normal distribution (also called z-distribution) because for small samples observations are more likely to fall beyond two standard deviations from the mean than under the normal distribution. In other words, the standard error is less accurate for a small sample. The thicker tails of the t-distribution are a correction to account for the poorly estimated standard error. As the sample size is inreased, the t-distribution becomes similar to a normal distribution. 3. Why is the degrees of freedom 'n-1' and not just 'n'? Imagine we have n = 3, and so we have three numbers a, b, c. Their mean is 10. - Let's assume that a = 5 => mean of (5, b, c) = 10. - Then we assume that b = 15 => mean of (5, 15, c) = 10. - Can we assume a value for c? No, we can't. c has to be 10 to satisfy the condition that the mean of (a, b, c) = 10. Therefore, for n = 3, we only have the freedom to choose values for two observations. Thus, n-1 is the number of degrees of freedom for measuring the sample mean. `35:T46f,md` The null, ${tex`H_0`}, and alternative hypotheses, ${tex`H_1`}, are two mutually exclusive statements. The null hypothesis states that a population parameter, such as the mean, is equal to a hypothesized value. The null hypothesis is often an initial claim that is based on previous analyses or specialized knowledge. The alternative hypothesis states that a population parameter is smaller, greater, or different than the hypothesized value in the null hypothesis. The alternative hypothesis is what you might believe to be true or hope to prove true by rejecting the null hypothesis. The default null hypothesis is that the means are equal.

For example:
${getTex(h0Tex)}: The average marks for the 8th grade = 75%.
${getTex(h1Tex)}: The average marks for the 8th grade <> 75%.

We never “accept” the null hypothesis. It is safer to say that we “fail to reject” the null hypothesis, i.e., that there is insufficient statistical evidence to reject it. It turns out it's much easier to disprove a hypothesis than to positively prove one. We live in a sad little world, don't we? :P `36:T9c5,md`If we get a sample t-value close to 0, then we fail to reject the null hypothesis. However, if the t-value is greater than a critical t-value, then the sample likely came from a population with a different mean to the hypothesised mean. 1. To determine the critical t-value, we need first to determine the significance level we're after. The significance level, also denoted as alpha or α, is the probability of rejecting the null hypothesis when it is true. For example, a significance level of 0.05 indicates a 5% risk of concluding that a difference exists when there is no actual difference. It is also equivalent to a [95% confidence level](https://en.wikipedia.org/wiki/Confidence_interval). 2. The next step is to determine whether we are doing a two-tailed test or one-tailed test. - A two-tailed test is non-directional; it allows us to determine if the sample mean is different from the hypothetical mean, i.e., either greater or less than. - A one-tailed test is directional; it allows us to determine if the sample mean is greater or less than the hypothetical mean, but not both. One-tailed tests are only used when we are not worried about missing an effect in the untested direction. The advantage is that the one-tailed test achieves a higher level of significance for a smaller t-value. For example, say we devised a training program to help students better understand maths. We want to understand if there is a statistically significant improvement in their marks after the training program. However, we're not worried about them being worse off after the training program. So, a one-tailed test may be used. 3. We can now use an [online calculator](https://goodcalculators.com/student-t-value-calculator/) to work out the critical t-value for the given degrees of freedom and significance level. 4. Finally, we compare the calculated t-value for the sample to the critical t-value. - If the calculated t-value for the sample is greater than the critical t-value from the table, we can conclude that the difference between the means for the two groups is significantly different. We reject the null hypothesis in favour of the alternative hypothesis. - On the other hand, if the calculated t-value for the sample is lower than the critical t-value, we can conclude that the difference between the means for the two groups is not significantly different. We fail to reject the null hypothesis. Let's see this in action. :) `37:T68d,{ const refresh = refresh_4; const n = n_4; const mu = mu_4; const mu0 = mu0_4; const sigma = sigma_4; const numberOfSamples = 1; const xaxis = "x1"; const yaxis = "y1"; const showlegend = true; const sample = "sample"; const trace_x = { name: xText, x: Array(n).fill(sample), y: experiment.flatData, type: "scatter", mode: "markers", marker: { color: "black" } }; const trace_mean = { name: meanText, x: [sample], y: [experiment.means[0]], type: "scatter", mode: "markers", marker: { color: "red" } }; const trace_mu = { name: muText, x: [sample], y: [mu], type: "scatter", mode: "lines", marker: { color: "green" } }; const trace_mu0 = { name: mu0Text, x: [sample], y: [mu0], type: "scatter", mode: "lines", marker: { color: "blue" } }; const data = [trace_x, trace_mean, trace_mu, trace_mu0]; var layout = { title: "sample data", yaxis: { title: scoreText }, width: width, showlegend: true, shapes: [ { type: "line", x0: 0, y0: mu, x1: 1, y1: mu, line: { color: "green", width: 2 }, xref: "paper", yref: "y" }, { type: "line", x0: 0, y0: mu0, x1: 1, y1: mu0, line: { color: "blue", width: 2 }, xref: "paper", yref: "y" } ] }; console.log({ data, layout }); var div = DOM.element("div"); Plotly.newPlot(div, data, layout); return div; }38:T4bc,md` Now that we understand how the one-sample t-test works, it'll be easier to appreciate the other types. - **The one-sample t-test** is used to compare the mean of a population with a theoretical value.
e.g.compare the average marks of a sample of students with a hypothesised value of 60%. - **The unpaired two-sample t-test** is used to compare the mean of two independent samples.
e.g. compare the average marks between samples of English and Indian students, to understand if there is a significant difference in academic ability between the two populations. - **The paired t-test** is used to compare the means between two related groups of samples.
e.g. compare the average marks of a sample of students before and after a training program, to understand its effectiveness. The approach/formulas may be slightly different, but the principles are the same. You can read more about the three types here - [Understanding t-tests: 1-sample, 2-sample, and paired t-tests](https://blog.minitab.com/blog/adventures-in-statistics-2/understanding-t-tests-1-sample-2-sample-and-paired-t-tests).`39:T5e2,md` 1. [StatsCast: What is a t-test?](https://www.youtube.com/watch?v=0Pd3dc1GcHc). YouTube video. 2. [Student's t-test](https://www.youtube.com/watch?v=pTmLQvMM-1M). YouTube video. 3. [t-test](https://www.investopedia.com/terms/t/t-test.asp). Investopedia. 4. [What Is a t-test? And Why Is It Like Telling a Kid to Clean Up that Mess in the Kitchen?](https://blog.minitab.com/blog/statistics-and-quality-data-analysis/what-is-a-t-test-and-why-is-it-like-telling-a-kid-to-clean-up-that-mess-in-the-kitchen). The Minitab Blog. 5. [What Are T Values and P Values in Statistics?](https://blog.minitab.com/blog/statistics-and-quality-data-analysis/what-are-t-values-and-p-values-in-statistics). The Minitab Blog. 6. [Understanding t-Tests: 1-sample, 2-sample, and Paired t-Tests](https://blog.minitab.com/blog/adventures-in-statistics-2/understanding-t-tests-1-sample-2-sample-and-paired-t-tests). The Minitab Blog. 7. [Understanding Hypothesis Tests: Significance Levels (Alpha) and P values in Statistics](https://blog.minitab.com/blog/adventures-in-statistics-2/understanding-hypothesis-tests-significance-levels-alpha-and-p-values-in-statistics). The Minitab Blog. 8. McDonald, J.H. [Student's t–test for two samples](http://www.biostathandbook.com/twosamplettest.html), 2014. Handbook of Biological Statistics (3rd ed.). Sparky House Publishing, Baltimore, Maryland. 9. [Hypothesis testing](https://onlinecourses.science.psu.edu/statprogram/reviews/statistical-concepts/hypothesis-testing). PennState. `3a:T453,getExperiment = function({ numberOfSamples = numberOfSamplesDefault, n = nDefault, mu = muDefault, sigma = sigmaDefault } = {}) { var samples = Array.from({ length: numberOfSamples }, (x, i) => i); var data = samples.map(() => { return d3.range(n).map(() => d3.randomNormal(mu, sigma)()); }); var means = data.map(sample => math.mean(sample)); var stds = data.map(sample => math.std(sample)); var stdErrors = stds.map(std => std / math.sqrt(n)); var stdErrorPopulation = math.std(means); var stdErrorsPopulation = Array(numberOfSamples).fill(stdErrorPopulation); // Now let's flatten the data for plotting const flatSamples = samples.reduce((res, current) => { return res.concat(Array(n).fill(current)); }, []); const flatData = [].concat.apply([], data); const mus = Array(numberOfSamples).fill(mu); const sigmas = Array(numberOfSamples).fill(sigma); return { numberOfSamples, n, samples, flatSamples, data, flatData, means, stds, stdErrors, stdErrorPopulation, stdErrorsPopulation, mus, sigmas }; }3b:Tc9a,getPValuePlot = ({ v, alpha, tail, tValue }) => { const { x, y: yt } = getPDF({ v, dist: "t" }); const pdfValue = t.pdf(tValue, v); const pValue = getPValue({ tValue, v, tail }); const isRejected = pValue > alpha ? false : true; let tLeft; let tRight; if (tValue >= 0 && tail == "two-tailed") { tLeft = -tValue; tRight = tValue; } else if (tValue < 0 && tail == "two-tailed") { tLeft = tValue; tRight = -tValue; } else { tLeft = tValue; tRight = tValue; } const { x: xRight, y: yRight } = getPDF({ v, dist: "t", range: [tRight, 5] }); const { x: xLeft, y: yLeft } = getPDF({ v, dist: "t", range: [-5, tLeft] }); const data = [ { name: "t-distribution", x: x, y: yt, mode: "lines" }, { name: `t-value`, x: [tValue, tValue], y: [0, 0.5], yref: "paper", marker: { color: "black" }, line: { dash: "dash" }, mode: "lines" } ]; const traceLeft = { name: `P-value`, x: xLeft, y: yLeft, marker: { color: "red" }, mode: "lines", fill: "tozeroy", showlegend: tail == "one-tailed-left" ? true : false }; const traceRight = { name: `P-value`, x: xRight, y: yRight, marker: { color: "red" }, mode: "lines", fill: "tozeroy", showlegend: tail == "one-tailed-right" ? true : false }; if (tail == "two-tailed") { data.push(traceLeft, traceRight); } else if (tail == "one-tailed-left") { data.push(traceLeft); } else { data.push(traceRight); } var layout = { title: `t-distribution
ν = ${v}, α = ${alpha}`, xaxis: { title: tText, range: [-5, 5] }, yaxis: { title: probabilityDensityText, range: [0, 0.5] }, width: width, annotations: [ { x: tValue, y: 1, yref: "paper", yanchor: "bottom", text: `t-value = ${tValue.toPrecision(4)}`, showarrow: false }, { x: tail == "two-tailed" ? 0 : tValue, y: pdfValue, yanchor: "bottom", xanchor: tValue >= 0 ? "left" : "right", xshift: tValue >= 0 ? 5 : -5, text: `P-value = ${pValue.toPrecision(2)}`, showarrow: false }, { x: 0.1, y: 0.8, xref: "paper", yref: "paper", yanchor: "middle", text: isRejected ? "Reject
null hypothesis" : "Fail to reject
null hypothesis", showarrow: false, font: { family: "sans serif", size: largeFont, color: isRejected ? "green" : "red" } }, { x: 0.9, y: 0.8, xref: "paper", yref: "paper", yanchor: "middle", text: isRejected ? `${pValue.toPrecision(2)} ≤ ${alpha}` : `${pValue.toPrecision(2)} > ${alpha}`, showarrow: false, font: { family: "sans serif", size: largeFont, color: isRejected ? "green" : "red" } } ] }; var div = DOM.element("div"); Plotly.newPlot(div, data, layout); return div; }3c:T10d7,getCriticalTValuePlot = ({ v, alpha, tail, tValue }) => { const { x, y: yt } = getPDF({ v, dist: "t" }); const tValuePrecise = tValue.toPrecision(2); const tValuePreciseAbs = Math.abs(tValuePrecise); let pCritical; if (tail == "two-tailed") { pCritical = alpha / 2; } else if (tail == "one-tailed-left") { pCritical = -alpha; } else { pCritical = alpha; } let tCritical; const tLeft = t.quantile(Math.abs(pCritical), v); const tRight = t.quantile(1 - Math.abs(pCritical), v); tCritical = tail == "one-tailed-left" ? tLeft : tRight; const tCriticalPrecise = tCritical.toPrecision(2); const tCriticalPreciseAbs = Math.abs(tCriticalPrecise); let isRejected; if (tail == "two-tailed") { isRejected = Math.abs(tValue) <= Math.abs(tCritical) ? false : true; } else if (tail == "one-tailed-left") { isRejected = tValue >= tCritical ? false : true; } else { isRejected = tValue <= tCritical ? false : true; } let conclusion; if (tail == "two-tailed") { isRejected ? (conclusion = `${tValuePreciseAbs} ≥ ${tCriticalPreciseAbs}`) : (conclusion = `${tValuePreciseAbs} < ${tCriticalPreciseAbs}`); } else if (tail == "one-tailed-left") { isRejected ? (conclusion = `${tValuePrecise} ≤ ${tCriticalPrecise}`) : (conclusion = `${tValuePrecise} > ${tCriticalPrecise}`); } else { isRejected ? (conclusion = `${tValuePrecise} ≥ ${tCriticalPrecise}`) : (conclusion = `${tValuePrecise} < ${tCriticalPrecise}`); } const { x: xRight, y: yRight } = getPDF({ v, dist: "t", range: [tRight, 5] }); const { x: xLeft, y: yLeft } = getPDF({ v, dist: "t", range: [-5, tLeft] }); const data = [ { name: "t-distribution", x: x, y: yt, mode: "lines" }, { name: `critical t-value`, x: [tCritical, tCritical], y: [0, 0.5], yref: "paper", marker: { color: "red" }, line: { dash: "dash" }, mode: "lines" }, { name: `t-value`, x: [tValue, tValue], y: [0, 0.25], yref: "paper", marker: { color: "black" }, line: { dash: "dash" }, mode: "lines" } ]; const traceLeft = { name: `critical region`, x: xLeft, y: yLeft, marker: { color: "red" }, mode: "lines", fill: "tozeroy", showlegend: tail == "one-tailed-left" ? true : false }; const traceRight = { name: `critical region`, x: xRight, y: yRight, marker: { color: "red" }, mode: "lines", fill: "tozeroy", showlegend: tail == "one-tailed-right" ? true : false }; if (tail == "two-tailed") { data.push(traceLeft, traceRight); } else if (tail == "one-tailed-left") { data.push(traceLeft); } else { data.push(traceRight); } var layout = { title: `t-distribution
ν = ${v}, α = ${alpha}`, xaxis: { title: tText, range: [-5, 5] }, yaxis: { title: probabilityDensityText, range: [0, 0.5] }, width: width, annotations: [ { x: tValue, y: 0.5, yref: "paper", yanchor: "bottom", text: `t-value = ${tValue.toPrecision(4)}`, showarrow: false }, { x: tCritical, y: 1, yref: "paper", yanchor: "bottom", text: `critical t-value = ${tCritical.toPrecision(4)}`, showarrow: false }, { x: 0.1, y: 0.8, xref: "paper", yref: "paper", yanchor: "middle", text: isRejected ? "Reject
null hypothesis" : "Fail to reject
null hypothesis", showarrow: false, font: { family: "sans serif", size: largeFont, color: isRejected ? "green" : "red" } }, { x: 0.9, y: 0.8, xref: "paper", yref: "paper", yanchor: "middle", text: conclusion, showarrow: false, font: { family: "sans serif", size: largeFont, color: isRejected ? "green" : "red" } } ] }; var div = DOM.element("div"); Plotly.newPlot(div, data, layout); return div; }3d:[] 30:{"id":"2aac45f604225781","slug":"an-interactive-explanation-of-the-t-test-to-compare-means","title":"An interactive explanation of the statistical t-test used to compare sample means","thumbnail":"6dcd2ca0378fc11432641449155e45c3a5d2e627681f22d15b11b9ace35c7863","access_level":"public","create_time":"$D2019-06-12T22:47:19.255Z","update_time":"$D2019-08-12T06:33:27.432Z","view_time":null,"first_public_time":"$D2019-07-20T23:44:02.203Z","fork":false,"stars":8,"starred":false,"owner":{"type":"team","id":"1dbca818aab3cef4","login":"bluprince13","name":"bluprince13","avatar_url":"https://avatars-next.observableusercontent.com/6293bad2ab82f32701b437bded214389444b0cc33ed2edcd76137e7acff2318f","tier":"starter_2024"},"version":1980,"publish_version":1980,"role":null,"authored":false,"authors":[{"type":"individual","id":"d8028b6d4a88d81e","login":"bluprince13","name":"bluprince13","avatar_url":"https://avatars-next.observableusercontent.com/6293bad2ab82f32701b437bded214389444b0cc33ed2edcd76137e7acff2318f","tier":null}],"members":[{"type":"individual","id":"d8028b6d4a88d81e","login":"bluprince13","name":"bluprince13","avatar_url":"https://avatars-next.observableusercontent.com/6293bad2ab82f32701b437bded214389444b0cc33ed2edcd76137e7acff2318f","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":10,"collection":{"id":"4aa6eb52a3549079","slug":"explorables","title":"Explorables","owner":{"id":"1dbca818aab3cef4","login":"bluprince13","name":"bluprince13"}},"type":"notebook","body":{"title":"An interactive explanation of the statistical t-test used to compare sample means","cells":[{"id":0,"value":"md`# An interactive explanation of the statistical t-test used to compare sample means`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":15,"value":"md`\nIn statistics, t-tests are a type of hypothesis test that allows you to compare sample means. They are called t-tests because they boil your sample data down to one number, the t-value. In this explorable, we will develop an intuitive understanding of the t-test. I do assume that you're familiar with the normal distribution, standard deviation and the standard error of the mean. If not, I recommend checking out my article on the [standard error of the mean](https://www.vipinajayakumar.com/standard-error-of-the-mean) first. \n\nThere are three types of t-tests, but we'll focus on the one-sample t-test, used to compare the mean of a population with a theoretical value. Once we understand it, I promise that the other types will be a breeze. Shall we get started then?\n\nIt's recommended that you view this page on a wide screen. The plots don't look very good on smaller screens.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":31,"value":"toc()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":53,"value":"md`## A t-test is a statistical test used to determine if there is a significant difference between the means of two groups. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1679,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":431,"value":"md`## The one-sample t-test is used to compare the mean of a population with a theoretical value.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":438,"value":"md`The plot below shows two samples of data and their means. We want to compare the sample means with a hypothesised mean. For the sample on the left, we can see that there is a difference between its mean and the hypothesised mean. However, the sample data is so spread out; we can't easily tell if the difference is real. For the sample on the right, the data is much more tightly clustered, and the gap between the sample and the hypothesised mean is visually quite obvious. A t-test allows us to quantify the statistical significance of a difference between the means.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":419,"value":"viewof refresh_1 = button({ value: \"Resample\" })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":280,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":78,"value":"md`## The t-value can be interpreted as the difference between the means expressed as units of standard error.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":121,"value":"md`\nThe first step in a t-test is to calculate the t-value, a type of [inferential statistic](https://en.wikipedia.org/wiki/Statistical_inference).\n\n${tex.block`\nt=\\frac{\\overline{x}-\\mu_{0}}{s / \\sqrt{n}}`}\n\nwhere:\n- ${tex`\\overline{x}`} is the sample mean,\n- ${tex`\\mu_{0}`} is the hypothesised mean,\n- ${tex`s`} is the standard deviation,\n- ${tex`n`} is the sample size.\n\nFeeling a little overwhelmed by the above formula? Don't worry! It was the same for me. It might help if you interpret the t-value like this:\n\n${tex.block`\n\\text{t}\n=\\frac{\\text { signal }}{\\text { noise }}\n`}\n\nThe numerator, the signal, is the difference between the sample mean and the hypothesised mean. The denominator, the noise, is the standard error of the mean. Therefore, the t-value is simply the difference between the means expressed as units of standard error. We can infer that:\n\n- A large t-value indicates that the difference between the means is significantly different.\n- A small t-value suggests that the means are similar.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1477,"value":"md`## The shape of the t-distribution, and thus the probability of getting a particular t-value, depends on the number of degrees of freedom, ν.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":553,"value":"md`\nThe t-distribution is a probability distribution that is used to estimate population parameters when the sample size is small or when the population variance is unknown. If we took multiple random samples of data from the same populations and compared the sample means to the population mean, we'd get slightly different t-values each time due to random sampling error. Most of the time, we expect to get a t-value of zero. \n\nThe shape of the t-distribution, and thus the probability of getting a particular t-value, depends on the number of degrees of freedom, ν. For a one-sample t-test:\n\n${tex.block`\n\\nu = n - 1\n`}\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1932,"value":"md`The t-distribution is plotted below, alongside the normal z-distribution for comparison.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":524,"value":"viewof v_1 = setV()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":480,"value":"{\n const v = v_1;\n\n const { x, y: yt } = getPDF({ v, dist: \"t\" });\n const { _, y: yz } = getPDF({ v, dist: \"normal\" });\n console.log({ x, yt, yz });\n\n const data = [\n {\n name: \"t-distribution\",\n x: x,\n y: yt,\n mode: \"lines\"\n },\n {\n name: \"Normal z-distribution\",\n x: x,\n y: yz,\n mode: \"lines\"\n }\n ];\n\n var layout = {\n title: `t-distribution
degrees of freedom, ν = ${v}`,\n xaxis: {\n title: tText,\n range: [-5, 5]\n },\n yaxis: {\n title: probabilityDensityText,\n range: [0, 0.5]\n },\n width: width\n };\n\n var div = DOM.element(\"div\");\n Plotly.newPlot(div, data, layout);\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":840,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":198,"value":"md`## A hypothesis test uses sample data to determine whether to reject the null hypothesis.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":204,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":712,"value":"md`## Critical t-value approach: Compare the calculated t-value to the critical t-value for the hypothesis for a given degrees of freedom and significance level.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":858,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":879,"value":"viewof v_2 = setV()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":883,"value":"viewof a_2 = setA()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1130,"value":"viewof t_2 = setT()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":939,"value":"viewof tail_2 = setTail()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":862,"value":"{\n const v = v_2;\n const alpha = a_2;\n const tail = tail_2;\n const tValue = t_2;\n\n return getCriticalTValuePlot({v, alpha, tail, tValue})\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":703,"value":"md`If you don't have access to a computer, it's possible to look up t-values required for a given confidence level in a t-table as shown below. Why don't you try working out the critical t-value in the plot above using the t-table?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":696,"value":"html`

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1474,"value":"md`## P-value approach: Compare the P-value corresponding to the calculated t-value to the significance level.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":985,"value":"md`\nThe P-value approach is sort of the opposite of the critical t-value approach. \n\n1. We determine the probability, assuming the null hypothesis were true, of observing a more extreme test statistic in the direction of the alternative hypothesis than the one observed. This probability is called the P-value.\n\n2. We then compare the P-value to the significance level, α:\n - If P-value ≤ α, reject the null hypothesis in favour of the alternative hypothesis. \n - If P-value > α, do not reject the null hypothesis. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1004,"value":"viewof v_3 = setV()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1007,"value":"viewof a_3 = setA()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1017,"value":"viewof t_3 = setT()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1010,"value":"viewof tail_3 = setTail()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1000,"value":"{\n const v = v_3;\n const alpha = a_3;\n const tail = tail_3;\n const tValue = t_3;\n\n return getPValuePlot({ v, alpha, tail, tValue });\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1173,"value":"md`## T-test demo`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1635,"value":"md`Now that we're t-test experts, let's give it a go :)\n\nUse the inputs below to define a population, a sample size and a hypothesised mean to compare the sample to. Following the sample visualisation, there are further inputs to customise the t-test to be applied. You can then see the familiar t-distribution curve and the outcome of the t-test.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1192,"value":"viewof mu_4 = slider({\n min: -10,\n max: 10,\n value: 0,\n step: 1,\n description: muText\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1195,"value":"viewof sigma_4 = slider({\n min: 0,\n max: 10,\n value: 5,\n step: 1,\n description: sigmaText\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1182,"value":"viewof n_4 = slider({\n min: 3,\n max: 100,\n value: 10,\n step: 1,\n description: nText\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1204,"value":"viewof mu0_4 = slider({\n min: -10,\n max: 10,\n value: 5,\n step: 1,\n description: mu0Text\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1213,"value":"viewof refresh_4 = button({ value: \"Resample\" })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1177,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1220,"value":"viewof a_4 = setA()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1224,"value":"viewof tail_4 = setTail()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1244,"value":"viewof approach_4 = radio({\n options: [\n { label: \"Critical t-value approach\", value: \"t-value\" },\n { label: \"P-value approach\", value: \"P-value\" }\n ],\n value: \"P-value\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1349,"value":"{\n const n = n_4;\n const mu = mu_4;\n const mu0 = mu0_4;\n const sigma = sigma_4;\n const approach = approach_4;\n\n const v = n - 1;\n const alpha = a_4;\n const tail = tail_4;\n\n const mean = experiment.means[0];\n const s = experiment.sigmas[0];\n const tValue = getOneSampleTValue({ mean, mu0, s, n });\n console.log({ mean, s, tValue });\n\n if (approach == \"t-value\") {\n return getCriticalTValuePlot({ v, alpha, tail, tValue });\n } else {\n return getPValuePlot({ v, alpha, tail, tValue });\n }\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":160,"value":"md`## A t-test on a larger sample is more likely to achieve statistical significance.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1407,"value":"md`Take care when interpreting t-tests. If a t-test doesn't achieve statistical significance, it could be because:\n\n1. The difference (signal) isn't large enough. \n2. The variation (noise) within the samples is too great. \n3. The sample is too small. \n\nIt may be possible to reduce the noise or increase the sample size to get better statistical significance. On the other hand, an extremely large sample can produce statistically significant results even when the difference is very small and has no practical consequence. In other words, a t-test does not tell us whether the difference in means is relevant. That's up to us to determine based on our knowledge of the problem domain.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1398,"value":"md`## T-tests can result in Type I and Type II errors.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1403,"value":"md`\n**Type I**: Rejecting the null hypothesis when it is in fact true. A false positive.\n\n**Type II**: Not rejecting the null hypothesis when in fact the alternate hypothesis is true. A false negative.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1650,"value":"html`

H₀: Subject is not pregnant.
H₁: Subject is pregnant.

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1654,"value":"md`The t-test isn't foolproof. There are lots of ways it could go wrong. Bad luck, for example! You could also come to wrong conclusions if you are applying the t-test to data that violates the underlying assumptions of a t-test:\n\n1. Dependant variable follows a continuous or ordinal scale.\n2. Sample data is collected from a representative, randomly selected portion of the population.\n3. Sample data follows a bell-shaped distribution curve.\n4. Reasonably large sample size is used. \n5. Homogeneity of variance: the standard deviations of samples are approximately equal.\n6. No significant outliers.\n\nType II errors, in particular, are easily made if the sample size is too small. To minimise the probability of Type II errors, we can find out the sample size required using power analysis.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":21,"value":"md`## There are three main types of t-tests.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":19,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1486,"value":"md`## Conclusion`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1489,"value":"md`Phew! That took forever, huh? We've covered a lot of content, but hopefully, you can now appreciate that t-tests are pretty simple after all. Most things are simple if we take the time to understand them properly. If you're still struggling, go through it one more time, and maybe check out some of the references I used (linked in the section below). \n\nI'm also more than happy to answer any questions/comments you may have. In particular, if you have any tips for how I could make this content more concise/clear, I'd be very grateful!\n\nMost of all, I hope you enjoyed learning t-tests with me. :)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":45,"value":"md`## References`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":6,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":36,"value":"md`## Appendix`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":39,"value":"md`#### Imports`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":33,"value":"import {toc} from \"@bluprince13/toc/2\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":295,"value":"import {slider, button, radio} from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":8,"value":"stdlib = require(\"https://unpkg.com/@stdlib/stdlib@0.0.32/dist/stdlib-flat.min.js\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":10,"value":"ttest2 = stdlib.ttest2","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":238,"value":"math = require(\"https://cdnjs.cloudflare.com/ajax/libs/mathjs/5.4.0/math.js\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":253,"value":"d3 = require(\"d3@5\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":281,"value":"Plotly = require(\"https://cdn.plot.ly/plotly-latest.min.js\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":485,"value":"// https://github.com/stdlib-js/stdlib/tree/develop/lib/node_modules/%40stdlib/stats/base/dists/t/pdf\n\nt = stdlib.base.dists.t","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":731,"value":"// https://github.com/stdlib-js/stdlib/tree/develop/lib/node_modules/%40stdlib/stats/base/dists/normal/pdf\n\nnormal = stdlib.base.dists.normal","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":500,"value":"_ = require(\"underscore\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1496,"value":"html`\n

Styles - expand to see CSS

\n \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":345,"value":"md`#### Defaults`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":241,"value":"numberOfSamplesDefault = 10","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":244,"value":"nDefault = 10","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":247,"value":"muDefault = 0","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":259,"value":"mu0Default = 5","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":289,"value":"muMin = -10","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":292,"value":"muMax = 10","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":250,"value":"sigmaDefault = 5","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":318,"value":"sigmaMin = 0","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":322,"value":"sigmaMax = 10","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":316,"value":"range = [-sigmaMax, sigmaMax]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":269,"value":"xText = \"observations (x)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":369,"value":"meanText = \"sample mean (x̅)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":273,"value":"muText = \"pop mean (μ)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":359,"value":"mu0Text = `hypothesised mean (μ0)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":307,"value":"scoreText = \"score\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":311,"value":"sampleText = \"sample (i)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":525,"value":"degFreedomText = \"degrees of freedom\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":576,"value":"h0Tex = \"H_0\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":582,"value":"h1Tex = \"H_1\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":510,"value":"tText = \"t-value\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1185,"value":"nText = \"sample size (n)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1197,"value":"sigmaText = \"pop std dev (σ)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":507,"value":"probabilityDensityText = \"Probability Density Function (PDF)\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":886,"value":"sigLevelText = \"signficance level, α\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1905,"value":"largeFont = 20","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":348,"value":"md`#### Slider generators`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":286,"value":"setMu = () =>\n slider({\n min: muMin,\n max: muMax,\n value: muDefault,\n step: 0.1,\n description: muText\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1232,"value":"setTail = () =>\n radio({\n options: [\n { label: `H1: μ <> ${mu0Text}`, value: \"two-tailed\" },\n { label: \"H1: μ < μ0\", value: \"one-tailed-left\" },\n { label: \"H1: μ > μ0\", value: \"one-tailed-right\" }\n ],\n value: \"two-tailed\"\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1365,"value":"setA = () =>\n slider({\n min: 0.01,\n max: 0.5,\n value: 0.05,\n step: 0.01,\n description: sigLevelText\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1876,"value":"setV = () =>\n slider({\n min: 1,\n max: 20,\n value: 10,\n step: 1,\n description: degFreedomText\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1883,"value":"setT = () =>\n slider({\n min: -5,\n max: 5,\n value: 1,\n step: 0.01,\n description: tText\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":623,"value":"md`#### Helper functions`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":626,"value":"getTex = symbolTex => html`${tex`${symbolTex}`}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":492,"value":"getPDF = ({ v = nDefault, dist = \"t\", range = [-5, 5] }) => {\n var x = _.range(range[0], range[1] + 0.01, 0.01);\n\n let y;\n if (dist == \"t\") {\n y = x.map(x_value => t.pdf(x_value, v));\n } else if (dist == \"normal\") {\n y = x.map(x_value => normal.pdf(x_value, 0, 1));\n }\n return { x, y };\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1024,"value":"getPValue = ({ tValue, v = nDefault, tail = \"two-tailed\" }) => {\n const cdf = t.cdf(tValue, v);\n\n let pValue;\n if (tail == \"two-tailed\") {\n pValue = cdf > 0.5 ? (1 - cdf) * 2 : cdf * 2;\n } else if (tail == \"one-tailed-left\") {\n pValue = cdf;\n } else {\n pValue = 1 - cdf;\n }\n\n return pValue;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1339,"value":"getOneSampleTValue = ({ mean, mu0, s, n }) => {\n return (mean - mu0) / (s / Math.sqrt(n));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":351,"value":"md`#### Generate samples and plots\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":236,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1248,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1332,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1344,"value":"experiment = getExperiment({\n mu: mu_4,\n n: n_4,\n sigma: sigma_4,\n numberOfSamples: 1\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"}],"files":"$W3d","stdlib":"1","titling":"auto"}} 31:"$30:body"