3e:null 3f:["$","title",null,{"children":"Can you trust sample data? Part 1 | Jamey Mccabe | Observable"}] 45:T42a,One thing you can do to get a better feel for a full population is to change the size using the radio button above. **40** is a fairly small population and not typical of the data we will usually look at. (although it was about all NASA had to access risk of O-ring failure). As another example the voting in 2022 Georgia Runoff for Senator (Warnock vs. Walker) the # of votes ended up being 3.5 Million. This siulation limits at most **40,000** as this is plenty to simulate a large census and won't slow down the page. ### Measures of Distribution The below histogram illustrates the full population again but with a few statistics added, - mean (the center of all the points) - 1.96 times Standard Deviation which for a normally distributed data set means that inside these limits is 95% of the data. Conversely, a value outside these limits is less than 5% likely. - 2.5% and 97.5% ranges of the data. For a small population there is often only 1 data point outside and it's not same as the 95% range, but for a large population they should be the same. 46:T629,Plot.plot({ y: { grid: true, ticks: 3}, marks: [ Plot.rectY(fullpopulation_array, Plot.binX({y: "count"},{fill: "grey",fillOpacity: .5,thresholds: 65})), //add mean value arrow and text Plot.arrow("[1]", {x1: mean, y1: full_pop_size/18+2, x2: mean, y2: 0, bend: false,stroke: "green"}), Plot.text([[70, "70"]], {x: mean, y: full_pop_size/18+2, text: ["mean = " + mean.toFixed(2)], fill: "green", dy: -6}), //add 2x std dev arrows and text Plot.arrow("[1]", {x1: mean+2*std_dev, y1: full_pop_size/40+1, x2: mean+2*std_dev, y2: 0, bend: false,stroke: "green"}), Plot.text([[70, "70"]], {x: mean+2*std_dev, y: full_pop_size/40+1, text: ["1.96 x std dev \n(97.5th%) = " + (mean+1.96*std_dev).toFixed(2)], fill: "green", dy: 6}), Plot.arrow("[1]", {x1: mean-2*std_dev, y1: full_pop_size/40+1, x2: mean-2*std_dev, y2: 0, bend: false,stroke: "green"}), Plot.text([[70, "70"]], {x: mean-2*std_dev, y: full_pop_size/40+1, text: ["1.96 x std dev \n(2.5th%) = -" + ((mean-1.96*std_dev).toFixed(2))], fill: "green", dy: 6}), //add 95th population range as arrow Plot.arrow("[2]", {x1: ten_pct, y1: (full_pop_size/18+2)*.9, x2: ninety_pct, y2: (full_pop_size/18+2)*.9 ,strokeWidth: 2,fill: "green",stroke:"green"}), Plot.text([[1]], {x: ninety_pct-4, y: (full_pop_size/18+2)*.9, text: ["95% range of full population =\n"+ten_pct+" to "+ninety_pct], dy: 8,fill: "green"}) ] })47:T78a,Plot.plot({ y: { grid: true, ticks: 3, }, color: { legend: false }, marks: [ Plot.rectY(sample, Plot.binX({y: "count"},{fill: "grey",fillOpacity: .5,thresholds: 60}),), //add mean value arrow and text Plot.arrow("[1]", {x1: sm, y1: 4, x2: sm, y2: 0, bend: false}), Plot.text([[70, "70"]], {x: sm, y: 4, text: ["sample mean =\n" + sm.toFixed(2)], fill: "black", dy: 8}), //add sample mean CI for pop mean Plot.arrow("[2]", {x1: sam_95bot, y1: 4.5, x2: sam_95top, y2: 4.5,strokeWidth: 2,fill: "green",stroke:"black"}), Plot.text([[1]], {x: sam_95top-4, y: 5, text: ["95% CI for pop mean =\n"+sam_95bot+" to "+sam_95top], dy: 8}), //Repeat from main graph in Green //add mean value arrow and text Plot.arrow("[1]", {x1: mean, y1: 7, x2: mean, y2: 0, bend: false,stroke: "green"}), Plot.text([[70, "70"]], {x: mean, y: 5.5, text: ["population\nmean = " + mean.toFixed(2)], fill: "green", dy: 8}), //add 2x std dev arrows and text Plot.arrow("[1]", {x1: mean+2*std_dev, y1: 5, x2: mean+2*std_dev, y2: 4, bend: false,stroke: "green"}), Plot.text([[70, "70"]], {x: mean+2*std_dev, y: 4, text: ["2 x std dev \n(97.5th%) = " + (mean+2*std_dev).toFixed(2)], fill: "green", dy: 6}), Plot.arrow("[1]", {x1: mean-2*std_dev, y1: 5, x2: mean-2*std_dev, y2: 4, bend: false,stroke: "green"}), Plot.text([[70, "70"]], {x: mean-2*std_dev, y: 4, text: ["2 x std dev \n(2.5th%) = -" + ((-mean-2*std_dev).toFixed(2))], fill: "green", dy: 6}), //add 95th population range as arrow Plot.arrow("[2]", {x1: ten_pct, y1: 5.5, x2: ninety_pct, y2: 5.5,strokeWidth: 2,fill: "green",stroke:"green"}), Plot.text([[1]], {x: ninety_pct-4, y: 6, text: ["95% range of full population =\n"+ten_pct+" to "+ninety_pct], dy: 8,fill: "green"}) ] })48:T6ad,Some notes about the above: - In green for comparison and carried down from the full population is the mean of close to 0 and 2xStd Deviation of close to 20. - The green vertical arrow is showing the 95% **[confidence interval (CI) for the full population mean](https://en.wikipedia.org/wiki/Confidence_interval#Summary_statistics)**. This is calculated using the samples standard deviation reduced by the size of the sample. Equation for how big is this interval is ${tex`\bar{x}\pm t\times s/sqrt(n)`} in a sentence form: sample mean plus or minus **[t statistic](https://en.wikipedia.org/wiki/Student's_t-distribution#)** multiplied by the sample standard deviation and divided by the square root of the count of sample data. - In the Confidence Interval, look for: -- how close this interval comes to the population mean. In the strictest sense it just means that from this sample there is a 95% confidence that the population mean is in this range. When confidence (a.k.a. uncertainty) is cited in a news article, this is often what they are citing. If you sample 20 times you should find a case where the CI doesn't include the mean. If we wanted to be more sure of the range we could use a higher t statistic. e.g. a 99% CI would use a t statistic of -- how big and different range is the interval on resampling -- how the interval shrinks with a larger sample - Note how close the sample Median is to the full population. Typical data results cite this number and often don't give the confidence interval numbers at all. - Note how often sample data is outside the 95% range of the full population. These are called outliers. They get more apparent in the boxplots in the next graph.49:T4bb,// program to pull a database made for facets of given number of random items without replacement from an array function getRandomItemsFaceted(arr,num_items=1, num_facets=1) { let result_string = 'facet_id,num_from_fullpop, seq_in_fullpop,\n' //start by adding all the original data so it can be a facet also // for (let index = 0; index < arr.length; ++index) { // result_string = result_string.concat('full pop',",",arr[index],",",index,"\n"); } //loop through the requested facets for (let i = 0; i < num_facets; i++) { //setup without replacement check array let check_array = [] //Loop until we've pulled the requested num of items while (check_array.length < num_items) { // get random index value const randomIndex = Math.floor(Math.random() * arr.length); if (check_array.includes(randomIndex)) { continue; } else {check_array.push(randomIndex);} // add random item to the result result_string = result_string.concat(i,",",arr[randomIndex],",",randomIndex,"\n"); }} let sample_facets = d3.csvParse(result_string.slice(0,result_string.length-1)) return result_string; }4a:[{"id":"715db1223e067f00500780077febc6cebbdd90c151d3d78317c802732252052ab0e367039872ab9c77d6ef99e5f55a0724b35ddc898a1c99cb14c31a379af80a","href":"https://static.observableusercontent.com/files/715db1223e067f00500780077febc6cebbdd90c151d3d78317c802732252052ab0e367039872ab9c77d6ef99e5f55a0724b35ddc898a1c99cb14c31a379af80a","name":"penguins.csv","type":"text/csv","lastModified":1681830137085.837,"size":13506},{"id":"55dd27656202f92708f3d08a5e8c99bbae5f0f0ee1ce8089891aa5160cc3e606992a71f42d8c4f152f12bd5029031ec4853d19d8a93dcc1e991383f5672160ac","href":"https://static.observableusercontent.com/files/55dd27656202f92708f3d08a5e8c99bbae5f0f0ee1ce8089891aa5160cc3e606992a71f42d8c4f152f12bd5029031ec4853d19d8a93dcc1e991383f5672160ac","name":"1EVy1JZ_BTes3WWB0XgwfvhjtI-P4oiD_","type":"application/octet-stream","lastModified":1682708668875.999,"size":2021},{"id":"591f5ee2dedc6f1dbe14499759620ca9f8e410857bbae746b2ad3845773b5ad0b3d249fb56dc47a391b87a949e8afeb7d8bae1a3b190f3a6eb02646f5dfc5c06","href":"https://static.observableusercontent.com/files/591f5ee2dedc6f1dbe14499759620ca9f8e410857bbae746b2ad3845773b5ad0b3d249fb56dc47a391b87a949e8afeb7d8bae1a3b190f3a6eb02646f5dfc5c06","name":"T-stat-1leg.csv","type":"text/csv","lastModified":1682708975068.615,"size":2021}] 43:{"id":"f7c213647fe6a84e","slug":null,"title":"Can you trust sample data? Part 1","thumbnail":"3b1eb2a78ba0997754eb0195d104719f2bf2588c3cad6f0d470c983ecd8ac6a7","access_level":"public","create_time":"$D2023-04-18T15:02:17.085Z","update_time":"$D2023-06-27T19:50:48.523Z","view_time":null,"first_public_time":"$D2023-04-18T15:02:17.085Z","has_unpublished_edits":null,"fork":{"id":"7d2e8de76aa8294f","slug":null,"title":"Plot Histogram","owner":{"id":"853ac1670986a907","login":"jamemcc","name":"Jamey Mccabe"}},"stars":0,"starred":false,"owner":{"type":"team","id":"853ac1670986a907","login":"jamemcc","name":"Jamey Mccabe","avatar_url":"https://avatars-next.observableusercontent.com/f71fb64d089523b5722569ad18d3fcd17e4f10296d0750b0979baa77eb206d22","tier":"starter_2024","bio":"","following":false},"version":1467,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"75084426f534713f","login":"jamemcc","name":"Jamey Mccabe","avatar_url":"https://avatars-next.observableusercontent.com/f71fb64d089523b5722569ad18d3fcd17e4f10296d0750b0979baa77eb206d22","tier":null,"bio":"","following":false}],"members":[{"type":"individual","id":"75084426f534713f","login":"jamemcc","name":"Jamey Mccabe","avatar_url":"https://avatars-next.observableusercontent.com/f71fb64d089523b5722569ad18d3fcd17e4f10296d0750b0979baa77eb206d22","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Can you trust sample data? Part 1","cells":[{"id":4,"value":"# Can you trust sample data? Part 1 \nWalkthru Census vs Sample, Standard deviations of the population, CI for estimating Population Mean as a way of thinking about data. \n \nThough if you are over these please jump forward to the [Part 2: census and sample generator](https://observablehq.com/d/f0779561f39f2b45).\n\nCredit to and your invitation to try [Plot.bin](https://observablehq.com/@observablehq/plot-bin) and [Plot.rect](https://observablehq.com/@observablehq/plot-rect).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":505,"value":"Lets start by creating a full data population also called a census.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":508,"value":"### This is a full population - shown as a histogram\n(all the data values in the dataset counted) ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":510,"value":"The data was randomly generated when you opened this page or change the number above. The routine is below in the Setup and Appendix sections. This generated dataset has ${full_pop_size} total data points. The values range from 0 to 20 which could be thought of as the number of times when you flip a coin 20 times you get heads.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":5,"value":"\nPlot.plot({\n y: { grid: true, ticks: 3}, \n marks: [\n Plot.rectY(fullpopulation_array, Plot.binX({y: \"count\"},\n {fill: \"grey\",fillOpacity: .5,thresholds: 70}))\n ]\n})\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":854,"value":"In the above you should see the count of the values spread in a sort of bell shape:\n- There are relatively few data-points in the far left or far right\n- but there are more for the numbers near the center.\n \nIn fact to be more clear - here's all the ${full_pop_size} numbers themselves sorted from low to high","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":868,"value":"fullpop_table = fullpopulation_array","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"fullpop_table","database":"$undefined"},{"id":293,"value":"viewof full_pop_size = Inputs.radio([40, 1000,40000], {label: \"Enter full population size default 40\", value: 40})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":876,"value":"$45","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":776,"value":"$46","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":515,"value":"## Lets play with sampling\nThe meat of the exercise is to now pull random subsets from the full population and in fact it's been done once already below as this page was opened.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":37,"value":"viewof sample_size = Inputs.radio(sample_size_avail,{label: \"How many samples do you want\", value: 10});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1008,"value":"viewof button = Inputs.button(\"Resample\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":48,"value":"$47","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":700,"value":"$48","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1059,"value":"## Conclusion\nThat's the walkthough of sampling - feel free to play more in [Part 2: census and sample generator](https://observablehq.com/d/f0779561f39f2b45) - which is less of a story and more just a playground.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":705,"value":"# ignore all the sections below if you aren't interested in changing the stuff above","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":444,"value":"## Setup the data - Full Population","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1234,"value":"### Different Population Distributions\nWe've used the normal distribution for the population and from which we draw samples to keep it simple but statistics has many different distributions and starting with a different one maybe more helpful depending on the subject you are trying to get a feel for. There is one other I am finding usefull to get a feel for when there are 2 outcomes e.g. voting on the U.S. president or senators. \n#### [Bernoulli Distribution](https://en.wikipedia.org/wiki/Bernoulli_distribution)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1265,"value":"viewof desired_distr = Inputs.radio([\"Normal\", \"Bernoulli\"], {label: \"Select some\", value: \"Normal\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1290,"value":"Next two values can be changed for Bernoulli distribution","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1284,"value":"viewof trials = Inputs.range([30,2000],{label: \"Count of trials (e.g. people polled, # in medical trial)\", \n value: 500, step:10});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1294,"value":"viewof normal = Inputs.range([1,2000],{label: \"Count of normal situation (e.g. half trial in an even vote, # who received placebo in medical)\", value: 250, step:1})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":31,"value":"fullpopulation_array = {\n let pop = []\n if (desired_distr == \"Normal\"){\n //randomly generated gaussian normal rounded to integer\n pop = Array.from({length: full_pop_size}, () => Math.round(gaussianRandom(9.5,3)));}\n else {\n //next is to generate multiple trial results using bernoulli distribution\n //usefull for evaluating 2 outcome trial results rounded to integer\n pop = Array.from({length: trials}, () => nextBinomial(normal/trials,Number(trials)))}\n return pop;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":238,"value":"mean = sstat.mean(fullpopulation_array)\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":269,"value":"std_dev = sstat.standardDeviation(fullpopulation_array)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":351,"value":"ninety_pct = sstat.quantile(fullpopulation_array,.975)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1300,"value":"ninety_pctage = ninety_pct/trials","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":355,"value":"ten_pct = sstat.quantile(fullpopulation_array,.025)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1303,"value":"ten_pctage = (ten_pct/trials).toFixed(2)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":452,"value":"## Setup the Sample Data and Statistics","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1431,"value":"// routine to limit available sample sizes to those less than the population size\nfunction sample_size_limiter(sample_size_alternates) {\n let sample_size_avail=[]\n for (let i of sample_size_alternates) {\n if (i < full_pop_size){ sample_size_avail.push(i)}\n }\n return sample_size_avail\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1428,"value":"sample_size_avail = sample_size_limiter([10,25,70,200,5000])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":61,"value":"sample = {\n button;\n let sample = getRandomItem(fullpopulation_array,sample_size); \n return sample;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":323,"value":"sm = sstat.mean(sample)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":326,"value":"sample_std_dev = sstat.sampleStandardDeviation(sample)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1067,"value":"sample_95thCI = 1.96 * sample_std_dev","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1222,"value":"returns the student t test value depending on sample size of ${sample_size} and if it's 30 or less in which case we use a t-statistic to be used in calculating confidence interval \nsee https://en.wikipedia.org/wiki/Student's_t-distribution#","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1344,"value":"tStat1leg = await FileAttachment(\"T-stat-1leg.csv\").csv({typed: \"auto\"})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"tStat1leg","database":"$undefined"},{"id":1347,"value":"SELECT \".975\" FROM \"T-stat-1leg\" where DF = ${sample_size-1};","pinned":true,"hidden":false,"mode":"sql","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"t_stat_lookup","database":"T-stat-1leg.csv"},{"id":1204,"value":"cut_point = {\n let cut = 1.96;\n if(sample_size<30){\n //next 2 lines to workaround database result being sql table object\n let cut2 = (t_stat_lookup[0]+\"\")\n cut = Number(cut2.slice(cut2.search(\":\")+2,cut2.length -1));\n }\n return cut;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1074,"value":"sam_95top = parseFloat((sm + cut_point * sample_std_dev/Math.sqrt(sample_size)).toFixed(1))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1078,"value":"sam_95bot = parseFloat((sm - cut_point * sample_std_dev/Math.sqrt(sample_size)).toFixed(1))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":372,"value":"samp_top_pct = sstat.quantile(sample,.95)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":376,"value":"samp_bottom_pct = sstat.quantile(sample,.05)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1052,"value":"the below table- sample faceted was developed to produce and then graph multiple samples at the same time - but graphin using boxX doesn't seem to work accurately, leaving for hopeful later use after debugging","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":595,"value":"sample_faceted = d3.csvParse(getRandomItemsFaceted(fullpopulation_array,sample_size,5))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":994,"value":"sample_faceted","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":146,"value":"## Appendix - Libraries for Math, Statistics and graphing","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":157,"value":"#### mathjs provides a range of mathematical functions and constants`","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":160,"value":"mathjs = require(\"https://cdnjs.cloudflare.com/ajax/libs/mathjs/5.4.0/math.js\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":149,"value":"#### jStat provides a wide range statistical capabilities","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":155,"value":"sstat = require(\"https://unpkg.com/simple-statistics@7.8.3/dist/simple-statistics.min\");","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1200,"value":"jstat = require(\"//cdn.jsdelivr.net/npm/jstat@latest/dist/jstat.min.js\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":180,"value":"#### Custom coded function to generate random numbers from the normal (Gaussian) distribution","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":23,"value":"// Standard Normal variate using Box-Muller transform.\nfunction gaussianRandom(mean=0, stdev=1) {\n let u = 1 - Math.random(); // Converting [0,1) to (0,1]\n let v = Math.random();\n let z = Math.sqrt( -2.0 * Math.log( u ) ) * Math.cos( 2.0 * Math.PI * v );\n // Transform to the desired mean and standard deviation:\n return z * stdev + mean;\n // next line used to substitutue to algorithm above to check results using different one\n // return ((Math.random() + Math.random() + Math.random() + Math.random() + Math.random() + Math.random()) - 3) / 3;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":182,"value":"#### Custom coded function to pull a array of random entries from a larger array","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":43,"value":"// program to pull a given number of random items without replacement from an array\nfunction getRandomItem(arr,num_items=1) {\n\n //setup without replacement check array\n const check_array = []\n const result_array = []\n\n //Loop until we've pulled the requested num of items\n while (check_array.length < num_items) {\n // get random index value\n const randomIndex = Math.floor(Math.random() * arr.length);\n if (check_array.includes(randomIndex)) { continue; } else {check_array.push(randomIndex);}\n // get random item\n result_array.push(arr[randomIndex]);\n }\n return result_array;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":588,"value":"$49","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":565,"value":"#### Custom function to generate data from binomial distribution","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":544,"value":"// credit to http://nodegame.github.io/JSUS/docs/lib/random.js.html\nnextBinomial = function(p, trials) {\n var counter, sum;\n Number(trials);\n if ('number' !== typeof p) {\n throw new TypeError('nextBinomial: p must be number.');\n }\n if ('number' !== typeof trials) {\n throw new TypeError('nextBinomial: trials must be number.');\n }\n if (p < 0 || p > 1) {\n throw new TypeError('nextBinomial: p must between 0 and 1.');\n }\n if (trials < 1) {\n throw new TypeError('nextBinomial: trials must be greater than 0.');\n }\n counter = 0;\n sum = 0;\n while(counter < trials){\n if (Math.random() < p) {\n sum += 1;\n }\n counter++;\n }\n return sum;\n }","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":681,"value":"import {Button} from \"@observablehq/inputs\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":806,"value":"// Import the function into your notebook\nimport { SummaryTable } from \"@observablehq/summary-table\"\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":858,"value":"import {Icicle} from \"@d3/icicle\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W4a","stdlib":"1"}} 44:"$43:body"