3e:null 3f:["$","title",null,{"children":"Project 2: Data Mapping - Process Doc | IXD Sheridan | Observable"}] 45:T5de,When searching for a suitable dataset for my data visualization practice, I often come across statistical data that can be confusing and difficult for people without a background in statistics to understand or apply. One of the most common statistical concepts is the Confidence Interval (CI). My project aims to introduce designers to what a CI is and how we can use it.
To do this, I use the example of "daily mood rates" to introduce the normal distribution, mean, standard deviation, and other statistical concepts related to the CI. I also use the [Pachinko Simulator](https://observablehq.com/@mbostock/pachinko-simulator) model to illustrate the concept of the normal distribution. Based on this, I use d3.js to generate random normal distribution mood value verses, introducing the concepts of CI and confidence level in a simple and practical way to the designer audience.
To help the audience better understand the actual applications of the CI, I also use sleep survey data from 104 Americans to create a confidence interval chart, explaining the relationship between the error of sample mean and population mean and the relationship between CI and confidence level.
As someone without professional statistical knowledge, all my understanding of CI comes from basic statistic learning years ago and Google searches, so there may be mistakes in my project.
Project Link: [How do designers understand CI (Confidence Intervals)?](https://observablehq.com/d/3a3de54c0c0889d8)46:T465,Plot.plot({ marginBottom:60, height:400, // inset:80, width: 1000, x: { tickRotate: -45, ticks: 5, tickSize: 2, line: true, tickPadding: 7, labelAnchor: "left", labelOffset: 17, // nice: true, label: "Date", tickFormat: d3.utcFormat("%b,%Y"), // insetLeft:15, }, color: { type: "diverging", scheme: "burd", pivot: moodValueMean, legend: true, domain: [Math.min(...randomMoodValue), Math.max(...randomMoodValue)], range: [0, 1], // reverse:true, }, marks: [ Plot.ruleY([0,100]), // Plot.ruleX([2022-10-01]), Plot.areaY(finalData,{ //CI Area //The mood rate has a range of [0,100] x: "date", y1: d => ((moodValueMean - CI_Level * sigma) < 0 ) ? 0 : moodValueMean - CI_Level * sigma, y2: d => ((moodValueMean + CI_Level * sigma) > 100 ) ? 100 : moodValueMean + CI_Level * sigma, fill:"green", fillOpacity: 0.05, }), Plot.dot(finalData, { x: "formattedDate", y: "moodValue", fill: "moodValue" }), Plot.line(finalData,{x:"date", y:moodValueMean,stroke:"green"}), ], }) 47:T41f,Plot.plot({ marginBottom:20, height:400, // inset:80, width: 1000, x: { domain: finalData.map(d => d.date), axis:null, }, color: { type: "diverging", scheme: "burd", pivot: moodValueMean, legend: true, domain: [Math.min(...randomMoodValue), Math.max(...randomMoodValue)], range: [0, 1], // reverse:true, }, marks: [ Plot.ruleY([0,100]), Plot.areaY(finalData,{ //CI Area //The mood rate has a range of [0,100] x: "date", y1: d => ((moodValueMean - CI_Level * sigma) < 0 ) ? 0 : moodValueMean - CI_Level * sigma, y2: d => ((moodValueMean + CI_Level * sigma) > 100 ) ? 100 : moodValueMean + CI_Level * sigma, fill:"green", fillOpacity: 0.05, }), Plot.dot(finalData, { x: "date", y: "moodValue", fill: "moodValue" }), Plot.line(finalData,{x:"date", y:moodValueMean,stroke:"green"}), Plot.text([transCI(CI_Level)], { fontSize: 12, y: 90, fill:"green", // stroke:"#000", } ), ], })48:T4e4,Plot.plot({ marginBottom:20, height:400, // inset:80, width: 1000, x: { // tickRotate: -45, ticks: 10, tickSize: 2, line: true, // tickPadding: 7, labelAnchor: "right", labelOffset: 32, nice: true, label: "Participant No.", }, color: { type: "diverging", scheme: "purples", pivot: sleepMean, legend: true, // domain: [Math.min(...sleepHoursFilt), Math.max(...sleepHoursFilt)], domain: [0, Math.max(...sleepHoursFilt)], // range: [0, 1], // reverse:true, }, marks: [ Plot.ruleX([0,110]), Plot.ruleY([0,15]), Plot.areaY(sleepData,{ //CI Area //Sleep Hours can't less than 0 hour y1: d => ((sleepMean - CI_Level2 * sleepSigma) < 0 ) ? 0:sleepMean - CI_Level2 * sleepSigma, y2: sleepMean + CI_Level2 * sleepSigma, x: "participant", fill:"green", fillOpacity: 0.04, }), Plot.dot(sleepData, { y: "sleepHours", x: "participant", fill: "sleepHours" }), Plot.line(sleepData,{y:sleepMean, x:"participant",stroke:"green"}), Plot.text([transCI(CI_Level2)], { fontSize: 12, y: 13, x: 55, fill:"green", // stroke:"#000", } ), ], })49:T550,[Standard error of the mean](https://observablehq.com/@bluprince13/standard-error-of-the-mean)
[What exactly is a confidence interval?](https://www.scribbr.com/statistics/confidence-interval/#:~:text=about%20confidence%20intervals-,What%20exactly%20is%20a%20confidence%20interval%3F,a%20certain%20level%20of%20confidence.)
[Mean Estimation & Confidence Interval](https://observablehq.com/@pabloruizcuevas/mean-estimation-confidence-interval)
[Plot Cheatsheets](https://observablehq.com/@observablehq/plot-cheatsheets?collection=@observablehq/plot-cheatsheets)
[D3.js randomNormal() Function](https://www.geeksforgeeks.org/d3-js-randomnormal-function/)
[d3-random](https://observablehq.com/@d3/d3-random#normal)
[Elegant method to generate array of random dates within two dates](https://stackoverflow.com/questions/9035627/elegant-method-to-generate-array-of-random-dates-within-two-dates)
[Foster Cats - Length of Stay (Gantt Chart)](https://observablehq.com/d/b7d93beb0e1de80a)
[Toronto Cat Rescue - Adoption Stats](https://observablehq.com/d/a8ce860535ea6c06)
[D3.js | d3.deviation() function](https://www.geeksforgeeks.org/d3-js-d3-deviation-function/)
[How do I create JavaScript array (JSON format) dynamically?](https://stackoverflow.com/questions/2250953/how-do-i-create-javascript-array-json-format-dynamically)4a:[{"id":"890f45ecd3de2ab32ba7a74b41aa9e06a43721a9cd97176b8d7475a8cb00929974b520156c306d76973f07e54e2d2b7ac971f5c0703b227dc6e0da666f7fc4a0","href":"https://static.observableusercontent.com/files/890f45ecd3de2ab32ba7a74b41aa9e06a43721a9cd97176b8d7475a8cb00929974b520156c306d76973f07e54e2d2b7ac971f5c0703b227dc6e0da666f7fc4a0","name":"Screen Shot1.png","type":"image/png","lastModified":1677198133981.844,"size":77067},{"id":"96945497bfe3479b195611d287be9696a8eff74856a21c9fd691c39288c1c326bdc129dc12bfdc023171edcd3d52714d3c4bb23a134d53a5b665dac51ba0748e","href":"https://static.observableusercontent.com/files/96945497bfe3479b195611d287be9696a8eff74856a21c9fd691c39288c1c326bdc129dc12bfdc023171edcd3d52714d3c4bb23a134d53a5b665dac51ba0748e","name":"Screen Shot2.png","type":"image/png","lastModified":1677198133981.844,"size":143357},{"id":"f3cba51aa5e3ecee121fb0a3a58b3e6973ea37e59dcb9fd2a7b7bc033ee695eb8436b7f6f2fad288de6e71b191b31a31f89bfb2593f9041757d4679f6b2150d8","href":"https://static.observableusercontent.com/files/f3cba51aa5e3ecee121fb0a3a58b3e6973ea37e59dcb9fd2a7b7bc033ee695eb8436b7f6f2fad288de6e71b191b31a31f89bfb2593f9041757d4679f6b2150d8","name":"Screen Shot3.png","type":"image/png","lastModified":1677198133981.844,"size":430303},{"id":"5e7a5723e1df14100d73a6112cc94f611912639b6dbc1f386ce9a0640ca0f26d12e125eed7cfa68ed81fd56adbfbb23430977d210500c6488635caabf6539ee0","href":"https://static.observableusercontent.com/files/5e7a5723e1df14100d73a6112cc94f611912639b6dbc1f386ce9a0640ca0f26d12e125eed7cfa68ed81fd56adbfbb23430977d210500c6488635caabf6539ee0","name":"SleepStudyData.csv","type":"text/csv","lastModified":1677198133981.844,"size":2395}] 43:{"id":"e867d5605b3aef74","slug":null,"title":"Project 2: Data Mapping - Process Doc","thumbnail":"af50ff0c455751621cdf9f574ac24aa4cf76a5b893904fbd5772ea6b43920111","access_level":"public","create_time":"$D2023-02-24T00:22:13.981Z","update_time":"$D2023-09-12T14:48:01.360Z","view_time":null,"first_public_time":"$D2023-05-05T22:58:57.477Z","has_unpublished_edits":null,"fork":{"id":"ad8751d8b5691ed3","slug":null,"title":"","owner":{"id":"fe4dd134fc47e319","login":"dairuixin","name":"Ruixin Dai"}},"stars":0,"starred":false,"owner":{"type":"team","id":"0a8b950203491c0f","login":"ixd-sheridan","name":"IXD Sheridan","avatar_url":"https://avatars-next.observableusercontent.com/5d0881196900a7350c64d546dd5099a2532e7060f4150f97c1375dae8cb92d45","tier":"education_2026","bio":"Interaction Design at Sheridan College","following":false},"version":394,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"61fb8670c873e632","login":"dairuixin","name":"Ruixin Dai","avatar_url":"https://avatars-next.observableusercontent.com/abe1016db008932c58b2ed6fffea97a811984fa8b57de8558da2ec6caf3471b5","tier":null,"bio":"","following":false}],"members":[{"type":"individual","id":"61fb8670c873e632","login":"dairuixin","name":"Ruixin Dai","avatar_url":"https://avatars-next.observableusercontent.com/abe1016db008932c58b2ed6fffea97a811984fa8b57de8558da2ec6caf3471b5","tier":null,"role":"editor"},{"type":"team","id":"0a8b950203491c0f","login":"ixd-sheridan","name":"IXD Sheridan","avatar_url":"https://avatars-next.observableusercontent.com/5d0881196900a7350c64d546dd5099a2532e7060f4150f97c1375dae8cb92d45","tier":"education_2026","role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Project 2: Data Mapping - Process Doc","cells":[{"id":0,"value":"# Project 2: Data Mapping - Process Doc","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":5,"value":"### VDES39915 Visualizing Information: Dynamic Data Models","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":10,"value":"Ruixin Dai","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":12,"value":"Winter 2023","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":14,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":16,"value":"## Design Statement","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":19,"value":"$45","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":21,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":228,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":23,"value":"## Statistic Concept Research","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":48,"value":"I found some great notebooks that explain the standard deviation with confidence intervals. [Mean Estimation & Confidence Interval](https://observablehq.com/@pabloruizcuevas/mean-estimation-confidence-interval) created by Pablo Ruiz talked in detail about the significance of the sample mean and the overall mean in the confidence interval. The audience can even keep increasing the sample size to observe the change in standard deviation.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":54,"value":"screenShot20230223At125155 = FileAttachment(\"Screen Shot1.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":57,"value":"[Standard error of the mean](https://observablehq.com/@bluprince13/standard-error-of-the-mean) created by bluprince13 instead uses numerous graphs and formulas to explain in depth the estimation between the sample and the population and the explanation of the various statistical concepts involved.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":61,"value":"screenShot2 = FileAttachment(\"Screen Shot2.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":65,"value":"These notebooks allowed me to learn the basic concepts of statistics related to CI. Their works are quite professional. Trying to use their formulas directly or apply the data to theirs charts could be a super difficult task. I want to make these concepts as simple as possible by leaving behind the complex formulas and operations and just using narratives and charts to talk about CI concepts.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":25,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":27,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":30,"value":"## Dataset Generation","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":68,"value":"I would like to start with the word \"Confidence\" to show the relationship between statistics and everyday life. For example, \"How confident are you today?\" Or \"How are you feeling today?\" Quantifying moods and feelings, recording and analyzing these data might build a good dataset for people to understand statistic easily.

\nI believe that people's mood values over time should be normally distributed. Everyone has happy and sad times, but there should be an average value that represents the mood over a spercific period of time.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":75,"value":"Building a narrative is a good way to get the audience to follow my lead and understand how to use mood values to calculate confidence intervals. This means that I would not have a dataset to use for the calculation, I had to generate data. This data could be random, but it had to fit a normal distribution.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":73,"value":"I found [Mike Bostock's Pachinko Simulator](https://observablehq.com/@mbostock/pachinko-simulator) data model while searching for a way to generate random normally distributed data using d3.randomNormal() method. D3's own [examples of random data](https://observablehq.com/@d3/d3-random#normal) all use his stacked blob effect code.\n\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":83,"value":"ScreenShot3 = FileAttachment(\"Screen Shot3.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":79,"value":"I learned how to control the sample size, mean and standard deviation to interact with the chart. And I also changed the color and the size of the bulbs. I originally wanted to implement the effect of the little ball dropping from above that the author used in his Notebook. I didn't see how the chart was implemented though.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":117,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":100,"value":"viewof n = Inputs.range([30, 365], {value:100,step: 1, label: \"Sample Size\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":88,"value":"viewof sigma = Inputs.range([0, 15], {step: 1, label: \"Sigma σ\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":107,"value":"viewof moodValueMean = Inputs.range([0, 100], {step: 1, label: \"Sample Mean\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":114,"value":"pachinko(d3.randomNormal(moodValueMean,sigma),[0,100]); \n// Learn more at https://observablehq.com/@mbostock/pachinko-simulator","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"Since the data is generated, both the mean and standard deviation need to be given to use d3.randomNormal(). I have a little concern here that the audience will not understand the order of the data analysis, so I also used another set of given data (sleep duration analysis) to calculate the mean and standard deviation.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":125,"value":"// randomMoodValue = [d3.randomNormal(moodValue,sigma)()]; // Generate a single number\nrandomMoodValue = Array.from({length: n}, () => Math.floor(d3.randomNormal(moodValueMean,sigma)() )); //Generate an array of number","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":130,"value":"Here I have a great difficulty that building an x-axis for my chart. It is difficult to make a chart with only one variable. Of course I can also divide the mood values of [0, 100] into intervals by counting how many records there are in each interval.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":132,"value":"Plot.plot({\n height:700,\n weight:800,\n x:{\n axis: \"bottom\",\n grid: true,\n ticks: 10,\n }, \n marks: [\n Plot.ruleX([Math.min(...randomMoodValue)-5,Math.max(...randomMoodValue)+5]),\n Plot.ruleY([0,n/2.5]),\n Plot.rectY(\n randomMoodValue, \n Plot.binX(\n { y: \"count\" }, \n {\n fill: \"blue\", \n fillOpacity: 0.5, \n thresholds: 10,\n }\n )\n )\n ], \n marginLeft: 100, \n height: 200,\n width: 666\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":164,"value":"This normal distribution is similar to Pachinko Simulator, but intuitively it expresses the normal distribution of \"frequencies\" rather than the data itself. I wanted to use a scatter plot like [bluprince13's charts](https://observablehq.com/@bluprince13/standard-error-of-the-mean) to show the dispersion tendency of the data. This means that I need a new variable for the x-axis. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":167,"value":"I came up with dates. In a real situation, if I wanted to record my mood values every day, there would certainly be days when I would forget to do so. Or maybe this data logging started at some point and is not something I've been doing. Using time would give this example a more realistic feel.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":170,"value":"I have to say that I spent a lot of time on how to generate random dates, which is probably unnecessary. I followed tons related questions answered on stackoverflow, and there are some that seem to successful. The following array is one of the results:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":139,"value":"randomDateArray = (randomDateArrayReap.reduce(function(a,b){if(a.indexOf(b)<0)a.push(b);return a;},[]));","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":174,"value":"This variable is used to store arrays that I have generated. The result of a large number of attemps here is, the final generated data sometimes has many duplicates, sometimes the generated dates are too early (even to 80s), and even a lot of future times! One idea I thought to solve this is asking users to enter a date to indicate when they started recording mood data. It works for a while, but more problems occured.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":241,"value":"viewof firstDay = Inputs.date({value:\"2020-12-10\",label: \"First Day I rate my mood:\", max:new Date()}) ","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":247,"value":"Even I can generate right dates, I can't weed out the duplicate dates in the array. Because according to their ISO-8601 format (1973-01-31T02:26:51.408Z), some elements have the same date but different time. And when I want to convert them all to yyyy-mm-dd format, the ticks in the table show NaN. Looks like it can't recognize dates besides ISO-8601.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":252,"value":"randomDateArrayRaw = randomDateArray.flatMap(n => n.toISOString().split('T')[0]); //Format the date to yyyy-mm-dd","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":190,"value":"$46","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":192,"value":"viewof CI_Level = Inputs.radio(new Map([[\"1σ interval = 68% CI\",1], [\"2σ interval = 95% CI\",2], [\"3σ interval = 99.7% CI\",3], [\"4σ interval = 99.99% CI\",4],[\"5σ interval = 99.99994% CI (Paticle Physics standard)\",5]]), {value:2, label: \"Confidence Interval Level\"})\n\n// About standard error and confidence level, learn more at: https://observablehq.com/@bluprince13/standard-error-of-the-mean","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":180,"value":"To add to the despair, the date data used as the x-axis conflicted with the confidence interval area mark I did later on. (As shown above, the confidence interval is not fully displayed in the chart). I found out that this error was related to the generated random date data when I debugged the confidence interval issue. I eventually had to give up showing the x-axis (because it doesn't make sense at all), it's just a tool to help spread out the anchor points on the chart and try not to overlap each other.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":34,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":227,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":37,"value":"## Data Visualization","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":210,"value":"$47","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":219,"value":"The scatter plot nicely shows how a set of normally distributed data diverges from the center of the mean like the two ends. I use the burd color scheme to express good and bad emotions, with blue representing bad emotions and red representing happy emotions.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":221,"value":"I also constructed a function where the confidence level interacts with the area mark, and the audience can choose different confidence levels to observe the change of the confidence interval.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":213,"value":"function transCI(CIL) {\n // const CIL_txt = 0,\n if(CIL === null){\n return \"Please select a CI level\"\n }else if(CIL == 1){\n return \"1σ interval = 68%CI\"\n }else if(CIL == 2){\n return \"2σ interval = 95%CI\"\n }else if(CIL == 3){\n return \"3σ interval = 99.7%CI\"\n }else if(CIL == 4){\n return \"4σ interval = 99.99%CI\"\n }else if(CIL == 5){\n return \"5σ interval = 99.99994%CI (Paticle Physics standard)\"\n } \n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":233,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":235,"value":"As mentioned above, I introduced a set of sleep survey data in order to deepen the audience's understanding of the relevance of CI. This data set comes from [Michael Lomuscio](https://www.kaggle.com/datasets/mlomuscio/sleepstudypilot) on Kaggle.com","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":231,"value":"sleepstudydata = await FileAttachment(\"SleepStudyData.csv\").csv({typed: \"auto\"})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"sleepstudydata","database":"$undefined"},{"id":238,"value":"The variable \"Participant\" was not in the original data. Again, I added this variable to give the chart an x-axis to support it. This is much simpler than generating random date.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":269,"value":"Get the sleep hours and participant number data from the doc.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":326,"value":"sleepHours = sleepstudydata.filter(sleepstudydata => sleepstudydata[\"Hours\"] > 0) //Extract data from the csv doc to array, and filtered out missing data.","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":333,"value":"sleepHoursFilt = sleepHours.map(data => {\n return data.Hours\n})\n// sleepHourArr = sleepHours.map(d => [d.Participant, d.Hours]) //I tried to extract two variables from the original data, but it doesn't shows in a json format. So I still used my clumpsy method here: extract varibables separatly and combine them in a function called jsonRecords(). ","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":262,"value":"parNo = sleepHours.map(data => {\n return data.Participant\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":286,"value":"Calculate the mean and the standard deviation.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":288,"value":"sleepMean = Number(d3.mean(sleepHoursFilt).toString().match(/^\\d+(?:\\.\\d{0,2})?/))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":293,"value":"// sleepSigma = Math.floor(d3.deviation(hours))\nsleepSigma = Number(d3.deviation(sleepHoursFilt).toString().match(/^\\d+(?:\\.\\d{0,2})?/))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":279,"value":"We got valid sleep data here. And then we can build the chart.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":276,"value":"sleepData = jsonRecords2(parNo,sleepHoursFilt)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":281,"value":"$48","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":296,"value":"viewof CI_Level2 = Inputs.radio(new Map([[\"1σ interval = 68% CI\",1], [\"2σ interval = 95% CI\",2], [\"3σ interval = 99.7% CI\",3], [\"4σ interval = 99.99% CI\",4],[\"5σ interval = 99.99994% CI (Paticle Physics standard)\",5]]), {value:0, label: \"Confidence Interval Level\"})\n\n// About standard error and confidence level, learn more at: https://observablehq.com/@bluprince13/standard-error-of-the-mean","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":301,"value":"Because this is a deterministic array, the audience cannot use the slider to observe the effect of the different parameters on the chart as they did in the previous chart. However, I will remind the audience in the narrative that this example is closer to the real statistical application. Changes in the elements of the chart will depend on the changes in the sample size, thereby explaining the relationship between the confidence interval and the variation of the sample mean.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":39,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":41,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":44,"value":"## Reference","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":303,"value":"Many thanks to the wonderful works of others for their references and help with my project!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":305,"value":"$49","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":90,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":94,"value":"~~~\n\n~~~","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":96,"value":"## Appendix","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":98,"value":"function* pachinko(random, extent, height = 250,width = 1000) {\n const radius = 5;\n const dodge = dodger(radius * 2 + 1);\n const margin = {top: 0, right: 30, bottom: 20, left: 30};\n const values = Float64Array.from({length: n}, random);\n if (extent === undefined) extent = d3.extent(values);\n const x = d3.scaleLinear(extent, [margin.left, width - margin.right]).nice();\n\n const svg = d3.select(DOM.svg(width, height)).style(\"overflow\", \"visible\");\n yield svg.node();\n\n svg.append(\"g\")\n .attr(\"transform\", `translate(0,${height - margin.bottom})`)\n .call(d3.axisBottom(x));\n\n for (let i = 0; i < n; ++i) {\n if (i % 20 == 0) yield svg.node();\n const cx = x(values[i]);\n const cy = height - margin.bottom - dodge(cx) - radius - 1;\n if (cy < margin.top) break;\n svg.append(\"circle\")\n .attr(\"cx\", cx)\n .attr(\"cy\", -400)\n .attr(\"r\", radius)\n .attr(\"cy\", cy)\n .attr(\"fill\", \"#d3b4e0\");\n }\n\n yield svg.node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":111,"value":"dodger = radius => {\n const radius2 = radius ** 2;\n const bisect = d3.bisector(d => d.x);\n const circles = [];\n return x => {\n const l = bisect.left(circles, x - radius);\n const r = bisect.right(circles, x + radius, l);\n let y = 0;\n for (let i = l; i < r; ++i) {\n const { x: xi, y: yi } = circles[i];\n const x2 = (xi - x) ** 2;\n const y2 = (yi - y) ** 2;\n if (radius2 > x2 + y2) {\n y = yi + Math.sqrt(radius2 - x2) + 1e-6;\n i = l - 1;\n continue;\n }\n }\n circles.splice(bisect.left(circles, x, l, r), 0, { x, y });\n return y;\n };\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":134,"value":"function jsonRecords(arr1,arr2,arr3) {\n var moodRecords = []\nfor(var i in arr1) {\n moodRecords.push({\n \"date\" : arr1[i],\n \"moodValue\" : arr2[i],\n \"formattedDate\" : arr3[i],\n })\n}\n return moodRecords;\n }","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":141,"value":"randomDateArrayReap = Array.from({length: n}, () => (new getRandomDate()));","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":137,"value":"finalData = jsonRecords(randomDateArray,randomMoodValue,randomDateArrayRaw)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":143,"value":"function getRandomDate() {\n const maxDate = Date.now();\n const timestamp = Math.floor(Math.random() * maxDate);\n return new Date(timestamp);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":257,"value":"function jsonRecords2(arr1,arr2) {\n var sleepRecords = []\nfor(var i in arr1) {\n sleepRecords.push({\n \"participant\" : arr1[i],\n \"sleepHours\" : arr2[i],\n })\n}\n return sleepRecords;\n }","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W4a","stdlib":"1"}} 44:"$43:body"