2c:null 2d:["$","title",null,{"children":"Assignment (Data science workflows in JavaScript Workshop Series) | Angie Hjort | Observable"}] 32:T435,For this assignment, you will explore impacts of calcium addition on sugar maple seedlings in two watersheds at Hubbard Brook Experimental Forest using data from Juice and Fahey (2019). Sugar maple (_Acer saccharum_) growth can be inhibited by soil acidification from acid rain, which leaches calcium (a nutrient important for plant growth) from soils. Learn more: [Decades of acid rain is causing loss of valuable Northeast sugar maples, Cornell researchers warn](https://news.cornell.edu/stories/2006/05/acid-rain-causing-decline-sugar-maples-say-researchers). Stephanie Juice, Tim Fahey and colleagues at Hubbard Brook Long Term Ecological Research (LTER) site (Vermont, USA) measured “general sugar maple germinant health by height, leaf area, biomass, and chlorophyll content” (Juice and Fahey 2019) for seedlings in untreated and calcium-treated watersheds to investigate the impact of soil calcium addition on sugar maple seedling growth. In this assignment, you will explore the sugar maple data to gain insights about the impact of watershed calcium addition.33:[{"id":"84da9367be5328b6e4363a169e1504c723c3a6ba1b5510d2df1d2fe49d8922497c2e02c5ef4dad9258da9faea5455dbca05e7d6cea741f78f23c449adab29750","href":"https://static.observableusercontent.com/files/84da9367be5328b6e4363a169e1504c723c3a6ba1b5510d2df1d2fe49d8922497c2e02c5ef4dad9258da9faea5455dbca05e7d6cea741f78f23c449adab29750","name":"w1_acsa_seed_physical.txt","type":"text/plain","lastModified":1691151923642.654,"size":33070}] 30:{"id":"56cbd0cbe01b81d1","slug":"assignment-data-science-workflows-in-javascript-workshop","title":"Assignment (Data science workflows in JavaScript Workshop Series)","thumbnail":"37ff38c0334023c327a2dea3963df0cc6fcde14659a5b2c42d465e7f1a1ebd5f","access_level":"public","create_time":"$D2023-08-04T11:16:57.088Z","update_time":"$D2023-08-05T07:54:17.852Z","view_time":null,"first_public_time":"$D2023-08-04T11:16:57.088Z","fork":true,"stars":0,"starred":false,"owner":{"type":"team","id":"c555e02063f451b5","login":"angiehjort","name":"Angie Hjort","avatar_url":"https://avatars-next.observableusercontent.com/d3c0d6d22f1525b9f9f1d4c8187de1dc20adc65c5baeb538c15672fb523883fa","tier":"starter_2024"},"version":836,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"1245953aabb247aa","login":"angiehjort","name":"Angie Hjort","avatar_url":"https://avatars-next.observableusercontent.com/d3c0d6d22f1525b9f9f1d4c8187de1dc20adc65c5baeb538c15672fb523883fa","tier":null}],"members":[{"type":"individual","id":"1245953aabb247aa","login":"angiehjort","name":"Angie Hjort","avatar_url":"https://avatars-next.observableusercontent.com/d3c0d6d22f1525b9f9f1d4c8187de1dc20adc65c5baeb538c15672fb523883fa","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"subscribed":null,"forks":0,"collection":{"id":"bca6f2c110294361","slug":"observable-course","title":"Observable course","owner":{"id":"c555e02063f451b5","login":"angiehjort","name":"Angie Hjort"}},"type":"notebook","body":{"title":"Assignment (Data science workflows in JavaScript Workshop Series)","cells":[{"id":0,"value":"# Assignment (Data science workflows in JavaScript Workshop Series)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":5,"value":"## Logistics\n\n- **Fork this notebook** to make your own copy. Complete tasks below in your fork.\n- When you are ready to submit, ensure that your fork visibility is **Public unlisted** so that we can see your work. To update the setting, go to the Share button, the under Access choose “Can view (unlisted)” for Public visibility. Make sure to Save changes.\n- Submit assignment by sharing the URL in [this Google Form](https://forms.gle/cYLwZBsyvikVwgH59) by end of day **Friday, August 4th 2023**\n- Ask questions in the Observable Community Slack #learn-javascript channel.\n- Things in yellow are where you’re prompted to add content for the assignment.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":27,"value":"## About the data","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":29,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":33,"value":"**Data source:** Juice, S. and T. Fahey. 2019. Health and mycorrhizal colonization response of sugar maple (Acer saccharum) seedlings to calcium addition in Watershed 1 at the Hubbard Brook Experimental Forest ver 3. Environmental Data Initiative. https://doi.org/10.6073/pasta/0ade53ede9a916a36962799b2407097e","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":43,"value":"## Task 1: Access the data","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":45,"value":"Access the data from the [this data package on the Environmental Data Initiative](https://portal.edirepository.org/nis/mapbrowse?packageid=knb-lter-hbr.157.3). You will only need one file: **w1_acsa_seed_physical** (found in the Resources section on that page). You can either read in the remote file directly using the URL, or download the file and attach it in your notebook. Note that while this is a text file (.txt), values are comma-separated, so you can still use the CSV methods we used in the workshop.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":58,"value":"Below this cell, add a new JavaScript cell below, then add code to load data from the file above as `maples`.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":560,"value":"Changes i made in the task: i thought it would be interesting to keep both years and see what additional analysis it can give, thus i tried to only remove the missing measurements -999. \n\nThis however resulted in troubles performing PCA because it didn't like the missing values in 2004 and also it looked like the two years resulted in two vastly different sets of plants due to unaccounted factors. So i then decided to run the analysis separately for 2003 and 2004 and compare the results side by side.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":451,"value":"maplesRaw = FileAttachment(\"w1_acsa_seed_physical.txt\").csv({typed: true})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":66,"value":"---\n## Task 2: Take a look at the data in a Data table cell","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":74,"value":"Below, visualize the `maples` data in a Data table cell. Familiarize yourself with the variables. Here are variable summaries (for the variables you’ll use in this assignment) from the [data package metadata](https://portal.edirepository.org/nis/metadataviewer?packageid=knb-lter-hbr.157.3): \n\n- **Year**: year measured\n- **Watershed**: study watershed (Reference = untreated; W1 = Calcium treated)\n- **Elevation**: categorical description of watershed elevation\n- **Transect**: study transect\n- **Sample**: sugar maple seedling sample number\n- **StemLength**: seedling stem length (millimeters)\n- **Leaf1Area**: area of first leaf sampled from seedling (cm²)\n- **Leaf2Area**: area of second leaf sampled from seedling (cm²)\n- **LeafDryMass**: dry mass of both leaves (grams)\n- **StemDryMass**: dry mass of seedling stem (grams)\n- **CorrectedLeafArea**: total leaf area for both sampled leaves (Leaf1Area + Leaf2Area), corrected for the area removed for chlorophyll analysis (cm²)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":93,"value":"Below this cell, add a new Data table cell below, selecting maples as the data source:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":456,"value":"maplesRaw","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":97,"value":"---\n## Task 3: Data wrangling","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":99,"value":"Make a simplified version of the `maples` data that only includes observations from 2003. Using the method(s) of your choice (e.g. Array methods, Arquero, SQL, Data table cell, any other tools you prefer, or a combination!), create a simplified version of the `maples` data updated as follows:\n\n- Only keep (and rename) variables:\n - Year (rename to **year**)\n - Watershed (rename to **watershed**)\n - StemLength (rename to **stemLength**)\n - LeafDryMass (rename to **leafMass**)\n - StemDryMass (rename to **stemMass**)\n - CorrectedLeafArea (rename to **leafArea**)\n- Only keep observations from 2003","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":105,"value":"Below this cell, add a JavaScript, SQL, or Data Table cell to wrangle the data as described above, naming your new subset `maples2003`.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":575,"value":"Here i made a helper function to remove the missing measurements. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":539,"value":"validate = v => v < 0 ? null : v;","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":458,"value":"maples = maplesRaw.map(d => ({\n year: d.Year,\n watershed: d.Watershed,\n stemLength: validate(d.StemLength),\n leafMass: validate(d.LeafDryMass),\n stemMass: validate(d.StemDryMass),\n leafArea: validate(d.CorrectedLeafArea)\n}))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":641,"value":"maples2003 = maples.filter(f => f.year === 2003)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":644,"value":"maples2004 = maples.filter(f => f.year === 2004).map(({leafArea, ...rest}) => ({...rest})) //the entire empty column needs to be removed for PCA to work","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":463,"value":"maples","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":127,"value":"---\n## Task 4: Exploratory data visualization","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":129,"value":"In the area below, create three exploratory data visualizations for the 2003 maple seedlings. Where prompted, add a Markdown cell with a 1-sentences summary of a takeaway or insight gleaned from the visualization.\n\n**Note:** Throughout the rest of the assignment, you will use the 2003 subset `maples2003` you created above.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":131,"value":"Below this cell, use the **Chart cell** to create histogram of leafArea for seedlings recorded in 2003. Facet (vertically) by watershed to compare the distributions in the untreated (Reference) and calcium treated (W1) watersheds. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":469,"value":"Plot.plot({\n color: {\n legend: true\n },\n marks: [\n Plot.auto(maples, {x: \"leafArea\", fx: \"year\", fy: \"watershed\", color: \"watershed\"})\n ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":210,"value":"Below this cell, add a new Markdown cell, then describe and compare the distributions for leaf areas in untreated and calcium treated watersheds based on the histograms.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":474,"value":" Based on the histograms looks like adding calcium makes maple leaves have a considerably larger area on average. The data on leaf area is only available for 2003","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":181,"value":"---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":183,"value":"Below this cell, insert a **Plot snippet from the Add cell menu to create a Scatterplot** of **stemLength** and **stemMass**, with fill color mapped to **watershed**. Then, visualize the linear trends by adding a `linearRegressionY` mark atop the scatterplot mark, with stroke color mapped to **watershed**.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":481,"value":"Plot.plot({\n width: 900,\n facet: {\n data: maples,\n x: mergeYears.length ? undefined : \"year\"\n },\n color: { legend: true },\n marks: [\n Plot.dot(maples, {\n x: \"stemLength\",\n y: \"stemMass\",\n fill: \"watershed\",\n tip: true\n }),\n Plot.linearRegressionY(maples, {x: \"stemLength\", y: \"stemMass\", stroke: \"watershed\"})\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":219,"value":"Below this cell, add a new Markdown cell, then describe and compare the relationship between stem length and mass in untreated and calcium treated watersheds.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":493,"value":" First, there is a clear positive correlation between stem length and stem mass, which makes sense. Then, comparing the two watersheds, we can see that the one with added calcium has the maple stems considerably heavier although not any longer. The effect can be observed both in 2003 and 2004\n\n Year 2004 yielded much heavier stems overall and especially for longer plants in both watersheds\n\n If we merge the data from the two years the effect of added calcium on stem weight becomes much less pronounced due to a much larger overall variation of weights","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":764,"value":"viewof mergeYears = Inputs.checkbox([\"Merge years\"], {label: \"Try this!\", value: []})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":198,"value":"---\nBelow this cell, add a JavaScript cell to import PlotMatrix from [Fil’s Autoplot matrix notebook](https://observablehq.com/@observablehq/autoplot-matrix). Then in a second JavaScript cell, use PlotMatrix to create an interactive matrix of charts to visualize the different variable relationships and distributions.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":495,"value":"import {PlotMatrix as PM2003} with {maples2003 as data} from \"@observablehq/autoplot-matrix\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":650,"value":"import {PlotMatrix as PM2004} with {maples2004 as data} from \"@observablehq/autoplot-matrix\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":497,"value":"PM2003()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":652,"value":"PM2004()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":204,"value":"---\n## Task 5: Analysis / modeling","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":228,"value":"**Choose one** of the following to explore differences and patterns in sugar maple morphology in untreated and calcium-treated watersheds: \n\n- **Option 1:** Perform simple linear regression on stem dimensions (stemLength versus stemMass) for the untreated and calcium-amended watershed. For the purposes of this assignment, you can do these separately (e.g. create separate subsets for Reference and W1 watersheds, then use simple linear regression to estimate parameters for each). \n\n- **Option 2:** Perform k-means clustering with two clusters to see if, based on the four quantitative variables in `maples2003`, the seedlings are largely within separate clusters based on watershed. Add a visualization or table to describe the outcomes of cluster analysis.\n \n- **Option 3:** Perform principal component analysis using the four quantitative variables in `maples2003`, and create a biplot showing both observation scores (points) on PC1 and PC2, overlayed by the variable vectors (arrows).\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":237,"value":"Below this cell, add any cells necessary to perform the analysis you chose from the options above.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":588,"value":"ML = require(\"https://www.lactame.com/lib/ml/6.0.0/ml.min.js\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":591,"value":"import {scale, asMatrix} from \"@chrispahm/hierarchical-clustering\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":594,"value":"mapleScaled2003 = scale(maples2003.map(({year, watershed, ...rest}) => rest))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":655,"value":"mapleScaled2004 = scale(maples2004.map(({year, watershed, ...rest}) => rest))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":598,"value":"maplePCA = ({\n y2003: new ML.PCA(mapleScaled2003.map(Object.values)), \n y2004: new ML.PCA(mapleScaled2004.map(Object.values))\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":798,"value":"explainedVariance = maplePCA.y2003.getExplainedVariance().map((d, i) => ({year: 2003, v: d, pc: i+1}))\n .concat(maplePCA.y2004.getExplainedVariance().map((d, i) => ({year: 2004, v: d, pc: i+1})))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":606,"value":"Plot.plot({\n facet: {\n data: explainedVariance,\n y: \"year\"\n },\n y: {tickFormat: (d) => \"PC\" + d},\n marks: [\n Plot.barX( explainedVariance, {y: \"pc\", x: \"v\"}),\n Plot.ruleX([0])\n ]\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":610,"value":"import {viewof loadings as loadings2003} with {mapleScaled2003 as food_scaled} from \"@chrispahm/principal-component-analysis\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":668,"value":"import {viewof loadings as loadings2004} with {mapleScaled2004 as food_scaled} from \"@chrispahm/principal-component-analysis\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":612,"value":"import {viewof scores as scores2003} with {mapleScaled2003 as food_scaled, maples2003 as food} from \"@chrispahm/principal-component-analysis\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":671,"value":"import {viewof scores as scores2004} with {mapleScaled2004 as food_scaled, maples2004 as food} from \"@chrispahm/principal-component-analysis\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":813,"value":"loadingsAndScores = {\n const scoresCombined2003 = scores2003.map((d, i) => ({year: maples2003[i].year, watershed: maples2003[i].watershed, ...d}));\n const scoresCombined2004 = scores2004.map((d, i) => ({year: maples2004[i].year, watershed: maples2004[i].watershed, ...d}));\n const scoresCombined = scoresCombined2003.concat(scoresCombined2004);\n const loadings = loadings2003.map(d => ({year: 2003, ...d})).concat(loadings2004.map(d => ({year: 2004, ...d})))\n\n //putting loadings and scores together in one array is an ugly hack that allows facets to work in a chart below\n //would appreciate a hint to a more elegant solution\n return loadings.concat(scoresCombined);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":706,"value":"chart = Plot.plot({\n width: 900,\n facet: {\n data: loadingsAndScores,\n x: \"year\"\n },\n x: { domain: [-4, 4] },\n y: { domain: [-4, 4] },\n color: { legend: true },\n marks: [\n Plot.density(loadingsAndScores, {\n x: \"PC1\",\n y: \"PC2\",\n fill: \"watershed\",\n stroke: \"watershed\",\n fillOpacity: (d) => (d.watershed ? 0.02 : 0),\n strokeOpacity: (d) => (d.watershed ? 0.8 : 0),\n tip: true\n }),\n Plot.ruleX([0], { stroke: \"lightgrey\" }),\n Plot.ruleY([0], { stroke: \"lightgrey\" }),\n Plot.arrow(loadingsAndScores, {\n x1: 0,\n x2: (d) => d.PC1 * scalingFactor,\n y1: 0,\n y2: (d) => d.PC2 * scalingFactor,\n opacity: (d) => (d.Variable ? 1 : 0)\n }),\n Plot.text(loadingsAndScores, {\n x: (d) => d.PC1 * scalingFactor * 1.3,\n y: (d) => d.PC2 * scalingFactor * 1.1,\n text: \"Variable\",\n opacity: (d) => (d.Variable ? 1 : 0)\n })\n ]\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":619,"value":"viewof scalingFactor = Inputs.range([1, 10], {label: \"Arrow scaling\", step: 1, value: 4})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":242,"value":"--- \n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":244,"value":"Below this cell, add a new Markdown cell and in 1-2 sentences summarize the findings from your analysis for Task 5. Then, in 1-2 sentences based on **all** of the exploration you have done in this assignment, share your preliminary thoughts on if and how calcium addition impacts sugar maple seedlings. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":608,"value":" Looking at the explained PCA variance bar chart, one principle component would be enough to explain most of the variance, thus our dimensionality reduction was very successful. We still have space on the screen however so adding PC2 in charts also wouldn't hurt. \n\n Then, looking at the directions and the length of arrows, all 4 variables contribute to that single most important principle component PC1. We can define the two clusters for the watersheds both in 2003 and 2004, density plot instead of scatterplot helps to see them better.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":633,"value":" Looking at the entire analysis we can conclude that adding calcium helps to make maple seedlings have larger leaves and heavier stems, so the research hypothesis is confirmed","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":246,"value":"---\n## Congratulations! Submit your assignment.\n\nOnce you’re done with the assignment, make sure to: \n\n- Update your notebook settings (click the Share button) so that Public access is “View (unlisted)”\n- Submit the URL to your notebook in [this Google form](https://forms.gle/cYLwZBsyvikVwgH59) by midnight on August 4, 2023\n- Await your course certificate, which we’ll send out in mid-August! \n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":262,"value":"_**Thank you** for joining us for the Data Science Workflows in JavaScript workshop series! We look forward to seeing you in our upcoming courses and workshops, and in our Community Slack!_ ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"}],"files":"$W33","stdlib":"1","titling":"auto"}} 31:"$30:body"