28:null 29:["$","title",null,{"children":"CSE 442 Tutorial 1: Observable, JavaScript, Data Wrangling and Arquero | Zening | Observable"}] 2e:T4f7,md`
### Table Construction from CSV Finally, [\`fromCSV()\`](https://uwdata.github.io/arquero/api#fromCSV) parses comma-separated values files. Using the \`delimiter\` option, it can be configured to parse related formats such as tab, pipe (\`|\`), or semi-colon (\`;\`) separated values. The \`decimal\` option sets the decimal character for number parsing (default \`'.'\`). The \`fromCSV()\` method performs parsing; use [\`loadCSV()\`](https://uwdata.github.io/arquero/api#loadJSON) to both load and parse a file. The related [\`table.toCSV()\`](https://uwdata.github.io/arquero/api/table#toCSV) method writes a table to a CSV string. The CSV file \`beer.csv\` has data for over 1,000 beers and is attached to this notebook. Let's retrieve the attachment and pass it to \`fromCSV()\` to instantiate a table. We pass a \`height\` option to \`view()\` to set the maximum height of the table viewer. We do _not_ pass a \`limit\` option, allowing us to scroll the view and see data up to the default limit of 100 rows. *The data includes beer \`name\` and \`style\`, a \`brewery_id\`, the percentage of alcohol by volume (\`abv\`), and a bitterness measure in [international bittering units](https://en.wikipedia.org/wiki/Beer_measurement#Bitterness) (\`ibu\`).*`2f:[{"id":"2a5494b9560f0b4253c7307423666be97a19d781414694e0bbcfa3335f7061cc4f958b7d606ce4632b748329265b2f84312499442b802a880384fc4dad0c0350","href":"https://static.observableusercontent.com/files/2a5494b9560f0b4253c7307423666be97a19d781414694e0bbcfa3335f7061cc4f958b7d606ce4632b748329265b2f84312499442b802a880384fc4dad0c0350","name":"schedule-OH.json","type":"application/json","lastModified":1704124597086.071,"size":369},{"id":"07341b87a5bc022a9f0bcb27f16181b88ac6011e29bbf3ecac7153ae0cb8fb49f4a8cedabf30508ec61744a982968a8cfa930361b221abee4f9cfbd227840e41","href":"https://static.observableusercontent.com/files/07341b87a5bc022a9f0bcb27f16181b88ac6011e29bbf3ecac7153ae0cb8fb49f4a8cedabf30508ec61744a982968a8cfa930361b221abee4f9cfbd227840e41","name":"beer.csv","type":"text/csv","lastModified":1704127500811.682,"size":58140},{"id":"6d3f41e014e5b62e0a1b02e9a8483fae7c1ff1d75dac8d7b308d00c3915b946d1d36a6938afdd70214cc0b4df41ccea447f4bd7532a1c4ad64b4d9d61fe4cd6b","href":"https://static.observableusercontent.com/files/6d3f41e014e5b62e0a1b02e9a8483fae7c1ff1d75dac8d7b308d00c3915b946d1d36a6938afdd70214cc0b4df41ccea447f4bd7532a1c4ad64b4d9d61fe4cd6b","name":"breweries.csv","type":"text/csv","lastModified":1704160426272.269,"size":22519},{"id":"c8722c1f2dc71939de9d7f171f741ecec5544128fc0e602dface8293adef1c0831f2f0e76d867024a9ea89b641be110df218fa5053196aa0b5cdd730ca63114a","href":"https://static.observableusercontent.com/files/c8722c1f2dc71939de9d7f171f741ecec5544128fc0e602dface8293adef1c0831f2f0e76d867024a9ea89b641be110df218fa5053196aa0b5cdd730ca63114a","name":"schedule-OH.csv","type":"text/csv","lastModified":1704124585120.661,"size":126}] 2c:{"id":"dc4f0a7bef837c69","slug":null,"title":"CSE 442 Tutorial 1: Observable, JavaScript, Data Wrangling and Arquero","thumbnail":"281fcad7cf70875edc23cdf3f758064ed3d4e83a565d59248cc5f7b1eab91d59","access_level":"public","create_time":"$D2023-12-29T22:12:38.317Z","update_time":"$D2024-01-06T01:20:53.722Z","view_time":null,"first_public_time":"$D2023-12-29T22:12:38.317Z","fork":{"id":"d05ef8a5ac6483c1","slug":"cse442-tutorial1","title":"CSE 442 Tutorial 1: Observable and Data Wrangling (student version)","owner":{"id":"5148a1fc070969c6","login":"nussarafirn","name":"Nussara Tieanklin"}},"stars":1,"starred":false,"owner":{"type":"team","id":"bab6482ef7c91e6f","login":"zeningqu","name":"Zening","avatar_url":"https://avatars-next.observableusercontent.com/9bb70c46593f31ea5274495a69d2e1d8a65885184046055673b2e16e1612a52a","tier":"starter_2024"},"version":2266,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"a844db2e26a4fcef","login":"zeningqu","name":"Zening","avatar_url":"https://avatars-next.observableusercontent.com/9bb70c46593f31ea5274495a69d2e1d8a65885184046055673b2e16e1612a52a","tier":null}],"members":[{"type":"individual","id":"27473fe6eda74079","login":"sqhang","name":"sqhang","avatar_url":"https://avatars-next.observableusercontent.com/559e0fd3e80432ba7284bafbb0d5ea940be92195eff079138ba234fbdd316d49","tier":null,"role":"editor"},{"type":"individual","id":"a844db2e26a4fcef","login":"zeningqu","name":"Zening","avatar_url":"https://avatars-next.observableusercontent.com/9bb70c46593f31ea5274495a69d2e1d8a65885184046055673b2e16e1612a52a","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":43,"collection":null,"type":"notebook","body":{"title":"CSE 442 Tutorial 1: Observable, JavaScript, Data Wrangling and Arquero","cells":[{"id":0,"value":"md`
\n# CSE 442 Tutorial 1: Observable, JavaScript, Data Wrangling and Arquero` ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":4,"value":"md`Welcome to the tutorial! We're happy to have you! 😊 \n\nOur tutorial today will focus on:\n1. Observable Basics\n2. Intro to JavaScript\n3. Basic Data Processing\n4. Advanced Data Wrangling with Arquero`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":9,"value":"md`
\n## 1. Getting familiar with Observable\n\nWe will do some exploring here, but refer back to the official [Observable walkthrough](https://observablehq.com/d/d69e24aa78dabd68) if you have more questions!\n\n### The Basics\n\nEach page is a **notebook** that consists of editable, interactive **cells**. The cells can be filled with JavaScript, Markdown, or HTML. Use \\`Shift-Enter\\`, or the play button in the top right corner of each cell to run each cell. The output will appear at the top of the cell.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":12,"value":"md`👈 You can view or hide the source code of a cell by clicking the left-hand margin. Try it now!`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":14,"value":"md`Here is some Markdown. Edit me! You can add...\n# Headers of\n### different sizes,\n* bullet\n* points\n\n\\`fixed width font\\`\n\n1. Ordered\n2. Lists,\n\nand more. For reference, here is a [guide to formatting using the Markdown syntax](https://observablehq.com/@jaynel/markdown-summary).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":175,"value":"or using the new Markdown mode of Observable:","pinned":true,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":16,"value":"md`To pin/unpin the source code of the cell, hover over the pin icon in the left hand margin, or use the shortcut \\`Ctrl+Shift+P\\`.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":18,"value":"html`

Here is some HTML

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":215,"value":"

Here is some HTML

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined"},{"id":23,"value":"md`Every cell is editable. Changes you make to this document will not be saved, but if you [fork](https://observablehq.com/@observablehq/fork-share-merge) the document, a copy will be saved under your account and any changes made to the forked copy will be saved. You can fork this document by navigating to the three dot options icon and selecting the \"Fork\" option.\n\nIf you want to share your notebook, you can either Publish the notebook, which will make the content publicly visible, or you can enable link-sharing, to privately share the notebook. However, if you make changes to your notebook after the link has been shared, you will have to **Reshare** the link in order for others to see the changes.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":25,"value":"md`### 👉 Practice\n\nAdd a new cell and try adding some markdown, HTML, or JavaScript.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":27,"value":"md`### Cell Values\n\nCells can simply contain values:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1397,"value":"'hello cell'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1409,"value":"md`When you change the contents of a cell, click the Play button on the right-hand side, or press \\`shift-return\\` to re-evaluate the cell.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1399,"value":"md`### Cell Names\n\nCells can include a name, so that other cells can refer to them:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":29,"value":"currentWeek = 2","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":31,"value":"\"There are \" + (10 - currentWeek) + \" weeks left in the quarter\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":22,"value":"md`JavaScript cells are run one expression at a time, so if you want multiple expressions within a single cell, enclose the statements in curly braces and use \\`return\\` to provide the cell value.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":20,"value":"{\n const course = \"CSE442\";\n const message = \"Welcome to \" + course;\n return message;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1423,"value":"md`### Cells Are Reactive\n\nA particularly lovely feature of Observable is that cells are _reactive_: if a cell value changes, all cells that reference it automatically update. Try edit the \\`currentWeek\\` cell, and see what happens!\n \nHere's a more dynamic example. We can use a tagged template to generate HTML output, in the form of a slider. Observable's special \\`viewof\\` operator then exposes the \\`value\\` property of the slider as the value of the cell. (Though in general, I recommend using [Observable Inputs](https://observablehq.com/@observablehq/inputs) to generate interactive widgets.)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1425,"value":"viewof slider = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1429,"value":"md`We can now author dynamic content that reactively responds to the slider value:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1431,"value":"tex`f(x) = x^2 \\rightarrow f(${slider}) = ${slider}^2 = ${slider * slider}`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":310,"value":"md`
\n## 2. A Brief Introduction to JavaScript`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1793,"value":"blockOutput = {\n const myConstant = '_How do I love thee?_ Let me count the ways: ';\n let myVariable = [];\n \n // you didn't think you'd make it to the end without a for loop, did you?\n for (let i = 0; i < 3; ++i) {\n // push adds an element to an array, updating it in place\n myVariable.push(`**${i + 1}...**`); \n }\n \n // you can use tagged templates anywhere!\n return md`${myConstant + myVariable.join(' ')}`;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":319,"value":"As you may have noticed above, we declare new variables using `const` (for values that never change) and `let` (for mutable variables). This is JavaScript syntax! The `const` and `let` statements declare a block-scoped local variable. Here, let's learn some basic JavaScript that are specifically useful for data visualization. You can always explore more in [A Minimal Introduction to JavaScript and Observable](https://observablehq.com/@uwdata/a-minimal-introduction-to-javascript-and-observable).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":327,"value":"### JavaScript types","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":332,"value":"The set of types in the JavaScript language consists of [primitive values](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#primitive_values) and [objects](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data_structures#objects). Primitive values include Boolean type, Null type, Undefined type, Number type, BigInt type, String type, and Symbol type, which are all immutable.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":342,"value":"for example:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":346,"value":"442 // a number","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1886,"value":"{\n let number = 442;\n return number;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":348,"value":"\"Data Visualization\" // a string","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":356,"value":"true // a boolean","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":358,"value":"isNaN(442) // you may use isNaN(a) to check if a is Not-A-Number","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":364,"value":"(1 + Math.sqrt(5)) / 2 // you can also do math","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":366,"value":"Math.random() // or generate 'random' numbers","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":374,"value":"2 < 1 // you can compare things","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":377,"value":"\"A\" < \"a\" // including string comparisons","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":371,"value":"2 + 2 === 4 // more comparison","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":389,"value":"2 + 2 !== \"4\" // and more","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":391,"value":"Notice that we are using `===` and `!==` instead of `==` and `!=` above! Double equals will perform a type conversion when comparing two things, while triple equals won't automatically convert types and if the types differ, false is returned:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":399,"value":"2 + 2 == \"4\" // double equals","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":401,"value":"2 + 2 === \"4\" // triple equals","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":409,"value":"We recommend that you avoid `==` and `!=` unless [you know what you're doing](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Equality_comparisons_and_sameness).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":430,"value":"You will also need `Date` for time-based data, for example:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":440,"value":"date = new Date()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":435,"value":" We will discuss Date more in the _Basic Data Processing_ section later.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":448,"value":"### Functions","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":452,"value":"Like Java, we can write functions to reuse some code:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":458,"value":"function addMe(a, b) { // function signature: name and input arguments\n return a + b; // function body, with return statement for output\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":459,"value":"md`We can then invoke the function and pass it arguments:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":463,"value":"addMe(2, 2)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":464,"value":"addMe('data', ' visualization')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":466,"value":"addMe('💕', 'U', '😍') // sorry heart-eyes, extra arguments are ignored","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":477,"value":"Missing arguments default to `undefined`, indicating a pure absence of value.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":471,"value":"addMe('1') // something from nothing","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2251,"value":"addMe()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":481,"value":"An arrow function expression is a compact alternative to a traditional function expression which can make a function concise. For example, `addMe` can be written as an arrow function:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":484,"value":"addMe2 = (a, b) => a + b","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":488,"value":"Let's decompose addMe down to addMe2 step-by-step:\n\n`\nfunction addMe(a, b) { \n return a + b;\n}`\n\nRemove the keyword `function` and treat the function name as a variable:\n\n`addMe = (a, b) { return a + b; }`\n\nPlace an arrow between the argument and opening body bracket:\n\n`addMe = (a, b) => { return a + b; }`\n\nRemove the body braces and the word \"return\" – the return is implied:\n\n`addMe = (a, b) => a + b`","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":505,"value":"md`\nCan you use arrow functions for all traditional functions? There are indeed some differences between the two and you can check [this MDN document](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions) to see the limitations of arrow functions.\n\nOne worth-noting difference: the often-used [**this**](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/this) keyword is only runtime-binded in traditional functions. The value of \\`this\\` is determined by how a function is called and it may be different each time the function is called.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1830,"value":"{\n const demo = {\n soul: 'd3',\n callback: function() {\n return this },\n }\n\n return demo.callback()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1859,"value":"md`Arrow functions don't provide their own \\`this\\` binding (it retains the \\`this\\` value of the enclosing lexical context).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1832,"value":"{\n const demo = {\n soul: 'd3',\n callback: () => this,\n }\n return demo.callback()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1916,"value":"{\n function showContext() {\n return this;\n }\n\n // return showContext();\n\n // const obj1 = { name: \"Object1\", method: showContext };\n // return obj1.method();\n \n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1815,"value":"md`\nSo what is the difference, really, between a \"function\" function and an \"arrow\" function? Well... a \"function\" function possesses a unique sense of self realized via its \\`this\\` context. An \"arrow\" function, arguably being further along on the road to Enlightenment, has instead abandoned the notion of self entirely and so may only deflect to others when asked of the \\`this\\` context. Should you wish to consult a guide as you seek to decipher \\`this\\` Zen koan, [MDN is your friend](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/this).\n\n...and if none of that made any sense to you, no worries! Just \\`console.log(this)\\` when you debug...\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1936,"value":"Side note: to see logging from console, open browser developer tools and go to the console. In Chrome, press `Cmd+Option+I` (on Mac) or `Control+Shift+I` (Windows, Linux, ChromeOS)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1930,"value":"{\n console.log(\"Date now: \" + Date.now());\n for (let i = 0; i < 10; i++){\n console.log(i);\n }\n return \"example cell\";\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":271,"value":"### Array, Object, and Basic Data Processing","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":518,"value":"### Array","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":522,"value":"JavaScript also supports arrays. An element inside an array can be of any type, and different elements of the same array can be of different types.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":531,"value":"arr1 = [1, 2, 3, 'hi'] // you can initialize an array like this","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1980,"value":"arr1.length","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1983,"value":"arr1[0]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1890,"value":"arr1.slice(1, 3)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":537,"value":"{\n arr1.push('Husky'); // you can push elements to array, this updates the array content\n return arr1;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":558,"value":"There are many useful array methods. Here we will discuss `filter`, `map`, and `sort`, as they are all commonly used in data processing. You can check [this MDN document](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array) to explore more array methods.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":576,"value":"For example, if we want to double each of the values in arr1, we can use `map` to apply a function (in this case, the function is `x * 2`) to each element in the array and return the results as a new array.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":574,"value":"arr1.map(d => d * 2)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":582,"value":"md`What is happening here? Since it does not make sense to multiple strings, we get \\`NaN\\` when we try to multiply a string value by 2. Here we don't really care about these values, so let's first filter the data so that we only double valid number values (you may also map and then filter to get the same result):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":593,"value":"result = arr1.filter(d => !isNaN(d)).map(d => d * 2) // keep valid numbers only, then double them","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":600,"value":"result.slice().sort() // the default sort method is ascending","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":615,"value":"result\n .slice() // you can first make a copy of an array, so we don't overwrite earlier values\n .sort((a, b) => b - a) // you can define a custom sort comparator; here we sort numbers in descending order","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":620,"value":"#### Object","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":628,"value":"JavaScript is also happy to structure values into a set of named properties.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":632,"value":"object = ({key: \"value\", anotherKey: \"anotherValue\"})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1901,"value":"md`_Aside: Observable doesn't let you write object literals \\`{...}\\` at the top level of a cell ...and for good reason! We'll see why later below. Hence the (extra parentheses) here._`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":636,"value":"You can change the value of a key by using `object[key] = newValue` or add a key by `object[newKey] = newValue`, which is very useful when you want to change or add some information to a data record:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":640,"value":"object['key'] = 'new Value'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":644,"value":"object['newKey'] = 'new value'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":651,"value":"Object.keys(object) // return an array of property key strings","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":655,"value":"Object.values(object) // return an array of property values","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":657,"value":"md`Together, arrays and objects can be used to model data, and arrays of Objects is a common format for data tables.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":660,"value":"Let's load and play some data!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":683,"value":"#### CSV","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":694,"value":"A comma-separated values (CSV) file is a delimited text file that uses a comma to separate values. Each line of the file is a data record. Each record consists of one or more fields, separated by commas. The use of the comma as a field separator is the source of the name for this file format. You can load csv file with:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":899,"value":"csvText = FileAttachment(\"schedule-OH.csv\").text() // loading a text file attached in Observable","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":668,"value":"dataCSV = d3.csvParse(csvText) // if we have the text loaded already, we can use d3.csvParse","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":710,"value":"csvURL = FileAttachment(\"schedule-OH.csv\").url() // you can get the url of a file you attached to Observable with .url()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":703,"value":"dataCSV2 = d3.csv(csvURL) // if we have a URL to a CSV, we can use d3.csv to both load and parse","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":724,"value":"By default, the D3 methods will treat all values as strings. In most cases, we will want to also parse data values, for example to read numeric data as JavaScript numbers. You can assign data types to CSV values either by using `d3.autoType` or by doing it yourself, for example:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":728,"value":"auto = d3.csv(csvURL, d3.autoType) // can handle numbers and booleans, but not all date formats","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":731,"value":"manual = d3.csv(csvURL, ({Name, Job, date}) => ({Name: Name, Job: Job, 'OH Date': new Date(date)}))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":833,"value":"datee = manual[1]['OH Date'].getDay()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":686,"value":"#### JSON","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":878,"value":"JSON (JavaScript Object Notation) is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute–value pairs and arrays (or other serializable values).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":892,"value":"FileAttachment(\"schedule-OH.json\").text() // here's what JSON data looks like as text","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":671,"value":"dataJSON = FileAttachment(\"schedule-OH.json\").json() // load and parse JSON from a file attached in Observable","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":770,"value":"json_url = FileAttachment(\"schedule-OH.json\").url() // we can also get a URL for a file attachment","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":768,"value":"dataJSON2 = d3.json(json_url) // loading with url","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":778,"value":"You can convert JavaScript values to a JSON string using the `JSON.stringify()` methods. Conversely, `JSON.parse()` parses a JSON string and returns the JavaScript values it describes.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":782,"value":"strJson = JSON.stringify(dataJSON)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":784,"value":"JSON.parse(strJson)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":789,"value":"Once you finish processing data, you can also export it by clicking three dots on the left of the cell and choose download CSV/JSON.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":797,"value":"md`### 👉 Practice\n\nGiven the data below and your knowledge of arrays and objects, can you find all people who are TAs in this sample data and list them in alphabetical order?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":856,"value":"data = d3.csv(csvURL, ({Name, Job, date}) => ({Name: Name, Job: Job, 'OH Date': new Date(date)}))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":801,"value":"// your code goes here\n// uncomment cell below to see the solution\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":802,"value":"{ // solution:\n return data.filter(d => d['Job'] ==='TA').sort((a, b) => a[\"Name\"] > b[\"Name\"] ? 1 : -1)\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":836,"value":"Can you find 1) any TAs that hold office hour on Tuesday? 2) any instructor holding office hour on Tuesday? (hint: use `Date.getDay()`)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":848,"value":"// your code goes here\n// uncomment cell below to see the solution\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":851,"value":"{ // solution:\n // 1)\n return data.filter(d => d['Job'] === 'TA' && d['OH Date'].getDay() === 2)\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1235,"value":"{ // solution:\n // 2)\n return data.filter(d => d['Job'] === 'Instructor' && d['OH Date'].getDay() === 2)\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":37,"value":"md`
\n## 3. Thinking about data 💭 `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":56,"value":"md`### Data Formats\n\nData comes in many shapes and sizes. Three conventions that are useful to recognize are wide (tidy), folded (long), and pivoted data. \n\nA **wide (or tidy)** data format is data where each variable is present as a separate column and each observation forms a row; in this case the *values* are the cells of the table and the *attributes / variables* are provided in the header row. This format makes it easy to look up the value for a particular attribute and observation (Alice's age is 24) and generally works well for visualization use cases since we can refer to attributes on their own and extract the values fairly easily (e.g., extract just the ages and ignore the weight).\n\n| Person | Age | Weight |Height|\n| :------------- | :---------- | :----------- |\n| Bob | 32 |168| 180|\n| Alice | 24 | 150 |175|\n| Steve | 64 |144|165|`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":58,"value":"md`**Folded (or long)** data format is where the attributes are included in *one column* and the values are included in another. This format is harder to work with because we now have to traverse the entire column to identify the values for a particular attribute (such as the age). Imagine trying to filter the table below to remove people with height less than 170; you would need to keep track of the height for each person, check against 170, and remove the rows (this is a lot less straightforward).\n\n|Person|Variable|Value|\n|:--|:--|:--|\n|Bob|\tAge\t|32|\n|Bob|\tWeight|\t168|\n|Bob|\tHeight|\t180|\n|Alice|\tAge|\t24|\n|Alice|\tWeight|\t150|\n|Alice|\tHeight|\t175|\n|Steve|\tAge\t|64|\n|Steve|\tWeight\t|144|\n|Steve|\tHeight\t|165|`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":60,"value":"md`**Pivoted (or cross tabulation)** is a data format in which attributes are included in *both* rows and columns. The data value in each cell is dependent on both the row and column headers, so each column is not explicitly defined. The values can be aggregated, so to transform the pivoted data back into the tidy format is not always easy. For example, the table below shows total population data, but the value is not explicitly described, so it is hard to refer to the 'population' attribute itself.\n\n|Age|\t1850\t|1860\t|1870|\n|:-|:-|:-|:-|\n|0\t|2934165\t|4213008\t|5517185|\n|5\t|2770735\t|3583239\t|4822149|\n|10\t|2476213\t|3152990\t|4770011|\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":257,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined"},{"id":252,"value":"md`
\n\n## 4. Data Transformation with Arquero`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1245,"value":"md`Arquero is a library for query processing and transformation of array-backed data tables. Following the [relational algebra](https://en.wikipedia.org/wiki/Relational_algebra) and inspired by the design of [dplyr](https://dplyr.tidyverse.org/), Arquero provides a fluent API for manipulating column-oriented data frames. Arquero supports a range of data transformation tasks, including filter, sample, aggregation, window, join, and reshaping operations.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1249,"value":"md`To import Arquero into your own notebooks, add a cell with the following code:\n~~~ js\nimport { aq, op } from '@uwdata/arquero'\n~~~\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":938,"value":"import {aq, op} from '@uwdata/arquero'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1256,"value":"### Data Table Basics","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1258,"value":"The first core abstraction in Arquero is a *data table* (or *data frame*), which models each column as an array of values.\n\nThe [\\`table()\\`](https://uwdata.github.io/arquero/api/#table) method creates a new table for a set of column vectors. The provided arrays are used as-is and should be considered _read-only_. Beware: modifying array content after calling \\`table()\\` may corrupt the table state!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1260,"value":"dt = aq.table({\n u: ['a', 'a', 'b', 'b'],\n x: Int32Array.of(1, 2, 3, 4),\n y: Float64Array.of(5, 6, 7, 8)\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1264,"value":"The `view()` method displays the table content as an HTML table. This method was added especially for use here in Observable. _To instead print tables to the console in either the browser or node, use the [`print()`](https://uwdata.github.io/arquero/api/table#print) method._","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1262,"value":"dt.view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1267,"value":"Tables have methods for the column and row counts, and for retrieving values:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1269,"value":"[dt.numCols(), dt.numRows()]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1272,"value":"dt.columnNames()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1274,"value":"dt.get(\"y\", 2)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1276,"value":"dt.array('y')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1278,"value":"The [`objects()`](https://uwdata.github.io/arquero/api/table#objects) method exports the table content as an array of flat objects. This method is useful for passing data to other tools (such as [Vega-Lite](https://observablehq.com/@vega/vega-lite-api)) that expect row-oriented data, but it does incur the cost of copying the table data.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1281,"value":"dt.objects()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1283,"value":"$2e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1287,"value":"viewof beers = aq // viewof shows the table view, but assigns the table value\n .fromCSV(await FileAttachment('beer.csv').text())\n .view({ height: 240 }) // set maximum height of table viewer in pixels","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1291,"value":"
\n## Transform Verbs [in Illustration](https://observablehq.com/@uwdata/an-illustrated-guide-to-arquero-verbs) and in Action","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":2074,"value":"orderby = md`
\n### Orderby`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2062,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3] });\n const o = t.orderby('x', aq.desc('y'));\n \n const color = sortColors([2, 0, 1]);\n const opt = { color: (i, r) => i < 0 ? null : color(r) };\n \n return hconcat(\n t.view(),\n edge('orderby(x, desc(y))'),\n o.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2057,"value":"md`The [\\`orderby()\\`](https://uwdata.github.io/arquero/api/verbs#orderby) verb accepts one or more columns by which to sort the data.\n\n_What are the **least** alcoholic drinks in the beers data?_`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1307,"value":"beers\n .orderby('abv')\n .view(5) // shorthand for view({ limit: 5 })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1301,"value":"md`\nTo sort a column in descending order, use the \\`desc()\\` method.\n\n_What are the **most** alcoholic beers per style?_
\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1304,"value":"beers\n .orderby('style', aq.desc('abv'))\n .view(50)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2088,"value":"select = md`
\n### Select`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2098,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3], z: [4, 5, 6] });\n const s = t.select('z', 'y');\n const opt = {\n color: {\n y: (i, r) => i < 0 ? null : categories[0],\n z: (i, r) => i < 0 ? null : categories[1]\n }\n };\n \n return hconcat(\n t.view(opt),\n edge('select(z, y)'),\n s.view(opt),\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1313,"value":"md`\nThe [\\`select()\\`](https://uwdata.github.io/arquero/api/verbs#select) verb selects a subset of columns.\n\n_Tired of irrelevant columns? Let's **drop** them._\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1309,"value":"beers\n .orderby(aq.desc('abv'))\n .select('name', 'style', 'abv') // ['name', 'style', 'abv'] also works\n .view(5)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1317,"value":"filter = md`
\n### Filter`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2115,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3] });\n const f = t.filter(d => d.x === 'B');\n const bits = f.mask();\n const opt = { color: (i, r) => i < 0 ? null : bits.get(r) ? categories[0] : categories[1] }\n \n return hconcat(\n t.view(),\n edge('filter(x = \"B\")'),\n t.view(opt),\n edge('reify()'),\n f.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2108,"value":"md`The [\\`filter()\\`](https://uwdata.github.io/arquero/api/verbs#filter) verb removes rows that do not match a provided filter expression.\n\n_How many beers involve bad puns on the word **\"hop\"**?_\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1329,"value":"beers\n .filter(d => op.includes(op.lower(d.name), 'hop'))\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2126,"value":"derive = md`
\n### Derive`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2131,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3] });\n const d = t.derive({ z: d => d.y * d.y });\n const opt = {\n color: {\n y: i => i < 0 ? null : opacity(categories[0], 0.15),\n z: i => i < 0 ? null : categories[0]\n }\n };\n \n return hconcat(\n t.view(),\n edge('derive(z = y * y)'),\n d.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1331,"value":"md`\nDeriving new columns from existing ones is a common analysis task. The [\\`derive()\\`](https://uwdata.github.io/arquero/api/verbs#derive) verb computes one or more new columns defined by input table expressions.\n\nThe \\`derive\\` verb accepts an object: keys provide the names of new columns, values are table expressions. The result is a new table in which previous columns are reused, and new columns replace any old columns with the same name.\n\n_Which beers are the most **\"intense\"** in terms of bitterness and alcohol content?_\n\nWith little to no scientific basis, let's define the intensity of a beer as a combination of its \\`abv\\` (alcohol by volume) and \\`ibu\\` (bitterness score):\n\n${tex`intensity = 3 \\times abv + \\frac{1}{3} \\times log_{10}(ibu)`}
\n\nVery arbitrary, but it roughly balances the values. Whether or not this \"intensity\" is a good thing in a beverage is, much like our formula, entirely subjective...\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1335,"value":"beers\n .derive({ intensity: d => 3 * d.abv + op.log10(d.ibu) / 3 })\n .orderby(aq.desc('intensity'))\n .view(10)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2161,"value":"rollup = md`
\n### Aggregate and Rollup`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2149,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3] });\n const r = t.count();\n const opt = { color: i => i < 0 ? null : categories[0] };\n \n return hconcat(\n t.view(opt),\n edge('count()'),\n r.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2154,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3] });\n const r = t.rollup({ total: d => op.sum(d.y) });\n const opt = { color: i => i < 0 ? null : categories[0] };\n \n return hconcat(\n t.view(opt),\n edge('rollup(total = sum(y))'),\n r.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1337,"value":"md`\nThe [\\`rollup()\\`](https://uwdata.github.io/arquero/api/verbs#rollup) verb \"rolls up\" values using [aggregate functions](https://uwdata.github.io/arquero/api/op#aggregate-functions) (\\`count\\`, \\`sum\\`, \\`average\\`, *etc.*) to create a new summary table. Similar to \\`derive()\\`, \\`rollup()\\` accepts an object of named table expressions. However, all expressions must include aggregate operations: any non-aggregated column value *must* be an input to an aggregate.\n\n_What are the mean (average) values of \\`abv\\`, \\`ibu\\`, and \\`intensity\\`?
\nHow strongly do \\`abv\\` and \\`ibu\\` correlate?_\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1340,"value":"beers\n .rollup({\n mean_abv: d => op.mean(d.abv), // op.average is available as a synonym\n mean_ibu: d => op.mean(d.ibu),\n mean_intensity: d => op.mean(3 * d.abv + op.log10(d.ibu) / 3),\n corr_abv_ibu: d => op.corr(d.abv, d.ibu),\n count: d => op.count()\n })\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1342,"value":"md`_The average \\`abv\\` is ~6%, and we see a positive correlation between alcohol level and bitterness. Of course, averages can sometimes be skewed by outliers... what happens if you change \\`op.mean()\\` to \\`op.median()\\` in the example above?_\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2183,"value":"groupby = md`
\n### Groupby`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2172,"value":"{\n const t = aq.table({ x: ['A', 'B', 'A'], y: [1, 2, 3] });\n const g = t.groupby('x');\n const r = g.rollup({ sum: op.sum('y'), avg: op.average('y'), count: op.count() }).groupby('x');\n\n const keys = g.groups().keys;\n const color = groupColors(keys);\n const opt = { color: (i, r) => i < 0 ? null : color(keys[r]) };\n \n return hconcat(\n t.view(),\n edge('groupby(x)'),\n g.view(opt),\n edge('rollup(...)'),\n r.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2178,"value":"md`_Lumping all beers into a single set glosses over a lot! Different styles involve different ingredients and processes. How does \\`intensity\\` vary by the style of beer?_
\n\nTo answer this question, we can apply the [\\`groupby()\\`](https://uwdata.github.io/arquero/api/verbs#groupby) verb, which groups the rows based on the values of one or more columns. Subsequent operations such as \\`rollup()\\` are then sensitive to these groups:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1345,"value":"beers\n .groupby('style')\n .rollup({\n mean_abv: d => op.mean(d.abv),\n mean_ibu: d => op.mean(d.ibu),\n mean_intensity: d => op.mean(3 * d.abv + op.log10(d.ibu) / 3),\n count: op.count()\n })\n .filter(d => d.count > 20) // let's not trust results with little support...\n .orderby(aq.desc('mean_intensity'))\n .view(10)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1347,"value":"md`Internally, Arquero computes group indices for all rows of a table and reuses them across subsequent operations. If you plan to make multiple \\`rollup()\\` calls for the same groups, call \\`groupby()\\` once and reuse the resulting table.\n\nLike \\`orderby()\\`, groupings persist across subsequent \\`select()\\` and \\`filter()\\` calls. To undo a grouping, call [\\`ungroup()\\`](https://uwdata.github.io/arquero/api/verbs#ungroup).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1350,"value":"md`
\n### Rank, Lead, Lag... Window Functions\n\nThe aggregation functions we used with \\`rollup()\\` can also be used with \\`derive()\\` to produce values that are functions both of an individual row and aggregates of rows.\n\n_Which beers are most \"intense\" within their respective style category? For example, what is the most \"intense\" Hefeweizen or the most \"intense\" Porter?_\n\nWe first \\`derive\\` intensity values as before. We then \\`groupby\\` style and compute a [standardized value (or *z*-score)](https://en.wikipedia.org/wiki/Standard_score): we center the values at their group mean and scale by the group standard deviation. Our calculation involves multiple group aggregates (mean and standard deviation) within a formula applied to each row of the table.\n\nAfter all those operations, we apply window functions!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1474,"value":"beers\n .groupby('style')\n .derive({ intensity: d => 3 * d.abv + op.log10(d.ibu) / 3 })\n .derive({ zscore: ({ intensity: x }) => (x - op.mean(x)) / op.stdev(x) || 0 })\n .orderby(aq.desc('zscore'))\n .filter(d => op.rank() === 1 && op.count() > 20) // filter by rank and count\n .select(aq.not('brewery_id', 'intensity'))\n .view({ height: 400 }) // set max height to show full table without scroll","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1476,"value":"md`\nCalculating rank values is an example of a [_window_ operation](https://uwdata.github.io/arquero/api/op#window-functions): a calculation involving a sliding \"window frame\" over sorted data values. Here, we added a \\`filter()\\` that uses the \\`op.rank()\\` window function to retain only the top-ranked rows within each group and uses the \\`op.count()\\` aggregate function to filter out smaller groups. This filtering step relies on the preceding \\`groupby()\\` and \\`orderby()\\` calls: all calculations are relative to the sorted groups.\n\nBoth \\`derive()\\` and \\`filter()\\` support normal functions, aggregate functions, and window functions to enable a variety of calculations. Though not shown here, windowed aggregate functions such as rolling averages are also supported; for more see the [Working with Window Queries](https://observablehq.com/@uwdata/working-with-window-queries) and [Arquero Cookbook](https://observablehq.com/@uwdata/arquero-cookbook) notebooks.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2201,"value":"join = md`
\n### Join`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2205,"value":"{\n const l = aq.table({ x: ['A', 'B', 'C'], y: [1, 2, 3] });\n const r = aq.table({ u: ['A', 'B', 'D'], v: [4, 5, 6] });\n const j = l.join(r, ['x', 'u']);\n \n const [cl, cr] = categories;\n const color = (f, c) => (i, r) => i < 0 || j.get(f, r) == null ? null : c;\n const format = v => v == null ? '∅' : v + '';\n const opt = {\n color: { x: color('x', cl), y: color('y', cl), u: color('u', cr), v: color('v', cr) },\n format: { x: format, y: format, u: format, v: format }\n };\n \n return hconcat(\n l.view({ color: i => i < 0 ? null : cl }),\n call(r.view({ color: i => i < 0 ? null : cr }), 'join', ', x = u'),\n j.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2207,"value":"{\n const l = aq.table({ x: ['A', 'B', 'C'], y: [1, 2, 3] });\n const r = aq.table({ u: ['A', 'B', 'D'], v: [4, 5, 6] });\n const j = l.join_left(r, ['x', 'u']);\n \n const [cl, cr] = categories;\n const color = (f, c) => (i, r) => i < 0 || j.get(f, r) == null ? null : c;\n const opt = {\n color: { x: color('x', cl), y: color('y', cl), u: color('u', cr), v: color('v', cr) },\n null: () => '∅'\n };\n \n return hconcat(\n l.view({ color: i => i < 0 ? null : cl }),\n call(r.view({ color: i => i < 0 ? null : cr }), 'join_left', ', x = u'),\n j.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2209,"value":"{\n const l = aq.table({ x: ['A', 'B', 'C'], y: [1, 2, 3] });\n const r = aq.table({ u: ['A', 'B', 'D'], v: [4, 5, 6] });\n const j = l.join_right(r, ['x', 'u']);\n\n const [cl, cr] = categories;\n const color = (f, c) => (i, r) => i < 0 || j.get(f, r) == null ? null : c;\n const opt = {\n color: { x: color('x', cl), y: color('y', cl), u: color('u', cr), v: color('v', cr) },\n null: () => '∅'\n };\n \n return hconcat(\n l.view({ color: i => i < 0 ? null : cl }),\n call(r.view({ color: i => i < 0 ? null : cr }), 'join_right', ', x = u'),\n j.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2211,"value":"{\n const l = aq.table({ x: ['A', 'B', 'C'], y: [1, 2, 3] });\n const r = aq.table({ u: ['A', 'B', 'D'], v: [4, 5, 6] });\n const j = l.join_full(r, ['x', 'u']);\n \n const [cl, cr] = categories;\n const color = (f, c) => (i, r) => i < 0 || j.get(f, r) == null ? null : c;\n const opt = {\n color: { x: color('x', cl), y: color('y', cl), u: color('u', cr), v: color('v', cr) },\n null: () => '∅'\n };\n \n return hconcat(\n l.view({ color: i => i < 0 ? null : cl }),\n call(r.view({ color: i => i < 0 ? null : cr }), 'join_full', ', x = u'),\n j.view(opt)\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1492,"value":"md`\n_Beyond looking at individual beers and their styles, what if we want to incorporate information about the breweries that produce them?_\n\nUntil now, we've ignored the \\`brewery_id\\` column, which is a [foreign key](https://en.wikipedia.org/wiki/Foreign_key) to another table about breweries. Let's load the brewery data (the attached \\`breweries.csv\\`). We can then use the [\\`join()\\`](https://uwdata.github.io/arquero/api/verbs#join) verb to blend the datasets together.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1496,"value":"viewof breweries = aq\n .fromCSV(await FileAttachment('breweries.csv').text())\n .view(10)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1502,"value":"beers\n .join(breweries)\n .view(5)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1506,"value":"md`With no other arguments provided, \\`join()\\` compares just the columns in both tables that have the same names. If each of those values match for a given pair of rows, all corresponding column values are included in the output. In addition, the verb defaults to an [inner join](https://en.wikipedia.org/wiki/Join_(SQL%29#Inner_join): rows without any matches are excluded from the output.\n\nWhile the one-argument \\`join()\\` call is convenient, it can cause unintended matches if otherwise unrelated columns have the same name. Let's perform the \\`join\\` again, this time with more parameters:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1525,"value":"beers\n .join(breweries, ['brewery_id', 'brewery_id']) // <- just 'brewery_id' also works!\n .view(5)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1529,"value":"md`We now make it explicit that we want to match only the \\`brewery_id\\` columns. The join condition uses a two-element array: the first element corresponds to the \"left\" table (beers) and the second element to the \"right\" table (breweries). We could also use just \\`'brewery_id'\\`, as here the key columns have the same name across tables.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2220,"value":"fold = md`
\n### Fold`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2223,"value":"{\n const t = aq.table({ A: [1], B: [2], C: [3] });\n const f = t.fold(aq.all());\n\n return hconcat(\n t.view({\n color: i => i < 0 ? categories[0] : categories[1]\n }),\n edge('fold(all())'),\n f.view({\n color: {\n key: i => i < 0 ? null : categories[0],\n value: i => i < 0 ? null : categories[1]\n }\n })\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2226,"value":"{\n const t = aq.table({ A: [1], B: [2], C: [3] });\n const f = t.fold(['B', 'C']);\n\n return hconcat(\n t.view({\n color: {\n B: i => i < 0 ? categories[0] : categories[1],\n C: i => i < 0 ? categories[0] : categories[1]\n }\n }),\n edge('fold([B, C])'),\n f.view({\n color: {\n key: i => i < 0 ? null : categories[0],\n value: i => i < 0 ? null : categories[1]\n }\n })\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1536,"value":"md`\nRemember previous discussions on \"Thinking about data 💭 \"? Here, we could convert the beers data table in **wide** format into **folded** or **pivoted** form.\n\n_Why? Because we can!_\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1651,"value":"Here is beers data table in **wide** format.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1606,"value":"beers\n .view(5)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1653,"value":"We could apply [fold()](https://uwdata.github.io/arquero/api/verbs.html#fold) verb to collapse one or more columns into two key-value pair columns.\n\n_Can we transform `abv` and `ibu` columns into key-value pairs?_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1545,"value":"beers\n .fold([\"abv\", \"ibu\"])\n .view(5)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2230,"value":"pivot = md`
\n### Pivot`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2233,"value":"{\n const t = aq.table({ key: ['A', 'B', 'C'], value: [1, 2, 3] });\n const p = t.pivot('key', 'value');\n\n return hconcat(\n t.view({\n color: {\n key: i => i < 0 ? null : categories[0],\n value: i => i < 0 ? null : categories[1]\n }\n }),\n edge('pivot(key, value)'),\n p.view({\n color: i => i < 0 ? categories[0] : categories[1]\n })\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2236,"value":"{\n const t = aq.table({ type: [1, 1, 1, 2, 2], name: ['A', 'A', 'B', 'A', 'B'], count: [1, 2, 3, 4, 5] });\n const g = t.groupby('type');\n const p = g.pivot('name', { value: d => op.sum(d.count) });\n\n const keys = g.groups().keys;\n const color = groupColors(keys);\n \n return hconcat(\n t.view(),\n edge('groupby(type)'),\n g.view({ color: (i, r) => i < 0 ? null : color(keys[r]) }),\n edge('pivot(name, sum(count))'),\n p.view({ color: i => i < 0 ? null : categories[i < 1 ? 0 : 1] })\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1660,"value":"On the other hand, [pivot()](https://uwdata.github.io/arquero/api/verbs.html#pivot) is the reverse of [fold()](https://uwdata.github.io/arquero/api/verbs.html#fold). The pivot() verb transforms columns into a cross-tabulation. The resulting table has new columns for each unique combination of the provided `keys`, populated with the provided `values`. The provided values must be aggregates, as a single set of keys may include more than one row. It is advisable to explicitly define the aggregation function.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1668,"value":"_What is the average ibu for each style?_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1634,"value":"beers\n .pivot(\n {style : d => d.style},\n {avg_ibu : d => op.mean(d.ibu)}\n ).view(5)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1670,"value":"_fold() and then pivot(). What is the average abv and ibu?_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":1553,"value":"beers\n .fold([\"abv\", \"ibu\"])\n .pivot(\n {key : d => d.key},\n {value : d => op.mean(d.value)}\n )\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":950,"value":"md`
\n## Transformation Verb Exercise\n\nLet's transform the cars dataset to answer a simple question: *In this dataset, which USA-made cars have the highest miles per gallon?*`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":942,"value":"md`Below is the Cars dataset. We see that it is stored in a JavaScript array, with each row as an Object. Expand the Array and take a look at the data.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":944,"value":"cars = (await require('vega-datasets@1'))['cars.json']()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":952,"value":"md`First, we will create an Arquero table \\`from\\` our current \\`cars\\` array:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":954,"value":"viewof cars_table = aq.from(cars).view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":955,"value":"md`Next, we're interested in cars where the country of origin is USA, so we would like only see rows where the Origin = USA. We can achieve this using the \\`filter\\` function. In the example below, \\`d\\` represents a single row from the cars table. In order to apply the filter, we must specify the conditions for our filter, which is to only include the rows where the field Origin is \"USA\". To do so we use the \\`op\\` object from Arquero (short for operator) to get the \\`includes\\` filter operator. This performs a simple equivalence check between a data field value and a specified value. So, we check if \\`d.Origin\\` matches with our desired term, \"USA\":`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1016,"value":"// to be filled -- filter only USA. \n // op.upper(): Returns the string value converted to upper case.\n // op.includes(array, value[, index]): Determines whether an array includes a certain value among its entries, returning true or false as appropriate.","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1736,"value":"// Shengqi's solution:\ncars_table\n // .filter(d => op.includes(op.upper(d.Origin), 'USA'))\n .filter(d => op.upper(d.Origin) === \"USA\")\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":960,"value":"md`Now let's see which of these USA made cars has the highest miles per gallon. Since we want the **highest** MPG, we want to sort the data based on the \\`Miles_per_Gallon\\` column. We can achieve this using the \\`orderby\\` function. We can chain our function calls to execute the \\`orderby\\` after the filter.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":962,"value":"// to be filled -- filter and get the highest mpg in the USA.\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1756,"value":"// Shengqi's solution:\ncars_table\n .filter(d => op.includes(op.upper(d.Origin), 'USA'))\n // .filter(d => op.upper(d.Origin) === \"USA\")\n .orderby(aq.desc(\"Miles_per_Gallon\"))\n .view()\n\n// Answer: plymouth champ","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":964,"value":"md`The default sort direction is an ascending sort (lowest-to-highest). To reverse the sort, we can use \\`aq.desc()\\` to specific a highest-to-lowest sort order. We can also use the \\`select\\` verb to select a subset of columns. Finally, we can include 5 in the \\`.view(5)\\` call to only display the first 5 rows.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":966,"value":"// to be filled -- filter for cars only made from USA, sort in decending order, and select a subset of collumn to show for 5 rows only. \n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1766,"value":"// Shengqi's solution:\ncars_table\n .filter(d => op.includes(op.upper(d.Origin), 'USA'))\n // .filter(d => op.upper(d.Origin) === \"USA\")\n .orderby(aq.desc(\"Miles_per_Gallon\"))\n .select([\"Name\", \"Miles_per_Gallon\", \"Horsepower\", \"Year\", \"Origin\"])\n .view(5)\n\n// Answer: plymouth champ","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":968,"value":"md`Looks like the Plymouth Champ is the top performer in terms of miles per gallon for USA-made cars!`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":970,"value":"### Summarize data with *rollup()* and *groupby()*\n\nThe [`rollup()`](https://uwdata.github.io/arquero/api/verbs#rollup) verb \"rolls up\" values using [aggregate functions](https://uwdata.github.io/arquero/api/op#aggregate-functions) (*count*, *sum*, *average*, *etc.*) to create a new summary table. The *rollup()* verb accepts an object of named table expressions. However, all expressions must include aggregate operations: any non-aggregated column value *must* be an input to an aggregate.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":973,"value":"- *Which region has the most cars in this dataset? How many cars each region produced?*\n- *What is the median number for cylinders?*\n- *What are the minimum and maximum years of cars included in this data?*\n- *What are the mean (average) values of miles per gallons, displacement, horsepower, weight, and acceleration?*\n\nTo answer these questions for each origin region, we can apply the [`groupby()`](https://uwdata.github.io/arquero/api/verbs#groupby) verb, which groups the rows based on the values of one or more columns. Subsequent operations such as `rollup()` are then sensitive to these groups:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":975,"value":"// to be filled -- find answer for each region\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1769,"value":"// Shengqi's answer\n// 1) Which origin has the most cars in this dataset? How many cars each origin produced?\ncars_table\n .groupby(\"Origin\")\n .rollup({\n count: op.count()\n })\n .orderby(aq.desc(\"count\"))\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1778,"value":"// Shengqi's answer\n// 2) What is the median number for cylinders in each region?\ncars_table\n .groupby(\"Origin\")\n .rollup({\n median_cylinders: d => op.median(d.Cylinders),\n count: op.count()\n })\n .orderby(aq.desc(\"count\"))\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1783,"value":"// Shengqi's answer\n// 3) What is the min and max year in each region?\ncars_table\n .groupby(\"Origin\")\n .rollup({\n min_year: d => op.min(d.Year),\n max_year: d => op.max(d.Year),\n median_cylinders: d => op.median(d.Cylinders),\n count: op.count()\n })\n .orderby(aq.desc(\"count\"))\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1786,"value":"// Shengqi's answer\n// 4) What are the mean (average) values of miles per gallons, displacement, horsepower, weight, and acceleration in each region?\ncars_table\n .groupby(\"Origin\")\n .rollup({\n mean_mpg: d => op.mean(d.Miles_per_Gallon),\n mean_displacement: d => op.mean(d.Displacement),\n mean_horsepower: d => op.mean(d.Horsepower),\n mean_weight: d => op.mean(d.Weight_in_lbs),\n mean_acc: d => op.mean(d.Acceleration),\n min_year: d => op.min(d.Year),\n max_year: d => op.max(d.Year),\n median_cylinders: d => op.median(d.Cylinders),\n count: op.count()\n })\n .orderby(aq.desc(\"count\"))\n .view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":978,"value":"What if we would like to change the weight to be in the metric system (kg)?\n\nSince we know that 1 lb = 0.453592 kg, we can then use the `derive()` verb to create a new column that changes the weight unit for the cars.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":980,"value":"// to be filled -- change the unit for the weights","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1789,"value":"// Shengqi's answer\n// 5) Add a new column for weight in kg system.\ncars_table\n .derive({ Weight_in_kg: d => 0.453592 * d.Weight_in_lbs\t})\n .orderby(aq.desc(\"Weight_in_kg\"))\n .view()\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":983,"value":"The examples above only scratch the surface. Arquero includes additional transformation verbs for combining multiple tables together (such as `join` and `union`), reshaping data among the different formats described above (`fold` and `pivot`), random sampling, and much more.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined"},{"id":986,"value":"md `
\n## Resources\n\nInterested in becoming a more proficient data wrangler? Read these notebooks next:\n\n- [Introduction to Arquero](https://observablehq.com/@uwdata/introducing-arquero)\n- [An Illustrated Guide to Arquero Verbs](https://observablehq.com/@uwdata/an-illustrated-guide-to-arquero-verbs)\n- [Tidy Data in JavaScript](https://observablehq.com/@uwdata/tidy-data-in-javascript)\n\nThe Vega-Lite visualization language also includes related mechanisms for data transformation. You will learn more about those later on in the [Data Transformation](https://observablehq.com/@uwdata/data-transformation) notebook reading.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":988,"value":"md`
\n## Acknowledgements\n\nThis notebook builds off of the work of [@uwdata](https://observablehq.com/@uwdata), [@jheer](https://observablehq.com/@jheer), [@anjana](https://observablehq.com/@anjana), [@jonfroehlich](https://observablehq.com/@jonfroehlich), Sonya Lao, Jane Hoffswell, \nNussara 'Firn' Tieanklin, Yueqian Zhang, Shengqi Hang, and Zening Qu.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2012,"value":"md`
\n## Appendix`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2015,"value":"function hconcat(...elem) {\n const span = html``;\n\n elem.forEach(e => {\n e.style.display = 'inline-block';\n e.style.verticalAlign = 'top';\n e.style.marginRight = '1em';\n span.appendChild(e);\n });\n\n return span;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2213,"value":"call = (arg, prefix, suffix = '') => {\n const elem = [md`— \\`${prefix}(\\``, arg, md`\\`${suffix})\\` ⟶`];\n const span = html``;\n \n elem.forEach(e => {\n e.style.display = 'inline-block';\n e.style.verticalAlign = 'top';\n e.style.marginRight = '0.4em';\n span.appendChild(e);\n });\n \n return span;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2019,"value":"edge = label => md` — \\`${label}\\` ⟶`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2029,"value":"opacity = (c, value) => {\n c = d3.color(c);\n c.opacity = value;\n return c + '';\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2034,"value":"categories = d3.schemeTableau10\n .filter((c, i) => i !== 2)\n .map(c => opacity(d3.color(c).brighter(), 0.3))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2038,"value":"sortColors = domain => d3.scaleOrdinal()\n .domain(domain)\n .range(d3.schemeBlues[domain.length].map(c => opacity(c, 0.5)))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2042,"value":"groupColors = domain => d3.scaleOrdinal()\n .domain(domain)\n .range(categories)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"}],"files":"$W2f","stdlib":"1","titling":"auto"}} 2d:"$2c:body"