2b:null 2c:["$","title",null,{"children":"[IAT355] WORKING COPY OF Tutorial 2: Working with data in d3 and Vega | TUTORIAL | IAT 355 Instructors | Observable"}] 31:T4d7,## The Detailed Titanic Dataset Here's the data dictionary of our more titanic dataset. **TitanicPassengers.csv** is from one of [Kaggle's datasets](https://www.kaggle.com/c/titanic/data) used to teach Machine Learning concepts | Variable | Defintion | Key | |---|---|---| | **PassengerId** || | **SurvivEd** | Survival| 0 = No, 1 = Yes | | **CClass** | Ticket class | 1 = 1st, 2 = 2nd, 3 = 3rd | **Sex** | Sex | male/female | **Age** | Age in years | **SibSp** | # of siblings / spouses aboard the Titanic | **Parch** | # of parents / children aboard the Titanic | **Ticket** | Ticket number | **Fare** | Passenger fare | **Cabin** | Cabin number | **Embarked** |Port of Embarkation | C = Cherbourg, Q = Queenstown, S = Southampton Notes: - **Age** if fractional if less than 1. Estimated ages are in the form of xx.5 - **sibsp**: The dataset defines family relations in this way... - Sibling = brother, sister, stepbrother, stepsister - Spouse = husband, wife (mistresses and fiancés were ignored) **parch**: The dataset defines family relations in this way... - Parent = mother, father - Child = daughter, son, stepdaughter, stepson - Some children travelled only with a nanny, therefore parch=0 for them.32:T882,We can start visualizing some of our information in Vega-Lite. ### What is Vega-Lite? ^{Content from [Introduction to Vega-Lite](https://observablehq.com/@uwdata/introduction-to-vega-lite) by Jeffrey Heer} [Vega-Lite](https://vega.github.io/vega-lite/) is a declarative language for interactive data visualization. Vega-Lite offers a powerful and concise visualization grammar for quickly building a wide range of statistical graphics. By declarative, you can provide a high-level specification of what you want the visualization to include, in terms of data, graphical marks, and encoding channels, rather than having to specify how to implement the visualization in terms of for-loops, low-level drawing commands, etc. The key idea is that you declare links between data fields and visual encoding channels, such as the x-axis, y-axis, color, etc, and the rest is handled automatically. The rest of the plot details are handled automatically. #### Vega-Lite JSON vs Vega-Lite API At its core, Vega-Lite uses [JavaScript Object Notation (JSON)](https://vega.github.io/vega-lite/) as a stand-alone file format for describing charts. You might encounter the JSON spec quite often in older notebooks, as well as parts of the documentation. However, to create Vega-Lite visualizations in a more convenient and programmatic way, we will be using the [Vega-Lite API](https://vega.github.io/vega-lite-api/), a set of JavaScript methods that produce Vega-Lite JSON specifications as output. The big benefit of the API is that compared to the JSON version, which can get _very_ messy especially when we move into interaction, the API is much more simple to read and straightforward. We are also in the process of [converting some of the older JSON examples](https://observablehq.com/collection/@krispybird/vega-lite-api-example-gallery) that show up in the Vega Lite example gallery into the API version. Starting in the next tutorial, we will cover Vega-Lite in more depth. For now, we'll show you how you can create a basic chart that you'll need for your second assignment. Let's take a look at this tick chart that shows the cost of fares to ride on the Titanic. 33:[] 2f:{"id":"61ee5c24e3639ecd","slug":null,"title":"[IAT355] WORKING COPY OF Tutorial 2: Working with data in d3 and Vega | TUTORIAL","thumbnail":"c92c50cf68f1f62abb26d7c3a3dbb9945f316fd23f511831d058407a01789a3f","access_level":"public","create_time":"$D2023-01-27T00:38:39.131Z","update_time":"$D2023-02-02T00:49:42.396Z","view_time":null,"first_public_time":"$D2023-01-27T00:38:39.131Z","fork":{"id":"19fb0ca35b90b636","slug":"02t-data","title":"[IAT355] Tutorial 2: Working with data in d3 and Vega | TUTORIAL","owner":{"id":"7e6e24f56083b659","login":"iat355","name":"IAT 355 Instructors"}},"stars":0,"starred":false,"owner":{"type":"team","id":"7e6e24f56083b659","login":"iat355","name":"IAT 355 Instructors","avatar_url":"https://avatars-next.observableusercontent.com/456faf8cd1b3834762f41707a963f2b80c59955f609529918a53471725486440","tier":"starter_2024"},"version":1447,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"9ee804e25e26989e","login":"krispybird","name":"Tiffany","avatar_url":"https://avatars-next.observableusercontent.com/eea8824eb4daf2c7d4d80b8d11e0cf12663a10cbf2e77b40837c5d88261142ae","tier":null}],"members":[{"type":"team","id":"7e6e24f56083b659","login":"iat355","name":"IAT 355 Instructors","avatar_url":"https://avatars-next.observableusercontent.com/456faf8cd1b3834762f41707a963f2b80c59955f609529918a53471725486440","tier":"starter_2024","role":"editor"},{"type":"individual","id":"9ee804e25e26989e","login":"krispybird","name":"Tiffany","avatar_url":"https://avatars-next.observableusercontent.com/eea8824eb4daf2c7d4d80b8d11e0cf12663a10cbf2e77b40837c5d88261142ae","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":2,"collection":null,"type":"notebook","body":{"title":"[IAT355] WORKING COPY OF Tutorial 2: Working with data in d3 and Vega | TUTORIAL","cells":[{"id":0,"value":"# [IAT355] WORKING COPY OF Tutorial 2: Working with data in d3 and Vega | TUTORIAL\nContent adapted from [Lyn Bartram](https://observablehq.com/@drlynb)\n\n**THIS IS THE TUTORIAL VERSION**. The solution version is found [here](https://observablehq.com/@iat355/02s-data?collection=@iat355/iat-355-tutorials-solution)\n\n^{I am covering some of the basic in how we can work with data in the Observable/d3/Vega environment.\nYou should look at Mike Bostocks's Introduction to Data workbook before reading this.}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1298,"value":"toc(\"h2, h3, h4\") // alternatively toc(\"h1, h2, h3, h4\") if you wanted to include the title heading","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":11,"value":"## Modules, Libraries and Data Files\nImport the libraries and data files you need. In this workbook we'll be continuing with the titanic dataset. I will be using a number of datasets and showing you how to merge, join and/or reference them in the same visualizations.\n\nFirst, we will import a few necessary libraries, such as D3 and the Vega-lite library. We'll cover more about Vega-lite in Tutorial book 3, but let's get a brief look of it at the end. For best practices, imports should go at the very bottom of your notebook.\n\n_Refresher_: D3 is a data-driven library created by Mike Bostock. It's a powerful library for visualizing data with HTML, CSS, and SVG. D3 allows you to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document, allowing for powerful interactivity with elements.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":6,"value":"d3 = require(\"d3@7\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":252,"value":"import {vl} from '@vega/vega-lite-api'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":840,"value":"Now, we'll look at another more detailed titanic dataset and load it. Let's assign the value **titanic**","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":517,"value":"//IMPORT YOUR GIST HERE\ntitanic = d3.csv(\"https://gist.githubusercontent.com/krispybird/56bbacb2dfd84c7c6083c62d62a5417a/raw/1b937a4e7404920ea6d3f729611ec06d43e7ae8e/titanic_train.csv\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":507,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":49,"value":"### Get the data into shape\n\n\nAll csv data are imported as text so I need to clean them. There are several ways:\n\n1. iterate through and change text fields to appropriate formats;\n2. change format while importing the d3.csv() function\n3. use the ```map()``` function\n\nExamples follow of each.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":124,"value":"md ` ### 1. Iterate through and change text fields to appropriate formats`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":83,"value":"- Use the + operator to iteratively change the text *Age* string in the *titanic* array to a number.\n- What other fields in *titanic* should also be converted to a number? Take 2 mins and try it out","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":57,"value":"// titanicParsed = {\n// for (let i = 0; i < titanic.length; i++) {\n// titanic[i].Age = +titanic[i].Age;\n\n// titanic[i].SibSp = +titanic[i].SibSp;\n// titanic[i].Survived = +titanic[i].Survived;\n// titanic[i].CClass = +titanic[i].CClass;\n// titanic[i].Fare = +titanic[i].Fare;\n// titanic[i].Parch = +titanic[i].Parch;\n// titanic[i].PassengerId = +titanic[i].PassengerId;\n\n// //what other fields in titanic should be converted to a number? Add them below\n// //Be careful of the capitalization of fields - Javascript is case sensitive!\n// }\n// return titanic;\n// }","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":94,"value":"md `### 2. Change the format when importing using the d3.csv function parameters`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":82,"value":"titanicCleanedD3 = d3.csv(\n \"https://gist.githubusercontent.com/krispybird/56bbacb2dfd84c7c6083c62d62a5417a/raw/1b937a4e7404920ea6d3f729611ec06d43e7ae8e/titanic_train.csv\",\n function (d) {\n return {\n age: +d.Age,\n cabin: +d.CClass,\n fare: +d.Fare\n\n //what values should we convert? fill in the blanks\n\n //think about what data fields we want to return --- not all are cloned automatically!\n };\n }\n)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":103,"value":"### 3. Use the ```array.map()``` function to change all types in the array","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":603,"value":"```.map()``` calls a function once on every element in an array, and returns a new array. \n```map()``` does not change the original array.\n\n_**Note:**_ the [```=>``` Arrow function](https://www.w3schools.com/js/js_arrow_function.asp) is a shorthand way of writing a function\n\nIt is equivalent to writing: \n\n```d = function(){```\n``` ...\n return a;\n \n}```","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":113,"value":"//the => Arrow function is a shorthand way of writing a function\n//It is equivalent to d = function(){\n// ...\n// return a;\n//}\n\n//a function is called on every element in array titanic2, and creates a new array called titanicMap\n//for every element in titanicMap, the age key is assigned a value corresponding to the numerical value of titanic2\ntitanicMap = titanic.map((d) => {\n var a = d;\n a.Age = +d.Age;\n a.SibSp = +d.SibSp;\n a.Parch = +d.Parch;\n a.Fare = +d.Fare;\n\n //remember what other fields we want to add in\n return a;\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1370,"value":"Let's try to get some more details out of our Titanic Dataset.\n\n**Try this question:**\n\nWho was the oldest person aboard the Titanic? \nWe can get this in one of several ways: through looping, or with D3","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":135,"value":"//Let's get the max age of who was on the titanic\n\n//try creating a variable called ageMax and find the age of the oldest person on the titanic\nageMax = {\n //write a loop here\n let maxAge = 0;\n let name = \"\";\n\n for (let i = 0; i < titanicMap.length; i++) {\n if (titanicMap[i].Age > maxAge) {\n maxAge = titanicMap[i].Age;\n name = titanicMap[i].Name;\n }\n }\n\n return maxAge + \" \" + name;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1365,"value":"The D3 library provides a set of [handy functions](https://observablehq.com/@d3/d3-mean-d3-median-and-friends) to do some of these statistical calculations. We can use `d3.mean` to get the mean of the ages. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1133,"value":"{\n //Let's do this with d3.mean() now. We'll first have to use map to create an array with just the ages.\n let ages = titanicMap.map(function (d) {\n return +d.Age;\n });\n\n return d3.mean(ages);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1381,"value":"#### Try this question:\nFind the `median` and `mode` of the passengers' ages using `d3.median` and `d3.mode`","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1384,"value":"//return your answers here\n\n{\n //Let's do this with d3.mean() now. We'll first have to use map to create an array with just the ages.\n let ages = titanicMap.map(function (d) {\n return +d.Age;\n });\n\n return \"Median: \" + d3.median(ages);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1430,"value":"//return your answers here\n\n{\n //Let's do this with d3.mean() now. We'll first have to use map to create an array with just the ages.\n let ages = titanicMap.map(function (d) {\n return +d.Age;\n });\n\n return \"Mode: \" + d3.mode(ages);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":462,"value":"Let's make an array of who was in cabin class 1 to see how we can start filtering some data","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":471,"value":"firstClass = titanicMap.filter(function(d) {\n return d.CClass == 1});","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1404,"value":"\"The number of passengers in first class is: \" + firstClass.length","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":721,"value":"Try getting the average age of the passengers in first class now. \nHint, create a new array with only the ages of the passengers in first class, then use ```d3.mean()```\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":479,"value":"//Try it here\n{\n let firstClassAges = firstClass.map(function (d) {\n return +d.Age;\n });\n\n return (\n \"The average age in first class is: \" + Math.round(d3.mean(firstClassAges))\n );\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":724,"value":"#### Try this question:\nWhich cabin class had highest average age of its passengers? We can find the average ages for second and third class to find out.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":752,"value":"//try to find the answer here for second class\n//hint: you can use d3.mean to find the average age of passengers in second class\n\n{\n let secondClass = titanicMap.filter(function (d) {\n return d.CClass == 2;\n });\n let secondAges = secondClass.map(function (d) {\n return +d.Age;\n });\n\n return d3.mean(secondAges);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":757,"value":"//try to find the answer here for third class\n//hint: you can use d3.mean to find the passengers in third class\n\n//try to find the answer here for second class\n//hint: you can use d3.mean to find the average age of passengers in second class\n\n{\n let thirdClass = titanicMap.filter(function (d) {\n return d.CClass == 3;\n });\n let thirdAges = thirdClass.map(function (d) {\n return +d.Age;\n });\n\n return d3.mean(thirdAges);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":744,"value":"#### Try this question:\nWho had the most expensive ticket aboard the Titanic, and how much was it?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":764,"value":"function calculateTicket() {\n //calculate the ticket inside this function\n let ticket = 0;\n let name = \"\";\n\n for (let i = 0; i < titanicMap.length; i++) {\n if (titanicMap[i].Fare > ticket) {\n ticket = titanicMap[i].Fare;\n name = titanicMap[i].Name;\n }\n }\n\n return name + \" paid: $\" + ticket;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":769,"value":"calculateTicket();","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1319,"value":"#### Try this question:\nHow much money was spent on fares in total from all tickets? \nWe can use ```d3.sum()``` to find this quickly","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1321,"value":"//\n{\n let totalSum = d3.sum(titanicMap, function (d) {\n return d.Fare;\n });\n\n return totalSum.toFixed(2);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1293,"value":"## Visualizing the Information","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":132,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":262,"value":"vl.markTick()\n .data(titanicMap)\n .encode(\n vl.x().fieldQ('Fare')\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":818,"value":"We can even start to make some charts, like this histogram showing age and the number of passengers in each group that survived.\n\nRemember that histograms are a specific type of visualization that groups continuous data into bins. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":297,"value":"vl.markBar()\n .data(titanicMap)\n .encode(\n vl.x().fieldQ(\"Age\").bin(true),\n vl.y().count(\"Survived\") //count\n )\n .render()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1400,"value":"Let's make this histogram together","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1401,"value":"//follow along here\nvl\n .markBar()\n .data(titanicMap)\n .encode(vl.x().fieldQ(\"Age\").bin({ step: 20 }), vl.y().count())\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1362,"value":"We can try to manually set bin size by passing in the parameter `{step: 2}` inside `.bin()`.\n\nTry changing the step to 2, 5, 10, then 20. See how the appearance of the distribution changes. What do you think is the best centrality measurement?\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1306,"value":"---\n## MORE RESOURCES\n- The filled version of the notebook can be found [here](https://observablehq.com/d/3e4899e8e2957a66)\n- For a more in-depth explanation of Obervable, read [A Taste of Observable](https://observablehq.com/@observablehq/a-taste-of-observable?collection=@observablehq/introduction)\n- A more detailed explanation of Observable cells in [Introduction to Code](https://observablehq.com/@observablehq/introduction-to-code)\n- How does [Observable differ from Javascript](https://observablehq.com/@observablehq/observables-not-javascript)\n- The [Markdown Cheat Sheet](https://observablehq.com/@observablehq/markdown-quick-reference)\n- For a collection of resources on Observable, read [Observable User Manual](https://observablehq.com/@observablehq/user-manual?collection=@observablehq/introduction)\n- The [D3](https://d3js.org/) library - we will cover this in the following weeks","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1308,"value":"---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":293,"value":"vega_explicit = {\n const [Vega, VegaLite] = await Promise.all([\n require(\"vega@3/build/vega.min.js\"),\n require(\"vega-lite@3/build/vega-lite.min.js\")\n ]);\n return async spec => {\n const div = document.createElement(\"div\");\n const view = new Vega.View(Vega.parse(VegaLite.compile(spec).spec));\n await view.initialize(div).runAsync();\n return div;\n };\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1203,"value":"embed = require('vega-embed@3')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":374,"value":"embedWithSpecificVersions = require.alias({\n \"vega\": \"vega@5.3.1\",\n \"vega-lite\": \"vega-lite@3.0.0\"\n})(\"vega-embed@4\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1302,"value":"import { toc } from \"@nebrius/indented-toc\" // This component generates the Table of Contents","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W33","stdlib":"1","titling":"auto"}} 30:"$2f:body"