2b:null 2c:["$","title",null,{"children":"[IAT355] Tutorial 2: Working with data in d3 and Vega | SOLUTION | Sooa Mo | Observable"}] 31:T4d7,## The Detailed Titanic Dataset Here's the data dictionary of our more titanic dataset. **TitanicPassengers.csv** is from one of [Kaggle's datasets](https://www.kaggle.com/c/titanic/data) used to teach Machine Learning concepts | Variable | Defintion | Key | |---|---|---| | **PassengerId** || | **SurvivEd** | Survival| 0 = No, 1 = Yes | | **CClass** | Ticket class | 1 = 1st, 2 = 2nd, 3 = 3rd | **Sex** | Sex | male/female | **Age** | Age in years | **SibSp** | # of siblings / spouses aboard the Titanic | **Parch** | # of parents / children aboard the Titanic | **Ticket** | Ticket number | **Fare** | Passenger fare | **Cabin** | Cabin number | **Embarked** |Port of Embarkation | C = Cherbourg, Q = Queenstown, S = Southampton Notes: - **Age** if fractional if less than 1. Estimated ages are in the form of xx.5 - **sibsp**: The dataset defines family relations in this way... - Sibling = brother, sister, stepbrother, stepsister - Spouse = husband, wife (mistresses and fiancés were ignored) **parch**: The dataset defines family relations in this way... - Parent = mother, father - Child = daughter, son, stepdaughter, stepson - Some children travelled only with a nanny, therefore parch=0 for them.32:T882,We can start visualizing some of our information in Vega-Lite. ### What is Vega-Lite? ^{Content from [Introduction to Vega-Lite](https://observablehq.com/@uwdata/introduction-to-vega-lite) by Jeffrey Heer} [Vega-Lite](https://vega.github.io/vega-lite/) is a declarative language for interactive data visualization. Vega-Lite offers a powerful and concise visualization grammar for quickly building a wide range of statistical graphics. By declarative, you can provide a high-level specification of what you want the visualization to include, in terms of data, graphical marks, and encoding channels, rather than having to specify how to implement the visualization in terms of for-loops, low-level drawing commands, etc. The key idea is that you declare links between data fields and visual encoding channels, such as the x-axis, y-axis, color, etc, and the rest is handled automatically. The rest of the plot details are handled automatically. #### Vega-Lite JSON vs Vega-Lite API At its core, Vega-Lite uses [JavaScript Object Notation (JSON)](https://vega.github.io/vega-lite/) as a stand-alone file format for describing charts. You might encounter the JSON spec quite often in older notebooks, as well as parts of the documentation. However, to create Vega-Lite visualizations in a more convenient and programmatic way, we will be using the [Vega-Lite API](https://vega.github.io/vega-lite-api/), a set of JavaScript methods that produce Vega-Lite JSON specifications as output. The big benefit of the API is that compared to the JSON version, which can get _very_ messy especially when we move into interaction, the API is much more simple to read and straightforward. We are also in the process of [converting some of the older JSON examples](https://observablehq.com/collection/@krispybird/vega-lite-api-example-gallery) that show up in the Vega Lite example gallery into the API version. Starting in the next tutorial, we will cover Vega-Lite in more depth. For now, we'll show you how you can create a basic chart that you'll need for your second assignment. Let's take a look at this tick chart that shows the cost of fares to ride on the Titanic. 33:[] 2f:{"id":"5b2ac4cedc2d6c02","slug":null,"title":"[IAT355] Tutorial 2: Working with data in d3 and Vega | SOLUTION","thumbnail":"cd2570fb739524f0aafa4ed0b17ba179744c2b253da843bcf973852a70459016","access_level":"public","create_time":"$D2023-02-05T05:56:10.875Z","update_time":"$D2023-02-05T05:56:10.875Z","view_time":null,"first_public_time":"$D2023-02-05T05:56:10.875Z","fork":{"id":"cca23e1e29ca8b94","slug":"02s-data","title":"[IAT355] Tutorial 2: Working with data in d3 and Vega | SOLUTION","owner":{"id":"7e6e24f56083b659","login":"iat355","name":"IAT 355 Instructors"}},"stars":0,"starred":false,"owner":{"type":"team","id":"6e3266892950d6ff","login":"sooamo","name":"Sooa Mo","avatar_url":"https://avatars-next.observableusercontent.com/132a26eabd5134a62a64ff73823721e2bca31fa293c744f76ef1892b0c2e78f1","tier":"starter_2024"},"version":1418,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"ca9e56dd7622b70d","login":"sooamo","name":"Sooa Mo","avatar_url":"https://avatars-next.observableusercontent.com/132a26eabd5134a62a64ff73823721e2bca31fa293c744f76ef1892b0c2e78f1","tier":null}],"members":[{"type":"individual","id":"ca9e56dd7622b70d","login":"sooamo","name":"Sooa Mo","avatar_url":"https://avatars-next.observableusercontent.com/132a26eabd5134a62a64ff73823721e2bca31fa293c744f76ef1892b0c2e78f1","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"title":"[IAT355] Tutorial 2: Working with data in d3 and Vega | SOLUTION","cells":[{"id":0,"value":"# [IAT355] Tutorial 2: Working with data in d3 and Vega | SOLUTION\nContent adapted from [Lyn Bartram](https://observablehq.com/@drlynb)\n\n#### This is the solution version of the notebook. The tutorial version with skeleton code is available [here](https://observablehq.com/@iat355/02t-data?collection=@iat355/iat-355-tutorials-exercises)\n\n^{I am covering some of the basic in how we can work with data in the Observable/d3/Vega environment.\nYou should look at Mike Bostocks's Introduction to Data workbook before reading this.}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":11,"value":"## Modules, Libraries and Data Files\nImport the libraries and data files you need. In this workbook we'll be continuing with the titanic dataset. I will be using a number of datasets and showing you how to merge, join and/or reference them in the same visualizations.\n\nFirst, we will import a few necessary libraries, such as D3 and the Vega-lite library. We'll cover more about Vega-lite in Tutorial book 3, but let's get a brief look of it at the end. For best practices, imports should go at the very bottom of your notebook.\n\n_Refresher_: D3 is a data-driven library created by Mike Bostock. It's a powerful library for visualizing data with HTML, CSS, and SVG. D3 allows you to bind arbitrary data to a Document Object Model (DOM), and then apply data-driven transformations to the document, allowing for powerful interactivity with elements.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":6,"value":"d3 = require(\"d3@7\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":252,"value":"import {vl} from '@vega/vega-lite-api'","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":840,"value":"Now, we'll look at another more detailed titanic dataset and load it. Let's assign the value **titanic**","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":517,"value":"//IMPORT YOUR GIST HERE\ntitanic = d3.csv('https://gist.githubusercontent.com/krispybird/56bbacb2dfd84c7c6083c62d62a5417a/raw/1b937a4e7404920ea6d3f729611ec06d43e7ae8e/titanic_train.csv')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":507,"value":"$31","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":49,"value":"### Get the data into shape\n\n\nAll csv data are imported as text so I need to clean them. There are several ways:\n\n1. iterate through and change text fields to appropriate formats;\n2. change format while importing the d3.csv() function\n3. use the ```map()``` function\n\nExamples follow of each.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":124,"value":"md ` ### 1. Iterate through and change text fields to appropriate formats`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":83,"value":"- Use the + operator to iteratively change the text *Age* string in the *titanic* array to a number. This will modify the original array\n- What other fields in *titanic* should also be converted to a number? Take 2 mins and try it out","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":57,"value":"titanicParsed = { \n for (let i=0;id3.csv function parameters`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":82,"value":"titanicCleanedD3 =\nd3.csv(\"https://gist.githubusercontent.com/krispybird/56bbacb2dfd84c7c6083c62d62a5417a/raw/1b937a4e7404920ea6d3f729611ec06d43e7ae8e/titanic_train.csv\", function(d) {\n return {\n age: +d.Age,\n \n //what values should we convert? fill in the blanks\n //think about what data fields we want to return --- not all are cloned automatically!\n };\n })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":103,"value":"### 3. Use the ```array.map()``` function to change all types in the array","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":603,"value":"```.map()``` calls a function on every element in an array, and returns a new array.\n\n_**Note 1:**_ because we used the + operator to convert several values in method 1, some of these fields are already integers\n\n\n_**Note 2:**_ the ```=>``` Arrow function is a shorthand way of writing a function\nIt is equivalent to: \n\n```d = function(){```\n``` ...\n return a;\n}```","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":113,"value":"//the => Arrow function is a shorthand way of writing a function\n//It is equivalent to d = function(){\n// ...\n// return a;\n//} \n\n//a function is called on every element in array titanic, and creates a new array called titanicMap\n//for every element in titanicMap, the age key is assigned a value corresponding to the numerical value of titanic\ntitanicMap = titanic.map(d => {\n var a = d;\n a.Age = +d.Age;\n a.SibSp = +d.SibSp;\n a.Parch = +d.Parch;\n a.Fare = +d.Fare;\n\n //remember what other fields we want to add in\n return a;\n\n \n})\n ","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1405,"value":"What if I wanted to use map to do something like in method 2, and return only a subset of fields? I can do something like this","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1400,"value":"titanicMap2 = titanic.map(d => {\n return {\n name: d.Name,\n age: +d.Age,\n sibsp: +d.SibSp,\n parch: +d.Parch,\n fare: +d.Fare\n };\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1407,"value":"Let's try to get some more details out of our Titanic Dataset.\n\n**Try this question:**\n\nWhat was the age of the oldest person aboard the Titanic? \n\nWe can get this in one of several ways: through a for loop, or with D3","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":135,"value":"//Let's get the max age of who was on the titanic\n\n//try creating a variable called ageMax and find the age of the oldest person on the titanic\nageMax = {\n let maxAge = 0;\n for (let i = 0; i maxAge){\n maxAge = titanic[i].Age\n }\n }\n\n return (maxAge)\n}\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1133,"value":"{\n //Let's do this with d3.mean() now. We'll first have to use map to create an array with just the ages. \n let ages = titanicMap.map(function(d){ return +d.Age});\n return d3.mean(ages)\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1342,"value":"#### Try this question:\nFind the `median` and `mode` of the passengers' ages using `d3.median` and `d3.mode`","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1344,"value":"{\n let ages = titanicMap.map(function (d) {\n return +d.Age;\n });\n return (\n \"Statistics of Passengers' Ages: \\n Mean: \" +\n d3.mean(ages) +\n \" \\n Median: \" +\n d3.median(ages) +\n \" \\n Mode: \" +\n d3.mode(ages)\n );\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":462,"value":"Let's make an array of who was in cabin class 1 to see how we can start filtering some data","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":471,"value":"firstClass = titanicMap.filter(function(d) {return d.CClass == 1});","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1362,"value":"\"The number of passengers in first class is: \" + firstClass.length","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":721,"value":"Try getting the average age of the passengers in first class now. \nHint, create a new array with only the ages of the passengers in first class, then use ```d3.mean()```\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":479,"value":"//Try it here\n{\n let firstClassAges = firstClass.map(function (d) {\n return +d.Age;\n });\n return \"The average age in first class is: \" + d3.mean(firstClassAges);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":724,"value":"#### Try this question:\nWhich cabin class had highest average age of its passengers? We can find the average ages for second and third class to find out.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":752,"value":"//try to find the answer here\n//hint: you can use d3.mean to find the average age of passengers in second class\n\n{\n let secondClass = titanicMap.filter(function(d){ return d.CClass == 2});\n let secondClassAges = secondClass.map(function(d){ return +d.Age});\n return d3.mean(secondClassAges);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":757,"value":"//try to find the answer here\n//hint: you can use d3.mean to find the passengers in third class\n\n{\n let thirdClass = titanicMap.filter(function(d){ return d.CClass == 3});\n let thirdClassAges = thirdClass.map(function(d){ return +d.Age});\n return d3.mean(thirdClassAges);\n }\n\n// Here's another shorthand way of writing the function above using arrow notation\n// {\n// let tc_avg = titanicMap.filter(d => { return d.CClass == 3}).map(d => { return +d.Age});\n// return d3.mean(tc_avg);\n// }","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":744,"value":"#### Try this question:\nWho had the most expensive ticket aboard the Titanic, and how much was it?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":764,"value":"function calculateTicket()\n{\n let ticket = 0;\n let name = \"\"\n for ( let i=0; i < titanicMap.length;i++)\n {\n if (+titanicMap[i].Fare > ticket){\n ticket = titanicMap[i].Fare;\n name = titanicMap[i].Name;\n }\n }\n return name + \", $\" + ticket\n}\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":769,"value":"calculateTicket();","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1307,"value":"#### Try this question:\nHow much money was spent on fares in total from all tickets? \nWe can use ```d3.sum()``` to find this quickly","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1286,"value":"//Amount of money spent on fares from all tickets\n//toFixed(2) rounds the number to two decimal places - note that toFixed is turning this back into a string!\n\n{ \n let totalSum = d3.sum(titanicMap, function(d){return +d.Fare})\n return totalSum.toFixed(2) \n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":132,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":262,"value":"vl.markTick()\n .data(titanicMap)\n .encode(\n vl.x().fieldQ('Fare')\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":818,"value":"We can even start to make some charts, like this histogram showing age and the number of passengers in each group that survived.\n\nRemember that histograms are a specific type of visualization that groups continuous data into bins. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":297,"value":"vl\n .markBar()\n .data(titanicMap)\n .encode(vl.x().fieldQ(\"Age\").bin(true), vl.y().count())\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1353,"value":"We can try to manually set bin size by passing in the parameter `{step: 2}` inside `.bin()`.\n\nTry changing the step to 2, 5, 10, then 20. See how the appearance of the distribution changes. What do you think is the best centrality measurement?\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":923,"value":"---\n## MORE RESOURCES\n- The filled version of the notebook can be found [here](https://observablehq.com/d/3e4899e8e2957a66)\n- For a more in-depth explanation of Obervable, read [A Taste of Observable](https://observablehq.com/@observablehq/a-taste-of-observable?collection=@observablehq/introduction)\n- A more detailed explanation of Observable cells in [Introduction to Code](https://observablehq.com/@observablehq/introduction-to-code)\n- How does [Observable differ from Javascript](https://observablehq.com/@observablehq/observables-not-javascript)\n- The [Markdown Cheat Sheet](https://observablehq.com/@observablehq/markdown-quick-reference)\n- For a collection of resources on Observable, read [Observable User Manual](https://observablehq.com/@observablehq/user-manual?collection=@observablehq/introduction)\n- The [D3](https://d3js.org/) library - we will cover this in the following weeks","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":293,"value":"vega_explicit = {\n const [Vega, VegaLite] = await Promise.all([\n require(\"vega@3/build/vega.min.js\"),\n require(\"vega-lite@3/build/vega-lite.min.js\")\n ]);\n return async spec => {\n const div = document.createElement(\"div\");\n const view = new Vega.View(Vega.parse(VegaLite.compile(spec).spec));\n await view.initialize(div).runAsync();\n return div;\n };\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1203,"value":"embed = require('vega-embed@3')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":374,"value":"embedWithSpecificVersions = require.alias({\n \"vega\": \"vega@5.3.1\",\n \"vega-lite\": \"vega-lite@3.0.0\"\n})(\"vega-embed@4\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W33","stdlib":"1","titling":"auto"}} 30:"$2f:body"