29:null 2a:["$","title",null,{"children":"Data visualization with Vega-Lite | Ayaan Kazerouni | Observable"}] 2f:T902,

It's worth explaining what's happening above. If you're curious, expand this.

First, the function call

    import('https://cdn.jsdelivr.net/npm/vega-datasets@3.1.0/+esm')

executes. It is an _asynchronous function call_. This means, unlike other functions which will finish executing before moving on, this function kicks off its execution and moves on to the next line before it finishes. Therefore, the function's return value is a [_Promise_](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise).

Promises are special values in JavaScript that will _eventually_ resolve in a value. You can use them to perform long-running tasks like downloading data or making web requests. The long-running task is kicked off in the background—that way other behaviors (like your other cells in this notebook) don't all have to wait for it unless they really need to. This model of concurrent execution is what lets your browser do a bunch of things concurrently (like stream a YouTube video in one tab while downloading a file in another tab while tracking your Uber's location in a third tab).

In this particular case, we want to _wait_ for the Promise to resolve because we need to use its value immediately. So we `await` it.

 
    await import('https://cdn.jsdelivr.net/npm/vega-datasets@3.1.0/+esm')

Now, the result of the expression above is a plain old Object. From that Object, we want the value associated with the `default` key, so we ask for the `default` key from the _whole expression_. We use parentheses to do this:

    (await import("https://cdn.jsdelivr.net/npm/vega-datasets@3.1.0/+esm")).default;

We now have _another_ object whose values are _other_ asynchronous functions meant for downloading individual datasets.

In the `cars` cell, we call one of those asynchronous functions (`cars.json`). It downloads the `cars` dataset. _Once again, we `await` the result_ so that we can use it as a parameter to another function. This time, we want to make a table out of it using `aq.from`.

30:[{"id":"f4a132c88d02a4d753acdf11b8a9f8ad3e320b6b9bfab64c8c7417e2a4bae210cb14c461269e0e63c9517a3abfcb00e4901210b6549b03f79950ba58fae42ae9","href":"https://static.observableusercontent.com/files/f4a132c88d02a4d753acdf11b8a9f8ad3e320b6b9bfab64c8c7417e2a4bae210cb14c461269e0e63c9517a3abfcb00e4901210b6549b03f79950ba58fae42ae9","name":"scatterplot@1.png","type":"image/png","lastModified":1744323415928.406,"size":65138},{"id":"300b093b9c70867f647f54b6edf6cca7c8e8167959008aa02d243d12e6002551103e05c8dc04140722ec8f689490d2d25841edb4233fc25925a81e8c4664c7f2","href":"https://static.observableusercontent.com/files/300b093b9c70867f647f54b6edf6cca7c8e8167959008aa02d243d12e6002551103e05c8dc04140722ec8f689490d2d25841edb4233fc25925a81e8c4664c7f2","name":"image@1.png","type":"image/png","lastModified":1776296345368.615,"size":23463}] 2d:{"id":"694b2b9e8e4f843c","slug":"csc477-vega-lite-intro","title":"Data visualization with Vega-Lite","thumbnail":"87f8f9beccb6e01d5288c3111193093847de0a5b2b580186e44fe4fa0068c39d","access_level":"unlisted","create_time":"$D2025-04-08T00:23:06.410Z","update_time":"$D2026-05-05T16:50:00.794Z","view_time":null,"first_public_time":null,"has_unpublished_edits":null,"fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"1d83bfce92a9c574","login":"ayaankazerouni","name":"Ayaan Kazerouni","avatar_url":"https://avatars-next.observableusercontent.com/0e57571babb328d16b661fb3c0f94c9629fc275ae03e484ce19dbcf72fa1be0d","tier":"starter_2024"},"version":864,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"7a2aaf0dcd8f4999","login":"ayaankazerouni","name":"Ayaan Kazerouni","avatar_url":"https://avatars-next.observableusercontent.com/d56bdf990c9be43beacbf55c7ca358b0aa7c36eb9211e9fc23f3b50bc80603bd","tier":null}],"members":[{"type":"team","id":"1d83bfce92a9c574","login":"ayaankazerouni","name":"Ayaan Kazerouni","avatar_url":"https://avatars-next.observableusercontent.com/0e57571babb328d16b661fb3c0f94c9629fc275ae03e484ce19dbcf72fa1be0d","tier":"starter_2024","role":"editor"},{"type":"individual","id":"7a2aaf0dcd8f4999","login":"ayaankazerouni","name":"Ayaan Kazerouni","avatar_url":"https://avatars-next.observableusercontent.com/d56bdf990c9be43beacbf55c7ca358b0aa7c36eb9211e9fc23f3b50bc80603bd","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":56,"collection":{"id":"6c810503155b0a33","slug":"csc-477-cal-poly","title":"CSC 477","owner":{"id":"1d83bfce92a9c574","login":"ayaankazerouni","name":"Ayaan Kazerouni"}},"type":"notebook","body":{"title":"Data visualization with Vega-Lite","cells":[{"id":0,"value":"# Data visualization with Vega-Lite","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":5,"value":"The previous lectures have talked about data-wrangling, marks, and channels.\n\n- **Data** are the abstractions we choose to represent visually to allow users to perform some analytic, exploratory, or explanatory task.\n- **Marks** are the shapes or \"glyphs\" drawn in the visualization.\n- **Channels** are the visually perceptive properties of marks, like their horizontal or vertical positions, color, size, etc.\n\nWith a basic framework of data, marks, and channels, we can design fairly sophisticated visualizations.\n\nThe tool we will use in this class, Vega-Lite, makes use of this framework to provide a simple _declarative_ grammar in which we can specify our visualizations.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":10,"value":"Like Arquero, Vega-Lite is available by default in Observable notebooks, through the variable `vl`.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":795,"value":"toc(\"h2,h3\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":15,"value":"---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":806,"value":"## Introduction\n\nThis first section provides an introduction to Vega-Lite's basic concepts and grammar.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":19,"value":"### The dataset","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":124,"value":"We'll use the `cars` dataset from the vega-datasets collection. It has a good mix of data types to help showcase some of Vega-Lite's core capabilities.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":118,"value":"cars.view(10)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":128,"value":"### Our first chart","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":130,"value":"To begin with, let's try to make a simple scatterplot that looks like this. This is just a screenshot of our target visualization.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":338,"value":"scatterplot1 = FileAttachment(\"scatterplot@1.png\").image({ width: 300 })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":146,"value":"First, stop and ask:\n1. What _mark_ is being drawn?\n2. What _data_ is being visualized?\n3. Which _visual channels_ are being used to communicate those data?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":148,"value":"// Our code goes here: JSON version.\n{\n // Write the chart as a JSON object\n const chartSpec = {\n data: {\n values: cars.objects()\n },\n mark: 'point',\n encoding: {\n x: {\n field: 'Miles_per_Gallon',\n type: 'quantitative'\n },\n y: {\n field: 'Horsepower',\n type: 'quantitative'\n }\n },\n width: 250,\n height: 250\n }\n\n // Create and render the chart\n return vl.spec(chartSpec)\n .render()\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":151,"value":"In addition to the JSON syntax, there is also the [Vega-Lite JavaScript API](https://vega.github.io/vega-lite-api/). It's more concise, and gives you some autocomplete suggestions (at least for top-level `vl` functions), but is less documented.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":154,"value":"// Our code goes here: JavaScript version.\nvl.markPoint()\n .data(cars)\n .encode(\n vl.x().fieldQ('Miles_per_Gallon'),\n vl.y().fieldQ('Horsepower')\n )\n .width(250)\n .height(250)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":179,"value":"**Can we make improvements over the initial chart?** Right now it looks like there's a fair bit of overlap in the middle due to \"overplotting\". Lots of points are drawn on top of each other, obscuring the points underneath.\n\nIncreasing the size of the plot may help. We can specify the plot's width as a factor of Observable's in-built `width` property. That way our chart size will be responsive to the window size (reactivity for the win!).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":757,"value":"width","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":181,"value":"// Code: Plot with responsive dimensions\nvl.markPoint()\n .data(cars)\n .encode(\n vl.x().fieldQ('Miles_per_Gallon'),\n vl.y().fieldQ('Horsepower')\n )\n .width(0.5 * width)\n .height(0.3 * width)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":535,"value":"### Nominal fields","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":192,"value":"Our dataset contains cars from the following `Origin`s.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":194,"value":"cars.groupby('Origin').count().view()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":347,"value":"Let's incorporate that information into the chart. **Which visual channel do you think we should use?**","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":202,"value":"// Code: Plot with responsive dimensions\nvl.markCircle({ size: 50, opacity: 0.6 })\n .data(cars)\n .encode(\n vl.x().fieldQ('Miles_per_Gallon')\n .title(\"Fuel efficiency (mpg)\"),\n vl.y().fieldQ('Horsepower'),\n vl.color().fieldN('Origin')\n )\n .width(0.5 * width)\n .height(0.3 * width)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":261,"value":"The \"`Miles_per_Gallon`\" title for the x-axis is a little ugly. There's no reason why our table's column names need to appear in our charts. You can set _axis_ options for the position encodings.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":493,"value":"#### Faceting ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":495,"value":"// Code: can we separate origin further? We've already \"used up\" the x and y coordinates.\nvl.markCircle({ size: 50, opacity: 0.6 })\n .data(cars)\n .encode(\n vl.x().fieldQ('Miles_per_Gallon')\n .title(\"Fuel efficiency (mpg)\"),\n vl.y().fieldQ('Horsepower'),\n vl.color().fieldN('Origin'),\n vl.column().fieldN('Origin')\n )\n .width(0.2 * width)\n .height(0.2 * width)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":507,"value":"### Ordinal fields\n\nIs `Cylinders` quantitative or ordinal?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":597,"value":"cars.sample(5).view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":600,"value":"cars.groupby('Cylinders').count().view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":499,"value":"// Number of cars of different cylinders\nvl.markBar({ fill: 'burlywood' })\n .data(cars)\n .encode(\n vl.x().fieldO('Cylinders'),\n vl.y().count()\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":510,"value":"// Average acceleration of cars with different numbers of cylinders\nvl.markBar({ fill: 'burlywood' })\n .data(cars)\n .encode(\n vl.x().fieldO('Cylinders'),\n vl.y().mean('Acceleration')\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":533,"value":"// Turning quantitative fields into Ordinal fields through binning\n// What are the data?\n// What are the channels?\n// What's the mark?\n\n// data.groupby('mpg_bin').count()\nvl.markBar({ fill: 'burlywood' })\n .data(cars)\n .encode(\n vl.x().fieldQ('Miles_per_Gallon').bin({ maxbins: 20 }),\n vl.y().count(),\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":641,"value":"## Vega-Lite transformations","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":642,"value":"Examples used in class for the lecture on data transformations.\n\nLet's look at reports of UFO \"sightings\" around the world. Data from the [UFO sightings dataset from the CORGIS collection](https://corgis-edu.github.io/corgis/json/ufo_sightings/).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":644,"value":"ufoSightings.view(6)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":645,"value":"That's ${ufoSightings.numRows()} UFO sightings!","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":647,"value":"### Aggregations","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":648,"value":"#### Counting\n\n_How frequently have UFO sightings been reported been over the years?_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":649,"value":"vl.markPoint()\n .data(ufoSightings)\n .encode(\n vl.x().year('sightingTime') // <--- Also using Vega-Lite's facilities for Time Units\n )\n .width(800)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":650,"value":"If there were multiple reports in a year, the points are simply drawn on top of each other. No good.\n\nThe chart below uses the `.count()` aggregation, and encodes it using the y-axis.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":651,"value":"vl.markBar()\n .data(ufoSightings)\n .encode(\n vl.x().year('sightingTime'),\n vl.y().count() // <-- The count aggregation is applied per x-axis value.\n )\n .height(250)\n .width(700)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":652,"value":"_Note: Do you think we suddenly solved our \"UFO problem\" in 2015? What's with the big dip?_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":653,"value":"The `.count()` aggregation is applied to the `.y()` encoding, and no field name needs to be specified. Counting is performed per value on the x-axis. In this case, per year.\n\nWe also changed our mark to bars instead of points. Lines could work as well, to show the uptick in reports.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":654,"value":"#### Binning","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":655,"value":"_How long are encounters, typically?_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":656,"value":"vl.markPoint()\n .data(ufoSightings)\n .encode(\n vl.x().fieldQ('encounterDuration')\n )\n .render()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":657,"value":"Woah, that's pretty skewed. Instead of adjusting the scale, let's omit those outlying values. Condolences to those whose encounters lasted, uh, over 2 years.\n\nLet's omit outliers above the [high outlier threshold](https://en.wikipedia.org/wiki/Interquartile_range#Outliers), i.e., anything over `Q3 + 1.5 * (Q3 - Q1)`.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":658,"value":"In this case, we will be omitting encounters that lasted more than ${highOutlierThreshold} seconds (around ${(highOutlierThreshold / 60).toFixed()} minutes). After omitting those outliers, we're left with ${shorterEncounters.numRows()} close encounters of the third kind.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":659,"value":"highOutlierThreshold = {\n const encounterLengths = ufoSightings.array('encounterDuration');\n const firstQuartile = d3.quantile(encounterLengths, 0.25);\n const thirdQuartile = d3.quantile(encounterLengths, 0.75);\n return thirdQuartile + (1.5 * (thirdQuartile - firstQuartile));\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":660,"value":"shorterEncounters = ufoSightings\n .params({ outlier: highOutlierThreshold }) // See these docs https://idl.uw.edu/arquero/api/table#params for why we do this\n .filter((d, $) => d.encounterDuration < $.outlier)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":661,"value":"vl.markCircle()\n .data(shorterEncounters)\n .encode(\n vl.x().fieldQ('encounterDuration')\n )\n .width(400)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":662,"value":"The chart above gives us some sense that most encounters range from 1 second to 1400 seconds (around 23 minutes).\nLike the years, it would be good to count _how many_ encounters there were of a given length.\n\nHere's that `.count()` aggregation again.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":663,"value":"vl.markCircle()\n .data(shorterEncounters)\n .encode(\n vl.x().fieldQ('encounterDuration'),\n vl.y().count()\n )\n .width(400)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":664,"value":"Unfortunately, we get a pretty strange-looking chart. The basic problem is that the `.count()` aggregation counts _per value_ from the x-axis. When the x-axis encodes a quantitative field, _that's a lot of unique values_! Is it really useful to distinguish between encounters that were 200 seconds long vs. encounters that were 190 or 210 seconds long?\n\nThat's where the `.bin()` aggregation comes in. We can ask Vega-Lite to aggregate `encounterDuration` values into discrete buckets, and then let the y-axis work with those buckets.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":665,"value":"vl.markBar()\n .data(shorterEncounters)\n .encode(\n vl.x().fieldQ('encounterDuration')\n .bin(true), // <-- Automatically create equal-length bins of data. Defaults to \"close to 10\" bins.\n vl.y().count()\n )\n .width(400)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":666,"value":"The `.bin()` aggregation takes the quantitative `encounterDuration` data and makes it ordinal data, grouping up data into bins at equal intervals. By default, we get 10 bins (0–200, 200–400, 400–600, and so on). We can also configure the number of bins using the `maxbins` option.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":667,"value":"vl.markBar()\n .data(shorterEncounters)\n .encode(\n vl.x().fieldQ('encounterDuration')\n .bin({ maxbins: 20 }),\n vl.y().count()\n )\n .width(400)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":668,"value":"### Other summary aggregations\n\nIn addition to `.count()`, Vega-Lite provides a number of other aggregations, like `.sum()`, `.mean()`, `.min()`, `.max()`, and `.median()`. See the [Vega-Lite Transform docs for details](https://vega.github.io/vega-lite/docs/aggregate.html).\n\nThe chart below shows the **year** on the x-axis and the **sum of encounter durations** for each year on the y-axis. Looks like folks spent a total of **16 days** or so being ogled by UFOs in 2012.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":669,"value":"vl.markBar()\n .data(shorterEncounters)\n .encode(\n vl.x().year('sightingTime'),\n vl.y().mean('encounterDuration'),\n )\n .width(800)\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":838,"value":"shorterEncounters.view(6)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":670,"value":"*Change the `.sum()` aggregation above to see the other aggregations in action.*","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":671,"value":"### In-class activity\n\n${flag()} Can you draw a chart to figure out at what _time of day_ encounters tended to take place? You'll use the `.hours()` transform on the `sightingTime` field to bin the data by _hour_.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":672,"value":"// Your code here\nvl.markBar()\n .data(shorterEncounters)\n .encode(\n vl.x().hours('sightingTime'),\n vl.y().count()\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":673,"value":"### Other transforms","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":674,"value":"#### Sort\n\n_Which states have the most UFO sightings?_ Using the `.count()` aggregation, we can simply draw a barchart with states on the x-axis, and the number of sightings on the y-axis.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":675,"value":"// Code: Plot with states sorted in different orders\nvl.markBar()\n .data(ufoSightings)\n .encode(\n vl.x().fieldN('state')\n // .sort('y') // sort by the y encoding, ascending \n // .sort('-y') // sort by the y encoding, descending\n .sort(vl.median('encounterDuration').order('descending')) // sort by something else (but then you should make it clear in the axis title)\n ,\n vl.y().count()\n )\n .render()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":676,"value":"By default, Vega-Lite will sort the states in alphabetical order. This is sometimes, but not always, the best choice.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":677,"value":"Consider:\n\n- If the user wants to _locate_ a particular state, sorted order is best (well, giving them the ability to filter is best, but we'll get to interactivity next week). With a sorted list, it's much easier to find, say, California by visually searching alphabetically. We can quickly see California's frankly alarming number of UFO sighting reports.\n- But if the user wants to _identify extremes_ (minimum and maximum), alphabetical order isn't the best.\n\nWe can change the order of the x-axis by using the `.sort()` transform as part of the encoding.\n\nExpand the cell above and uncomment the different `sort` transforms to see their effects.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":678,"value":"### Top-level data transforms\n\n#### Calculate\n\nRemember `.derive()` in Arquero? We can also use Vega-Lite to derive fields \"on the fly\".\n\nLet's stop ignoring those unfortunate folks who had days-long UFO encounters. Instead of simply omitting those fields, let's compute a _new_ ordinal field for encounter durations: short (< 30 seconds), medium (>= 30 seconds and < 600 seconds), and long (>= 600 seconds). 30 and 600 are the 1st and 3rd quartiles, respectively.\n\nThis is kind of what we want to do:\n\n```js\nufoSightings\n .derive({\n encounterOrdinal:\n d => d.encounterDuration < 30 ? 'short' : (d.encounterDuration < 600 ? 'medium' : 'long')\n })\n```","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":679,"value":"*Take a second to understand what the ternary expressions are doing above.*","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":680,"value":"But instead of preprocessing the data ourselves, we can do it in the Vega-Lite chart spec using the `.calculate()` transform.\n\nUnlike the other transforms we've seen, this doesn't happen as part of an encoding. It happens _before_ the encodings are applied.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":681,"value":"vl.markBar()\n .data(ufoSightings)\n .transform(\n vl.calculate('datum.encounterDuration < 30 ? \"short\" : \\\n datum.encounterDuration < 600 ? \"medium\" : \"long\"')\n .as('encounterOrdinal')\n )\n .encode(\n vl.y().fieldO('encounterOrdinal')\n .sort(['short', 'medium', 'long']), // Specifying a sort order for an ordinal field\n vl.x().count()\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":682,"value":"The calculate transform lets us compute a new value based on a given row in the table. We do this by evaluating an _expression_ (using [Vega-Lite's allowed expressions](https://vega.github.io/vega-lite/docs/calculate.html)). The \"current\" row is referred to as `datum` within the expression.\n\nOnce the new field is calculated, it's available to use in encodings (or subsequent transforms).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":683,"value":"#### Filter\n\nSimilarly, we can apply filters \"on the fly\". Earlier, we had filtered out encounters that lasted more than ${highOutlierThreshold} seconds. We did that using Arquero and saved the result in a whole new table called `shorterEncounters`. The `vl.filter()` transform lets us do this in the Vega-Lite chart spec.\n\nAgain, the transform takes in a Vega-Lite expression, using `datum` to access each record.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":684,"value":"// Code: Add a filter to the previous chart to only keep encounters shorter than 1455 seconds\nvl.markBar()\n .data(ufoSightings)\n .transform(\n vl.filter(`datum.encounterDuration <= ${highOutlierThreshold}`),\n vl.calculate('datum.encounterDuration < 30 ? \"short\" : datum.encounterDuration < 600 ? \"medium\" : \"long\"')\n .as('encounterOrdinal')\n )\n .encode(\n vl.y().fieldO('encounterOrdinal')\n .sort(['short', 'medium', 'long']), // Specifying a sort order for an ordinal field\n vl.x().count()\n )\n .render()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":685,"value":"Notice that we used the original `ufoSightings` table in the above chart, not the filtered `shorterEncounters` table.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":686,"value":"$${flag('important', undefined, 'firebrick')} Transforms are applied in the order in which they appear. This is important to remember when one transform might affect the later ones, e.g., if you're calculating a new field that you might use in subsequent transforms. **Vega-Lite will fail silently in cases like these, making it hard to know what exactly went wrong.**","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":687,"value":"## In-class activity\n\n${flag()} Use your knowledge of Vega-Lite and its transformations to re-create the following heatmap, which shows the frequency with which different UFO shapes are reported at various times of day.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":688,"value":"ufoSightingsScreenshot = FileAttachment(\"image@1.png\").image()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":689,"value":"Tips:\n- `hours(datum.sightingTime)` will get you the _hour_ from the timestamp. In our dataset, times are in 24-hour format, i.e., 16 is 4pm, 19 is 7pm, etc.\n- It may make your life easier to break up `.calculate` transforms into multiple steps—the field you calculate in one step becomes available to use in the next step.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":713,"value":"ufoSightings.view()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":690,"value":"// Your code here","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":691,"value":"---","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":28,"value":"## Imports","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":692,"value":"ufoSightings = {\n const ufoJSON = await (await fetch(\"https://corgis-edu.github.io/corgis/datasets/json/ufo_sightings/ufo_sightings.json\")).json()\n return aq.from(\n ufoJSON.map(u => ({\n city: u.Location.City,\n state: u.Location.State,\n country: u.Location.Country,\n latitude: u.Location.Coordinates['Latitude '],\n longitude: u.Location.Coordinates['Longitude '],\n shape: u.Data.Shape,\n encounterDuration: u.Data['Encounter duration'],\n description: u.Data['Description excerpt'],\n sightingTime: `${u.Dates.Sighted.Year}-${u.Dates.Sighted.Month}-${u.Date.Sighted.Day} ${u.Dates.Sighted.Hour}:${u.Dates.Sighted.Minute}`,\n documentedDate: `${u.Dates.Documented.Year}-${u.Dates.Documented.Month}-${u.Dates.Documented.Day}`,\n }))\n )\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":694,"value":"import { vl } from '@vega/vega-lite-api-v5'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":695,"value":"import {toc} from '@jonfroehlich/collapsible-toc'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":696,"value":"import {flag} from '@ayaankazerouni/utils'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":113,"value":"import { aq, op } from '@uwdata/arquero'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":100,"value":"cars = aq.from(await datasets['cars.json']())","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":444,"value":"world = datasets['world-110m.json']()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":22,"value":"datasets = (await import('https://cdn.jsdelivr.net/npm/vega-datasets@3.1.0/+esm')).default;","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":30,"value":"$2f","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"}],"files":"$W30","stdlib":"1","titling":"auto"}} 2e:"$2d:body"