Student at HKUST VisLab, Data Visualization Researcher
Learning Vis Tools Tutorial 6: Javascript basics and ObservableLearning Vis Tools Tutorial 8: Visualization with D3.jsLearning Vis Tools Tutorial 9: Visualization and Interaction with D3.js
Learning Vis Tools Tutorial 7: Vega-Lite and data processing libraries
// For example, we have a function that converts color names into hex code
colorToHex('red')
// We have an array of color names, and we want to have them in hex code
colorNames = ['Tomato', 'Orange', 'DodgerBlue', 'MediumSeaGreen', 'Gray', 'SlateBlue', 'Violet', 'LightGray']
// We may use loop
{
let colors = []
for (let i = 0; i < colorNames.length; i++) {
colors.push(colorToHex(colorNames[i]))
}
return colors
}
// It is much easier to use "map".
// It calls the "colorToHex" function for each item in the array
colorNames.map(colorToHex)
/******************************************************************************
* TODO: *
* Try to make an array of your favorite colors and use map to find out their *
* hex codes! You may take a look into the link to "HTML Color Names" above. *
******************************************************************************/
{
var myFavColor = ['purple', 'grey', 'white', 'pink', 'crimson'];
return myFavColor
}
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Reduce an array into a single value, it has the signature:
// arr.reduce(callback[, initialValue])
// where initialValue is optional. In our case, we pass in 0
[1, 2, 3, 4, 5].reduce(
(accumulator, currentValue) => { return accumulator + currentValue },
0
)
// For better illustrating what reduce is doing, we "print" the values for
// inspection. Be warned that, we usually do not modify variables in the callback
// function passing into map/reduce/filter functions, it violates the functional
// programming principles. To know more about functional programming in Javascript,
// see https://medium.com/javascript-scene/master-the-javascript-interview-what-is-functional-programming-7f218c68b3a0
{
mutable reducePrint = '\n'
return [1, 2, 3, 4, 5].reduce(
(accumulator, currentValue) => {
mutable reducePrint += `accumulator: ${accumulator} currentValue: ${currentValue}\n`
return accumulator + currentValue
},
0
)
}
// One of the powerful usage of reduce is to convert an array into an object,
// so that it can be used for dictionary lookup
colorMapping = colorNames.reduce(
(mapping, color) => {
mapping[color] = colorToHex(color)
return mapping
},
{}
)
// We can now use "colorMapping" for dictionary lookup
colorMapping['Violet']
/******************************************************************************
* TODO: *
* Try to reduce the array of your favorite colors into an object! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Filter function takes only one argument, which is a callback that calls on
// each item, and gives true or false as return value. All the items with true
// are kept, dropped otherwise.
// The following statement returns an array with color names with length larger
// than 6. Reminded that, the original array is not modified.
colorNames.filter(color => color.length > 6)
// By changing the callback function, we can filter the unwanted values in the array
colorNames.filter(color => colorToHex(color).indexOf('#ff') === -1)
/******************************************************************************
* TODO: *
* Try to filter the colorName array, so that only colors with "Blue" in *
* their names are kept! i.e. "DodgerBlue" and "SlateBlue" *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// It is very often that texts in html are wrapped with space/new line characters
wikiQuote = ` Data visualization is both an art and a science.[3] It is viewed as a branch of descriptive statistics by some, but also as a grounded theory development tool by others.\n`
// .trim() is very convenient for removing these unwanted characters
wikiQuote.trim()
// Spliting names from a list of names separated by comma
'John Doe, Richard Roe'.split(',')
// There is a space in front of "Richard Roe", we would like to remove that
// by str.trim()
'John Doe, Richard Roe'.split(',').map(s => s.trim())
colorsInString = 'Tomato, Orange, DodgerBlue, MediumSeaGreen, Gray, SlateBlue, Violet, LightGray'
/******************************************************************************
* TODO: *
* Try to split "colorsInString" to colors! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// "#ff" is at the beginning of the string "#ff6347", the hex code of "Tomato"
colorToHex('Tomato').indexOf('#ff')
// "Blue" starts at the sixth character of "StaleBlue"
'StaleBlue'.indexOf('Blue')
// indexOf() returns -1 if not found
colorToHex('StaleBlue').indexOf('#ff')
// It has the signature str.substring(indexStart[, indexEnd]),
// where indexEnd is optional, when not provided, it returns the string
// from indexStart to the end
['LightGreen', 'LightBlue', 'LightGray'].map(str => str.substring(5))
/******************************************************************************
* TODO: *
* Try to get the hex code for CMYK (cyan, magenta, yellow and black) without *
* the "#"! E.g. "00ffff" for cyan *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// It has the signature str.replace(regexp|substr, newSubstr|function)
// This will become a long passage to explain all of it, we simply look into
// the basic usage of it.
// Replacing "Light" by "Dark" in a string
['LightGreen', 'LightBlue', 'LightGray'].map(str => str.replace('Light', 'Dark'))
// The default is to replace the first occurance, while some of the time, we want
// to replace all the occurances
'LightGreen, LightBlue, LightGray'.replace('Light', 'Dark')
// We need to use regex (short term for REGular EXpression) with a "g" flag at
// the end, which means global
'LightGreen, LightBlue, LightGray'.replace(/Light/g, 'Dark')
/******************************************************************************
* TODO: *
* Try to replace "Green" by "Blue" of the following string: *
* 'LightGreen, DarkGreen, DeepGreen, DimGreen, MediumGreen, SlateGreen' *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Simply require('lodash'), it is conventionally assigned to the variable "_",
// which is pronounced as "underscore" (which is also the name of the predecessor
// of the lodash library, but that is another story.)
_ = require('lodash')
// An array of objects, it can be declared by array/object literal,
// or use what we have learnt to split up a string and make it organized
colors = 'Navy Blue, Light Blue, Dark Green, Light Green, Dark Blue, Ruby Red, Rose Red, Light Red'
.split(',')
.map(str => str.trim().split(' '))
.map(([ modifier, color ]) => ({ color, modifier }))
// The above statement is equivalent to:
// colors = [
// { color: 'Blue', modifier: 'Navy' },
// { color: 'Blue', modifier: 'Light' },
// { color: 'Green', modifier: 'Dark' },
// { color: 'Green', modifier: 'Light' },
// { color: 'Blue', modifier: 'Dark' },
// { color: 'Red', modifier: 'Ruby' },
// { color: 'Red', modifier: 'Rose' },
// { color: 'Red', modifier: 'Light' }
// ]
// Group colors by their main color
_.groupBy(colors, 'color')
/******************************************************************************
* TODO: *
* Try to group colors by their modifiers! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Extract the attribute "modifiers" from the "colors" array
_.map(colors, 'modifier')
// The equivalent in plain Javascript
colors.map(color => color.modifier)
// Zipping up two arrays, resulting in an array of arrays
colorPairs = _.zip(_.map(colors, 'modifier'), _.map(colors, 'color'))
// Then we can join them up to make the full name of the colors in strings
colorPairs.map(color => color.join(' '))
/******************************************************************************
* TODO: *
* Try to zip up the first names and last names, then join them up with a *
* space in between! Like "Jake Smith", "Margaret Jones"! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
weather = [
{ date: "1997-01-01 00:00:00", temperature: 18 },
{ date: "1997-01-01 01:00:00", temperature: 18 },
{ date: "1997-01-01 02:00:00", temperature: 17 },
{ date: "1997-01-01 03:00:00", temperature: 17 },
{ date: "1997-01-01 04:00:00", temperature: 16 },
{ date: "1997-01-01 05:00:00", temperature: 16 },
{ date: "1997-01-01 06:00:00", temperature: 17 },
{ date: "1997-01-01 07:00:00", temperature: 16 },
{ date: "1997-01-01 08:00:00", temperature: 18 },
{ date: "1997-01-01 09:00:00", temperature: 20 }
]
// Find the record with minimum temperature
_.minBy(weather, 'temperature')
// .termperature to get the minimum temperature value
_.minBy(weather, 'temperature').temperature
// meanBy does not return the record, but the value
_.meanBy(weather, 'temperature')
/******************************************************************************
* TODO: *
* Try to find the maximum temperature in "weather" using lodash! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Prettily print out the date and time of the moment
moment().format('Do MMMM YYYY, h:m a')
/******************************************************************************
* TODO: *
* Try to print the date in a format like "26-03-2019 17:49:59"! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Parse string into Date, then add 5 days to it
moment('05-03-2019', 'MM-DD-YYYY').add(5, 'days').format('Do MMMM YYYY, h:m a')
/******************************************************************************
* TODO: *
* Try to write down your birthday in the format "14-2-2019", parse it and *
* print it in a format like "8th May 2019"! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Weather data of Hong Kong International Airport (which ICAO airport code is VHHH) from 1997-01-01 to 2006-12-31
VHHH1997to2006 = d3.csv('https://raw.githubusercontent.com/leoyuholo/learning-vis-tools/master/tutorial07/lab7/VHHH_1997-01-01_2006-12-31.csv')
// Weather data of Hong Kong International Airport from 2007-01-01 to 2017-10-28
VHHH2007to2017 = d3.csv('https://raw.githubusercontent.com/leoyuholo/learning-vis-tools/master/tutorial07/lab7/VHHH_2007-01-01_2017-10-28.csv')
VHHH1997to2006MonthlyTempRange = {
// Remove data with invalid temperature, check the variable above, we can find that
// some of the records have temperature as empty string (i.e. "")
let filterByValidTemp = VHHH1997to2006.filter(record => record.temperature !== '')
// Convert string to number, as all the columns are in string data type, we need to
// convert them into numbers, so that we do not compare the temperature alphabetically
let tempAsNumber = filterByValidTemp.map(record => ({ ...record, temperature: +record.temperature }))
// Group by month, which the return object has date as keys and records of that day as values
let groupByMonth = _.groupBy(tempAsNumber, record => record.date.substring(0, 7))
// Map over the values of the object, which are the records of each month, we find
// out the min and max of each month, alongside with the year and the month
// We use _.map() instead of the built-in array.map() because it provides the "date"
// in the arguments passing into the callback function
let monthlyTempRange = _.map(groupByMonth, (monthRecords, date) => {
let dateMoment = moment(date)
return {
year: dateMoment.year(),
month: dateMoment.month() + 1,
min: _.minBy(monthRecords, 'temperature').temperature,
max: _.maxBy(monthRecords, 'temperature').temperature
}
})
return monthlyTempRange
}
/******************************************************************************
* TODO: *
* Try to preprocess the dataset from 2007 to 2017, which is stored in the *
* variable "VHHH2007to2017"! *
* The return value should be an array with 130 items. *
******************************************************************************/
VHHH2007to2017MonthlyTempRange = {
return []
}
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Concatenate the two datasets together
data = VHHH1997to2006MonthlyTempRange.concat(VHHH2007to2017MonthlyTempRange)
vegaEmbed({
data: { values: data },
width: 500,
height: 400,
title: 'Heatmap of Hong Kong Temperature',
mark: 'rect',
encoding: {
x: { field: 'year', type: 'o' },
y: { field: 'month', type: 'o' },
color: {
field: 'max',
type: 'q',
// We use "blueorange" color scheme instead of the default "viridis"
// because it is closer to the convention that red means hot and
// blue means cold. We also set the domain to between 0 and 40, to
// have a consistent color encoding when plotting the minimum
// temperature
scale: { scheme: 'blueorange', domain: [0, 40] }
}
}
})
/******************************************************************************
* TODO: *
* Try to plot the heatmap with the minimum temperature instead of maximum! *
* Remember to fill in your code to complete the cell calculating the values *
* in "VHHH2007to2017MonthlyTempRange" array first! *
******************************************************************************/
/******************************************************************************
* END OF YOUR CODE *
******************************************************************************/
// Warning: The following plot is rather computation intensive, comment it out if you experience lagging.
// vegaEmbed({
// data: { values: VHHH1997to2006.concat(VHHH2007to2017) },
// width: 500,
// height: 400,
// title: 'Heatmap of Hong Kong Temperature',
// mark: 'rect',
// transform: [
// { filter: 'datum.temperature !== ""' }
// ],
// encoding: {
// x: { timeUnit: 'year', field: 'date', type: 'o' },
// y: { timeUnit: 'month', field: 'date', type: 'o' },
// color: {
// aggregate: 'max',
// field: 'temperature',
// type: 'q',
// scale: { scheme: 'blueorange', domain: [0, 40] }
// }
// }
// })
Purpose-built for displays of data
Observable is your go-to platform for exploring data and creating expressive data visualizations. Use reactive JavaScript notebooks for prototyping and a collaborative canvas for visual data exploration and dashboard creation.
