The avatar for @lnicoletti
leonardo Nicoletti
Social scientist and data nerd with a passion for storytelling and journalism.
Published
Edited
Jan 10, 2022
1 fork
6 stars
5
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
// html`<div class="FigTitle" style="font-weight:500; font-size: 21px;text-align:justify">
// The Social Cost of the pandemic: <span style="color:#cbb35c;font-weight:900">Wealthy Brits</span> Work From Home While <span style="color:#9c75b4;font-weight:900">Poor Brits</span> Continue To Commute
// <div>
// <div class="FigSubtitle">This figure illustrates the relationship between household income and the intensity in travel to workplaces over the course of the COVID-19 pandemic. In this map, areas coloured in gold represent areas that are both high income and that experienced lower levels of travel to workplaces throughout the pandemic. Here, residents of these wealthier areas were able to successfully reduce their mobility and adopt a work-from-home lifestyle. In contrast, areas coloured in purple represent areas that are both low income and that experienced higher levels of travel to workplaces throughout the pandemic. Residents of these poorer areas were not able to reduce their mobility as much, and they continued to commute to work. Finally, shades of brown represent (the few) local authorities that are both high income and that experienced higher levels of travel to workplaces throughout the pandemic. Regardless of the wealthy status of these areas, residents continued to commute to work.</div>
// `
Insert cell
// html`<select name="view" id="chartView">
// <option value="map">Map</option>
// <option value="chart">Chart</option>
// </select>`
Insert cell
Insert cell
Insert cell
Insert cell
// d3.renderHexJSON(hex_la, 100, 100).filter(d=>boroughMob.map(c=>c.code).includes(d.key))
Insert cell
Insert cell
ukUpd_tot_dec_2020
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
labels = ["low", "", "high"]
Insert cell
n = Math.floor(Math.sqrt(colors.length))
Insert cell
xBivar = d3.scaleQuantile(Array.from(ukUpd_tot_map.values(), d => d[0]), d3.range(n))
Insert cell
yBivar = d3.scaleQuantile(Array.from(ukUpd_tot_map.values(), d => d[1]), d3.range(n))
Insert cell
Insert cell
colors
Insert cell
// formatBivar = (value) => {
// if (!value) return "N/A";
// let [a, b] = value;
// return `${a}% ${ukUpd_tot_map.title[0]}${labels[x(a)] && ` (${labels[x(a)]})`}
// ${b}% ${data.title[1]}${labels[y(b)] && ` (${labels[y(b)]})`}`;
// }
Insert cell
ukUpd_tot_map = Object.assign(new Map(ukUpd_tot.map(d=>[d.area_code, [d["workplaces_percent_change_from_baseline"], d["Total annual income (£)"]]])), {title: ["Travel to Workplaces", "Household Income"]})
Insert cell
ukUpd_tot
Insert cell
Insert cell
d3.rollups(ukUpd, v=>d3.median(v, d => d.workplaces_percent_change_from_baseline), d => d.place_id)
Insert cell
ukUpd_2020 = d3.csvParse(await FileAttachment("2020_GB_Region_Mobility_Report_0803.csv").text(), d3.autoType)
Insert cell
ukUpd_2021 = d3.csvParse(await FileAttachment("2021_GB_Region_Mobility_Report_1213.csv").text(), d3.autoType)
Insert cell
ukUpd = d3.merge([ukUpd_2020, ukUpd_2021])
Insert cell
// ukUpd.filter(d=>(d.place_id === "ChIJqZHHQhE7WgIReiWIMkOg-MQ")).map((d,i)=>ukUpd[i-1])[1]
Insert cell
// ukUpd.filter(d=>(d.place_id === "ChIJqZHHQhE7WgIReiWIMkOg-MQ"))
Insert cell
Insert cell
Insert cell
place = "ChIJoynJR59ldkgRwGnsoi2uDgQ"
Insert cell
// d3.rollup(ukUpdPct, v=> d3.mean(v, d=> d.retailPctChange), d => d.place_id)
Insert cell
Insert cell
// (ukUpdPct.filter(d=>d.retail_and_recreation_percent_change_from_baseline===0).length + ukUpdPct.filter(d=>d.grocery_and_pharmacy_percent_change_from_baseline===0).length + ukUpdPct.filter(d=>d.transit_stations_percent_change_from_baseline===0).length + ukUpdPct.filter(d=>d.workplaces_percent_change_from_baseline===0).length + ukUpdPct.filter(d=>d.residential_percent_change_from_baseline===0).length)/ukUpdPct.length
Insert cell
Insert cell
// d3.median(ukUpdPct.filter(d=>(d.place_id === place)&& (d.retailPctChange!==Infinity) && (d.retailPctChange!==-Infinity)), d => d.retailPctChange)
Insert cell
Insert cell
ukUpd_tot = makeAggData(ukUpd)
Insert cell
ukUpd_tot_dec_2021 = makeAggData(ukUpd.filter(d=>d.date>new Date("2021-12-01")))
Insert cell
ukUpd_tot_dec_2020 = makeAggData(ukUpd.filter(d=>(d.date>new Date("2020-12-01"))&&(d.date<new Date("2021-01-01"))))
Insert cell
Insert cell
Insert cell
uk = FileAttachment("local_authority_boundaries.json").json()
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
// new Date(lockdowns[0].start.getFullYear(), lockdowns[0].start.getMonth(), lockdowns[0].start.getDate()+7).toISOString().split('T')[0]
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
// ukUpd_lockdowns.map(d=> d.data.map(c=>
// [d3.max(d.data,d=>d['workplaces_percent_change_from_baseline']),
// d3.min(d.data,d=>d['workplaces_percent_change_from_baseline'])]))
Insert cell
hexes_t = d3.renderHexJSON(hex_la, colScale.bandwidth(), rowScale.bandwidth())//BIND THIS TO UKUPD_LOCKDOWNS
Insert cell
numberOfCols = 4
Insert cell
numberOfRows = 3
Insert cell
colScale = d3.scaleBand()
.domain(d3.range(numberOfCols))
.range([0, smDimensions.visWidth])
.padding(0.05)
Insert cell
rowScale = d3.scaleBand()
.domain(d3.range(numberOfRows))
.range([0, smDimensions.visHeight])
.padding(0.05)
Insert cell
Insert cell
Insert cell
Insert cell
formatYearMonth = d3.timeFormat("%b %Y")
Insert cell
Insert cell
// d3.json("https://pastelsky.github.io/covid-19-mobility-tracker/output/BR/mobility.json")
Insert cell
// t = d3.csv("https://raw.githubusercontent.com/ActiveConclusion/COVID19_mobility/master/google_reports/mobility_report_europe.xlsx")
Insert cell
// wb = xlsx.read(new Uint8Array(d3.csv("https://raw.githubusercontent.com/ActiveConclusion/COVID19_mobility/master/google_reports/mobility_report_europe.xlsx")), {type: 'array'})
Insert cell
// ws = wb.Sheets.Sheet1
Insert cell
// as_json = xlsx.utils.sheet_to_json(ws)
Insert cell
// t.filter(d=>(d.country === "United Kingdom")&&(d.region === "Cumbria"))
Insert cell
// loadExcel("https://raw.githubusercontent.com/ActiveConclusion/COVID19_mobility/master/google_reports/mobility_report_europe.xlsx", {
// type: "array"
// })
Insert cell
// fetch("https://google-mobility-data.p.rapidapi.com/api/data?name=France", {
// "method": "GET",
// "headers": {
// "x-rapidapi-key": "bd5010b7eemsh00ab5684fa6a7efp15b791jsnd00e2d098829",
// "x-rapidapi-host": "google-mobility-data.p.rapidapi.com"
// }
// })//.then(res => res.text())
// .then(response => {
// console.log(response);
// })
// // .catch(err => {
// // console.error(err);
// // });
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
loa_google.filter(c=>c.lad19nm==="City of London")[0]//.lad19cd
Insert cell
Insert cell
boroughIncMob
Insert cell
hexes_[0].n.slice(0,3)
Insert cell
hexes_ = d3.renderHexJSON(hex_la, width, height)
Insert cell
boundarySegments_ = d3.getBoundarySegmentsForHexJSON(hex_la, width, height, "region");
Insert cell
Insert cell
Insert cell
function renderLegend(svg, colorscale, heightLegend, widthLegend, barHeight, legendTicks, legendTickformat, id) {
const defs = svg.append("defs");
const linearGradient = defs.append("linearGradient")
.attr("id", id);
linearGradient.selectAll("stop")
.data(colorscale.ticks().map((t, i, n) => ({ offset: `${100*i/n.length}%`, color: colorscale(t) })))
.enter().append("stop")
.attr("offset", d => d.offset)
.attr("stop-color", d => d.color);
svg.append('g')
.attr("transform", `translate(0,${heightLegend})`)
.append("rect")
// .attr('transform', `translate(${margin.left}, 0)`)
.attr("width", widthLegend)
.attr("height", barHeight)
.style("fill", "url(#"+id+")");

const axisScale = d3.scaleLinear()
.domain(colorscale.domain())
.range([0, widthLegend])

const axisBottom = g => g
.attr("class", `legx-axis`)
.attr("transform", `translate(0,${heightLegend+barHeight})`)
.call(d3.axisBottom(axisScale)
.ticks(legendTicks)
.tickSize(-barHeight)
.tickFormat(d3.format(legendTickformat)))
svg.append('g')
.call(axisBottom);
}
Insert cell
loadExcel = async (uri, option = { type: "array" }) => {
const wb = await loadExcelWorkbook(uri, option);
const ws = wb.Sheets[wb.SheetNames[0]];
const json = xlsx.utils.sheet_to_json(ws);
return json;
}
Insert cell
loadExcelWorkbook = async (uri, option = { type: "array" }) => {
const buffer = await d3.buffer(uri);
const array = new Uint8Array(buffer);
const wb = xlsx.read(array, option);
return wb;
}
Insert cell
function wrap2(text, wrapWidth, yAxisAdjustment = 0) {
text.each(function() {
var text = d3.select(this),
words = text.text().split(/\s+/).reverse(),
word,
line = [],
lineNumber = 0,
lineHeight = 1.1, // ems
y = text.attr("y"),
dy = parseFloat(text.attr("dy")) - yAxisAdjustment,
tspan = text.text(null).append("tspan").attr("x", 0).attr("y", y).attr("dy", `${dy}em`);
while (word = words.pop()) {
line.push(word);
tspan.text(line.join(" "));
if (tspan.node().getComputedTextLength() > wrapWidth) {
line.pop();
tspan.text(line.join(" "));
line = [word];
tspan = text.append("tspan").attr("x", 0).attr("y", y).attr("dy", ++lineNumber * lineHeight + dy + "em").text(word);
}
}
});
return 0;
}
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
AQIextentGLA = [-58.76655569991438, 15]//d3.extent(AQdataGLA, d=>d.AQindex_pc_change_from_baseline)
Insert cell
Insert cell
Insert cell
Insert cell
colorDomainW = [0.25, d3.max(mobilityData, d=>Number(d['White']))];
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
latestCovidUpdate = d3.extent(d3.merge([ukCovidApiLoaLower, ukCovidApiLoaUpper]), d=>d.date)[1]
Insert cell
ukCovidApiLoa2020 = d3.rollup(d3.merge([ukCovidApiLoaLower, ukCovidApiLoaUpper]).filter(
d=>(d.date> new Date("2020-12-01"))&&
(d.date< new Date("2020-12-13"))),
v => d3.sum(v, d => +d["newCasesByPublishDate"]), d => d["areaCode"])
Insert cell
ukCovidApiLoa2021 = d3.rollup(d3.merge([ukCovidApiLoaLower, ukCovidApiLoaUpper]).filter(d=>d.date> new Date("2021-12-01")), v => d3.sum(v, d => +d["newCasesByPublishDate"]), d => d["areaCode"])
Insert cell
ukCovidApiLoaLower = d3.csv("https://api.coronavirus.data.gov.uk/v2/data?areaType=ltla&metric=newCasesByPublishDate&format=csv", d3.autoType)
Insert cell
ukCovidApiLoaUpper = d3.csv("https://api.coronavirus.data.gov.uk/v2/data?areaType=utla&metric=newCasesByPublishDate&format=csv", d3.autoType)
Insert cell
londonCovidApi = d3.csv("https://api.coronavirus.data.gov.uk/v2/data?areaType=region&areaCode=E12000007&metric=newCasesBySpecimenDate&format=csv", d3.autoType)
Insert cell
ukCovidApi = d3.csv("https://api.coronavirus.data.gov.uk/v2/data?areaType=overview&metric=newCasesBySpecimenDate&format=csv", d3.autoType)
Insert cell
Insert cell
Insert cell
Insert cell
incomeLOA_raw = d3.csv("https://gist.githubusercontent.com/lnicoletti/fb38b11e3ee566f5fc2dd4a8ee4a7fc4/raw/a37b54dc60d5a8f3297a9c83d6a5baa172193dc2/incomeestimatesforsmallareas.csv")
Insert cell
Insert cell
Insert cell
Insert cell
policies_tot = d3.merge([nationalPolicies, policies])
Insert cell
Insert cell
covidBoroughs = d3.csv('https://gist.githubusercontent.com/lnicoletti/3c31d0a0b9dac5506ca17f2a03f7fda4/raw/46ca93e0ba3b56fbdd7b8605eff553dc7ac5f038/london_covid_tot_0222.csv')

// https://gist.githubusercontent.com/lnicoletti/3c31d0a0b9dac5506ca17f2a03f7fda4/raw/46ca93e0ba3b56fbdd7b8605eff553dc7ac5f038/london_covid_tot_0222.csv

// https://gist.githubusercontent.com/lnicoletti/a85be0474bc536faa9a899183e8afc1c/raw/7ed6553560ff3f11c81159ab2abeeff38ce5cad1/london_covid_tot.csv
Insert cell
boroughEMob = d3.csv("https://gist.githubusercontent.com/lnicoletti/aab0206acee57bf8e3569da88694eb8a/raw/7829d0d8d8e1f656a4415d22b0aca8a57a8ca455/uk_mobility_boroughs_vs_ethnicity_0222.csv")

// https://gist.githubusercontent.com/lnicoletti/aab0206acee57bf8e3569da88694eb8a/raw/7829d0d8d8e1f656a4415d22b0aca8a57a8ca455/uk_mobility_boroughs_vs_ethnicity_0222.csv

// https://gist.githubusercontent.com/lnicoletti/838d77f2f6ee09fde646aaf1d51dff54/raw/fc21c5f0e0564030bf0ffe62db78f16eafd22ea1/uk_mobility_boroughs_vs_ethnicity_0206.csv
Insert cell
boroughIncMob = d3.csv("https://gist.githubusercontent.com/lnicoletti/d64584223e7dd2f1cb692e7e7916b7b8/raw/474b29a9364c171fe8844ce9f6369e5a1b428d7a/uk_mobility_boroughs_vs_income_0222.csv")

// https://gist.githubusercontent.com/lnicoletti/d64584223e7dd2f1cb692e7e7916b7b8/raw/474b29a9364c171fe8844ce9f6369e5a1b428d7a/uk_mobility_boroughs_vs_income_0222.csv

// https://gist.githubusercontent.com/lnicoletti/5a75ccc8a280fa4f27b89c527a31819d/raw/049024e3d6975b41188a582e2f85e429dbac4529/uk_mobility_boroughs_vs_income_0206.csv
Insert cell
covidData = d3.csv("https://gist.githubusercontent.com/lnicoletti/eb3b0baae3055f2123aa662c7f587fb6/raw/e307786321ab971c6821983f3086af65ad560f3c/london_covid_time_0222.csv")
// https://gist.githubusercontent.com/lnicoletti/eb3b0baae3055f2123aa662c7f587fb6/raw/e307786321ab971c6821983f3086af65ad560f3c/london_covid_time_0222.csv

// https://gist.githubusercontent.com/lnicoletti/ba017024023a9a6aa175103ec6083dd9/raw/dcedaa8fad629108bfa025f7f24aa7a25f409fed/london_covid_time_0206.csv
Insert cell
mobilityCities = d3.csv("https://gist.githubusercontent.com/lnicoletti/0912e160d0f5a3b8641569df8d44ed00/raw/d7dcddb955056c9fe8ce17a5130e639d9503777c/cities_mob_0222.csv")
// https://gist.githubusercontent.com/lnicoletti/0912e160d0f5a3b8641569df8d44ed00/raw/d7dcddb955056c9fe8ce17a5130e639d9503777c/cities_mob_0222.csv

// https://gist.githubusercontent.com/lnicoletti/f86f9a22d10223ec92f4c8a99482b75a/raw/32c7ca74fd3af83a1aed8964c886f6568896d851/cities_mob_0206.csv
Insert cell
mobilityData = d3.csv("https://gist.githubusercontent.com/lnicoletti/7f72c5d8cdedfa752a4ed6dfc31073fc/raw/b14450c3e3ed844230397968487283019768ae18/london_mobility_boroughs_demographics_agg_0222.csv")
// https://gist.githubusercontent.com/lnicoletti/7f72c5d8cdedfa752a4ed6dfc31073fc/raw/b14450c3e3ed844230397968487283019768ae18/london_mobility_boroughs_demographics_agg_0222.csv

// https://gist.githubusercontent.com/lnicoletti/7a4e79a0442116613d30b18de6fdd96d/raw/1666e3bd79e429890f270fa7be00e3cfa3a15220/london_mobility_boroughs_demographics_agg_0206.csv
Insert cell
Insert cell
Insert cell
Insert cell
mobilityDataUK = d3.csv("https://gist.githubusercontent.com/lnicoletti/01d481244751de64f9a45eea4a6223d5/raw/6dc0e44045c3c5f40120f925654a5b33b310e442/uk_mobility_boroughs_demographics_agg_0222.csv")

// https://gist.githubusercontent.com/lnicoletti/01d481244751de64f9a45eea4a6223d5/raw/6dc0e44045c3c5f40120f925654a5b33b310e442/uk_mobility_boroughs_demographics_agg_0222.csv

// https://gist.githubusercontent.com/lnicoletti/0316fa8d74fcadc6e187e3fffbc7980b/raw/4f0049c4611216e773f89adf96eda25de9c7d496/uk_mobility_boroughs_demographics_agg_0206.csv
Insert cell
Insert cell
baselineTickAQGLA = 6;
Insert cell
baselineTickAQ = 6;
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
import {Plot} from "@observablehq/plot"
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
xlsx = require('xlsx@0.16.9/dist/xlsx.full.min.js')
Insert cell

One platform to build and deploy the best data apps

Experiment and prototype by building visualizations in live JavaScript notebooks. Collaborate with your team and decide which concepts to build out.
Use Observable Framework to build data apps locally. Use data loaders to build in any language or library, including Python, SQL, and R.
Seamlessly deploy to Observable. Test before you ship, use automatic deploy-on-commit, and ensure your projects are always up-to-date.
Learn more