2a:null 2b:["$","title",null,{"children":"DATA EXPLORATION - 1 | Spe Chen | Observable"}] 30:T553,md`The purpose is to ease the navigation through the dataset and better analyze the variables. The multi-line chart has been choosen for two reasons. - It provides quite a complete vision of how data ranges and flows in the time-series. Although being capable of plotting a lot of information, has to be taken into account that if the data is not discriminable from others by being able to highlight it or filter it can be a very messy chart. D3.js is for that case a great tool. \b - The other reason for choosing this plot was that when doing the class exercices this concept was managed particularly much more different than expected from the line charts we plotted in d3 before. After struggling for the exercices, we have gone further this time. Reference is for the struggle, not particularly the results. Also we've been experiencing on plotting tables. Inspired by one notebook using svg tables which the Professor shared with us to illustrate as example of the multiple view charts. Tables, while might not be related directly to visualization, always will be a great an effective way to easily seize the underlying nature of data relative to a problem to solve. When tables are powerful the possibilities are much. Couldn't obtain the result we expected but happy to get lost exploring how different web technologies approach solutions for that. `31:T5c7,multiline = { const svg = d3 .create("svg") .attr("viewBox", [0, 0, width + margin.right, height]) .style("overflow", "visible"); svg.append("g").call(xAxis); svg.append("g").call(yAxis); const path = svg .append("g") .attr('id', 'allpaths') .attr("fill", "none") .attr("stroke", color) .attr("stroke-width", 1.5) .attr("stroke-linejoin", "round") .attr("stroke-linecap", "round") .selectAll("path") .data(data.series) .join("path") .style("mix-blend-mode", "multiply") .attr("id", d => d.name) .attr("d", d => line(d.values)); svg .append('text') .attr('x', x(data.dates.slice(-1)[0])) .attr('y', 20) .attr('dx', 40) .attr('dy', '.05em') .style('text-anchor', 'end') .text('Deaths per million plotted by countries'); svg.call(hover, path); // Annotation // svg // .append('rect') // .lower() // .attr('x', x(new Date(2020, 2, 24))) // .attr('y', y()) // .attr('width', x(new Date(2020, 5, 2)) - x(new Date(2020, 2, 24))) // .attr("y", margin.top + 60) // .attr('fill', '#21424bf0'); svg .append('text') .attr('x', 20) .attr('y', 20) .text('COVID-19'); function update(dropdown) { d3.selectAll('path').attr('stroke-width', null); d3.select(`#${dropdown}`) .raise() .attr('stroke-width', highlightWidth) .attr('stroke', highlightColor); } return Object.assign(svg.node(), { update }); }32:Tac6,md`### 2. Data preparation The data is not that complex as we have worked with it quite deeply during the project. Nevertheleess, the management for information nested deeply is counter-intuitive and quite a challenge for those not familiarized with javascript. One of the frustrated challenges for plotting the multiseries is that ingest mainly two arrays, one for the time series and one for the values. All the examples we could found for this plot were using datasets which came the same length for both arrays, missing data was handled to keep values in sync through the indexes of each array. As it can be observed, some lines are longer than others, and some just don't fit at all the range. It probably could be easily solved by reducing minor countries wich amount many dates without values and slicing the first weeks of the data set, when the first wave was experienced worldwide. We have had great time playing and understanding the rollUp function and other aggregating methods provided by d3. Another difficulty in data which has affected the final result was to understand well how javascript and d3 manages time and dates, as well as other formats. An exercise during classes was about managing dates from a dataset but unexpectedly theh values came already parsed from source and didn't really got messy with them. The time for it came finally and understand better the underlying complexity. ### Features Many ideas and creative possible solutions came to mind when thinking how to execute visualizations and interactions with the user. Maybe because our knowledge is yet immature we hardly could execute anything in comparision of the many brilliant thoughts we come with. We achieved the tooltip for the curves and to make it work properly with the desired effect to be shown smoothly as tue mouse approaches rather than when the mouse is just over the plotted pixel. Other things like selectables or sliders, might not work solidly and we were confident in being able to better solve it. Note, never get too confident even for that what loooks that much easy. ### Conlusions Working with something you like and that you know you will enjoy no matter what the strive required along the path. Maybe the result is not the expected, but the effort and the many notebooks opened, ideas tested and long periods of hours struggling for getting the result that you want achieve is what we obtained, a deep dive into the nature of d3.js and better handling the not that 'colorful' concept underlying to visualization. Closing the eyes and thinking in arrays is what we get, and I consider it a great result. Honoured for the collegues from the course and for the professors who proficiently mentor the knowlege of this discipline. `33:T67d,// data = { // let plot = d3 // .nest() // .key(function(d) { // return d.date.split('-')[0]; // }) // key is the year // .sortKeys(d3.ascending) // .key(function(d) { // var splitDate = d.date.split('-'); // return splitDate[0] + '-' + splitDate[1]; // key is year-month // }) // .sortKeys(d3.ascending) // .rollup(function(d) { // return d.length; // }) // .entries(raw); // let yMFormat = d3.time('%Y-%m'); // let makeAllKeys = function(year) { // let allKeys = []; // for (var i = 0; i < 12; i++) { // // 12 months in a year // allKeys.push(yMFormat(new Date(year, i, 1))); // } // return allKeys; // let nestedData = nestedData.map(function(yearObj) { // return { // values: makeAllKeys(+yearObj.key).map(function(k) { // let value = yearObj.values.filter(function(v) { // return v.key == k; // })[0]; // return value || { key: k, values: 0 }; // }) // }; // }); // }; // } data = { let places = Array.from(d3.groups(rawdata, d => d.location), d => ({ name: d[0], values: d[1].map(e => e.deaths), dates: d[1].map(e => e.date) })); let omissions = ['International', 'World']; places = places.filter(d => !omissions.includes(d.name)); let [minDate, maxDate] = d3.extent(rawdata, d => d.date); let dates = d3.timeDay.range( new Date(minDate), d3.timeDay(new Date(maxDate)) ); console.log(places.filter(d => d.dates.length === dates.length)); return { series: places.filter(d => (d.dates.length = dates.length)), dates: dates }; }34:T671,function hover(svg, path) { if ("ontouchstart" in document) svg .style("-webkit-tap-highlight-color", "transparent") .on("touchmove", moved) .on("touchstart", entered) .on("touchend", left); else svg .on("mousemove", moved) .on("mouseenter", entered) .on("mouseleave", left); const dot = svg.append("g").attr("display", "none"); dot .append("circle") .attr("r", 4) .attr("fill", highlightColor); dot .append("text") .attr("font-family", "sans-serif") .attr("font-size", "1em") .attr("text-anchor", "middle") .attr("y", -16); function moved() { d3.event.preventDefault(); const mouse = d3.mouse(this); const xm = x.invert(mouse[0]); const ym = y.invert(mouse[1]); const i1 = d3.bisectLeft(data.dates, xm, 1); const i0 = i1 - 1; const i = xm - data.dates[i0] > data.dates[i1] - xm ? i1 : i0; const s = d3.least(data.series, d => Math.abs(d.values[i] - ym)); path .attr("stroke", d => (d.name === s.name ? highlightColor : "#6fd8f2")) .filter(d => d.name === s.name) .raise(); dot.attr("transform", `translate(${x(data.dates[i])},${y(s.values[i])})`); dot.select("text").text(`${s.name} (${numFmt(s.values[i])})`); } function entered() { path.style("mix-blend-mode", null).attr("stroke", "#6fd8f2"); dot.attr("display", null); } function left() { path.style("mix-blend-mode", "multiply").attr("stroke", null); dot.attr("display", "none"); d3.select(`#${dropdown}`) .raise() .attr('stroke-width', highlightWidth) .attr('stroke', highlightColor); } }35:[] 2e:{"id":"d9d96a073812b475","slug":"data-exploration-1","title":"DATA EXPLORATION - 1","thumbnail":"70a7018723dea70f06ae5c417b73e0308aa805c38c2283782f2e7a0e540b5983","access_level":"public","create_time":"$D2021-01-08T18:14:36.615Z","update_time":"$D2021-01-08T18:16:05.743Z","view_time":null,"first_public_time":"$D2021-01-08T18:16:10.561Z","has_unpublished_edits":null,"fork":true,"stars":1,"starred":false,"owner":{"type":"team","id":"875efdc7be078385","login":"spepechen","name":"Spe Chen ","avatar_url":"https://avatars-next.observableusercontent.com/f9b28d0e0e81c7c6d6e8167c86d9659b16cf7d4d736bb8ff502ccf4b97fa1528","tier":"starter_2024"},"version":1093,"publish_version":1093,"role":null,"authored":false,"authors":[{"type":"individual","id":"48c34b5377e95cdb","login":"spepechen","name":"Spe Chen ","avatar_url":"https://avatars-next.observableusercontent.com/f9b28d0e0e81c7c6d6e8167c86d9659b16cf7d4d736bb8ff502ccf4b97fa1528","tier":null}],"members":[{"type":"individual","id":"48c34b5377e95cdb","login":"spepechen","name":"Spe Chen ","avatar_url":"https://avatars-next.observableusercontent.com/f9b28d0e0e81c7c6d6e8167c86d9659b16cf7d4d736bb8ff502ccf4b97fa1528","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":2,"collection":{"id":"22b00eb53ea80219","slug":"mvtec","title":"MVTEC","owner":{"id":"875efdc7be078385","login":"spepechen","name":"Spe Chen "}},"type":"notebook","body":{"titling":"auto","title":"DATA EXPLORATION - 1","cells":[{"id":1089,"value":"md`# DATA EXPLORATION - 1`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":369,"value":"md`# COVID-19 deaths by country\n\n## 1. Introduction`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":983,"value":"$30","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"viewof dropdown = select({\n title: \"Countries\",\n description: \"Pick a place to highlight\",\n options: data.series.map(d => d.name).sort((a, b) => d3.ascending(a, b)),\n value: \"Spain\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":55,"value":"$31","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":53,"value":"viewof N = number({\n title: 'Show top N counties based on cases', \n value: 10, min: 2, max: 90,\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":59,"value":"viewof slider2 = {\n const days = d3.timeDay.count(\n tableData[0][0],\n tableData[tableData.length - 1][0]\n );\n return slider({\n title: \"Pick a Date\",\n value: days,\n min: 0,\n max: days,\n step: 1,\n display: v => d3.timeFormat('%Y-%m-%d')(data.dates[v])\n });\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":51,"value":"html`${table(tableData.slice(0, 20))}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":996,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1012,"value":"md`#### References\n\n- [Svg table](https://observablehq.com/@analyzer2004/svgtable)\n- [Multi-line chart by Mike Bostock](https://observablehq.com/@d3/multi-line-chart)\n- [Excellent exeuction of multi-lines, very tidy and solid code](https://observablehq.com/@chris-dangerfield/active-covid19-cases-by-country-region)`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":49,"value":"tableData = rawdata.sort((a, b) => Number(b.deaths) - Number(a.deaths))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":47,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":45,"value":"rawdata = {\n let raw = await d3.csv(\n \"https://covid.ourworldindata.org/data/owid-covid-data.csv\"\n );\n let covid = Object.values(raw);\n\n covid = Array.from(\n d3.rollup(\n covid,\n v => ({\n date: v[0].date,\n location: v[0].location,\n\n 'case rate':\n (d3.sum(v, d => +d['total_cases_per_million']) / v[0].population) *\n 100000,\n deaths: d3.sum(v, d => numFmt(+d['total_deaths_per_million'])),\n 'death rate':\n (d3.sum(v, d => +d['total_deaths_per_million']) / [v[0].population]) *\n 100000,\n population: +v[0].population\n }),\n d => d.date,\n d => d.location\n ),\n ([_, v]) => {\n return Array.from(v, ([_, value]) => value);\n }\n ); //.sort((a, b) => { return b.date - a.date; });\n\n return covid.flat().filter(d => d.date > \"2020-02-15\");\n // .sort((a, b) => d3.descending(b.cases, a.cases));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"highlightWidth = 7","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":103,"value":"tickValues = [1,2,5,10,20,50,100,200,500,1000,2000,5000,10000,20000,50000,100000]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":39,"value":"margin = ({ top: 60, right: 40, bottom: 30, left: 30 })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":37,"value":"height = 600","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":106,"value":"highlightColor = \"LightCoral\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":108,"value":"color = \"#0cc3fe\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"// type = d => {\n// d.date = parseTime(d.date);\n// d.cases = +d.cases > 0 ? +d.cases : undefined;\n// d.deaths = +d.deaths || 0;\n// return d;\n// }","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":381,"value":"// const cdata = covid.map(d =>\n// d.data.map(k => {\n// return {\n// location: d.location,\n// population: +d.population,\n// life_expectancy: +d.life_expectancy,\n// date: Date(k.date),\n// cases: isNaN(k.total_cases_per_million)\n// ? 0\n// : +k.total_cases_per_million,\n// deaths: isNaN(k.total_deaths_per_million)\n// ? 0\n// : +k.total_deaths_per_million\n// };\n// })\n// );\n\n// return cdata.flat();","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":294,"value":"numberFormat = function(n) {\n return(d3.format(\".2~s\")(n)\n .replace(\"M\", \"m\")\n .replace(\"G\", \"b\"));\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":288,"value":"timeFormat = d3.timeFormat(\"%e %B %Y\");","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":33,"value":"updateMultiline = multiline.update(dropdown)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":22,"value":"x = d3\n .scaleUtc()\n .domain(d3.extent(data.dates))\n .range([margin.left, width - margin.right])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":20,"value":"y = d3\n .scaleLinear()\n .domain([0, d3.max(data.series, d => d3.max(d.values))])\n .nice()\n .range([height - margin.bottom, margin.top])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":18,"value":"xAxis = g => g\n .attr(\"transform\", `translate(0,${height - margin.bottom})`)\n .call(d3.axisBottom(x).ticks(width / 80).tickSizeOuter(0))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":16,"value":"yAxis = g =>\n g\n .attr(\"transform\", `translate(${width - margin.right},0)`)\n .call(d3.axisRight(y).tickFormat(numFmt))\n .call(g => g.select(\".domain\").remove())\n .call(g =>\n g\n .select(\".tick:last-of-type text\")\n .clone()\n .attr(\"x\", 3)\n .attr(\"text-anchor\", \"start\")\n .attr(\"font-weight\", \"bold\")\n .text(data.y)\n )","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"line = d3\n .line()\n .curve(d3.curveCardinal.tension(0.5))\n .defined(d => d > 0)\n .x((d, i) => x(data.dates[i]))\n .y(d => y(d))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":10,"value":"numFmt = d3.format(',')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"parseTime = d3.timeParse('%Y-%m-%d')\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"d3 = require(\"d3@5\", \"d3-array@^2.2\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":83,"value":"import {select, slider, number} from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2,"value":"// Build the html table\nfunction table(data) {\n let cols = Object.keys(data[0]);\n\n //Map over columns, make headers,join into string\n let headerRow = cols.map(col => `${col}`).join(\"\");\n\n //map over array of json objs, for each row(obj) map over column values,\n //and return a td with the value of that object for its column\n //take that array of tds and join them\n //then return a row of the tds\n //finally join all the rows together\n let rows = data\n .map(row => {\n let tds = cols.map(col => `${row[col]}`).join(\"\");\n return `${tds}`;\n })\n .join(\"\");\n\n //build the table\n const table = `\n\t\n\t\t\n\t\t\t${headerRow}\n\t\t\n\t\t\n\t\t\t${rows}\n\t\t\n\t

`;\n\n return table;\n}\n// Source: https://dev.to/boxofcereal/how-to-generate-a-table-from-json-data-with-es6-methods-2eel","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":391,"value":"html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W35","stdlib":"1"}} 2f:"$2e:body"