2c:null 2d:["$","title",null,{"children":"Visualization components for OWID data | Ernesto Laval | Observable"}] 32:T472,Once we have the data on the client, we would like to visualize it in order to explore and find relevant "messages" that can be extracted from the relationships between time and entities. We can definitively use already available Plotting tools such as Observable Plot (more on this below) or we can built our own visualizations in the DOM. We could build visualizations from scratch using D3 but we can also build an OWID visuallziation framework that would allow a community to build components that are specifically designed to visualize OWID data. Some of these components could be general tools (such as trend charts, maps or barcharts) but in the future we could create some specific visualizations that allow to better communicate specific variables or the retationship between more than one variable. Here I present a Proof of Concept for such framework, using as an example a **preliminary** version of a tool that allows us to build Trend Charts and Bar Charts for OWID data. *Note: this is not production ready code and has many pending issues. It was created in a couple of days just to illustrates a potential approach*33:[{"id":"5d4f906715f326692ce7d3376613c828334113d424efa4195c3188249da5945abb0e7296d8e9caa80d4e4162edf9db7a7d40ddbbfe277ff2fa6db46b21a878a2","href":"https://static.observableusercontent.com/files/5d4f906715f326692ce7d3376613c828334113d424efa4195c3188249da5945abb0e7296d8e9caa80d4e4162edf9db7a7d40ddbbfe277ff2fa6db46b21a878a2","name":"image@2.png","type":"image/png","lastModified":1664632194936.358,"size":36259},{"id":"c8a9281a8a7bf5cc18f17efcfa7c2289449be234c89375868cbd4998c250b66056e19a2e5ab810b0ec8477e68a1d48bcaf7e353d920b23ddbf53774fe9fcd96c","href":"https://static.observableusercontent.com/files/c8a9281a8a7bf5cc18f17efcfa7c2289449be234c89375868cbd4998c250b66056e19a2e5ab810b0ec8477e68a1d48bcaf7e353d920b23ddbf53774fe9fcd96c","name":"image.png","type":"image/png","lastModified":1664573933324.338,"size":266263},{"id":"3839eb085f5438c4a90fd1752a1875bdb0005ca646f1d34e78ab8eb498d5c621e0cee84b973a2fc96ff69bbc04166bf94380c812ff803c1c8f13fcb09446dc3e","href":"https://static.observableusercontent.com/files/3839eb085f5438c4a90fd1752a1875bdb0005ca646f1d34e78ab8eb498d5c621e0cee84b973a2fc96ff69bbc04166bf94380c812ff803c1c8f13fcb09446dc3e","name":"image@3.png","type":"image/png","lastModified":1664632541590.785,"size":175455}] 30:{"id":"1c8ecfe52c864726","slug":"owid-visualisation-components-poc","title":"Visualization components for OWID data","thumbnail":"ddaf610093b917811a08d5a7740265e199e3237ea19c1d43fb446ae88f91d716","access_level":"public","create_time":"$D2022-09-27T02:46:26.985Z","update_time":"$D2022-12-02T02:00:09.821Z","view_time":null,"first_public_time":"$D2022-09-28T01:29:33.488Z","has_unpublished_edits":null,"fork":false,"stars":0,"starred":false,"owner":{"type":"team","id":"0843f5dc42ca68a0","login":"elaval","name":"Ernesto Laval","avatar_url":"https://avatars-next.observableusercontent.com/089f0414fc9025dadb05cc06d7a406bc9be10093187c46729cb4ba3a4098dc30","tier":"starter_2024"},"version":852,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"e04cd7b5f4351ae9","login":"elaval","name":"Ernesto Laval","avatar_url":"https://avatars-next.observableusercontent.com/089f0414fc9025dadb05cc06d7a406bc9be10093187c46729cb4ba3a4098dc30","tier":null}],"members":[{"type":"individual","id":"e04cd7b5f4351ae9","login":"elaval","name":"Ernesto Laval","avatar_url":"https://avatars-next.observableusercontent.com/089f0414fc9025dadb05cc06d7a406bc9be10093187c46729cb4ba3a4098dc30","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":2,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Visualization components for OWID data","cells":[{"id":0,"value":"# Visualization components for OWID data\n## Proof of Concept","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":308,"value":"Our World in Data (OWID) facilitates the access to an extensive collection of worldwide datasets through its website: https://ourworldindata.org/\n\nIn addition to articles in several topics, they provide a general purpose visualisation tool - [OWID Grapher](https://ourworldindata.org/owid-grapher) - which is a versatile tool to present underlying datasets in different formats (e.g. Trend Charts to understand evolution on time, Bar Charts to compare metrics for different countries on a single year, Maps to display the geographical distribution of indicators). Grapher provides valuable context (e.g. sources), access to raw data and functionality to download visualizations in reusable image formats. \n\n $\"drawing\"$ ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":320,"value":"Grapher is a great tool, and its code is open sourced https://github.com/owid/owid-grapher. There are many years of experience behind this tool which has allowed the OWID team to polish several details that allow the information to be presented in a clear and comprehensive way.\n\nFrom a technological point of view it serve the purposes of providing easy access to several data sources in a consistent way, allows the visualizations to be embedded in external pages (via iframes) and the exporting capabilities serves many purposes. \n\nBut some of us would like to have even more flexibility in the visualization architecture. It would be great to be able to have OWID visualization components (or access to data sources) in Observable, or available as building blocks for new web developments (beyond the iframe embedding capabilities).\n\nThis notebook presents some ideas in this direction and shows a germinal code that illustrates a possible way to build modularized visualisation components for OWID.\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":327,"value":"## Decoupling data from visualization","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":329,"value":"OWID data is managed in a MySQL database which contains thousands of datasets and variables (indicators) which have values associated to entities (e.g. countries) and years.\n\nGrapher code includes connections to the MySQL database and queries to get the data. One important step to provide flexibility is to decouple the visualization tools from the actual database that contains the data in the backend.\n\nFortunately OWID does provide a way to access the data via an API without a direct query to the data base. If you know the variable id (e.g. 96429 for 'Life expectancy (OECD)') you can use the following HTTP request to obtain a json file with the data associated to such variable.\n\n

\n  https://ourworldindata.org/grapher/data/variables/data/96429.json\n

\n\nAnd you obtain an object with this format:\n\n $\"image@2.png\"$ \n\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":383,"value":"OWID also provides an API that allows the client to obtain metadata associated to a variable, which can be used to display sources, units, and descriptions.\nFor example:\n\n

\n  https://ourworldindata.org/grapher/data/variables/metadata/96429.json\n

\n\n $\"image@3.png\"$ ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":736,"value":"Having obtained the data & metadata for a specific variable (using above mentioned http requests) and applying simple Javascript transformation to map entity names, you can obtain the the table below that can be used for visualization purposes","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":379,"value":"viewof table1 = Inputs.table(data)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":395,"value":"## Visualization components for OWID data","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":398,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":405,"value":"Example: Trend chart for Life Expectancy (OECD) in Rusia, China, UK, South Africa and Chile","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":448,"value":"### ${metaDataLifeExpectancy.datasetName}\nSource: ${metaDataLifeExpectancy.source.name}, ${metaDataLifeExpectancy.source.dataPublishedBy}
\n${metaDataLifeExpectancy.source.link}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":410,"value":"{\n const selectedData = dataLifeExpectancy.filter((d) =>\n [\"Russia\", \"China\", \"United Kingdom\", \"Chile\", \"South Africa\"].includes(\n d.entityName\n )\n );\n return owidVis.trendChart(selectedData).y({ grid: true }).unit(unit).node();\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":454,"value":"In this POC we are using a javascript package called **@elaval/owid-vis-poc**\n\n(package available via npm at @elaval/owid-vis-poc, source code at https://github.com/elaval/owid-vis-poc)\n\nIn order to use this package in an observable note (such as this one) you can import it via:\n\n

\n  owid = require(\"@elaval/owid-vis-poc@0.3.2\")\n

\n\nAnd then you can use to produce a visualization (it creates a DOM element) via\n\n

\n  owid.trendChart(data)\n  .node();\n

\n\nOr with additional configuration\n

\n  owid.trendChart(data)\n  .unit(\"years\")\n  .node();\n

","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":470,"value":"With the same dataset, we can produce a bar chart. For example:\n\n### Life expectancy in ${selectedYear}\nSource: ${metaDataLifeExpectancy.source.name}, ${metaDataLifeExpectancy.source.dataPublishedBy}
\n${metaDataLifeExpectancy.source.link}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":498,"value":"viewof selectedYear = {\n return Inputs.range(\n d3.extent(metaDataLifeExpectancy.dimensions.years.values.map((d) => d.id)),\n {\n label: \"Year\",\n step: 1,\n value: 1990\n }\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":473,"value":"{\n const selectedData = dataLifeExpectancy.filter((d) =>\n [\"Russia\", \"China\", \"United Kingdom\", \"Chile\", \"South Africa\"].includes(\n d.entityName\n )\n );\n\n return owidVis\n .barChart(selectedData)\n .width(width)\n .unit(unit)\n .year(selectedYear)\n .x({ grid: true })\n .node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":815,"value":"Inputs.bind(\n Inputs.range(\n d3.extent(metaDataLifeExpectancy.dimensions.years.values.map((d) => d.id)),\n {\n label: \"Year\",\n step: 1,\n value: 1990\n }\n ),\n viewof selectedYear\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":811,"value":"In a similar way we can produce a Map visualization","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":805,"value":"owidVis.map(data).unit(\"years\").year(selectedYear).width(1000).node()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":431,"value":"dataLifeExpectancy = {\n const dictEntities = {};\n metaDataLifeExpectancy.dimensions.entities.values.forEach(\n (d) => (dictEntities[d.id] = d)\n );\n\n return dataLifeExpectancyJson.values.map((value, i) => ({\n year: dataLifeExpectancyJson.years[i],\n entityId: dataLifeExpectancyJson.entities[i],\n entityName: dictEntities[dataLifeExpectancyJson.entities[i]].name,\n entityCode: dictEntities[dataLifeExpectancyJson.entities[i]].code,\n value: value\n }));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":422,"value":"dataLifeExpectancyJson = d3.json(\n `https://ourworldindata.org/grapher/data/variables/data/96429.json`\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":425,"value":"metaDataLifeExpectancy = d3.json(\n `https://ourworldindata.org/grapher/data/variables/metadata/96429.json`\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":708,"value":"## Allow the user to navigate through datasets / variables","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":37,"value":"viewof selectedIndicator = {\n const options = [\n { name: \"Life expectancy (OECD)\", id: 96429 },\n { name: \"Tax revenue\", id: 1382 },\n { name: \"Chartbook of Economic Inequality, Gini (2016)\", id: 1454 }\n ];\n\n return Inputs.select(options, {\n format: (d) => d.name,\n label: \"Select an indicatior\"\n });\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":91,"value":"## ${metaData.datasetName}\n${metaData.description}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":73,"value":"viewof selectedEntities = {\n const options = metaData.dimensions.entities.values.map((d) => d.name);\n return Inputs.select(options, {\n label: \"Select entities\",\n multiple: true,\n value: _.sampleSize(options, 5)\n\n /*.filter((d) =>\n [\"United Kingdom\", \"South Africa\", \"Japan\", \"Chile\"].includes(d))*/\n });\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":54,"value":"{\n return owidVis\n .trendChart(selectedData, {\n selectedYearCallback: handleSelectedYearCallback\n })\n .y({ grid: true })\n .unit(unit)\n .width(1000)\n .node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":801,"value":"Year ${selectedYear}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":192,"value":"{\n return owidVis\n .barChart(selectedData)\n .unit(unit)\n .year(selectedYear)\n .x({ grid: true })\n .width(1000)\n .node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":839,"value":"Year ${selectedYear}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":828,"value":"{\n return owidVis.map(data).unit(\"years\").year(selectedYear).width(1000).node();\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":93,"value":"Source: ${metaData.source.name}, ${metaData.source.dataPublishedBy}
\n${metaData.source.link}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":275,"value":"unit = metaData.shortUnit || metaData.unit","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":260,"value":"selectedData = data.filter((d) =>\n selectedEntities ? selectedEntities.includes(d.entityName) : true\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":65,"value":"data = {\n return dataJson.values.map((value, i) => ({\n year: dataJson.years[i],\n entityId: dataJson.entities[i],\n entityName: dictEntities[dataJson.entities[i]].name,\n entityCode: dictEntities[dataJson.entities[i]].code,\n value: value\n }));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":60,"value":"dictEntities = {\n const dict = {};\n\n metaData.dimensions.entities.values.forEach((d) => (dict[d.id] = d));\n\n return dict;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":56,"value":"dataJson = d3.json(\n `https://ourworldindata.org/grapher/data/variables/data/${selectedIndicator.id}.json`\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":32,"value":"metaData = d3.json(\n `https://ourworldindata.org/grapher/data/variables/metadata/${selectedIndicator.id}.json`\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":268,"value":"## Aux code","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":224,"value":"function handleSelectedYearCallback(year) {\n viewof selectedYear.value = year;\n viewof selectedYear.dispatchEvent(new Event(\"input\", { bubbles: true }));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":104,"value":"## Code libraries","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":123,"value":"This demo uses a graphing PoC tool that was created just for demo purposes (not production ready): @elaval/owid-vis-poc\n\nSource code: [https://github.com/elaval/owid-vis-poc](https://github.com/elaval/owid-vis-poc)\n\nNPM package: \"@elaval/owid-vis-poc\"\n\nUsage:\n

\n  // We assume that data has already being retrieved from OWID datasets with the following format\n  data = [{year: 1960, entityId: 1, entityName: \"United Kingdom\", entityCode: \"GBR\", value: 70.8}, ...]\n  \n  owid = require(\"@elaval/owid-vis-poc@0.3.2\")\n  chartEL = owid.OWIDPlot(data, {chartType:\"lineChart\", unit: \"years\"})  // returns a DOM Element that can be embedded in a web page\n

\n","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":131,"value":"owidVis = require(\"@elaval/owid-vis-poc@0.3.2\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W33","stdlib":"1"}} 31:"$30:body"