31:T610,{ // If this page is outside of Observable, set the default styles to approximate // the appearance of the Observable Look and Feel. if (ui === "custom") { return htl.html`

Note: This is an archived version of an Observable Notebook supplement for the RAMPVIS PhilTransA submission titled: 
"Visualization for Epidemiological Modelling: Challenges, Solutions, Reflections & Recommendations"
For the live version, see here: https://observablehq.com/@henryqw/multiple-linked-views

`; } else { return htl.html`

Note: This is a live version of the Observable Notebook supplement for the RAMPVIS PhilTransA submission titled 
"Visualization for Epidemiological Modelling: Challenges, Solutions, Reflections & Recommendations"

`; } }32:T418,Based on the requirements gathered from the initial meeting with a group of EMBs, we prototyped our system with MLVs and coordinated interactions, to support interactive comparison of different models by **input, output and configurations**. ${await FileAttachment("ensemblevis_mlv.png").image({width:800})} **Coordinations**: **A.** We use a PCP(top left) to represent the input data of simulation parameters, brushing on the PCP triggers an update in (1)the parameter space visualization in the middle, and (2)the dimensionality reduction plot (bottom right). **B.** Selecting one parameter configuration in the parameter space visualization (1)updates the line chart(top right) to show the overview of the selected simulation output(mean values of the 1,000 simulation runs), and (2)updates the bottom left PCP to represent the 1,000 individual simulation runs in details. **C.** Brushing the dimensionality reduction plot (bottom right) triggers an update in the input PCP(top left), and coordinately updates others in **Coordination A**. 33:[{"id":"c766f70935ae37d3818231370a1043402bda2e479e6c98f50968615def16d8dc3c078209cbb21ae255536333c7b1119666897a94a8b243b3f43ffd5e8cb9bacb","href":"https://static.observableusercontent.com/files/c766f70935ae37d3818231370a1043402bda2e479e6c98f50968615def16d8dc3c078209cbb21ae255536333c7b1119666897a94a8b243b3f43ffd5e8cb9bacb","name":"commongis.png","type":"image/png","lastModified":1629121793738.589,"size":434533},{"id":"53c625ae59034f5040ed247f226d734277429dba4918b41ec30c1cfe89c1c0f2f2a1bae36b97ae91710686acbb2445c7f9be5f81304594c16370994de3e471d1","href":"https://static.observableusercontent.com/files/53c625ae59034f5040ed247f226d734277429dba4918b41ec30c1cfe89c1c0f2f2a1bae36b97ae91710686acbb2445c7f9be5f81304594c16370994de3e471d1","name":"ensemblevis_mlv.png","type":"image/png","lastModified":1629126779131.742,"size":1294292},{"id":"e5ed14355b84221cdb480bb6f5c811edc843e753d5a895459f5b0660fa4b24301b69e79b8bebb1f78eb37d79800ed735437c7ed51bb56abdab00b1ccc2fe3080","href":"https://static.observableusercontent.com/files/e5ed14355b84221cdb480bb6f5c811edc843e753d5a895459f5b0660fa4b24301b69e79b8bebb1f78eb37d79800ed735437c7ed51bb56abdab00b1ccc2fe3080","name":"situ.png","type":"image/png","lastModified":1629126504837.045,"size":244585},{"id":"a6c203c891687e9778d3426a2c6f8495c03ebce4c65cbd91fcc16d561ae19b75d55e7c6f9f73050a16e8d232f316f19b7f970b9fa1f3c3f4eb9dff850fac03eb","href":"https://static.observableusercontent.com/files/a6c203c891687e9778d3426a2c6f8495c03ebce4c65cbd91fcc16d561ae19b75d55e7c6f9f73050a16e8d232f316f19b7f970b9fa1f3c3f4eb9dff850fac03eb","name":"ensemble_line_chart.png","type":"image/png","lastModified":1630704058214.457,"size":73217},{"id":"6940dc20af1c3cc002ab54fd18b297718042ec4b4e2bfccb286241c57d79cb986faf9036c12dfa9684bfc5382628676f7d55a6d0b45bf2f65004cbfd073aa298","href":"https://static.observableusercontent.com/files/6940dc20af1c3cc002ab54fd18b297718042ec4b4e2bfccb286241c57d79cb986faf9036c12dfa9684bfc5382628676f7d55a6d0b45bf2f65004cbfd073aa298","name":"MLV-3.png","type":"image/png","lastModified":1631830509474.638,"size":615697}] 2f:{"title":"RAMPVIS : Multiple Linked Views for Visual Exploration","cells":[{"id":0,"value":"md`# RAMPVIS : Multiple Linked Views for Visual Exploration`","pinned":false,"hidden":false,"mode":"ojs"},{"id":818,"value":"$31","pinned":false,"hidden":false,"mode":"ojs"},{"id":360,"value":"---\n## 1. TASK & USERS - User Story","pinned":false,"hidden":false,"mode":"md"},{"id":334,"value":"As **a visualization researcher**\n\nI would like to \n * develop interfaces that enable epidemiological model builders(EMB) to compare models with different input, output, and configurations.\n\nSo that EMBs can\n * can understand and adjust their models.\n","pinned":false,"hidden":false,"mode":"md"},{"id":128,"value":"---\n\n## 2. DATA","pinned":false,"hidden":false,"mode":"md"},{"id":346,"value":"Input data:\n\n* 160 different parameter configurations, each includes 16 parameters:\n * https://github.com/ScottishCovidResponse/EnsembleVis-UI/blob/master/public/assets/posterior_parameters.csv\n\nOutput data:\n\n* Average value from 1,000 runs for each configuration:\n * https://github.com/ScottishCovidResponse/EnsembleVis-UI/tree/master/public/assets/data/output","pinned":false,"hidden":false,"mode":"md"},{"id":7,"value":"---\n\n## 3. Multiple Linked Views for Visual Exploration","pinned":false,"hidden":false,"mode":"md"},{"id":824,"value":"### or how to Project for 2D - part 1","pinned":false,"hidden":false,"mode":"md"},{"id":365,"value":"**Multiple Linked Views (MLV)** or **multiple coordinated views** refers to *any instance where data is represented in multiple windows*.\n\nDifferent representations of data are placed in multiple windows and interactions with any of these views result in coordinated responses in others ([Roberts, 2007](https://ieeexplore.ieee.org/document/4269947)).\n\nBy adopting MLV, our implementation enables users to **rapidly explore the complex simulation data** ([North and Shneiderman, 2000](https://dl.acm.org/doi/10.1145/345513.345282)).","pinned":false,"hidden":false,"mode":"md"},{"id":371,"value":"---\n## 4. ORIGIN & CONTEXT","pinned":false,"hidden":false,"mode":"md"},{"id":798,"value":"[Roberts (2007)](https://ieeexplore.ieee.org/document/4269947) provides an overview of the state of the art of MLV. MLV as a specific exploratory visualization technique is constantly evolving for visualization and visual analytics, and its application in other domains is gaining popularity over the years. The user may want to explore complex data to generate new insights, but their familiarity with classic visual designs (bar chart, line chart etc.) makes that goal difficult. **Introducing MLVs therefore may help them perceive new and insightful information from their data.**","pinned":false,"hidden":false,"mode":"md"},{"id":13,"value":"___\n\n## 5. EXAMPLES","pinned":false,"hidden":false,"mode":"md"},{"id":388,"value":"**Improvise** [(Weaver, 2004)](https://ieeexplore.ieee.org/document/1382904) uses MLV to control various aspects of the visualization in a coordinated manner.","pinned":false,"hidden":false,"mode":"md"},{"id":851,"value":"await FileAttachment(\"commongis.png\").image({width:600})","pinned":false,"hidden":false,"mode":"ojs"},{"id":854,"value":"[Hadlak et al. (2011)](https://ieeexplore.ieee.org/document/6065000) used MLVs to enable exploration of a selected data dimension.","pinned":false,"hidden":false,"mode":"md"},{"id":858,"value":"await FileAttachment(\"situ.png\").image({width:600})","pinned":false,"hidden":false,"mode":"ojs"},{"id":53,"value":"---\n## 6. DESCRIPTION & USE","pinned":false,"hidden":false,"mode":"md"},{"id":400,"value":"$32","pinned":false,"hidden":false,"mode":"md"},{"id":410,"value":"---\n## 7. REFLECTION","pinned":false,"hidden":false,"mode":"md"},{"id":415,"value":"### 7.1 What Worked","pinned":false,"hidden":false,"mode":"md"},{"id":427,"value":"EMB feedback has been positive. To the best of our knowledge, the interactions and manipulations supported by our implementation under an epidemiological setting are novel. During our discussions, we demonstrated the capability and EMBs immediately started spotting and discussing unusual patterns in the visualizations, see figure below.\n\nThe demonstration started with applying brushing on the input PCP, this enables the user to filter input parameters. The results (input simulation configurations) were shown in the matrix view below, the user then selected one specific configuration and explore the output in the line chart. \n\nThe output of this configuration shows flat curves (see the figure below), which implies the input configuration is not trustworthy.","pinned":false,"hidden":false,"mode":"md"},{"id":863,"value":"await FileAttachment(\"MLV-3.png\").image({width:800})","pinned":false,"hidden":false,"mode":"ojs"},{"id":866,"value":"And with this particular simulation configuration, there is an unusual pattern around day 21 in the output (see the figure below).
**Similar patterns were observed with several other configurations, which prompted an investigation by the EMBs.**","pinned":false,"hidden":false,"mode":"md"},{"id":870,"value":"await FileAttachment(\"ensemble_line_chart.png\").image({width:600})","pinned":false,"hidden":false,"mode":"ojs"},{"id":873,"value":"EMBs are generally familiar with visual designs such as PCP and heatmap, they were surprised by the amount of information made available by combining them.","pinned":false,"hidden":false,"mode":"md"},{"id":877,"value":"Other than a screen sharing session from the developer during the meetings, the system is hosted live (in the form of a web application), this enables the EMBs to explore the system in real-time.
\n**We believe this fostered an excellent cross-disciplinary collaboration where we focus on a common objective.**","pinned":false,"hidden":false,"mode":"md"},{"id":434,"value":"### 7.2 What Didn't Work","pinned":false,"hidden":false,"mode":"md"},{"id":438,"value":"The data size is huge (256,000,000 data points), the data processing is heavy. And given the time we had, the performance is not optimized and the projection of data may be improved with dimensionality reduction (which we briefly experimented with).
\n**We are currently working together with the generic team on the optimization for data processing and storage.**","pinned":false,"hidden":false,"mode":"md"},{"id":884,"value":"Our MLVs at the moment use a balanced representation strategy [(Hadlak et al., 2015)](https://diglib.eg.org/handle/10.2312/eurovisstar.20151109.001-020), where each view is assigned the same amount of space, regardless of the amount of data the visualization is representing. The approach worked fine at the beginning, but we quickly realized the weakness as more and more new views were being introduced.
\n**The strategy isn't the best choice as certain views should take priority over others, and we are currently working on allocating the limited screen space to the MLVs.**","pinned":false,"hidden":false,"mode":"md"},{"id":461,"value":"### 7.3 Recommendations & Opportunities","pinned":false,"hidden":false,"mode":"md"},{"id":425,"value":"* Adopt a different MLV representation strategy (other than the balanced strategy used at the moment) that shows the hierarchical (an overview+detail representation)\n* The ability to compare the model output with the real-world benchmark data\n* Sensitivity analysis as a next step\n","pinned":false,"hidden":false,"mode":"md"},{"id":624,"value":"### 7.4 Who Was Involved?\n","pinned":false,"hidden":false,"mode":"md"},{"id":628,"value":"Rita Borgo, Hui Fang, Elif Firat, Robert Laramee, Cagatay Turkay, Tom Torsney-Weir, Qiru Wang ","pinned":false,"hidden":false,"mode":"md"},{"id":16,"value":"---\n\n## 8. REFERENCES","pinned":false,"hidden":false,"mode":"md"},{"id":484,"value":"**Hadlak, S., Schulz, H.-J., & Schumann, H. (2011).**
\n_In Situ Exploration of Large Dynamic Networks._
\nIEEE Transactions on Visualization and Computer Graphics, 17(12), 2334–2343.
\nhttps://doi.org/10.1109/TVCG.2011.213\n\n**Hadlak, S., Schumann, H., & Schulz, H.-J. (2015).**
\n_A Survey of Multi-faceted Graph Visualization._
\nEurographics Conference on Visualization (EuroVis) - STARs, 20 pages.
\nhttps://doi.org/10.2312/EUROVISSTAR.20151109\n\n**Roberts, J. C. (2007).**
\n_State of the Art: Coordinated & Multiple Views in Exploratory Visualization._
\nFifth International Conference on Coordinated and Multiple Views in Exploratory Visualization (CMV 2007), 61–71.
\nhttps://doi.org/10.1109/CMV.2007.20\n\n**Weaver, C. (2004).**
\n_Building Highly-Coordinated Visualizations in Improvise._
\nIEEE Symposium on Information Visualization, 159–166.
\nhttps://doi.org/10.1109/INFVIS.2004.12\n","pinned":false,"hidden":false,"mode":"md"},{"id":202,"value":"---\n\n### Imports","pinned":false,"hidden":false,"mode":"md"},{"id":815,"value":"import { ui } from \"@mootari/environment\"","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W33","stdlib":"1","titling":"auto"}