2b:null 2c:["$","title",null,{"children":"Analyzing the Design Space for Visualizing Neural Attention in Text Classification | Hernan VALDIVIESO LOPEZ | Observable"}] 31:T646,md` # Analyzing the Design Space for Visualizing Neural Attention in Text Classification Hernan Valdivieso, [Denis Parra](http://web.ing.puc.cl/~dparra), Andres Carvallo, [Gabriel Rada](https://medicina.uc.cl/persona/dr-gabriel-rada/), PUC, Chile
[Katrien Verbert](https://wms.cs.kuleuven.be/cs/onderzoek/augment/katrien-verbert), KU Leuven, Belgium
[Tobias Schreck](https://www.tugraz.at/institute/cgv/schreck/), TU Graz, Austria
## Introduction Deep Neural Networks (DNNs) are a type of machine learning model (Goodfellow et al, 2016) which have reported state-of-the-art results in several tasks in the past years. Despite the impressive results reported by these models in several fields such as computer vision (Krizhevsky et al., 2014), natural language processing (Mikolov et al., 2013) or recommender systems (Covington et al., 2016), one of their biggest drawbacks is their lack of interpretability and transparency. Some of the best performing DNN models have millions of parameters, so making sense of what these models learn is an active research challenge. These algorithms can help to solve and automate difficult and expensive tasks, but their adoption in critical domains, which usually requires liability, and depends on making their decision interpretable by humans. Some large funding programs such as DARPA XAI (Gunning and Aha, 2019) are addressing this problem, providing evidence of their importance. On the other side, recent legislation such as Europe's GDPR gives people the right to explainability of automated decisions regarding their private data. `32:T676,md` One of the most significant techniques introduced to DNNs in the latest years is the so called attention mechanism (Larrochelle and Hinton, 2010). The idea is inspired by our visual system, since humans focus selectively on parts rather than on a whole image, combining information from several fixations to form the full scene (Mnih et al, 2014). This mechanism allows the network to focus on a subset of inputs or parameters when trained on a task. Attention has improved the performance of these models, and it has also given them a chance to be more explainable. Inspecting what the model is paying attention to helps to make the model accountable in tasks such as image classification, document classification or automatic image captioning. Despite this potential, researchers in the area of machine learning usually use the traditional visualization idioms available in software packages, rather than studying all the options for visual encodings to represent models, results or parameters more effectively. We see a chance of using design principles from information visualization in order to improve the way that neural attention models are visually presented. This article focuses on the design space to analyze, inspect and understand what neural attention models are learning. In particular, we aim at contributing to the field of Explainable Artificial Intelligence (XAI), by describing the potential design space as well as informed decisions to take into account when presenting the results of neural networks using the attention mechanism. We also propose some initial ideas with a use case: classification of biomedical documents. `33:T430,md` **Node-link** The seminal article by Bahdanau et al. (2014) on neural machine translation provides one of the first idioms to present attention on a model using a node-link representation, in Figure 1 of the original article. The model shows that in order to predict the current word ${tex `Y_t`}, the model needs information from the current state ${tex `S_{t}`}, from the previous word ${tex `Y_{t-1}`}, as well as from a contextual vector ${tex `c_i`} (not in the diagram), which is calculated by a function that sums all the hidden vectors ${tex `h_i`} weighted by their respective attentions ${tex`\alpha_{t,i}`}. We observe links with attention labels ${tex`\alpha_{t,i}`} coming from the hidden states ${tex `h_i`}, which are combined to create the vector ${tex `c_i`}, all this after a softmax operation to ensure that all the attention values add up to 1.

`34:T5d9,md` **Image and text instances with highlights**. Some articles present the attention directly on the items. For instance, it is usual that in the case of object categorization, the visualization of attention shows the *location* where the model was paying attention to. The Figure 3 in the original article by Xu et al. (2015) shows attention directly on the image used to generate a caption automatically. The top row of images shows deterministic "soft attention" while the bottom row shows stochastic "hard" attention.

Likewise, it is traditional on attention-based text classification methods that the visualization shows to which words the model was paying attention to during the task, such as the Figure 6 in the article by Yang et al. (2016) presenting attention at the sentence level (red color hue) as well as at word level (blue color hue). In this case, color luminance is scaled inversely proportional to the level of attention, i.e., the more the luminance, the smaller the attention.

`35:T73c,md ` ## Summary of Idioms used for Visualizing Attention Munzner (2014) provides a framework for analysis and design of visualizations. She proposes a hierarchical model where the first step is identifying the domain situation. The next step is to identify the respective data and dataset types (*What*), followed by tasks to be solved as pairs {action, target} (*Why*). The final step is to identify the visual encoding and interaction idioms (*How*), considering the most appropriate marks and channels to encode the data. An example summarizing the surveyed works with the frameworks by Munzner, with a non-exhaustive list of tasks (*Why*) is presented in the following table. | Visualization | What | Why (tasks) | How(Marks & Channels) | | --------------------- | ------------ |------------ |------------------------------------------------------------------------------------- | | Heatmap | text | {compare,dependency} |2D matrix alignment of area marks; diverging colormap | | (static) Node-link | text | {identify,features} |Rect with text for nodes, connection marks for links. Quantitative attribute on links. | | Interactive node-link | text | {identify, paths} |Rect with text for nodes, connection marks for links. Links appear when hovering nodes. | | Text and image | text/image | {identify,features} |Color luminance on the image spots or words composing a text. |
In the coming section we will present a use case where a traditional task of document classification is utilized to explore the design space for visualizing the attention provided by the model. `36:T7a5,md` # Use Case: Inspecting Attention-Based Document Classification Document classification is a well-known task in the areas of information retrieval as well as machine learning. As a use case for this article we focus on classification of biomedical research papers. This task is important within the area of Evidence-based Healthcare, where physicians attempt to identify the most relevant articles to answer clinical or health-policy questions related to patients, treatments and their outcomes. In particular, we collaborate with the Epistemonikos foundation helping them to identify which systematic review documents are scientifically sound in order to be indexed in their database. Until now, they have relied on physicians to do this work. Despite the lack of scalability, they refrain from using automatic document classification algorithms since they cannot guarantee that the reasons used by the algorithm are valid or generalizable. Using a classification model which provides attention can be of great use in this task. Given a biomedical paper abstract like this one: "Low-dose Thimerosal (ethyl-mercury) is still used in infants' vaccines: Should we be concerned with this form of exposure?" by Dorea (2018) ![image](https://user-images.githubusercontent.com/208111/62179131-703b4e80-b319-11e9-99db-609dbde3131c.png) Which is the best way to visualize the attention that the model pays on the features (words) to classify a document technically sound systematic review? Is the traditional encoding of highlighted words with colored background, such as Yang et al. (2016), the best encoding? How can information visualization help us to explore the design space and choose among the options available? `37:T4a7,md` # The design space for visualizing attention The framework by Munzner (2014) describes marks and channels as the most basic elements to design a visualization. **Marks** are defined as geometric primitive objects categorized based number of spatial dimensions they require: points (0D), lines (1D), areas (2D), or volume mark (3D). On the other side, a **channel** is a way to control the appearance of marks, such as length, color, angle, etc. On documents, the channels used for representing attention by highlighting words on text are color luminance or color saturation. While this seems to be a reasonable decision, if we follow Steven's Psychological Power law (Munzner, 2014) shows that while the perception of certain channels such as length is perceived linearly, color brightness is not perceived linearly, but rather by a factor of 0.5. This means that, for instance, an increase of three times the brightness will not be perceived by people as three times the attention of the model on a word, but rather only 1.5 times more attention. `38:Tb7d,md` * **Font property - Size** This is the most commonly used channel in word clouds. It maps magnitude to the height of the font. However, the literature on visualization suggests that font size might not be an optimal channel to convey quantitative information because of the irregular shape of words and the non-direct relationship between height, width and painting area. * **Font property - Luminance** According to [(Bateman et al. 2008)](https://dl.acm.org/citation.cfm?id=1379130), after discarding color hue that only works for categorical values and font weight that has too little resolution to be effective, color intensity (contrast or saturation) is the next best font channel to convey importance of a keyword and support quantitative information. Depending on the color model used (HSL or HSV, for instance) and fixing the color font to a default black, we can map strong levels of attention to a font with small luminance (pure black) and small levels of attention to light grey. * **Additional Mark - Bars Length** Bars are one of the most used marks in visualization, they allow the use of very effective and precise channels like position and length. When bars are aligned they can clearly encode the value using position, the strongest channel for quantitative visualization. Like [(Felix et al. 2017)](https://ieeexplore.ieee.org/abstract/document/8017641), we use horizontal bars, since they align with the word reading order. One issue, though, is that in the original article they did not explain how to calculate the range of the bar's length, so we propose to use the length of the longer word as a maximum attention equal to 1, and the other bar lengths linearly proportional. Another option to explore is aligning the words in columns to improve the comparison of attention between words, as the result of the study by Felix suggests. * **Additional Mark - Circle Area** Circles are often used in scatter plots as bubbles to convey a third value. Also in other visualizations in which the spatial properties are already used to encode other information (e.g., bubble maps), circles are often used. Since the diameter of circles change together with area size, it is not possible to find a unique strategy to overlap circles with words the same way we do with bars. For this reason, in our proposal, words are placed to the right side of each circle. * **Aditional Mark - Background Color Intensity** This is the traditional way to present attention, as in the work by [Yang et. el (2016)](https://www.aclweb.org/anthology/N16-1174). Mapping directly attention to font color can hide the words if they receive too low attention, so this encoding use background color intensity to highlight the words with more attention. However, for certain tasks the direct encoding of attention by font color could be more effective, such as identifying only the top ${tex `n` } words used to classify a document. `39:T787,md ` The following is an example of an actual biomedical document, by [Farquharson et al. (2018)](https://www.nature.com/articles/s41467-018-03500-9), which our model correctly classified as *Not Relevant to be indexed* (**NR**). We present the title (*A Meta Analysis of birth origin on reproduction in diverse captive environments*), the prediction made by our neural neural classification model with attention (in this case, *Not Relevant*), the actual class of the document (ground truth, in this case *Not Relevant*), and the reason why this document was classified as NR. There are five reasons for labeling a document as not relevant in our dataset: - excluded_not_humans: article is about animals rather than human patients. - excluded_protocol: article which describes the method or protocol of a systematic review to be conducted in the future, not ready yet. - excluded_not_health_decision: the results of the article do not provide evidence towards or against treatments to make health decisions. - broad_synthesis: this article is a review of several systematic reviews, too broad for the goals of the system. - excluded_not_health: systematic review with conclusions or implications not directly related to health outcomes. Notice, in this example, that words like *bird*, *captive*, *invertebrates* and *mammal*s are paid attention to. This provides an evidence that the model is paying attention on the right words to classify the document as NR. Notice also that the same words in different sentences have different attention levels: this is a property of the neural attentive model, since attention uses also the context of the word, not only the word token. One aspect that requires further inspection though is that words like *for*, *the*, and *in* also have an important level of attention, which raises some concern about potential biases the model is learning in order to classify this article. `3a:T9c0,md` # Design space to visualize the attention when comparing between documents The second task we analyze for visualization design is that of comparing attention between documents. The traditional way to present attention is using a heatmap where the luminance on the cell grids indicate the level of attention (the darker the cell, the higher the attention). We propose alternatives in this design space that we can explore in the future work to compare attention for the same words between different documents. * **Text** The luminance has the problem of being a not very accurate channel: Two words with a small difference in attention might be perceived too similar due to the small accuracy of the luminance channel. One alternative is to change the luminance to text that indicate the attention value. This encoding allows us to easily compare the attention between documents and words, but to explore the result and identify particular values like the word with more attention will be more difficult than using simply luminance. * **Text/Luminance** This encoding combines luminance and text. With this encoding the task to explore the visualization is easier with the luminance and we allow the user to compare more precisely attention using the actual values. * **Bar (Horizontal or vertical)** Bars are one of the most used marks in visualization, as we indicated in the previous section as well as citing the work by Felix et al. (2017). We propose to change the luminance on the cell grids by horizontal or vertical bars because the alignment will allow to compare attention for a fixed words between different documents (vertical bar) or attention within a particular document between different words (horizontal bar). * **Area** This encoding is the merge of vertical and horizontal bar. Although less precise than bar length, surface area is a more precise channel than color luminance or saturation. Moreover, the bars within the cells can be too small, making attention difficult to compare across words or documents. * **Interactive hover** The previous encodings are all designed for static visualizations. Hence, we also explore the alternative to use interactive visualization that help us to compare with more precision the attention. If a user hovers the mouse pointer upon a cell, we will interactively display this bar or square on the other cells of the grid. This aprroch will help users to compare more precisely the atttention between close values. `3b:T2cc2,designSpace = { const w = width - margin.left - margin.right + 100; const h = 350; margin.top + margin.bottom + 30; const svg = d3.select(DOM.svg(w, h)); const data = [ { attention: 0.9, row: 0, column: 0 }, { attention: 0.5, row: 0, column: 1 }, { attention: 0.3, row: 1, column: 0 }, { attention: 0.4, row: 1, column: 1 } ]; const length = d3 .scaleLinear() .domain([0, 1]) .range([0, 39]); const color = d3 .scaleLinear() .domain([0, Math.max(...data.map(d => d.attention))]) .range([colors.lowAttention, colors.highAttention]); // rotate the text svg .append("text") .attr("x", 0) .attr("y", 0) .attr("transform", "translate(20, 130) rotate(-90)") .text("Static"); svg .append("text") .attr("x", 0) .attr("y", 0) .attr("transform", "translate(20, 270) rotate(-90)") .text("Interactive"); svg .append("text") .attr("x", 0) .attr("y", 0) .attr("transform", "translate(35, 250) rotate(-90)") .text("hover"); let text = "Encoding to compare attention in words between documents"; svg .append("g") .attr("transform", "translate(170, 25)") .append('text') .text(text); svg .append("text") .attr("x", 49) .attr("y", 60) .text("Luminance"); const V1 = svg .append("g") .attr("transform", "translate(50, 70)") .selectAll(".V1") .data(data) .enter(); V1.append("rect") .attr("class", "V1") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => color(d.attention)) .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); svg .append("text") .attr("x", 192) .attr("y", 60) .text("Text"); const V2 = svg .append("g") .attr("transform", "translate(170, 70)") .selectAll(".V2") .data(data) .enter(); V2.append("rect") .attr("class", "V2") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V2.append("text") .attr("class", "V2") .attr("x", d => d.column * 40 + 8) .attr("y", d => d.row * 40 + 25) .attr("fill", "black") .text(d => d.attention); svg .append("text") .attr("x", 265) .attr("y", 60) .text("Text/Luminance"); const V3 = svg .append("g") .attr("transform", "translate(290, 70)") .selectAll(".v3") .data(data) .enter(); V3.append("rect") .attr("class", "V3") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => color(d.attention)) .attr("width", 40) .attr("height", 40) .attr("stroke", "white"); V3.append("text") .attr("class", "V3") .attr("x", d => d.column * 40 + 8) .attr("y", d => d.row * 40 + 25) .attr("fill", "red") .text(d => d.attention); svg .append("text") .attr("x", 405) .attr("y", 60) .text("Vertical Bar"); const V4 = svg .append("g") .attr("transform", "translate(410, 70)") .selectAll(".V4") .data(data) .enter(); V4.append("rect") .attr("class", "V4") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V4.append("rect") .attr("class", "V4") .attr("x", d => d.column * 40 + 12.5) .attr("y", d => (d.row + 1) * 40 - length(d.attention)) .attr("fill", d => "gray") .attr("width", 15) .attr("height", d => length(d.attention)) .attr("stroke", "black"); svg .append("text") .attr("x", 520) .attr("y", 60) .text("Horizontal Bar"); const V5 = svg .append("g") .attr("transform", "translate(530, 70)") .selectAll(".V5") .data(data) .enter(); V5.append("rect") .attr("class", "V5") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V5.append("rect") .attr("class", "V5") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40 + 12.5) .attr("fill", d => "gray") .attr("width", d => length(d.attention)) .attr("height", 15) .attr("stroke", "black"); svg .append("text") .attr("x", 670) .attr("y", 60) .text("Area"); const V6 = svg .append("g") .attr("transform", "translate(650, 70)") .selectAll(".v6") .data(data) .enter(); V6.append("rect") .attr("class", "V6") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V6.append("rect") .attr("class", "V6") .attr("x", d => d.column * 40) .attr("y", d => (d.row + 1) * 40 - length(d.attention)) .attr("fill", d => "gray") .attr("width", d => length(d.attention)) .attr("height", d => length(d.attention)) .attr("stroke", "black"); const V4_2 = svg .append("g") .attr("transform", "translate(410, 190)") .selectAll(".V4_2") .data(data) .enter(); V4_2.append("rect") .attr("class", "V4_2") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V4_2.append("rect") .attr("class", "V4_2") .attr("x", d => d.column * 40 + 10) .attr("y", d => (d.row + 1) * 40 - length(d.attention)) .attr("fill", d => "gray") .attr("width", 15) .attr("height", d => length(d.attention)) .attr("stroke", "black") .on("mouseover", d2 => { V4_2.filter(d => (d.row != d2.row || d.column != d2.column) && d2.attention < d.attention) .append("rect") .attr("class", "V4_2 hover") .attr("x", d => d.column * 40 + 10) .attr("y", d => (d.row + 1) * 40 - length(d2.attention)) .attr("fill", d => color(0.5)) .attr("width", 15) .attr("height", length(d2.attention)) .attr("stroke", "black") V4_2.filter(d => (d.row != d2.row || d.column != d2.column) && d2.attention >= d.attention) .append("rect") .attr("class", "V4_2 hover") .attr("x", d => d.column * 40 + 10) .attr("y", d => (d.row + 1) * 40 - length(d2.attention)) .attr("fill", d => color(0.5)) .attr("width", 15) .attr("height", length(d2.attention)) .attr("stroke", "black") V4_2.filter(d => (d.row != d2.row || d.column != d2.column) && d2.attention >= d.attention) .append("rect") .attr("class", "V4_2 hover") .attr("x", d => d.column * 40 + 10) .attr("y", d => (d.row + 1) * 40 - length(d.attention)) .attr("fill", d => "gray") .attr("width", 15) .attr("height", d => length(d.attention)) .attr("stroke", "black") }) .on("mouseout", _ => { V4_2.selectAll(".V4_2.hover").remove(); }); const V5_2 = svg .append("g") .attr("transform", "translate(530, 190)") .selectAll(".V5_2") .data(data) .enter(); V5_2.append("rect") .attr("class", "V5_2") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V5_2.append("rect") .attr("class", "V5_2") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40 + 12.5) .attr("fill", d => "gray") .attr("width", d => length(d.attention)) .attr("height", 15) .attr("stroke", "black") .on("mouseover", d2 => { V5_2.filter(d => (d.row != d2.row || d.column != d2.column) && d2.attention < d.attention) .append("rect") .attr("class", "V5_2 hover") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40 + 12.5) .attr("fill", color(0.5)) .attr("width", d => length(d2.attention)) .attr("height", 15) .attr("stroke", "black"); V5_2.filter(d => (d.row != d2.row || d.column != d2.column) && d2.attention >= d.attention) .append("rect") .attr("class", "V5_2 hover") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40 + 12.5) .attr("fill", color(0.5)) .attr("width", d => length(d2.attention)) .attr("height", 15) .attr("stroke", "black"); V5_2.filter(d => (d.row != d2.row || d.column != d2.column) && d2.attention >= d.attention) .append("rect") .attr("class", "V5_2 hover") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40 + 12.5) .attr("fill", d => "gray") .attr("width", d => length(d.attention)) .attr("height", 15) .attr("stroke", "black"); }) .on("mouseout", _ => { V5_2.selectAll(".V5_2.hover").remove(); }); const V6_2 = svg .append("g") .attr("transform", "translate(650, 190)") .selectAll(".V6_2") .data(data) .enter(); V6_2.append("rect") .attr("class", "V6_2") .attr("x", d => d.column * 40) .attr("y", d => d.row * 40) .attr("fill", d => "white") .attr("width", 40) .attr("height", 40) .attr("stroke", "black"); V6_2.append("rect") .attr("class", "V6_2") .attr("x", d => d.column * 40) .attr("y", d => (d.row + 1) * 40 - length(d.attention)) .attr("fill", d => "gray") .attr("width", d => length(d.attention)) .attr("height", d => length(d.attention)) .attr("stroke", "black") .on("mouseover", d2 => { V6_2.filter( d => (d.row != d2.row || d.column != d2.column) && d2.attention < d.attention ) .append("rect") .attr("class", "V6_2 hover") .attr("x", d => d.column * 40) .attr("y", d => (d.row + 1) * 40 - length(d2.attention)) .attr("fill", color(0.5)) .attr("width", d => length(d2.attention)) .attr("height", d => length(d2.attention)) .attr("stroke", "black"); V6_2.filter( d => (d.row != d2.row || d.column != d2.column) && d2.attention >= d.attention ) .append("rect") .attr("class", "V6_2 hover") .attr("x", d => d.column * 40) .attr("y", d => (d.row + 1) * 40 - length(d2.attention)) .attr("fill", color(0.5)) .attr("width", d => length(d2.attention)) .attr("height", d => length(d2.attention)) .attr("stroke", "black"); V6_2.filter( d => (d.row != d2.row || d.column != d2.column) && d2.attention >= d.attention ) .append("rect") .attr("class", "V6_2 hover") .attr("x", d => d.column * 40) .attr("y", d => (d.row + 1) * 40 - length(d.attention)) .attr("fill", "gray") .attr("width", d => length(d.attention)) .attr("height", d => length(d.attention)) .attr("stroke", "black"); }) .on("mouseout", _ => { V6_2.selectAll(".V6_2.hover").remove(); }); return svg.node(); }3c:T851,md ` # Conclusion and Future Work In this interactive article, we studied a particular technique within the context of explainable AI, which is the attention mechanism in deep neural networks. We have conducted a non-exhaustive survey and analysis of the visual idioms used to represent neural attention, at the level of modeling architectures or presenting the prediction of for images, sentence translation or document classification. Then, using a usage case, classification of biomedical articles, we have explored some alternatives in the design space for visualizing attention. The first task addressed is related to identifying important words in a particular document (being attended by the neural model) and the second task is about comparing not only words within a document, but also comparing between documents. Based on what is presented here, our next steps are designing a user study, and then conduct it with actual physicians, specially considering those who use systems for evidence-based healthcare such as [Epistemonikos](https://www.epistemonikos.org/), [Cochrane](https://www.cochrane.org/) or [Embase](https://www.embase.com/#search). We also plan to provide a more structured and wider exploration of the options for the design space. For instance, in the idioms to visualize attention within the text, we used a linear scale to encode the attention in channels such as font size, or luminance. There are alternatives to study: changing the scale to logarithmic, exponential, or quadratic. Depending on the task, even inverting the scale to focus in the words receiving less attention could be an aspect for deeper investigation. We would also like to identify a broader range of tasks to be considered in order to identify a larger scope of idioms useful for task related to inspecting attention models. Another aspect we would like to study is going beyond the attention of neural networks, but testing other methods which provide explanations such as [LIME](https://github.com/marcotcr/lime) or [Shap](https://github.com/slundberg/shap), and to test the design space with these alternatives. `3d:Ta74,md` ## References Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. Bateman, S., Gutwin, C., & Nacenta, M. (2008). Seeing things in the clouds: the effect of visual features on tag cloud selections. In Proceedings of the nineteenth ACM conference on Hypertext and hypermedia (pp. 193-202). ACM. Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 191-198). ACM. Farquharson, K. A., Hogg, C. J., & Grueber, C. E. (2018). A meta-analysis of birth-origin effects on reproduction in diverse captive environments. Nature communications, 9(1), 1055. Felix, C., Franconeri, S., & Bertini, E. (2017). Taking word clouds apart: An empirical investigation of the design space for keyword summaries. IEEE transactions on visualization and computer graphics, 24(1), 657-666. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press. Gunning, D., & Aha, D. W. (2019). DARPA's Explainable Artificial Intelligence Program. AI Magazine, 40(2), 44-58. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105). Larochelle, H., & Hinton, G. E. (2010). Learning to combine foveal glimpses with a third-order Boltzmann machine. In Advances in neural information processing systems (pp. 1243-1251). Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111-3119). Mnih, V., Heess, N., & Graves, A. (2014). Recurrent models of visual attention. In Advances in neural information processing systems (pp. 2204-2212). Munzner, T. (2014). Visualization Analysis and Design. A K Peters/CRC Press. Olah, C., & Carter, S. (2016). Attention and augmented recurrent neural networks. Distill, 1(9), e1. http://doi.org/10.23915/distill.00001 Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel & Bengio, Y. (2015). Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning (pp. 2048-2057). Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies (pp. 1480-1489). `3e:T91f,md` # Appendix ## The classification model We left the details of our neural attentive document classification model in the appendix to avoid users diverting their focus from the main contribution of the article, which is on the area of information visualization. The next Figure shows the model.

The input is document title and abstract of length *n* words, where *n* is also a parameter of the model that we set to 150 (the median amount of words in the abstracts of our dataset). We do this since in some cases the documents have abstracts of more than 2,000 words, making the model too complex only to manage these outliers. Each word is represented with a word embedding vector (word2vec or Glove) ${tex`w_i`}. Next, each word embedding gets into an LSTM cell, which ouputs a hidden state vector ${tex`h_i`}. For each hidden state, the attention layer learns a value (attention ${tex`\alpha_i`}) depending on how much it contributes to the final decision, i.e, the document classification. Finally the product of each hidden state vector ${tex`h_i`} with their respective attention parameter ${tex`\alpha_i`} are aggregated through a linear layer to finally obtain the predicted class. Note that we can identify which words were payed more attention to because the index of the attention parameters are the same as the index of the words used as inputs for the model. These attentions are then used in the visualizations, to explain the class prediction. ## Results

The next table shows the results of the model, which improves the performance of our previous model based on random forest by around 10% on different metrics (precision, recall and f1-score). However, it has better performance when predicting documents that are scientifically relevant (class 1 in the table) compared to those that are not relevant (class 0 in the table), explained by the lower recall levels of the last one. The goal of the visualizations in this document is helping on inspecting the results of the model, specially those cases wrongly classified. `3f:T1503,buildVisualization = (dataset, interactive) => { const w = width - margin.left - margin.right + 100; const gridSize = Math.floor(width / 38); const h = dataset.finals_words.length * (gridSize + 2) + margin.top + margin.bottom + 30; const h_legend = h + 100; const svg = d3.select(DOM.svg(w, h_legend)); const data = []; const documents = []; const key = Object.keys(dataset.heatmap); key.sort(); key.forEach((key, row) => { dataset.heatmap[key].forEach((d, column) => { data.push({ value: d, row: row, column: column, word: key }); }); }); for (let i = 0; i <= Math.max(...data.map(d => d.column)); i++) { documents.push(i); } const bottomText = svg .selectAll() .data(documents) .enter() .append("g") .attr("transform", "translate(0," + (margin.left + gridSize / 1.5) + ")"); const predictionColor = d3 .scaleLinear() .domain([0, 0.5, 1]) .range([colors.not_relevant, "white", colors.relevant]); const lengthBarCertainty = d3 .scaleLinear() .domain([0, 1]) .range([0, 20]); // % of prediction bottomText .append("rect") .attr("x", (_, i) => i * (gridSize + 2) + 90) .attr("y", h - 82) .attr("width", gridSize) .attr("height", gridSize + 3) .attr("stroke", "black") .attr("fill", 'white'); // TextDocument bottomText .append("text") .text(d => (d < 10 ? "0" + String(d) : String(d))) .attr("x", (_, i) => i * (gridSize + 2) + 93) .attr("y", h - 73) .attr("font-size", "7pt") .attr("fill", "black"); // TextDocument bottomText .append("text") .text(d => { let prediction = dataset.random_sample[d].predicted_label?"R":"NR"; return `${prediction}` }) .attr("x", (_, i) => i * (gridSize + 2) + 93) .attr("y", h - 65) .attr("font-size", "6pt") .attr("fill", "black"); bottomText .append("text") .text(d => { console.log(d); let ground = dataset.random_sample[d].label?"R": "NR"; return `${ground}` }) .attr("x", (_, i) => i * (gridSize + 2) + 93) .attr("y", h - 58) .attr("font-size", "6pt") .attr("fill", "black"); // Words svg .selectAll() .data(key) .enter() .append("text") .text(d => d) .attr("x", 0) .attr("y", (_, i) => i * (gridSize + 2)) .attr("font-size", "9pt") .attr("fill", colors.text) .attr("transform", "translate(0," + (margin.left + gridSize / 1.5) + ")"); const color = d3 .scaleLinear() .domain([0, Math.max(...data.map(d => d.value))]) .range([colors.lowAttention, colors.highAttention]); const cards = svg.selectAll().data(data); let clicked = null; const cardsEnter = cards .enter() .append("rect") .attr("x", d => d.column * (gridSize + 2)) .attr("y", d => d.row * (gridSize + 2)) .attr("transform", "translate(90, 20)") .attr("rx", 4) .attr("ry", 4) .attr("class", "bordered") .attr("width", gridSize) .attr("height", gridSize) .style("fill", d => color(d.value)); svg .append("text") .attr("x", 250) .attr("y", h) .attr("font-size", "10pt") .text("Documents"); if (interactive) { cardsEnter .on("click", (d, i, all) => { let result = findIndex(dataset, d); if (result.index >= 0) { if (clicked) { d3.select(clicked).style("stroke", "#F7F7F7"); } d3.select(all[i]).style("stroke", colors.clicked); clicked = all[i]; buildBar(result, documents, gridSize, d, svg, 0); } }) .on("mouseover", (d, i, all) => { if (clicked == all[i]) { return; } d3.select(all[i]).style("stroke", colors.hover); let result = findIndex(dataset, d); if (result.index >= 0) { buildBar(result, documents, gridSize, d, svg, 1); } }) .on("mouseout", (d, i, all) => { if (clicked == all[i]) { return; } d3.select(all[i]).style("stroke", "#F7F7F7"); }); } //Append a defs (for definition) element to your SVG let defs = svg.append("defs"); let linearGradient2 = defs .append("linearGradient") .attr("id", "linear-gradient-2") .attr("x1", "0%") .attr("y1", "0%") .attr("x2", "100%") .attr("y2", "0%") .selectAll("stop") .data(color.range()) .enter() .append("stop") .attr("offset", (_, i) => i / (color.range().length - 1)) .attr("stop-color", d => d); //Draw the rectangle and fill with gradient const marginBottom = 40; svg .append("rect") .attr("x", 90) .attr("y", h + 25) .attr("width", documents.length * (gridSize + 2)) .attr("height", 20) .style("fill", "url(#linear-gradient-2)"); svg .append("text") .attr("x", 88) .attr("y", h + 22 + marginBottom) .attr("font-size", "9pt") .text("0"); svg .append("text") .attr("x", 230) .attr("y", h + 22 + marginBottom) .attr("font-size", "9pt") .text("% of attention"); svg .append("text") .attr("x", documents.length * (gridSize + 2) + 75) .attr("y", h + 22 + marginBottom) .attr("font-size", "9pt") .text(String(Math.max(...data.map(d => d.value))).slice(0, 6)); return svg.node(); }; 40:Tb88,_buidBar = (words, attention, svg, width, version, mainWord, document) => { let newWords = []; words.forEach((d, i) => { newWords.push(`${i + 1}-${d}`); }); let newIndex = 6; let haveSeen = false; while (!haveSeen && newIndex >= 0) { if (words[newIndex] == mainWord) { haveSeen = true; break; } newIndex--; } const translate = 130; svg.selectAll(`.barchart-${version}`).remove(); let height = 30; if (version == 1) { height = 330; } let newsvg = svg .append("g") .attr("class", `barchart-${version}`) .attr("transform", `translate(${width + translate}, ${height})`); const x = d3 .scaleLinear() .domain([0, 0.1]) .range([0, 300]); const y = d3 .scaleBand() .domain(newWords) .range([0, 200]) .padding(0.2); const yAxis = g => g .attr("transform", `translate(${width + translate}, ${height})`) .attr("class", `barchart-${version}`) .call(d3.axisLeft(y).tickSizeOuter(0)); const xAxis = g => g .attr("transform", `translate(${width + translate}, ${height + 200})`) .attr("class", `barchart-${version}`) .call(d3.axisBottom(x)) .call(g => g.select(".domain").remove()); const border = newsvg .append("rect") .attr("x", 0) .attr("y", 0) .attr("fill", version == 0 ? colors.clicked : colors.hover) .attr("height", 200) .attr("width", 300); const bars = newsvg .selectAll(".rect") .data(attention) .enter() .append("rect") .attr("x", x(0)) .attr("y", (_, i) => y(newWords[i])) .attr("class", "rect") .attr("fill", colors.bars) .attr("height", y.bandwidth()) .attr("width", d => x(d) - x(0)); svg.append("g").call(xAxis); svg.append("g").call(yAxis); newsvg .append("text") .attr("class", "title") .attr("x", 90) .attr("y", 235) .text(`Document ${document.index}`); // Define the div for the tooltip let div = d3 .select("body") .append("div") .attr("class", "tooltip") .style("opacity", 0); newsvg .append("text") .attr("class", "title") .attr("x", 10) .attr("y", -15) .text(`${document.title.slice(0, 30)}...`); newsvg .append("text") .attr("class", "title") .attr("x", 265) .attr("y", -15) .text(`(?)`) .style("cursor", "pointer") .on("click", d => { div.transition() .duration(200) .style("opacity", 0.9); div.html(document.title) .style("left", d3.event.pageX - 100 + "px") .style("top", d3.event.pageY - 70 + "px"); }) .on("mouseout", d => { div.transition() .duration(500) .style("opacity", 0); }); svg .selectAll(`.barchart-${version}`) .selectAll("text") .attr("fill", colors.text) .filter((d, i) => d == `${newIndex + 1}-${mainWord}`) .attr("fill", colors.focusText) .attr("font-weight", 900); }; 41:T2493,buildAttentionIdioms = (d, dataset) => { const letterWidth = {}; const medicalDocument = d; const text = dataset.random_sample[medicalDocument].text; const attentions = dataset.random_sample[medicalDocument].attention; const w = width - margin.left - margin.right; const div = document.createElement("div"); div.style.float = "left"; document.body.appendChild(div); let i = text.length; while (i--) { let letter = text.charAt(i); div.innerText = letter; let computedWidth = window .getComputedStyle(div, null) .getPropertyValue("width") .match(/\d+/)[0]; letterWidth[letter] = parseInt(computedWidth); } let maxLength = 0 text.split(" ").forEach(d => { let length = 0; var i = d.length; while (i--) { length += letterWidth[d.charAt(i)]; } if (length > maxLength){ maxLength = length; } }) let xText = 0; let yText = 0; let xBar = 0; let yBar = 0; let xCircle = 0; let yCircle = 0; let xFont = 0; let yFont = 0; const color = d3 .scaleLinear() .domain([0, d3.max(attentions)]) .range(["#f7f7f7", "red"]); const lengthBar = d3 .scaleLinear() .domain([0, d3.max(attentions)]) .range([0, maxLength]); const lengthCircle = d3 .scaleLinear() .domain([0, d3.max(attentions)]) .range([0, 20]); const sizeFont = d3 .scaleLinear() .domain([0, d3.max(attentions)]) .range([8, 20]); const words = text.split(" ").map((d, index) => { let length = 0; var i = d.length; while (i--) { length += letterWidth[d.charAt(i)]; } let newWidthText = xText + 10 + length; let newWidthCircle = xCircle + 12 + 2 * lengthCircle(attentions[index]) + length; let newWidthBar = xBar + 12 + length; let newWidthFont = xFont + 10 + (length * sizeFont(attentions[index])) / 17; if (lengthBar(attentions[index]) > 8 + length) { newWidthBar = xBar + 10 + lengthBar(attentions[index]); } if (newWidthText > w - 50) { xText = 0; yText += 20; newWidthText = 10 + length; } if (newWidthFont > w - 50) { xFont = 0; yFont += 30; newWidthFont = 10 + (length * sizeFont(attentions[index])) / 17; } if (newWidthCircle > w - 50) { xCircle = 0; yCircle += 30; newWidthCircle = 12 + 2 * lengthCircle(attentions[index]) + length; } if (newWidthBar > w - 50) { xBar = 0; yBar += 20; newWidthBar = 12 + length; if (lengthBar(attentions[index]) > 10 + length) { newWidthBar = 12 + lengthBar(attentions[index]); } } let value = { xText: xText, yText: yText, word: d, attention: attentions[index], width: length + 6, xBar: xBar, yBar: yBar, xCircle: xCircle, yCircle: yCircle, xFont: xFont, yFont: yFont }; xText = newWidthText; xBar = newWidthBar; xCircle = newWidthCircle; xFont = newWidthFont; return value; }); // Font property - Size const translate1 = 55; // Font property - Color Intensity const translate2 = translate1 + (yFont + 85); // Additional Mark - Bars Length const translate3 = translate2 + (yText + 85); // Additional Mark - Circle Area const translate4 = translate3 + (yBar + 85); // Aditional Mark - Background Color Intensity const translate5 = translate4 + (yCircle + 85); const h = translate5 + (yText + 25); const svg = d3.select(DOM.svg(w, h)); ////////////////////////////////////////////////////////////////////////////// // Font property - Size svg .append("g") .attr("transform", `translate(50, ${translate1})`) .selectAll("text") .data(words) .enter() .append("text") .attr("class", d =>`V1 ${d.word}`) .attr("x", d => d.xFont) .attr("y", d => d.yFont) .attr("font-size", d => sizeFont(d.attention)) .attr("fill", "black") .text(d => d.word) .on('mouseover', d => svg.selectAll(`.V1.${d.word}`).attr('fill', 'magenta')) .on('mouseout', d => svg.selectAll(`.V1.${d.word}`).attr('fill', 'black')) .style('cursor', 'pointer'); svg .append("g") .attr("transform", `translate(50, ${translate1 - 30})`) .append("text") .attr("x", 0) .attr("y", 0) .attr("font-size", 20) .attr("font-weight", "bold") .text("Font property - Size"); ////////////////////////////////////////////////////////////////////////////// // Font property - Color Intensity const color2 = d3 .scaleLinear() .domain([0, d3.max(attentions)]) .range(["#f7f7f7", "black"]); svg .append("g") .attr("transform", `translate(50, ${translate2})`) .selectAll("rect") .data(words) .enter() .append("rect") .attr("class", d =>`V2 ${d.word}`) .attr("x", d => d.xText - 2) .attr("y", d => d.yText - 15) .attr('height', 20) .attr('width', d => d.width) .attr("fill", 'white') .attr("stroke", "magenta") .style("display", "none"); svg .append("g") .attr("transform", `translate(50, ${translate2})`) .selectAll("text") .data(words) .enter() .append("text") .attr("x", d => d.xText) .attr("y", d => d.yText) .attr("fill", d => color2(d.attention)) .text(d => d.word) .on('mouseover', d => svg.selectAll(`.V2.${d.word}`).style('display', 'block')) .on('mouseout', d => svg.selectAll(`.V2.${d.word}`).style('display', 'none')) .style('cursor', 'pointer'); svg .append("g") .attr("transform", `translate(50, ${translate2 - 30})`) .append("text") .attr("x", 0) .attr("y", 0) .attr("font-size", 20) .attr("font-weight", "bold") .text("Font property - Luminance"); ////////////////////////////////////////////////////////////////////////////// // Additional Mark - Bars Length svg .append("g") .attr("transform", `translate(50, ${translate3})`) .selectAll("rect") .data(words) .enter() .append("rect") .attr("x", d => d.xBar) .attr("y", d => d.yBar - 15) .attr("width", d => lengthBar(d.attention)) .attr("height", 18) .attr("fill", "#dbdbdb") svg .append("g") .attr("transform", `translate(50, ${translate3})`) .selectAll("text") .data(words) .enter() .append("text") .attr("class", d =>`V3 ${d.word}`) .attr("x", d => d.xBar) .attr("y", d => d.yBar) .attr("fill", "black") .text(d => d.word) .on('mouseover', d => svg.selectAll(`.V3.${d.word}`).attr('fill', 'magenta')) .on('mouseout', d => svg.selectAll(`.V3.${d.word}`).attr('fill', 'black')) .style('cursor', 'pointer'); svg .append("g") .attr("transform", `translate(50, ${translate3 - 30})`) .append("text") .attr("x", 0) .attr("y", 0) .attr("font-size", 20) .attr("font-weight", "bold") .text("Additional Mark - Bars Length"); ////////////////////////////////////////////////////////////////////////////// // Additional Mark - Circle Area svg .append("g") .attr("transform", `translate(50, ${translate4})`) .selectAll("circle") .data(words) .enter() .append("circle") .attr("cx", d => d.xCircle + lengthCircle(d.attention)) .attr("cy", d => d.yCircle - 5) .attr("r", d => lengthCircle(d.attention)) .attr("fill", "#dbdbdb"); svg .append("g") .attr("transform", `translate(50, ${translate4})`) .selectAll("text") .data(words) .enter() .append("text") .attr("class", d =>`V4 ${d.word}`) .attr("x", d => d.xCircle + 2 * lengthCircle(d.attention)) .attr("y", d => d.yCircle) .attr("fill", "black") .text(d => d.word) .on('mouseover', d => svg.selectAll(`.V4.${d.word}`).attr('fill', 'magenta')) .on('mouseout', d => svg.selectAll(`.V4.${d.word}`).attr('fill', 'black')) .style('cursor', 'pointer'); svg .append("g") .attr("transform", `translate(50, ${translate4 - 30})`) .append("text") .attr("x", 0) .attr("y", 0) .attr("font-size", 20) .attr("font-weight", "bold") .text("Additional Mark - Circle Area"); ////////////////////////////////////////////////////////////////////////////// // Aditional Mark - Background Color Intensity svg .append("g") .attr("transform", `translate(50, ${translate5})`) .selectAll("rect") .data(words) .enter() .append("rect") .attr("x", d => d.xText - 2) .attr("y", d => d.yText - 15) .attr("width", d => d.width) .attr("height", 18) .attr("fill", d => color(d.attention)) .attr("rx", 8) .attr("ry", 8); svg .append("g") .attr("transform", `translate(50, ${translate5})`) .selectAll("text") .data(words) .enter() .append("text") .attr("class", d =>`V5 ${d.word}`) .attr("x", d => d.xText) .attr("y", d => d.yText) .text(d => d.word) .on('mouseover', d => svg.selectAll(`.V5.${d.word}`).attr('fill', 'magenta')) .on('mouseout', d => svg.selectAll(`.V5.${d.word}`).attr('fill', 'black')) .style('cursor', 'pointer'); svg .append("g") .attr("transform", `translate(50, ${translate5 - 30})`) .append("text") .attr("x", 0) .attr("y", 0) .attr("font-size", 20) .attr("font-weight", "bold") .text("Aditional Mark - Background Color Intensity"); return svg.node() }42:[] 2f:{"id":"a2718d0a0fabfe52","slug":"analyzing-the-design-space-for-visualizing-neural-attenti","title":"Analyzing the Design Space for Visualizing Neural Attention in Text Classification","thumbnail":"6577e87c0e846911b63c1b52872394f2a30276fe35308eef556cb9199516f538","access_level":"public","create_time":"$D2019-08-01T21:59:24.171Z","update_time":"$D2019-08-01T21:59:24.171Z","view_time":null,"first_public_time":"$D2019-08-01T22:03:23.294Z","has_unpublished_edits":null,"fork":{"id":"be123758dafbaead","slug":"analyzing-the-design-space-for-visualizing-neural-attenti","title":"Analyzing the Design Space for Visualizing Neural Attention in Text Classification","owner":{"id":"5d66a17ef16118c8","login":"clpuc","name":"PUC Chile"}},"stars":2,"starred":false,"owner":{"type":"team","id":"f0b7dd8a6790ef7f","login":"hernan4444","name":"Hernan VALDIVIESO LOPEZ","avatar_url":"https://avatars-next.observableusercontent.com/aec9960f7351b8b733a359a5b17daa65117ca13efe33aed1130913e5fc6b7685","tier":"starter_2024"},"version":2605,"publish_version":2605,"role":null,"authored":false,"authors":[{"type":"individual","id":"baf872010e930bf9","login":"hernan4444","name":"Hernan VALDIVIESO LOPEZ","avatar_url":"https://avatars-next.observableusercontent.com/aec9960f7351b8b733a359a5b17daa65117ca13efe33aed1130913e5fc6b7685","tier":null}],"members":[{"type":"individual","id":"baf872010e930bf9","login":"hernan4444","name":"Hernan VALDIVIESO LOPEZ","avatar_url":"https://avatars-next.observableusercontent.com/aec9960f7351b8b733a359a5b17daa65117ca13efe33aed1130913e5fc6b7685","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":0,"collection":null,"type":"notebook","body":{"titling":"auto","title":"Analyzing the Design Space for Visualizing Neural Attention in Text Classification","cells":[{"id":497,"value":"$31","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2317,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2320,"value":"md`\n# Previous works visualizing attention\n\nIn the past, several visualizations have been used either to explain a model of neural networks using an attention mechanism, or to show what the network is paying attention to for a particular data instance (image, document, etc.) Often, the authors of these visualizations make several design choices and make assumptions on how people will perceive them, without much support from design principles of information visualization. In this section we identify the most common idioms used to depict attention, providing an example for each one.\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1964,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1974,"value":"md`\n**Interactive node-link**. Another interesting example is the one by Figure 10 from Olah and Carter (2016), presenting attention in an interactive way, within a translation task. Here the result of the interaction and the colors chosen allows to inspect an example rather than visualizing the full model. This implementation allows, for instance, to let the user inspect that when the article \"la\" in French was translated from English, the model was paying attention to both words \"the\" and \"Area\" in the original sentence.\n\n

\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1957,"value":"md`\n**Heatmap**. In the same work by Bahdanau et al. (2014) on neural machine translation, Figure 3 shows a heatmap idiom used to represent attention. The following image is taken from the original article, where the input text is presented in the x-axis (words rotated), while the output text, translation to French, is presented in the y-axis. The whiter the cell, the higher the attention paid; the darker the cell is the opposite. In the first plot we see that when the French word *économique* was generated, the model was paying attention to the English words *Economic* as well as *European*.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1940,"value":"md`\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1999,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2004,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2236,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1943,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2421,"value":"md`\n\nIf the task of identifying the features (words) requires a more precise alignment between channel and the actual measure of attention, we need to test alternative encodings. The framework by Munzner also identifies, based on the work by Barthes (1969), the accuracy of the different channels which can serve us as a reference to explore the space of design for visualizing attention. \n\n\n\n\nFor instance, using position on common or unaligned scales, length, tilt area or depth could provide a more precise representation than color luminance of saturation to represent attention. \n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2440,"value":"md`\n## Proposal of Marks and Channels for Visualizing Attention Directly on Text\n\nThe first task we address is idetifying the words that the model paid more attention to, directly on the text. When visualizing attention, we need to consider the data we are representing. In our case, attention of a neural model for text classification is a probability distribution over words, i.e., for each word we have a positive numerical value in the range ${tex` \n[0..1]`}, and attentions ${tex` \\alpha_w`} to all words in a document must sum up to 1, i.e., ${tex` \\sum_{w \\in Doc} \\alpha_w=1`}. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2469,"value":"md `\nThere is a similarity of our task with visualizing word clouds: text to visualize having weights assigned to words. Then, for defining our design space, a good starting point is the work by [(Felix et al. 2017)](https://ieeexplore.ieee.org/abstract/document/8017641), who studied different ways of visually encoding word clouds. We then propose five different encodings, cosidering: a) font properties (size, luminance), b) additional marks (bars, circles), and the traditional background color for each word. Our goal is to propose these options, and implement them in the current manuscript to eventually conduct a user study and validate them. We discuss the different channels in the remainder of this section.\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1562,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2147,"value":"md`\n## Explore design space in one document\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2500,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2191,"value":"html`\n

${attention_documents.random_sample[20].title}

Prediction: ${attention_documents.random_sample[20].prediction_label?\"Relevant (R)\": \"Not Relevant (NR)\"}

Ground truth: ${attention_documents.random_sample[20].label?\"Relevant (R)\": \"Not Relevant (NR)\"}

Classification ${attention_documents.random_sample[20].classification}

Document: ${attention_documents.random_sample[documentSelector.document].title}

Prediction: ${attention_documents.random_sample[documentSelector.document].predicted_label?\"Relevant (R)\": \"Not Relevant (NR)\"}

Ground truth: ${attention_documents.random_sample[documentSelector.document].label?\"Relevant (R)\": \"Not Relevant (NR)\"}

Classification ${attention_documents.random_sample[documentSelector.document].classification}

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1386,"value":"documentAttention = {\n return buildAttentionIdioms(documentSelector.document, attention_documents)\n }","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2181,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2187,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2200,"value":"md`# An idiom for comparing attention between documents and words\n\nThe previously introduced idiom allows to compare attention within a particular document. However, for the task of comparing between documents as well, a new form of encoding might be needed. In the following idiom, based on a traditional heatmap, we show in the x-axis several documents and on the y-axis words that the documents were paying attention to. This is a traditional way to present attention, where the luminance on the cell grids indicates the level of attention (the darker the cell, the higher the attention). \nOne innovative aspect of this idiom is presenting some words several times in order to show that the same word receives different levels of attention depending on their location in the document. \n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2523,"value":"md`\n### Document classification \nIn the x-axis, rather than a simple label we introduce an information block with three labels for each document. The top label is the index of the document (in the example below, 00), the middle text is the prediction of the model (in the example, **NR**), and the bottom label is the ground truth of the document (**R** in the example). For simplicity, in this article we do not go deeper into the design of this information box, but one aspect to further analysis could be introducing a glyph to present this same infomation. The glyph could allow us to present the model's uncertainty about the prediction, for instance. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2526,"value":"md`\n![image](https://user-images.githubusercontent.com/15641721/62243245-e3d96c00-b3aa-11e9-8e3e-34bf65f73e44.png)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2206,"value":"md` ### Idiom showing attention in context \n\nRegarding the words shown more than once in the y-axis, the format used is \"word-occurrence\", i.e., \"controlled-1\" indicates the first time the word \"controlled\" is shown, and \"controlled-2\" indicates the second time that the word is presented. With this encoding, these documents and words we show the same word have different attention and the idioms can't explain it. This example demostrate the level of attention will be dependent on the context, i.e., the words around (before and after) the word analyzed\n\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2216,"value":"md`\n\n### Incorporating interactivity for contextual inspection\n\nWe also integrate interactivity in order to enhance the presentation of attention in context. With the mouse we can hover or click on a cell to show the attention received by the word compared to its surrounding context, i.e., the five words around (before and after in the text) the word clicked or hovered. This information is presented by displaying an additional bar plot per each cell hovered or clicked upon, with up to two bar plots.\n\n![image](https://user-images.githubusercontent.com/15641721/62247203-56e6e080-b3b3-11e9-93d0-eed999cb650a.png)\n\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2561,"value":"md `\n## A playground for comparing attention between documents\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2599,"value":"md `\nYou can play with the next interactive visualization . Notice how the model, in general, pays special attention to words like *included* and *studies* in order to classify documents as relevant. While this is also a feature that humans take into account, it could also be a bias in the dataset, making the model easy to fool when finding these words in non relevant documents.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2219,"value":"\n\nchart4 = {\n return buildVisualization(same_words, true)\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2487,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1590,"value":"$3d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2596,"value":"$3e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":245,"value":"md`# Internal code to make visualizations`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1543,"value":"colors = ({\n \"bars\": \"#174487\",\n \"clicked\": \"#b1f0af\",\n \"hover\": \"#a2c6fc\",\n \"text\": \"#7a7a7a\",\n \"focusText\": \"#000000\",\n \"relevant\": \"#9eff61\",\n \"not_relevant\": \"#ff6161\",\n \"lowAttention\": \"#fdae61\",\n \"highAttention\": \"#2c7bb6\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":240,"value":"margin = ({ top: 20, right: 20, bottom:20, left: 20 })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":313,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":718,"value":"buildBar = (result, documents, gridSize, d, svg, version) => {\n let documentDict = {\n index: d.column,\n title: result.document.title\n }\n let max = Math.min(result.index + 6, result.text.length);\n let min = Math.max(result.index - 5, 0);\n let words = result.text.slice(min, max);\n let attention = result.document.attention.slice(min, max);\n let heatmapWidth = documents[documents.length - 1] * (gridSize + 4) + 90;\n let mainWord = result.text[result.index];\n _buidBar(words, attention, svg, heatmapWidth, version, mainWord, documentDict);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":330,"value":"$40","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2160,"value":"$41","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":98,"value":"style = html`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":750,"value":"findIndex = (dataset, d) => {\n let document = dataset.random_sample[d.column];\n let text = document.text.split(\" \");\n let word = d.word.split(\"-\")[0];\n let counter = d.word.split(\"-\")[1];\n let findWord = 0;\n let index = 0;\n while (findWord != counter && index >= 0) {\n index = text.indexOf(word, index == 0 ? 0 : index + 1);\n findWord++;\n }\n return { index, text, document };\n};","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2144,"value":"get_text = (d, i) => {\n let prediction = d.predicted_label?\"R\":\"NR\";\n let ground = d.label?\"R\": \"NR\";\n return `${i<10?\"0\"+i:i}: (${prediction} - ${ground}) ${d.title}`\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":75,"value":"d3 = require(\"d3@5\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1349,"value":"textwrap = require('d3-textwrap');","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":80,"value":"chroma = require('chroma-js')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":228,"value":"import {jsonInput} from \"@mbostock/file-input\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1530,"value":"import {form} from \"@mbostock/form-input\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":49,"value":"same_words","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2288,"value":"attention_documents","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":31,"value":"\nviewof same_words = jsonInput({\n initialUrl: \"https://gist.githubusercontent.com/Hernan4444/c737c7aef0969f9f29640a10836a39d5/raw/02e5fab9429e439a198321a90853b1cb9c0ce509/same_words.json\"\n});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2287,"value":"viewof attention_documents = jsonInput({\n initialUrl: \"https://gist.githubusercontent.com/Hernan4444/c737c7aef0969f9f29640a10836a39d5/raw/02e5fab9429e439a198321a90853b1cb9c0ce509/attention_documents.json\"\n});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W42","stdlib":"1"}} 30:"$2f:body"