34:["$","title",null,{"children":"An interactive, Javascript-based browser for microbial pangenome graphs | cofest-pangenome | Observable"}] 3c:T583,md`## Abstract Variant discovery is largely conducted by comparing sample genomes to a single canonical reference genome under the assumption that the sample of interest differs only slightly from its reference. However, many pathogens diversify their genomes so quickly that the sample and reference may differ greatly. When these differences are sufficiently large, existing algorithms fail to adequately align sample sequence data to the reference, and variation underlying virulence factors (e.g. drug resistance, immune evasion) may go undetected. These applications may be better served using a "pangenome graph" - a data structure encoding many reference sequences, thus representing much greater haplotypic variation than a single linear genome. However, departing from the linear genome model poses significant visualization challenges, particularly when one wishes to visualize the reads that originally gave rise to the underlying graph. Here, we explore methods to address two major - and intimately related - visualization problems: 1. How do we display a region (or "subgraph") of the pangenome graph? 2. How do we display the reads that were used to construct the subgraph? The prototypes developed here will hopefully provide useful guidance for the eventual development of a new graph-based genome browser. Sample data is located here: https://github.com/broadinstitute/PangenomeVis `3d:T66e,md`## Introduction Genomic data visualization is a critical step in the analysis of sequencing datasets and the development of new algorithms. As the amount of genomic data available to be displayed often far exceeds our ability to digest and interpret at once, effective visualization requires design decisions that limit the amount of data shown at once. Interactivity is a particularly powerful feature for these visualizations, providing a researcher to display, hide, or reorganize relevant information based on current analysis context. Dozens of excellent genome browser incarnations exist (see [Wang et al. 2012](https://academic.oup.com/bib/article/14/2/131/208726 "A brief introduction to web-based genome browsers ") for a detailed albeit slightly out-of-date listing). Among the most widely-used visualization tool is the Java-based and client-side Integrative Genomics Viewer, or IGV ([Robinson et al. 2011](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3346182/ "Integrative Genomics Viewer ")). Adopting a "Google Maps for Genomics" philosophy, IGV dynamically loads only those slices of large genomic file formats spanning a currently selected locus. A recent Javascript rewrite of IGV, termed IGV.js ([Robinson et al. 2020](https://www.biorxiv.org/content/10.1101/2020.05.03.075499v1 "igv.js: an embeddable JavaScript implementation of the Integrative Genomics Viewer (IGV)")), allows an embeddable IGV widget to be inserted into websites, providing flexible and powerful visualization capabilities for new datasets without requiring the client-side installation of software. An example visualization is provided below. `3e:T1211,{ window.generateIgv = params => { console.log('testing'); const div = document.getElementById("igv-div"); return igv.createBrowser(div, { ...defaultOptions, reference: params.reference, tracks: params.tracks }); }; window.trackList = { del_alt1: { tracks: [ { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_A.alt1_imperfect.bam", name: "sample_A.alt1_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_B.alt1_imperfect.bam", name: "sample_B.alt1_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_C.alt1_imperfect.bam", name: "sample_C.alt1_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_D.alt1_imperfect.bam", name: "sample_D.alt1_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_E.alt1_imperfect.bam", name: "sample_E.alt1_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_F.alt1_imperfect.bam", name: "sample_F.alt1_imperfect.bam" } ], reference: { fastaURL: "gs://broad-dsde-methods-pangenome-vis/variants/del/haps/alt1_hap.fasta", indexURL: "gs://broad-dsde-methods-pangenome-vis/variants/del/haps/alt1_hap.fasta.fai" } }, del_alt2: { tracks: [ { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_A.alt2_imperfect.bam", name: "sample_A.alt2_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_B.alt2_imperfect.bam", name: "sample_B.alt2_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_C.alt2_imperfect.bam", name: "sample_C.alt2_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_D.alt2_imperfect.bam", name: "sample_D.alt2_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_E.alt2_imperfect.bam", name: "sample_E.alt2_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_F.alt2_imperfect.bam", name: "sample_F.alt2_imperfect.bam" } ], reference: { fastaURL: "gs://broad-dsde-methods-pangenome-vis/variants/del/haps/alt2_hap.fasta", indexURL: "gs://broad-dsde-methods-pangenome-vis/variants/del/haps/alt2_hap.fasta.fai" } }, ref: { tracks: [ { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_A.ref_imperfect.bam", name: "sample_A.ref_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_B.ref_imperfect.bam", name: "sample_B.ref_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_C.ref_imperfect.bam", name: "sample_C.ref_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_D.ref_imperfect.bam", name: "sample_D.ref_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_E.ref_imperfect.bam", name: "sample_E.ref_imperfect.bam" }, { url: "gs://broad-dsde-methods-pangenome-vis/variants/del/reads/sample_F.ref_imperfect.bam", name: "sample_F.ref_imperfect.bam" } ], reference: { fastaURL: "gs://broad-dsde-methods-pangenome-vis/variants/del/haps/ref_hap.fasta", indexURL: "gs://broad-dsde-methods-pangenome-vis/variants/del/haps/ref_hap.fasta.fai" } } }; return html` `; }3f:T8b1,md`## Linear and non-linear layouts Nearly all browsers that visualize genomic reads rely on a linear genome model for organizing an effective display. While a linear layout is not without its drawbacks ([Nielsen and Wong, 2012](https://www.nature.com/articles/nmeth.1992)), it succeeds at genomic data visualization by being highly intuitive. Distance and relationships between different parts of the dataset can be quickly ascertained ([even if the true conformation of the genome is much more complicated](https://www.nature.com/collections/rsxlmsyslk "Collection: The 3D genome")). Horizontal display of a genomic region and vertical display of data aligned to that region make efficient use of available screen real-estate. ![A k-mer graph of several dozen HLA-A (red), HLA-B (yellow), and HLA-C (green) sequences](https://www.statnews.com/wp-content/uploads/2016/10/rainbow-1600x900.jpg) The display of a pangenome graph - a collection of multiple reference genomes and one or more genomic datasets - complicates matters considerably. Consider the graph above, made using [GraphViz](https://graphviz.org/) by decomposing several dozen HLA class A/B/C alleles into k-mers and plotting their relationships to one another along the length of the sequences ([Zimmer 2016, figure by Garimella, McVean, Paten](https://www.statnews.com/2016/10/07/dna-genome-sequencing-new-maps/)). Lines of the same color with increased thickness indicate shared sequence between multiple alleles, and lines of different colors that merge indicate shared sequence across classes. Thus, the graph qualitatively depicts low and high-frequency alleles along with some shared evolutionary history between the sequences. However, beyond the merged lines, the vertical dimension of the plot encodes no additional information; the graph plotting algorithm used here simply attempts to maximize certain *aesthetic* values (e.g. keeping lines horizontal and as non-overlapping as possible), rather than encoding some value relevant to the underlying data ([Gansner et al. 1993](https://ieeexplore.ieee.org/document/221135)). The display is also static; GraphViz does not permit nodes to be repositioned manually or inspected interactively.`40:T409,md` ![Bandage UI](https://rrwick.github.io/Bandage/images/wiki/bandage_gui.png) For a more interactive take on non-linear graph layout, [Bandage](https://rrwick.github.io/Bandage/) is a popular client-side software package for visualizing many graph formats. Also employing a non-linear layout, Bandage displays graph vertices as ribbons with length proportional to their sequence length. Vertices can be manipulated in various ways: highlighted, dragged and moved about the screen, used to initiate a BLAST search, etc. While Bandage is a general viewer of graphs in GFA format, the functionality provided is largely geared towards the inspection of *de novo* whole genome assemblies, not on inspecting variant calls and their underlying evidence. Most often this is for a single genome at a time or a metagenome, but not usually a *pan*genome. Bandage also typically doesn't display the sequence of the nodes themselves (though there are ways to rewrite the GFA in ways that trick Bandage into displaying this information). `41:T562,md`### Construction of an LdBG A simple de Bruijn graph (dBG) is (at least in theory) straightforward to build. An example is illustrated in the figure below. To build a dBG, we need simply to extract fixed-length subsequences of length k ("k-mer") with a stride of 1 from input reads and record each k-mer and its immediately adjacent k-mers in a graph data structure (a list of nodes and edges). Sequencing errors can be optionally removed from the graph by *correction* (i.e. replacing an erroneous k-mer with its more likely counterpart) or *cleaning* (i.e. removing an erroneous k-mer from the graph entirely). LdBGs extend dBGs in two important ways: 1. They add "stackability": by keeping k fixed across multiple datasets, the resulting graphs can combined using a full-join on common k-mers. 2. They add "connectivity": decomposition of reads into k-mers only retains k+2 bases of connectivity (only a k-mer's immediate predecessors and successors are recorded, and the fact that two k-mers in the same read but much further apart is lost). To restore this connectivity information, reads can first be used to construct the graph and then realigned after graph construction to provide an auxillary set of graph annotations that provide additional navigation information ([Turner et al. 2018](https://academic.oup.com/bioinformatics/article/34/15/2556/4938484)). `42:Tb77,md`Manual inspection of these sequences reveals two differences: a T/G mutation (position 6) and a A/G mutation (position 19). Assuming a k-mer size of 5, graph records (k-mer, coverage, and edge tuples) for these sequences can be represented as follows:

AAATG 1 0 0 .c...C.. ........ ........
AACGC 1 1 1 ...tA... ...tA... ...tA...
AATGC 1 0 0 a....C.. ........ ........
ACGCA 1 1 1 a...A... a.....G. a.....G.
ACTAA 1 1 1 ..g..C.. ..g..C.. ..g..C..
AGATG 0 1 1 ........ .c...C.. .c...C..
AGCCG 0 1 0 ........ ...tA... ........
AGTCG 1 1 1 ...t..G. ...t..G. ...t..G.
ATAGC 0 1 0 ........ ..g..C.. ........
ATATC 2 0 2 ..g.AC.. ........ ..g.AC..
ATCAA 1 1 1 ...t.... ...t.... ...t....
ATCCG 1 0 1 ...tA... ........ ...tA...
ATCTG 0 1 1 ........ .c...C.. .c...C..
ATGCC 1 1 1 a...A... ..g.A... ..g.A...
ATTTG 1 0 0 .c...C.. ........ ........
CCGAC 1 1 1 ...t...T ..g....T ...t...T
CGCAA 1 0 0 a...A... ........ ........
CGCAG 0 1 1 ........ a...A... a...A...
CGTTA 1 1 1 ..g...G. ..g...G. ..g...G.
CTAAC 1 1 1 a.....G. a.....G. a.....G.
CTATC 0 1 0 ........ ..g.A... ........
GACTA 1 1 1 .c..A... .c..A... .c..A...
GATGC 0 1 1 ........ a....C.. a....C..
GCAAA 1 0 0 .c.....T ........ ........
GCAGA 0 1 1 ........ .c.....T .c.....T
GCCGA 0 1 0 ........ a....C.. ........
GGATA 1 0 1 .c.....T ........ .c.....T
GGCTA 0 1 0 ........ .c.....T ........
TATCA 1 1 1 a...A... .c..A... a...A...
TCCGA 1 0 1 a....C.. ........ a....C..
TGCCA 1 1 1 a....... a....... a.......

Here, each row is a k-mer from the union of k-mers across all three samples (to reduce storage costs, we store the lexicographically lowest version of a k-mer rather than both the forward and reverse-complement k-mers). The following three columns represent the frequency of this k-mer from each sample's reads. A coverage of 0 indicates that the k-mer is absent from that sample. The final three columns represent incoming edge (first four positions) and outgoing edge (last four positions) information. Four incoming edges (a, c, g, t) and four possible outgoing edges (A, C, G, T) are possible. If a possible edge is absent in the reads, we write '.' to indicate this absence. Thus, one can see how the samples are stacked column-wise. The union of k-mers is considered across all samples, and the coverage/edge metadata tells us whether or not each sample actually contains this k-mer and the other k-mers to which it is connected. This process can be applied to both reference sequence(s) and sequenced reads for a sample (after a suitable process for removing sequencing errors is applied). We can see the k-mers that flank the left variant T/G variant here (TATCA and CCGAC) have slightly different edge information across the three samples, indicating the potential presence of a variant. But looking at this raw graph record information to discern the presence of a variant is rather painful. Let's see if we can improve upon this. `43:T498,chart = { const links = data.links.map(d => Object.create(d)); const nodes = data.nodes.map(d => Object.create(d)); const simulation = d3 .forceSimulation(nodes) .force("link", d3.forceLink(links).id(d => d.id)) .force("charge", d3.forceManyBody()) .force("center", d3.forceCenter(width / 2, height / 2)); const svg = d3.create("svg").attr("viewBox", [0, 0, width, height]); const link = svg .append("g") .attr("stroke-opacity", 0.6) .attr("stroke-width", 3) .selectAll("line") .data(links) .join("line") .attr("stroke", d => d.value); const node = svg .append("g") .attr("stroke", "#fff") .attr("stroke-width", 1.5) .selectAll("circle") .data(nodes) .join("circle") .attr("r", 5) .attr("fill", color) .call(drag(simulation)); node.append("title").text(d => d.id); simulation.on("tick", () => { link .attr("x1", d => d.source.x) .attr("y1", d => d.source.y) .attr("x2", d => d.target.x) .attr("y2", d => d.target.y); node.attr("cx", d => d.x).attr("cy", d => d.y); }); invalidation.then(() => simulation.stop()); return svg.node(); }44:T85e,/** displayGraph: display the haplotypes provided using a force-directed graph layout * @param {Array[String]} haplotypes: an array of haplotype sequences to display * @param {Number} k: the k-mer size to use for the graph */ function displayGraph(haplotypes, k = 5) { var colors = ["#DF5A49", "#334D5C", "#E27A3F", "#45B29D", "#EFC94C"]; var data = { nodes: toNodes(haplotypes, k), links: toLinks(haplotypes, colors, k) }; const links = data.links.map(d => Object.create(d)); const nodes = data.nodes.map(d => Object.create(d)); var drag = simulation => { function dragstarted(d) { if (!d3.event.active) simulation.alphaTarget(0.2).restart(); d.fx = d.x; d.fy = d.y; } function dragged(d) { d.fx = d3.event.x; d.fy = d3.event.y; } function dragended(d) { if (!d3.event.active) simulation.alphaTarget(0); d.fx = null; d.fy = null; } return d3 .drag() .on("start", dragstarted) .on("drag", dragged) .on("end", dragended); }; var height = 600; const simulation = d3 .forceSimulation(nodes) .force("link", d3.forceLink(links).id(d => d.id)) .force("charge", d3.forceManyBody()) .force("center", d3.forceCenter(width / 2, height / 2)); const svg = d3.create("svg").attr("viewBox", [0, 0, width, height]); const link = svg .append("g") .attr("stroke-opacity", 0.6) .attr("stroke-width", 3) .selectAll("line") .data(links) .join("line") .attr("stroke", d => d.value); const node = svg .append("g") .attr("stroke", "#fff") .attr("stroke-width", 1.5) .selectAll("circle") .data(nodes) .join("circle") .attr("r", 5) .attr("fill", color) .call(drag(simulation)); node.append("title").text(d => d.id); simulation.on("tick", () => { link .attr("x1", d => d.source.x) .attr("y1", d => d.source.y) .attr("x2", d => d.target.x) .attr("y2", d => d.target.y); node.attr("cx", d => d.x).attr("cy", d => d.y); }); invalidation.then(() => simulation.stop()); return svg.node(); }45:T463,md`### Limitations of this display There is value in this display - each individual k-mer is inspectable, variation can be observed fairly easily, and different sample identities can be discerned. However, there is much to be desired in this visualization: 1. Many samples share k-mers, but because the edges for each samples are drawn as straight lines from one node to the next, the edges all overlap and make it difficult to distinguish the individual samples. 2. The major/minor alleles in each "bubble" must be determined manually by inspecting the text of the k-mers, adding quite a bit of manual work to seeing variation in the graph. 3. We have no way of hiding/revealing individual samples should we only be interested in examining a subset of the available samples. 4. A lot of layout space is wasted in displaying k-mers rather than unitigs. 5. Some peculiar behavior with the layout (e.g. it starts off too large for the display area; clicking and holding a node repels the other nodes to the point where the display becomes empty). 6. *It is not obvious how to display reads on top of this visualization.* `46:T779,md`## TO DO The following to-dos are roughly ordered by things we think are relatively straightforward to things that are very challenging. * Replace the color array with ColorBrewer palettes with a size computed from the haplotypes array length. [minimal javascript] * Replace linear edges in graph with splines or arcs (with different radii) so that they no longer occlude one another. * Improve layout behavior such that clicking and holding a node no longer repels all the other nodes off-screen. * Provide a side panel of actions (e.g. turn on/off samples). [This item depends on other action items] * Allow the user to zoom in/out of the graph using the mouse wheel or trackpad swipe (see [d3-zoom](https://github.com/d3/d3-zoom)). * Allow graph colors to be chosen manually. * Better tool-tip information (including k-mer coverage). * Write a Javascript-based read simulator (accept read length, paired/unpaired, insert-size, error rate as parameters) [no visualization needed] * Accept a graph as kmers, but display them as unitigs by default and allow for the option for a unitig to be displayed as constituent k-mers on-demand. * Write converter for multi-color dBG text format to GFA2. [no javascript required] * Support GFA2. * Save a graph to PNG/PDF/SVG. * Highlight potential variants. * Path highlighting - select a source and destination node and highlight the per-color paths between the two selected nodes. Write out the implied haplotypes on a side panel. * Display McCortex link annotations on graph. * Provide ability to designate one of the samples to act as the reference (a "pseudo-reference"), such that the layout becomes linear and all of the other samples line up underneath it. [This item depends on other action items] * Display reads on the graph when the graph is linearized (i.e. when a sample is chosen to be the pseudo-reference). [This item depends on other action items] `47:[{"id":"0ceed4936afdb7ab40fa413255c53a43c7081321f1fb6cd9214edca3cde84b27386eaba2e99282d9ad0c954f7f5466086eba4595e71ac5ec4924abbb57dd9393","href":"https://static.observableusercontent.com/files/0ceed4936afdb7ab40fa413255c53a43c7081321f1fb6cd9214edca3cde84b27386eaba2e99282d9ad0c954f7f5466086eba4595e71ac5ec4924abbb57dd9393","name":"cycle-links.png","type":"image/png","lastModified":1595370546619.934,"size":361331},{"id":"7f5a3c8c5f61f2c36250a51ff4db624073194b8fc754a7ebec57b9311cb0f97f2b6d06daed1bee412752160c3d5d673e43f38e04a24df91e07f453b1bfc4136b","href":"https://static.observableusercontent.com/files/7f5a3c8c5f61f2c36250a51ff4db624073194b8fc754a7ebec57b9311cb0f97f2b6d06daed1bee412752160c3d5d673e43f38e04a24df91e07f453b1bfc4136b","name":"sim-screenshot.png","type":"image/png","lastModified":1595417678457.756,"size":1039196}] 38:{"id":"0c0d7484c67a27d1","slug":"an-interactive-javascript-based-browser-for-microbial-pan","title":"An interactive, Javascript-based browser for microbial pangenome graphs","thumbnail":"c2bad83063dcff378efa89b06cea1a287f74470ad731f12937f9c0a6f1f70610","access_level":"public","create_time":"$D2020-07-21T15:34:16.386Z","update_time":"$D2020-07-23T19:41:32.701Z","view_time":null,"first_public_time":"$D2020-07-22T14:49:50.376Z","has_unpublished_edits":null,"fork":true,"stars":3,"starred":false,"owner":{"type":"team","id":"0c319674e9a5596e","login":"cofest-pangenome","name":"","avatar_url":null,"tier":"starter_2024","bio":"","followed_by_you":false,"follows_you":false},"version":704,"publish_version":704,"role":null,"authored":false,"authors":[{"type":"individual","id":"9a27763f42f55fe1","login":"kvg","name":"Kiran V Garimella","avatar_url":"https://avatars-next.observableusercontent.com/7b41097914e32c952999d5b4dfa5ea3d88ca247413c17eca119be5d21ebc5cb5","tier":null,"bio":"Computational scientist working at the Broad Institute on long read methods development.","followed_by_you":false,"follows_you":false}],"members":[{"type":"team","id":"0c319674e9a5596e","login":"cofest-pangenome","name":"","avatar_url":null,"tier":"starter_2024","role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":15,"collection":{"id":"6799d6f1019a3d72","slug":"pangenome-prototypes","title":"Pangenome Prototypes","owner":{"id":"0c319674e9a5596e","login":"cofest-pangenome","name":""}},"type":"notebook","body":{"titling":"auto","title":"An interactive, Javascript-based browser for microbial pangenome graphs","cells":[{"id":0,"value":"md`# An interactive, Javascript-based browser for microbial pangenome graphs`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":79,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"$3d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":85,"value":"html`

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":372,"value":"igv = require('https://rawgit.com/turner/igv-observable/master/igv.js')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":36,"value":"div = document.getElementById(\"igv-div\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":49,"value":"defaultOptions = ({\n showRuler: true,\n fileLoadWidget: {\n hidden: false,\n embed: false\n }\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":692,"value":"function generateIgvInstance(div, params) {\n return igv.createBrowser(div, {\n ...defaultOptions,\n reference: params.reference,\n tracks: params.tracks\n });\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":694,"value":"function generateIgv(params) {\n console.log('here', params);\n return generateIgvInstance(document.getElementById(\"igv-div\"), params);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":698,"value":"$3e","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":94,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":581,"value":"$40","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":586,"value":"md`\nEffective display requires consideration of the ultimate aims of the analysis. If the goal is to simply detect the relationship between the sequences in the graph, then the illustration above serves our purpose. However, our goal lies beyond this horizon.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":327,"value":"md`## A de Bruijn-based pangenome graph\n\nOur ultimate goal is to use a pangenome graph to help identify variants that would be otherwise difficult to identify using only a canonical reference sequence. We wish not only to see the structure of the graph, but to be able to see the underlying evidence (the reads) that were used to construct the graph as well. We now discuss our underlying model for the representation of such data and explore the implications of this model for visualization.\n\nFor microbial datasets (typically less than a 25 Mbp genome) sequenced with short reads (~150 bp, paired end, high coverage, low error rate), a \"multi-color linked de Bruijn graph\" (or more simply \"LdBG\") is a convenient starting place for representing genomic data in a manner suitable for variant calling ([Iqbal et al. 2012](https://www.nature.com/articles/ng.1028), [Garimella et al. 2019](https://www.biorxiv.org/content/10.1101/698910v1)).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":276,"value":"$41","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":483,"value":"md`\n

${Object.assign(await FileAttachment(\"cycle-links.png\").image(), {\n alt: 'Graph connectivity',\n style: 'width: 100%; max-width: 1200px;'\n})}\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":491,"value":"md`### A toy example\n\nLet's look at a toy example with three samples and two simple variants. For simplicity, we'll omit the connectivity information for now.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":135,"value":"seq1 = \"TTGATATCCGACTAACGCAAATGCCA\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":166,"value":"seq2 = \"TTGATAGCCGACTAACGCAGATGCCA\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":262,"value":"seq3 = \"TTGATATCCGACTAACGCAGATGCCA\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":296,"value":"$42","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":330,"value":"md`### Visualizing the graph\n\nTo display the graph, let's construct a force-directed layout using the d3.js javascript visualization library below. We'll also assign each sample a characteristic color (sample 0 -> red, 1 -> green, 2 -> blue) so that we can see which samples contain which variants.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":496,"value":"md`First, we'll need a few utility methods to turn our three sequences into a graph and extract the union of nodes, etc.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":177,"value":"function toGraph(seq, color, k = 5) {\n var graph = [];\n var i;\n for (i = 0; i <= seq.length - k - 1; i++) {\n var source = seq.substring(i, i + k);\n var target = seq.substring(i + 1, i + k + 1);\n graph.push({ source: source, target: target, value: color });\n }\n\n return graph;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":203,"value":"function toNodes(seqs, k = 5) {\n var s = new Set();\n var i, j;\n for (j = 0; j < seqs.length; j++) {\n for (i = 0; i <= seqs[j].length - k; i++) {\n var source = seqs[j].substring(i, i + k);\n s.add(source);\n }\n }\n\n var nodes = [];\n for (var it = s.values(), val = null; (val = it.next().value); ) {\n nodes.push({ id: val, group: 1 });\n }\n\n return nodes;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":554,"value":"function toLinks(haps, colors, k = 5) {\n var links = toGraph(haps[0], colors[0], k);\n var i = 1;\n for (i = 1; i < haps.length; i++) {\n links = links.concat(toGraph(haps[i], colors[i], k));\n }\n\n return links;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":511,"value":"md`Now we can apply the force-directed graph layout in the excellent d3.js package (see [d3-force](https://github.com/d3/d3-force) for API reference).`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":146,"value":"d3 = require(\"d3@5\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":216,"value":"data = ({\n nodes: toNodes([seq1, seq2, seq3]),\n links: toLinks([seq1, seq2, seq3], [\"#ff0000\", \"#00ff00\", \"#0000ff\"], 5)\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":150,"value":"color = {\n const scale = d3.scaleOrdinal(d3.schemeCategory10);\n return d => scale(d.group);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":152,"value":"height = 600","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":148,"value":"drag = simulation => {\n function dragstarted(d) {\n if (!d3.event.active) simulation.alphaTarget(0.8).restart();\n d.fx = d.x;\n d.fy = d.y;\n }\n\n function dragged(d) {\n d.fx = d3.event.x;\n d.fy = d3.event.y;\n }\n\n function dragended(d) {\n if (!d3.event.active) simulation.alphaTarget(0);\n d.fx = null;\n d.fy = null;\n }\n\n return d3\n .drag()\n .on(\"start\", dragstarted)\n .on(\"drag\", dragged)\n .on(\"end\", dragended);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":144,"value":"$43","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":455,"value":"md`The two variants in our toy dataset are now clearly visible as \"bubbles\", as is their proximity to one another. Clicking and dragging nodes allows them to be repositioned for a more aesthetically pleasing layout. Hovering over a node provides a tool-tip with the sequence of the node's k-mer.\n`\n\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":529,"value":"md`### A second example: dynamically simulated sequences\n\nLet's look at another example. This time, we'll define some simulation functions to generate sequences containing variants for us.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":402,"value":"function generateHaplotype(length) {\n var result = '';\n var characters = 'ACGT';\n var charactersLength = characters.length;\n\n for (var i = 0; i < length; i++) {\n result += characters.charAt(Math.floor(Math.random() * charactersLength));\n }\n return result;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":430,"value":"function shuffle(a) {\n var j, x, i;\n for (i = a.length - 1; i > 0; i--) {\n j = Math.floor(Math.random() * (i + 1));\n x = a[i];\n a[i] = a[j];\n a[j] = x;\n }\n return a;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":400,"value":"/** simulateHaplotypes: simulate haplotype sequences, incorporating specified variants\n * @param {String} ref: a given reference\n * @param {String} alt: an alternate allele\n * @param {Number} ac: alternate allele count\n * @param {Number} n: and total number of samples\n */\n\nfunction simulateHaplotypes(ref, alt, flank = 50, ac = 1, n = 2) {\n var refLeft = generateHaplotype(flank);\n var refRight = generateHaplotype(flank);\n\n var refHaplotype = refLeft + ref + refRight;\n var altHaplotype = refLeft + alt + refRight;\n\n var haplotypes = [];\n var i = 0;\n for (i = 0; i < n - ac; i++) {\n haplotypes.push(refHaplotype);\n }\n for (i = 0; i < ac; i++) {\n haplotypes.push(altHaplotype);\n }\n\n return shuffle(haplotypes);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":409,"value":"haps = {\n // play with these settings to generate new sequences for variant 1\n var haps1 = simulateHaplotypes('A', 'C', 10, 2, 4);\n\n // play with these settings to generate new sequences for variant 2\n var haps2 = simulateHaplotypes('G', 'T', 10, 1, 4);\n\n // now, concatenate the simulated haplotypes for each sample\n var haps = [];\n var i = 0;\n for (i = 0; i < haps1.length; i++) {\n haps.push(haps1[i] + haps2[i]);\n }\n return haps;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":541,"value":"$44","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":550,"value":"displayGraph(haps, 21)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":318,"value":"$45","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":641,"value":"$46","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":658,"value":"md`## Useful resources\n\n1. [Gallery of other D3/Observable projects.](https://observablehq.com/@d3/gallery) Great place to find working code examples!\n2. [Simulated datasets.](https://github.com/broadinstitute/PangenomeVis/) Each is of ~10 kbp haplotypes that have simple variants (SNV, INS, DEL), and a more complicated example with multiple variants within a small window. Reads are simulated without and with errors, and premade graphs in a simple text format are available.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":666,"value":"md`\n

${Object.assign(await FileAttachment(\"sim-screenshot.png\").image(), {\n alt: 'IGV screenshot of simulated data',\n style: 'width: 100%; max-width: 1200px;'\n})}\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":397,"value":"md`## Scratch space\n\nBelow this line, feel free to use this space as a coding playground.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W47","stdlib":"1"}} 33:null