2e:null 2f:["$","title",null,{"children":"Evaluating the validity metric on a different layout | Mathieu Jacomy | Observable"}] 34:T525,md` In the [previous notebook](https://observablehq.com/@jacomyma/a-validity-metric-for-interpreting-distances-in-a-network-m) of this series I have proposed a validity metric for reading network maps using a force-directed placement algorithm. I reflected on just one case. It is now time to take a look at what happens in other contexts: different networks, different layouts. This notebook focuses on layouts, and remains text-heavy. I decided to "think out loud" and keep track of my process. For a more practice exploration of what the metric does in different context, head to the next notebook of this series (forthcoming). The validity metric has three components: * The **characteristic distance**, noted *D_Q* * The **share of unexpected edges**, noted *Q* and expressed as a percentage * The **share of edges shorter than *D_Q* **, noted *P_Q* and expressed as a percentage And the important fact is that *D_Q* is where *Q* is the highest, hence its name.
** *D_Q* is the distance at which the most edges are unexpected** (of that length or shorter). ### Practical illustration The network map below features *D_Q* as a legend element on the bottom-left. To help visualize its meaning, shorter edges are in green and longer ones in red. `35:Ta43,md`The green curve *Q* is basically flat, but its fluctuations still allow to find a maximum. The validity metric technically works, but what does it tell? Re-run the random layout a number of times to get an idea. *D_Q* varies a lot, because the maximum of a roughly flat curve is basically random. The important point here is that the maximal *Q* is close to zero, regardless of the final choice of *D_Q*. In other words, **there is no distance capable of capturing more edges than expected in this layout**. This leads to an important algorithm design point: *D_Q* only has a meaning insofar as the optimal *Q* means something. We can always compute the maximum, but when the curve is flat, it is meaningless. What to do? **Option 1:** write a paper focusing on evaluating force-directed layouts (*Q* always peaks), presenting the metric as if it is a formula (serious + easy to implement), providing a rationale based on an interpretation of that formula, and providing a spectacular illustration based on the most favorable cases. **Option 2:** complicate the metric so that it refuses to provide a result in certain cases. But then it is more than a simple formula, and publishing it requires a significant amount of additional work. I believe that many computer scientists favor option 1, but doing so is refusing to assume the role of algorithm designer. That is understandable because your reward is essentially publication. Implementing a metric in existing tools will be done by completely different people with a very different culture. But the problematic consequence of this system is that it produces *docile algorithms*. Docility means that down the road, nothing prevents users from misusing and misinterpreting the metric; because no one assumed the responsibility of the design, because it is too distributed between the author of the paper, the different implementation, the graphic design, and the users themselves. I aim to fight back against docility. In this case it is clear that for some layouts, *D_Q* is meaningless. I want the metric to resist *by design*, I want it to refuse to give the meaningless figure. I want the metric to tell *"I am meaningless"* and not just *"I am good/bad"* while hiding the meaninglessness of that statement. The discussion about design is too big for this notebook. But I will still draw practical consequences from it. Now that we have an idea of the problem, we can at least try to make things clear, and design the metric accordingly. To that end, let's rewind a bit and think of what makes it a *validity metric*. `36:T16ea,md`## What makes it a validity metric? My initial goal was to get an interpretable and quantized statement about how well the layout is doing. The problem is that it assumes we know *what* it is doing; but we do not, really. However, this is not an issue, as we can just take a stance on that matter. In the context of this metric, we look at a certain aspect of the layout, and we assume it. This aspect is inspired by the internal mechanic of force-directed placement algorithms, but it is not specific to it, it does not depend on it. This aspect is how much it succeeds at putting connected nodes closer. **The metric is based on whether or not the layout succeeds at bringing connected nodes closer.** A number of problems naturally come up with such statement. (1) It is not a *yes or no* question: success is a spectrum. (2) Success is also relative (we know that it is impossible to have *all* edges short). (3) "Closer" is a relative notion, it implies "closer than"; but than *what* exactly?. The metric solves these issues a certain way, depending on how it quantizes the statement. Another point requires attention. The metric does not tell us if the *layout* is valid. It tells us if *a given interpretation of the layout* is valid. It tells us how valid it is to read the network map a certain way. I do not state that the layouts should, do, or are intended to bring connected nodes together. It is a validity metric *for interpreting distances* in the picture. Which comes with its own question: which interpretation exactly are we talking about? As one can see, this issue has two interconnected layers: * **Does the layout bring connected nodes closer?** * ** *If that is the case*, where do we see it in the network map?** ***Q_max* tells us on the success of the layout at bringing connected nodes closer.** The *Q* metric depends on the distance, and one should rigourously note it *Q(D)*. The highest value of *Q* for a given layout is noted *Q_max* and the value of *D* where *Q(D) = Q_max* is by convention noted *D_Q* (because it maximizes *Q*), hence *Q_max = Q(D_Q)*. *Q* overall tells us about the success of the layout at bringing connected nodes closer, but *Q_max* is the best case scenario, the value of *D* where we see the layout succeeding the most (*D_Q*). I use *Q_max* because (1) it is an actual value and not a function, and (2) it relates to the second layer of the issue, the picture. Q(D) = [ CP(D) - ECP(D) ] / CP(∞) where: * *CP(D)* is the count of pairs of nodes closer than distance *D* * *ECP(D)* is the count of expected pairs of nodes closer than *D* * *CP(∞)* is the total count of connected pairs or nodes And ECP(D) = P(D) x CP(∞) / P(∞), where *P(D)* is the count of pairs of nodes closer than *D* and *P(∞)* the total number of pairs of nodes. *Q_max* is the highest number of edges we can find, that are shorter than a certain distance, and that are not already expected. Indeed, it is not surprising that for a given distance, some edges are shorter. If there are node pairs closer, then those have a chance to be connected. This is why we discount these pairs from *Q*, and only look at the *unexpected* edges, the count of edges above expectations. *Q_max* is the best we can find for a given layout, but it is hard to expose without exposing first what *D_Q* is first. One thing is easy to expose though: if *Q_max* is low, it means that *Q(D)* is always low, hence it means that the layout failed at bringing connected nodes together; it means that it did not succeed more than what chance naturally produces (i.e. some edges are short by chance). Note: We could have another measure of the success of the layout. For instance we could measure how much shorter are edges than expected on average. This alternative deserves consideration, although I will not explore this path here. This series of notebooks basically follows my train of thoughts, and I intend to keep it this way. I will use these notebooks as a trace of my process, and blackbox it elsewhere (in my [research blog](https://reticular.hypotheses.org/) first, then in a paper and a tool, if it works). So let me share this "work in progress" observation: I did not realize that *D_Q* could become meaningless until I tried on a random layout. This is why *Q* plays a more important role in this notebook than the last. I did not expect it, and it forces me to rethink the process. At the end of the day, it might well be more productive to use another measure of success, for the layout, than *Q*. We will see. ***D_Q* tells us how to interpret the network map.** If *Q_max* is low, it means that the layout did not bring connected nodes together. Hence it cannot be seen in the picture. If *Q_max* is high however, then that there is at least a certain distance (*D_Q*) under which we find significantly more edges than expected. *D_Q* is then a quantitative information we can use to forge useful statements about the map: * *P(D_Q)* percent of edges are shorter than *D_Q*. * *C(D_Q)* percent of those were expected (are due to chance), but * *Q_max* percent are unexpected, due to the layout. Note: P(D) = C(D) + Q(D) If *Q_max* is high then *P(D_Q)* is high too, and that is a simple and useful information on how to read the network map. A certain distance "captures" many edges, and that is specifically due to the layout (*Q_max*). The distance *D_Q* itself may be complicated to enounce, it is straightforward to grasp as it is just a line drawn on the picture. `37:T1d57,function draw(_g, _settings, data) { // SETTINGS let settings = _settings || {}; // Note: all sizes except canvas size are based on a 1000px x 1000px canvas // (this allows resizing the whole thing without losing tuning) // // 👇 Default values settings.zoom = settings.zoom || 0.9; // slight de-zoom for comfort settings.size = settings.size || 300; settings.node_size = settings.node_size || 5; settings.node_color = settings.node_color || 'rgba(0, 0, 0, 1)'; settings.edge_color_DQ = settings.edge_color_DQ || true; settings.edge_color = settings.edge_color || 'rgba(150, 150, 150, 0.8)'; settings.edge_thickness = settings.edge_thickness || 2; settings.margin_bottom = settings.margin_bottom || 140; let D = data.DQ; // Create canvas const canvas = document.createElement('canvas'); canvas.width = settings.size; canvas.height = settings.size + settings.margin_bottom; const ctx = canvas.getContext('2d'); // Clone network, so that we do not modify the original let g = _g.copy(); // Prepare network data addMissingVisualizationData(); rescaleGraphToGraphicSpace(); // Paint a white background ctx.fillStyle="white"; ctx.beginPath(); ctx.rect(0, 0, settings.size, settings.size); ctx.fill(); ctx.closePath(); // Scale canvas ctx.scale(settings.size/1000, settings.size/1000); // Shuffle edges (important for coloring) let edges = g.edges(); shuffle(edges); // Draw each edge ctx.lineCap="round"; ctx.lineJoin="round"; ctx.fillStyle = 'rgba(0, 0, 0, 0)'; ctx.lineWidth = settings.edge_thickness; if (!settings.edge_color_DQ) { ctx.strokeStyle = settings.edge_color; } edges.forEach(function(eid){ const ns = g.getNodeAttributes(g.source(eid)); const nt = g.getNodeAttributes(g.target(eid)); const d = Math.sqrt(Math.pow(ns.x-nt.x, 2)+Math.pow(ns.y-nt.y, 2)) if (settings.edge_color_DQ) { if (d<=D*1.01) { ctx.strokeStyle = 'rgba(96, 168, 98, 0.8)'; } else { ctx.strokeStyle = 'rgba(229, 120, 114, 0.8)'; } } ctx.beginPath(); ctx.moveTo(ns.x, ns.y); ctx.lineTo(nt.x, nt.y); ctx.stroke(); ctx.closePath(); }); // Draw each node ctx.lineWidth = 0; ctx.shadowColor = 'transparent'; g.nodes().forEach(function(nid){ const n = g.getNodeAttributes(nid); ctx.beginPath(); ctx.arc(n.x, n.y, settings.node_size * n.size, 0, 2 * Math.PI, false); ctx.fillStyle = settings.node_color; ctx.fill(); ctx.closePath(); }); // Reset current transformation matrix to the identity matrix ctx.setTransform(1, 0, 0, 1, 0, 0); // Draw the scale const D_scaled = D * ( settings.size/1000); drawScaleH(ctx, 10+D_scaled/2, settings.size+20, D_scaled); drawText(ctx, 'Dǫ', 10+D_scaled/2 - 8, settings.size + 10); drawText(ctx, Math.round(data.PQ)+'% of edges are Dǫ or shorter ('+Math.round(data.PQ-data.Q)+'% expected)', 8, settings.size + 60); drawText(ctx, 'Dǫ is the distance getting the most edges above expectations ('+Math.round(data.Q)+'%)', 8, settings.size + 90); drawText(ctx, 'Nodes closer than Dǫ have '+Math.round(data.CQ)+'% chances to be connected', 8, settings.size + 120); // Internal methods function rescaleGraphToGraphicSpace() { const referenceSize = 1000; // General barycenter resize let xbarycenter = 0; let ybarycenter = 0; let wtotal = 0; let dx; let dy; let ratio; g.nodes().forEach(function(nid){ let n = g.getNodeAttributes(nid); // We use node size as weight (default to 1) n.size = n.size || 1; xbarycenter += n.size * n.x; ybarycenter += n.size * n.y; wtotal += n.size; }) xbarycenter /= wtotal; ybarycenter /= wtotal; var dmax = 0; // Maximal distance from barycenter g.nodes().forEach(function(nid){ var n = g.getNodeAttributes(nid); var d = Math.sqrt( Math.pow(n.x - xbarycenter, 2) + Math.pow(n.y - ybarycenter, 2) ); dmax = Math.max(dmax, d); }) ratio = referenceSize / (2 * dmax); D *= ratio; // Resize g.nodes().forEach(function(nid){ let n = g.getNodeAttributes(nid); n.x = referenceSize / 2 + settings.zoom * (n.x - xbarycenter) * ratio; n.y = referenceSize / 2 + settings.zoom * (n.y - ybarycenter) * ratio; n.size *= ratio; }) } function addMissingVisualizationData() { let colorIssues = 0; let coordinateIssues = 0; g.nodes().forEach(function(nid){ let n = g.getNodeAttributes(nid); if (!isNumeric(n.x) || !isNumeric(n.y)) { const c = getRandomCoordinates(); n.x = c[0]; n.y = c[1]; coordinateIssues++; } if (!isNumeric(n.size)) { n.size = 1; } if (n.color == undefined) { n.color = '#665'; colorIssues++; } }) if (coordinateIssues > 0) { console.warn('Note: '+coordinateIssues+' nodes had coordinate issues. We carelessly fixed them.'); } function isNumeric(n) { return !isNaN(parseFloat(n)) && isFinite(n); } function getRandomCoordinates() { let candidates; let d2 = Infinity; const heuristicRatio = 5 * Math.sqrt(g.order); while (d2 > 1) { candidates = [2 * Math.random() - 1, 2 * Math.random() - 1]; d2 = candidates[0] * candidates[0] + candidates[1] * candidates[1]; } return candidates.map(function(d){return d * heuristicRatio}); } } function shuffle(array) { var currentIndex = array.length, temporaryValue, randomIndex; // While there remain elements to shuffle... while (0 !== currentIndex) { // Pick a remaining element... randomIndex = Math.floor(Math.random() * currentIndex); currentIndex -= 1; // And swap it with the current element. temporaryValue = array[currentIndex]; array[currentIndex] = array[randomIndex]; array[randomIndex] = temporaryValue; } return array; } function drawText(ctx, txt, x, y) { ctx.font = "500 18px sans-serif"; ctx.lineWidth = 4; ctx.fillStyle = '#FFFFFF'; ctx.strokeStyle = '#FFFFFF'; ctx.fillText( txt, x, y ); ctx.strokeText( txt, x, y ); ctx.lineWidth = 0; ctx.fillStyle = '#000'; ctx.fillText( txt, x, y ); } function drawScaleH(ctx, x, y, D) { const height = 12; ctx.strokeStyle = 'rgba(0, 0, 0, 1)'; ctx.lineCap="round"; ctx.lineJoin="round"; ctx.fillStyle = 'rgba(0, 0, 0, 0)'; ctx.lineWidth = 2; ctx.beginPath(); ctx.moveTo(x-D/2, y-height/2) ctx.lineTo(x-D/2, y+height/2) ctx.moveTo(x-D/2, y); ctx.lineTo(x+D/2, y); ctx.moveTo(x+D/2, y-height/2) ctx.lineTo(x+D/2, y+height/2) ctx.stroke(); ctx.closePath(); } // Return result return { canvas, ctx }; }38:Te5d,computeData = function(g, layout) { // Apply layout layout(g); // Compute list of pairs of nodes const pairs_of_nodes_index = (()=>{ let p = {}; const nlist = g.nodes(); for (let i=0; i{ return d; }); // Sort from longest to shortest pairs.sort(function(a,b){return b.d-a.d}); pairs.forEach((d,i)=>{d.rank = i}); return pairs; })(); // Compute indicators over D const indicators_over_D = (()=>{ const distances = d3.scaleLinear() .domain(d3.extent(pairs_of_nodes_list, d=>d.d)) .nice() .ticks(100); let ilist = distances.map(D=>{ const i = computeIndicators(D, g); return { D, E: i.E, P: i.P, P_all: i.P_all, Q: i.Q }; }); return ilist; })(); // Compute DQ const DQ = (()=>{ let Qmax = 0; let DQ = 0; indicators_over_D.forEach(d=>{ if (d.Q > Qmax) { Qmax = d.Q; DQ = d.D; } }); return DQ; })(); // Compute indicators for DQ const DQ_indicators = computeIndicators(DQ, g); // Internal methods function computeIndicators(D,g) { const connected_pairs = pairs_of_nodes_list.filter(d=>d.c); const total_of_connected_pairs = connected_pairs.length; const connected_pairs_below_D = connected_pairs.filter(d=>d.d<=D); const count_of_connected_pairs_below_D = connected_pairs_below_D.length; const disconnected_pairs = pairs_of_nodes_list.filter(d=>!d.c); const disconnected_pairs_below_D = disconnected_pairs.filter(d=>d.d<=D); const count_of_disconnected_pairs_below_D = disconnected_pairs_below_D.length; const total_of_pairs_below_D = count_of_connected_pairs_below_D + count_of_disconnected_pairs_below_D; const total_of_all_pairs = pairs_of_nodes_list.length; // Percentage of connected nodes below D const P = Math.floor(10000*count_of_connected_pairs_below_D/total_of_connected_pairs)/100; // False positive error (of interpreting D as a threshold under which all pairs are connected) const E = Math.floor(10000*(1-count_of_connected_pairs_below_D/total_of_pairs_below_D))/100; // Chances that a pair below D is connected (note: E+C=100%) const C = Math.floor(10000*(count_of_connected_pairs_below_D/total_of_pairs_below_D))/100; // Percentage of all pairs below D const P_all = Math.floor(10000*total_of_pairs_below_D/total_of_all_pairs)/100; // Quality measure, looking at how many connected pairs we grasp above expected const ratio_of_connected_pairs = total_of_connected_pairs / total_of_all_pairs const expected_count_of_connected_pairs_below_D = total_of_pairs_below_D*ratio_of_connected_pairs const Q = Math.floor(10000*(count_of_connected_pairs_below_D - expected_count_of_connected_pairs_below_D)/total_of_connected_pairs)/100; return { P, E, C, P_all, Q }; } return { g, pairs_of_nodes_index, pairs_of_nodes_list, indicators_over_D, DQ, PQ: DQ_indicators.P, EQ: DQ_indicators.E, CQ: DQ_indicators.C, Q: DQ_indicators.Q }; }39:[] 32:{"id":"a2914227115bfdc0","slug":"evaluating-the-validity-metric-on-a-different-layout","title":"Evaluating the validity metric on a different layout","thumbnail":"85ea5389c87183e2c0fc56935d0bc9d04f31d65dbb676bde0fc8ec9ae94eaf3a","access_level":"public","create_time":"$D2020-03-12T08:55:33.379Z","update_time":"$D2020-05-11T17:31:22.541Z","view_time":null,"first_public_time":"$D2020-03-13T11:03:56.008Z","has_unpublished_edits":null,"fork":{"id":"a27aabb1a0d2bf02","slug":"a-validity-metric-for-interpreting-distances-in-a-network-m","title":"A validity metric for interpreting distances in a network map","owner":{"id":"314a0a5985c09d6c","login":"jacomyma","name":"Mathieu Jacomy"}},"stars":0,"starred":false,"owner":{"type":"team","id":"314a0a5985c09d6c","login":"jacomyma","name":"Mathieu Jacomy","avatar_url":"https://avatars-next.observableusercontent.com/4496a2bc190392af306e4fcc2cc12dec9fe2acc458657e9bb35aecbeb5322d71","tier":"starter_2024"},"version":1297,"publish_version":1297,"role":null,"authored":false,"authors":[{"type":"individual","id":"5877bce6260ce3ea","login":"jacomyma","name":"Mathieu Jacomy","avatar_url":"https://avatars-next.observableusercontent.com/4496a2bc190392af306e4fcc2cc12dec9fe2acc458657e9bb35aecbeb5322d71","tier":null}],"members":[{"type":"individual","id":"5877bce6260ce3ea","login":"jacomyma","name":"Mathieu Jacomy","avatar_url":"https://avatars-next.observableusercontent.com/4496a2bc190392af306e4fcc2cc12dec9fe2acc458657e9bb35aecbeb5322d71","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":2,"collection":{"id":"e33184b07af44a70","slug":"quest-for-connected-closeness","title":"Quest for connected-closeness","owner":{"id":"314a0a5985c09d6c","login":"jacomyma","name":"Mathieu Jacomy"}},"type":"notebook","body":{"title":"Evaluating the validity metric on a different layout","cells":[{"id":1284,"value":"md`# Evaluating the validity metric on a different layout`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1279,"value":"md`***This notebook is part of a [series](https://observablehq.com/collection/@jacomyma/quest-for-connected-closeness) that retraces my quest for a metric capable of evaluating network maps. You may want to check or the final result, [connected-closeness](https://observablehq.com/@jacomyma/highlights-on-connected-closeness).***`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":0,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":936,"value":"{\n return draw(\n g1,\n {size: 640},\n data1\n ).canvas\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1058,"value":"{\n const form = html``;\n form.layoutAndRenderButton.onclick = () => {\n mutable data1_version++;\n }; \n \n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1041,"value":"md`The characteristic distance *D_Q* is found where the metric *Q* (in green below) is maximal.\n\n*Q* is the proportion of unexpected edges. More precisely, *Q(D)* is equal to the percentage of edges of length *D* or shorter, minus the expected percentage of edges of length *D* or shorter.\n\n*Q* is roughly equal to the gap between the actual number (in black) and the expected number (in blue) of connected pairs of nodes separated by no more than a given distance.\n\n ** *D_Q* is the distance where we find the most unexpected edges**.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":912,"value":"plot(data1);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":852,"value":"g1 = generateSBM(100, 3, .9, .1)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":856,"value":"layout1 = g => {\n let settings = {}\n\n // Apply a random layout\n g.nodes().forEach(nid => {\n g.setNodeAttribute(nid, 'x', 500-1000*Math.random());\n g.setNodeAttribute(nid, 'y', 500-1000*Math.random());\n });\n\n // Apply Force Atlas 2\n fa2.assign(g, {iterations: 1000, settings: {\n linLogMode: false,\n outboundAttractionDistribution: false,\n adjustSizes: false,\n edgeWeightInfluence: 0,\n scalingRatio: 10,\n strongGravityMode: true,\n gravity: 0.05,\n slowDown: 1,\n barnesHutOptimize: false,\n barnesHutTheta: 0.5\n }});\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":869,"value":"data1 = computeData(g1, layout1, data1_version)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1056,"value":"md`## Influence of the layout`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1211,"value":"md`The random layout is the opposite of a force-directed layout: it disregards the topology instead of visualizing it. How well does the validity metric do in that context?\n\nThe network below is the very same network, but rendered by a random layout.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1112,"value":"{\n return draw(\n g2,\n {size: 640},\n data2\n ).canvas\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1121,"value":"{\n const form = html``;\n form.layoutAndRenderButton.onclick = () => {\n mutable data2_version++;\n }; \n \n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1137,"value":"plot(data2);","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1116,"value":"g2 = g1.copy()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1096,"value":"layout2 = g => {\n let settings = {}\n\n // Apply a random layout, but contained to a circle\n const r = 300;\n g.nodes().forEach(nid => {\n let x = r-2*r*Math.random();\n let y = r-2*r*Math.random();\n let d2 = x*x+y*y;\n while (d2>r*r) {\n x = r-2*r*Math.random();\n y = r-2*r*Math.random();\n d2 = x*x+y*y;\n }\n g.setNodeAttribute(nid, 'x', x);\n g.setNodeAttribute(nid, 'y', y);\n });\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1099,"value":"data2 = computeData(g2, layout2, data2_version)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1214,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1219,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1266,"value":"md`## Take aways\n* The metric *D_Q* helps to make useful statements on the image\n* But *D_Q* is not always meaningful\n* *P* the percentage of edges shorter than *D* is not always useful, because it can be high purely due to chance\n* *Q_max* is the more meaningful metric, because it discounts expected edges\n* But *Q_max* is also hard to explain\n* It is important to contextualize *D_Q* with *Q_max* so that we know when it is meaningless\n\nWe see that just by looking at what it looks like on a random network, a lot of questions unfolded. It shows that the measure still requires a lot of design: better names, better symbols, better way to explain it, better ways to visualize it.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"md`

\n\n--- \n\n# Appendix\n\n*This is stuff we need to load in this notebook to make it work.*\n\n### Require Graphology stuff`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":27,"value":"Graph = require('graphology@0.17.1/dist/graphology.umd.min.js')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1291,"value":"GraphologyLibrary = require('graphology-library@0.3.0/dist/graphology-library.min.js')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":29,"value":"gexf = GraphologyLibrary.gexf","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":31,"value":"layout = GraphologyLibrary.layout","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":33,"value":"fa2 = GraphologyLibrary.layoutForceAtlas2","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"md`### Require d3`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":37,"value":"d3 = require(\"d3@5\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":842,"value":"md`### Import network generators`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":846,"value":"import {\n generateSBM\n} from \"@jacomyma/basic-network-generators\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":156,"value":"md`### Load Vega Lite for plotting`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":159,"value":"import {vl} from '@vega/vega-lite-api'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1046,"value":"md`### Routines`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1107,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":926,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1129,"value":"function plot(data) {\n\n const AREA = vl.markArea({interpolate: 'monotone', fill: 'rgba(50, 200, 100, 0.06)'})\n .encode(\n vl.x().fieldQ('D'),\n vl.y().fieldQ('P'),\n vl.y2().fieldQ('P_all'),\n );\n\n const P = vl.markLine({interpolate: 'monotone', stroke: 'black'})\n .encode(\n vl.x().fieldQ('D'),\n vl.y().fieldQ('P').title('P (BLACK)'),\n );\n\n const P_all = vl.markLine({interpolate: 'monotone', stroke: 'blue'})\n .encode(\n vl.x().fieldQ('D'),\n vl.y().fieldQ('P_all').title('P_all (BLUE)'),\n );\n\n const Q = vl.markLine({interpolate: 'monotone', stroke: 'green'})\n .encode(\n vl.x().fieldQ('D'),\n vl.y().fieldQ('Q').title('Q (GREEN)'),\n );\n\n return vl.layer(AREA, P, P_all, Q)\n .data(data.indicators_over_D)\n //.resolve({scale: {y: 'independent'}}) // resolve to independent y channel scales\n .title('Indicators over D')\n .width(640)\n .height(180)\n .render();\n }","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1086,"value":"md`Variables we track to retrigger certain routines`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1083,"value":"mutable data1_version = 0","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1101,"value":"mutable data2_version = 0","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W39","stdlib":"1","titling":"auto"}} 33:"$32:body"