2d:null 2e:["$","title",null,{"children":"Test Suite for Network Visualisation | tyr1981 | Observable"}] 33:T5c1,This is a VisUnit-compatible **data+tasks test-suite for network visualisation**. It contains **8 datasets** and **19 tasks** that network visualisations can connect to and evaluate against. The test-suite is built predominantly around materials used in three influencial empirical studies of network visualisations: - _Ghoniem et al._ (2005): [On the Readability of Graphs Using Node-Link and Matrix-Based Representations: A Controlled Experiment and Statistical Analysis](https://doi.org/10.1109/INFVIS.2004.1) - _Okoe et al._ (2018): [Node-Link or Adjacency Matrices: Old Question, New Insights](https://doi.org/10.1109/TVCG.2018.2865940) - _Di Giacomo_ et al. (2023): [Comparative Study and Evaluation of Hybrid Visualizations of Graphs](https://ieeexplore.ieee.org/abstract/document/10004748) And two task taxonomies: - _Lee at al._ (2006): [Task taxonomy for graph visualization](https://dl.acm.org/doi/abs/10.1145/1168149.1168168?casa_token=fjW1QMdduqgAAAAA:-DQlS0xYjNa05b6_NjvDm2H7vbadG5o9R4CsIdoydFxOVRKB9GZtXabH3PqJdGew-b1bOePqgxE0) - _Amar et al._ (2005): [Low-level components of analytic activity in information visualization](https://ieeexplore.ieee.org/abstract/document/1532136?casa_token=3XhiZShHylYAAAAA:OBNw93pz0wb7K_-leEcbfzRZwE1O4CbGGo1XUecV9xocQJcpLM7Z5bkl14iauqgm614Di1QVqg). The choice and design of datasets and tasks are discussed and al materials can be browsed interactively at the end of their respective sections using node-link diagrams.34:T43b,Ensure your Javascript network visualisation implements the **interface functions** below. Setting up the tasks in this suite relies on their existence. However, depending on which tasks you want to use, you may not need all functions: - _loadData(networkData)_ : loads one or a collection (array) of network datasets in the JSON format shown further below - _setNetwork(number)_ : if the dataset is a collection of networks, this will set the index of the "current" (currently shown) network. - _getData()_ : returns the currently loaded dataset (important to return the same object used when loading the data) - _clearTask()_ : removes any node selections or highlighting - _getSelectedNodes()_ : returns (an array of) node indeces selected interactively by user - _setHighlightedNodes([[first set of nodes],[second set of nodes],...])_ : sets up to three sets of distinct highlighted nodes using colours blue, orange, purple. - _setIntroduction(step, context)_ sets _context.gui.question_ (a div) to provide introductory information about the visualisation along multiple steps35:T4b1,Inputs.table(Object.values(networkTesting.data).map( d => ( { name : d.name, description : d.description, collection : Array.isArray(d) ? "yes, " + d.length + " networks" : "no", nodes : Array.isArray(d) ? d[0].nodes.length : d.nodes.length, edges : Array.isArray(d) ? d[0].edges.length : d.edges.length, lDensity : Array.isArray(d) ? (d[0].edges.length / d[0].nodes.length).toFixed(1) : (d.edges.length / d.nodes.length).toFixed(1), sqDensity : Array.isArray(d) ? Math.sqrt(d[0].edges.length / (d[0].nodes.length*d[0].nodes.length)).toFixed(1) : Math.sqrt(d.edges.length / (d.nodes.length*d.nodes.length)).toFixed(1), degrees : Array.isArray(d) ? d[0].nodes.map( (n,i) => nodeDegree(d[0],i)) : d.nodes.map( (n,i) => nodeDegree(d,i)) } ) ), { width : { name : 100, collection: 100, nodes : 50, edges : 50, }, format: { description : x => html`

${x}

`, degrees : row => Plot.plot({ height : 200, y: {grid: true}, color: {legend: true}, marks: [ Plot.rectY(row, Plot.binX({y: "count"}, {x: row, fill : "steelblue"})), Plot.ruleY([0]) ] }) } })36:T895,

**Discussion** (summarised from _Ghoniem et al._, _Okoe et al._, _Di Giacomo et al_.) **Synthetic vs. Real data:** Existing studies used both synthetic networks (_Ghoniem et al_.) and real ones (_Okoe et al._, _Di Giacomo et al._). The benefit of the former is that they can be tailored to the goals of evaluation (e.g. precise control of size and density) and can be produced en-masse. However, the danger with using synthetic data is that it may not be represantative of real one. Additionally, participants may relate to and engage less with datasets that are not grounded in reality (e.g., have meaningless numerical labels or appear to have random structure). **Representative real data:** _Okoe et al._ reviewed 17 real-world networks (e.g., trade exchanges between countries, co-authorship networks, protein-protein interactions) before picking two datasets - _Ingredients_ and _USAir97_ - that have densities and node-degre distributions similar to those reviewed. They find that the distribution of node degrees in real networks is uneven; real networks are often structured around "hub nodes" with significantly higher degrees than the network's average. This confirms an [analysis by _Melancon_](https://dl.acm.org/doi/10.1145/1168149.1168167) showing that real-life networks typically exhibit "small-word" effects (nodes and links cluster into 'regional' sub-networks). These observations are captured in network statistics listed above. **Density:** There are two ways to compute density: _linearDensity = edgeCount / nodeCount_ and _squareRootDensity = sqrt(edgeCount/nodeCount^2)_. [_Melancon_ finds](https://dl.acm.org/doi/10.1145/1168149.1168167) that the first better captures real-life scenarios where networks exhibit "small-world" effects. **Size:** Small networks (<100 nodes) are typically used in studies involving static, non-interactive visualisations. Larger networks (100-300 nodes) often require visualisations that implement zooming and panning. **Meaningful data:** Real networks with meaningful data can help participants maintain engagement. However, it may also introduce bias effects (e.g., participants may "read into" data that is not there). 37:T811,Inputs.table(Object.values(networkTesting.tasks).map( t => ( { name : t.name, description : t.description, inputs : "inputs" in t ? t.inputs.map( i => i.type == "vis" ? "vis.set"+i.state : (i.type == "question" ? "question param" : "output param")).join(", ") : "none", outputs : "outputs" in t ? t.outputs.map( i => i.type == "vis" ? "vis.get"+i.state : i.type).join(", ") : "none", used_in : { nodeCount : "Ghoniem et al., Lee et al. (adjacency, overview)", edgeCount : "Ghoniem et al., Lee et al. (adjacency, overview)", mostConnected : "Ghoniem et al., Lee et al. (adjacency, overview)", compareDegrees : "Ghoniem et al., Okoe et al., Lee et al. (adjacency)", findNode : "Ghoniem et al., Lee et al. (attribute)", nodesConnected : "Ghoniem et al., Okoe et al., Lee et al. (adjacency)", namedConnected : "Okoe et al., Lee et al. (adjacency, attribute)", selectNeighbors : "Okoe et al., Lee et al. (adjacency)", areNeighbors : "Okoe et al., Lee et al. (adjacency, attribute)", commonNeighbor : "Okoe et al., Lee et al. (common connection)", selectCommonNeighbors : "Okoe et al., Lee et al. (common connection)", namedCommonNeighbor : "Okoe et al., Lee et al. (common connection, attribute)", withinTwoHops : "Okoe et al., Lee et al. (accessability)", findPath : "Ghoniem et al., Okoe et al., Lee et al. (connectivity)", comparePaths : "Okoe et al., Lee et al. (connectivity)", rememberConnection : "Okoe et al., Lee et al. (connectivity)", rememberNeighbors : "Okoe et al., Lee et al. (common connection)" }[t.name], } ) ), { width : { name : 150, description : 200, inputs : 150, outputs : 150 }, format: { description : x => html`

${x}

`, used_in : x => html`

${x}

`, inputs : x => html`

${x}

`, } })38:T98c,

### Discussion **Coverage:** We mainly sought to select tasks used by well received studies on network visualisation - Ghoniem et al. (2005), Okoe et al. (2018), and Di Giacomo et al. (2023) - to support their reproduction and extension. Tasks also cover the network task taxonomy of Lee et al. (2006) and the taxonomy of Amar et al. (2006). **Ecological validity:** We think that a range of tasks have high ecological validiy - especially composite tasks that rely on multiple perceptual components or ones that leverage fast perceptual mechanisms. For example, _namedConnected_ requires participants to scan through a node's neighbors and compare their labels to a given node; _withinTwoHops_ requires participants to approximate the size of a nodes neighborhood. _pathCompare_ is mean to be delivered with a short time to answer and rely on a quick estimative judgement rather than on actual path search. On the other hand we think that some tasks are somewhat artificial. We think it is unlikely for users to manually search for nodes by label in large networks. We also think it's unlikely for users to search for paths longer than _2_, unless networks are very sparse. We point to Okoe et al. for a more comprehensive discussion. **Confounding factors:** While answering tasks via interaction (e.g., selecting nodes) is natural, in some cases the time this takes can be significantly longer than the time needed to resolve the task visually. For example, selecting a node's neighbors (_selectNeighbors_) can take a significant time that is also dependent on the node's degree. For this reason we added the _areNeighbors_ task which can be answered easily with a yes/no answer as soon as the task is visually resolved. Moreover, the number of candidates to be assessed as potential neighbors can be kept relatively constant to ensure tasks instances are of similar difficulty. **Visualisation capabilities:** We sought to design tasks so as to require a limited set of visualisation capabilities to be set up. All but one task in the suite build on the abilities of a visualisation to _setNetwork_, _setHighlightedNodes_, and _getSelectedNodes_ (see "Use" section above for details). The _introduceVis_ task builds on _setIntroduction_ to let visualisations introduce themselves to a participant in a series of explanatory steps. A few task specific discussion points are listed below (along with the actual task definitions):39:T438,mostConnected = { //return an accuracy function for this network+task let accuracy = (network) => (responses) => { if (responses[0].length != 1) return 0; let maxDegree = maxNodeDegree(network); return 1- Math.abs(nodeDegree(network, responses[0][0]) - maxDegree) / maxDegree; } //generate instances for this dataset (a [multi]network) function genInstances(d){ //if d is a regular network then transform it into a multi-network (i.e. array of networks) let mn = Array.isArray(d) ? d : [d]; return { data : d, instances : mn.map( (n,i) => ( {inputs : [i], accuracy : accuracy(n)})) } } return { name : "mostConnected", question: "Find and select the most connected node in the network.", description: "Finding the most connected in the network", inputs : [ { type : "vis", state : "Network"}], outputs : [ {type : "vis", state : "SelectedNodes"}], //create instances in bulk for all datasets in the test suite instances : Object.values(testData).map(genInstances) } }3a:T1606,compareDegrees = { //method to compute the accuracy of a compareDegrees answer let accuracy = (network, nodes) => (response) => { if (response.length != 1) //only one node should be selected return 0; let d1 = nodeDegree(network, nodes[0]); let d2 = nodeDegree(network, nodes[1]); if (d1 == d2) //task is wrong, nodes have same degree return 1; if ((response[0] == nodes[0] && d1 > d2) || (response[0] == nodes[1] && d2 > d1)) return 1; return 0; }; //helper function to assemble instances from a dataset and //associated instances. Instances are triples (node index 1, //node index 2, network index) for multi networks or tuples for //(node 1, node 2) for regular networks let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let nodes = [i[0],i[1]]; return {inputs : [networkIndex, [nodes]], accuracy : accuracy(network, nodes)} })} else return {data : data, instances : instances.map( i=> { let nodes = [i[0],i[1]]; return {inputs : [0, [nodes]], accuracy : accuracy(data, nodes)} })} }; return { name : "compareDegrees", question : "Select the blue node which has more connections.", description : "Given two highlighted nodes, participant selects the one with the larger degree.", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [{type : "vis", state : "SelectedNodes"}], instances : [ //the first element in those triples is the network index; the other two are the two nodes to highlight genInstances(testData.demoNetwork, [ [3, 4], [1, 5], [0, 2], [2, 4], [4, 5] ]), genInstances(testData.lesMis, [ [9, 69], [69, 58], [42, 28], [56, 65], [33, 21], [0, 11, 43], [0, 67, 41] ]), genInstances(testData.ingredients, [ [91, 156], [76, 203], [236, 1], [33, 183], [17, 80], [3, 172], [67, 13] ]), genInstances(testData.usair97, [ [100, 48], [163, 65], [193, 64], [90, 250], [290, 70], [43, 15], [91, 100], [122, 120], [183, 281], [297, 253] ]), genInstances(testData.ecoli, [ [21, 87], [96, 17], [45, 25], [52, 24], [79, 84], [73, 36], [6, 59], [32, 81], [75, 64], [8, 33] ]), genInstances(testData.weavers, [ [50, 10], [3, 9], [32, 8], [42, 2], [30, 32], [21, 63], [12, 8], [17, 1], [61, 34], [23, 52] ]), genInstances(testData.dblp, [ [77, 2], [47, 20], [58, 11], [48, 40], [55, 21], [80, 37], [44, 49], [49, 42], [47, 20], [23, 35] ]), genInstances(testData.ghoniem1, [ [3, 6, 0], [10, 1, 1], [7, 13, 2], [9, 18, 3], [3, 17, 4], [2, 15, 5], [2, 15, 6], [11, 18, 7], [17, 7, 8], [0, 7, 9] ]), genInstances(testData.ghoniem2, [ [0, 15, 0], [5, 1, 1], [12, 13, 2], [4, 3, 3], [18, 17, 4], [5, 9, 5], [8, 2, 6], [5, 16, 7], [8, 4, 8], [12, 11, 9] ]), genInstances(testData.ghoniem3, [ [7, 15, 0], [19, 9, 1], [8, 12, 2], [13, 15, 3], [14, 15, 4], [7, 10, 5], [3, 12, 6], [7, 1, 7], [17, 16, 8], [8, 19, 9] ]), genInstances(testData.ghoniem4, [ [24, 11, 0], [27, 40, 1], [4, 0, 2], [1, 9, 3], [43, 15, 4], [8, 20, 5], [25, 43, 6], [14, 13, 7], [34, 21, 8], [0, 17, 9] ]), genInstances(testData.ghoniem5, [ [44, 41, 0], [21, 47, 1], [34, 20, 2], [5, 7, 3], [40, 30, 4], [6, 22, 5], [48, 23, 6], [0, 16, 7], [47, 31, 8], [36, 8, 9] ]), genInstances(testData.ghoniem6, [ [44, 41, 0], [37, 33, 1], [48, 40, 2], [9, 46, 3], [25, 42, 4], [0, 21, 5], [42, 8, 6], [0, 8, 7], [39, 30, 8], [19, 28, 9] ]), genInstances(testData.ghoniem7, [ [44, 16, 0], [35, 67, 1], [68, 93, 2], [9, 46, 3], [65, 85, 4], [50, 67, 5], [42, 8, 6], [72, 39, 7], [32, 64, 8], [98, 75, 9] ]), genInstances(testData.ghoniem8, [ [14, 0, 0], [75, 77, 1], [30, 94, 2], [17, 67, 3], [54, 86, 4], [46, 96, 5], [40, 28, 6], [72, 39, 7], [46, 17, 8], [99, 74, 9] ]), genInstances(testData.ghoniem9, [ [35, 76, 0], [1, 90, 1], [56, 38, 2], [71, 84, 3], [0, 61, 4], [86, 65, 5], [19, 49, 6], [58, 67, 7], [26, 18, 8], [20, 35, 9] ]) ] } }3b:T12aa,findNode = { let accuracy = (network, label) => (response) => { if (response.length != 1) //only one node should be selected return 0; let nodeIndex = network.nodes.findIndex( n => n.label == label); if (response[0] == nodeIndex) return 1; else return 0; } let genInstances = (data, instances) => Array.isArray(data) ? ({data : data, instances : instances.map( i => { let networkIndex = i[1]; let network = data[networkIndex]; let label = i[0]; return {inputs : [networkIndex, label], accuracy : accuracy(network, label)} })}) : ({data : data, instances : instances.map( i => { let label = i; return {inputs : [0, label], accuracy : accuracy(data, label)} })}); return { name : "findNode", question: "Find and select the node labeled $1.", description: "Given the label of a node, the participant should find and select it.", inputs : [{ type : "vis", state : "Network"}, {type : "question"}], outputs : [ {type : "vis", state : "SelectedNodes"}], instances : [ genInstances(testData.demoNetwork, [ "node5", "node2", "node6", "node1", "node3" ]), genInstances(testData.lesMis, [ "Napoleon", "Myriel", "Cosette", "Isabeau", "Simplice" ]), genInstances(testData.ingredients, [ "mate", "cinnamon", "clove", "onion", "asparagus", "nut", "liver", "plum" ]), genInstances(testData.usair97, [ "Detroit City", "Youngstown-Warren Regional", "St Mary's", "Yuma Mcas/Yuma Intl", "Wichita Mid-Continent", "Chicago Midway", "Long Beach /Daugherty Field/", "Lanai", "Bert Mooney", "Rapid City Regional" ]), genInstances(testData.ecoli, [ "68", "71", "9", "27", "37", "23", "28", "9", "59", "69" ]), genInstances(testData.weavers, [ "VMWL", "WPXX", "VMRE", "UMVL", "VMLB", "UMPP", "FMBV", "YWFM", "VMYW", "YYFM" ]), genInstances(testData.dblp, [ "Fintan McGee", "Joseph Kotlarek", "Kai Xu", "Florian Ganglberger", "Oliver Kohlbacher", "Luca Grilli", "Danielle Szafir", "Min Zhu", "Mario Albrecht", "Benjamin Merlet" ]), genInstances(testData.ghoniem1, [ ["A", 0], ["K", 1], ["B", 2], ["F", 3], ["D", 4], ["I", 5], ["J", 6], ["Q", 7], ["O", 8], ["N", 9] ]), genInstances(testData.ghoniem2, [ ["A", 0], ["O", 1], ["P", 2], ["R", 3], ["C", 4], ["D", 5], ["T", 6], ["E", 7], ["J", 8], ["H", 9] ]), genInstances(testData.ghoniem3, [ ["N", 0], ["M", 1], ["D", 2], ["O", 3], ["K", 4], ["G", 5], ["A", 6], ["P", 7], ["S", 8], ["F", 9] ]), genInstances(testData.ghoniem4, [ ["E6", 0], ["A9", 1], ["A0", 2], ["D6", 3], ["C6", 4], ["B0", 5], ["E7", 6], ["C8", 7], ["E4", 8], ["C9", 9] ]), genInstances(testData.ghoniem5, [ ["D2", 0], ["A0", 1], ["B6", 2], ["A4", 3], ["E6", 4], ["C0", 5], ["D3", 6], ["C8", 7], ["E6", 8], ["D4", 9] ]), genInstances(testData.ghoniem6, [ ["A0", 0], ["A2", 1], ["D9", 2], ["A4", 3], ["D4", 4], ["A6", 5], ["B0", 6], ["E6", 7], ["C5", 8], ["E5", 9] ]), genInstances(testData.ghoniem7, [ ["B1", 0], ["A2", 1], ["C4", 2], ["B5", 3], ["H7", 4], ["A8", 5], ["H4", 6], ["I5", 7], ["E9", 8], ["J4", 9] ]), genInstances(testData.ghoniem8, [ ["A0", 0], ["H3", 1], ["I4", 2], ["A8", 3], ["C4", 4], ["J3", 5], ["G0", 6], ["C2", 7], ["E8", 8], ["F4", 9] ]), genInstances(testData.ghoniem9, [ ["A0", 0], ["H0", 1], ["C9", 2], ["J5", 3], ["C8", 4], ["H7", 5], ["A2", 6], ["J6", 7], ["A7", 8], ["F4", 9] ]) ] } }3c:T167a,nodesConnected = { //method to compute the accuracy of a nodesConnected answer let accuracy = (network, nodes) => (response) => { let connected = areConnected(network,nodes[0],nodes[1]); if ((connected && response[0] === "yes") || (!connected && response[0] === "no")) return 1; else return 0; }; //helper function to assemble instances from a dataset and //associated instances. Instances are triples (node index 1, //node index 2, network index) for multi networks or tuples for //(node 1, node 2) for regular networks let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let nodes = [i[0],i[1]]; return {inputs : [networkIndex, [nodes]], accuracy : accuracy(network, nodes)} })} else return {data : data, instances : instances.map( i=> { let nodes = [i[0],i[1]]; return {inputs : [0, [nodes]], accuracy : accuracy(data, nodes)} })} }; return { name : "nodesConnected", question : "Are the two highlighted nodes directly connected?", description : "Given two selected nodes the participant decides if they are connected.", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "select", options : ["select","yes","no"]}], instances : [ genInstances(testData.demoNetwork, [ [2, 3], [4, 5], [0, 5], [2, 0], [0, 2] ]), genInstances(testData.lesMis, [ [27, 58], [23, 43], [35, 38], [35, 74], [27, 29], [60, 24], [63, 66], [42, 72], [11, 70], [74, 36] ]), genInstances(testData.ingredients, [ [102, 67], [173, 184], [111, 123], [95, 27], // connected, [76, 67], // connected, [17, 138], // connected, [21, 116], [115, 32], [113, 30], [143, 234] // connected ]), genInstances(testData.usair97, [ [47, 46], // connected, [130, 310], // connected, [268, 30], [122, 79], [1, 142], [158, 151], // connected, [182, 31], [90, 146], // connected, [257, 64], // connected, [163, 90] ]), genInstances(testData.ecoli, [ [64, 52], [15, 49], [11, 19], // connected, [57, 89], [3, 2], // connected, [35, 1], // connected, [36, 60], [93, 90], [86, 49], [89, 93] // connected ]), genInstances(testData.weavers, [ [19, 20], // connected, [51, 4], [4, 22], // connected, [18, 44], [38, 13], [0, 1], // connected, [3, 17], [3, 32], [12, 30], [51, 59] ]), genInstances(testData.dblp, [ [33, 35], [54, 60], // connected, [44, 81], [32, 34], // connected, [54, 49], [58, 34], [14, 16], // connected, [58, 56], // connected, [72, 16], [49, 38] // connected ]), genInstances(testData.ghoniem1, [ [0, 5, 0], [1, 2, 1], [5, 8, 2], [4, 12, 3], [4, 10, 4], [11, 9, 5], [17, 10, 6], [1, 7, 7], [10, 18, 8], [14, 8, 9] ]), genInstances(testData.ghoniem2, [ [0, 2, 0], [14, 2, 1], [14, 12, 2], [5, 0, 3], [16, 12, 4], [19, 3, 5], [3, 6, 6], [8, 0, 7], [3, 7, 8], [10, 2, 9] ]), genInstances(testData.ghoniem3, [ [1, 10, 0], [11, 12, 1], [14, 0, 2], [1, 19, 3], [2, 12, 4], [18, 5, 5], [9, 14, 6], [8, 19, 7], [8, 17, 8], [12, 2, 9] ]), genInstances(testData.ghoniem4, [ [29, 10, 0], [11, 49, 1], [34, 47, 2], [49, 35, 3], [21, 37, 4], [33, 6, 5], [3, 48, 6], [28, 14, 7], [30, 15, 8], [15, 35, 9] ]), genInstances(testData.ghoniem5, [ [46, 10, 0], [22, 33, 1], [26, 37, 2], [13, 38, 3], [39, 4, 4], [43, 20, 5], [25, 10, 6], [0, 37, 7], [30, 15, 8], [17, 35, 9] ]), genInstances(testData.ghoniem6, [ [27, 29, 0], [5, 37, 1], [35, 3, 2], [13, 38, 3], [39, 4, 4], [25, 10, 5], [43, 20, 6], [0, 37, 7], [5, 33, 8], [19, 46, 9] ]), genInstances(testData.ghoniem7, [ [97, 86, 0], [39, 41, 1], [74, 47, 2], [52, 21, 3], [30, 78, 4], [43, 20, 5], [25, 10, 6], [88, 35, 7], [59, 52, 8], [43, 7, 9] ]), genInstances(testData.ghoniem8, [ [40, 18, 0], [33, 69, 1], [74, 47, 2], [84, 72, 3], [48, 98, 4], [43, 20, 5], [25, 10, 6], [88, 35, 7], [59, 52, 8], [84, 59, 9] ]), genInstances(testData.ghoniem9, [ [89, 81, 0], [33, 59, 1], [70, 27, 2], [12, 72, 3], [48, 95, 4], [55, 32, 5], [25, 10, 6], [59, 52, 7], [80, 15, 8], [99, 0, 9] ]) ]} }3d:T1760,namedConnected = { //method to compute the accuracy of a namedConnected answer let accuracy = (network, nodes) => (response) => { let labelIndex = network.nodes.findIndex( n => n.label === nodes[1]); let connected = areConnected(network,nodes[0],labelIndex); if ((connected && response[0] === "yes") || (!connected && response[0] === "no")) return 1; else return 0; }; let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; return {inputs : [networkIndex, [[i[0]]], i[1]], accuracy : accuracy(network, [i[0],i[1]])} })} else return {data : data, instances : instances.map( i=> { return {inputs : [0, [[i[0]]], i[1]], accuracy : accuracy(data, [i[0],i[1]])} })} }; return { name : "namedConnected", question : "Is node $2 connected to the (blue) highlighted node?", description : "Given one highlighted node and one named node, are they connected?", inputs : [ { type : "vis", state :"Network"}, { type : "vis", state : "HighlightedNodes"}, { type : "question"}], outputs : [ {type : "select", options : ["select","yes","no"]}], instances : [ genInstances(testData.demoNetwork, [ [2, "node1"], [2, "node6"], [0, "node5"], [2, "node2"], [3, "node1"] ]), genInstances(testData.lesMis, [ [55, "Mabeuf"], [74, "Gavroche"], [28, "Jondrette"], [54, "BaronessT"], [23, "Bamatabois"], [7, "Joly"], [58, "Bossuet"] ]), genInstances(testData.ingredients, [ [162, "nira"], [124, "parsnip"], [182, "tea"], [43, "cranberry"], [56, "cayenne"], [221, "cumin"], [44, "plum"] ]), genInstances(testData.usair97, [ [87, "Philadelphia Intl"], [289, "Dallas/Fort Worth Intl"], // connected, [280, "Dallas/Fort Worth Intl"], // connected, [161, "Monroe Regional"], [201, "Helena Regional"], [322, "Williamson County Regional"], [231, "Bush Field"], [307, "Houston Intercontinental"], // connected, [126, "Tallahassee Regional"], [293, "Memphis Intl"] ]), genInstances(testData.ecoli, [ [16, "11"], // connected, [25, "72"], [8, "88"], [45, "39"], // connected, [15, "53"], [89, "84"], // connected, [20, "29"], [71, "80"], [75, "1"], // connected, [50, "24"] ]), genInstances(testData.weavers, [ [30, "VMZO"], [5, "VMUU"], // connected, [37, "FMBV"], [2, "VMPO"], // connected, [12, "VMOZ"], [26, "FMBL"], // connected, [33, "YYVM"], [43, "UMOE"], // connected, [3, "FMOY"], // connected, [14, "FMLL"] // connected ]), genInstances(testData.dblp, [ [25, "Marnijati Torkel"], // connected, [27, "Peter Eades"], // connected, [78, "Bruno Pinaud"], // connected, [24, "Wulf Haubensak"], [30, "Marnijati Torkel"], // connected, [55, "Matthew O. Ward"], [45, "Giuseppe Liotta"], // connected, [26, "Kai Xu"], [66, "Takafumi Saito"], // connected, [13, "Fabien Jourdan"] ]), genInstances(testData.ghoniem1, [ [2, "A", 0], [2, "K", 1], [0, "F", 2], [1, "A", 3], [5, "C", 4], [13, "B", 5], [1, "M", 6], [10, "Q", 7], [0, "H", 8], [6, "M", 9] ]), genInstances(testData.ghoniem2, [ [2, "R", 0], [2, "H", 1], [19, "G", 2], [7, "O", 4], [13, "K", 5], [9, "D", 6], [13, "N", 7], [0, "P", 8], [3, "R", 9] ]), genInstances(testData.ghoniem3, [ [4, "A", 0], [10, "P", 1], [8, "N", 2], [7, "D", 4], [13, "K", 5], [9, "D", 6], [13, "N", 7], [4, "A", 8], [13, "T", 9] ]), genInstances(testData.ghoniem4, [ [4, "D4", 0], [10, "E1", 1], [8, "C2", 2], [7, "D5", 4], [13, "A0", 5], [9, "D1", 6], [13, "E7", 7], [4, "A3", 8], [13, "C0", 9] ]), genInstances(testData.ghoniem5, [ [4, "D4", 0], [10, "B7", 1], [18, "C2", 2], [27, "D5", 4], [48, "A5", 5], [29, "D9", 6], [20, "E7", 7], [40, "C3", 8], [3, "E0", 9] ]), genInstances(testData.ghoniem6, [ [12, "A5", 0], [6, "B3", 1], [39, "C9", 2], [8, "D1", 3], [10, "E4", 4], [7, "A2", 5], [19, "B6", 6], [5, "C0", 7], [14, "D8", 8], [11, "E3", 9] ]), genInstances(testData.ghoniem7, [ [15, "G8", 0], [15, "F1", 1], [12, "C5", 2], [52, "H2", 3], [20, "J6", 4], [10, "A7", 5], [7, "E9", 6], [37, "B2", 7], [14, "D4", 8], [6, "I3", 9] ]), genInstances(testData.ghoniem8, [ [18, "B4", 0], [9, "H7", 1], [12, "E6", 2], [7, "F9", 3], [16, "G3", 4], [55, "A8", 5], [10, "I2", 6], [83, "C1", 7], [20, "J5", 8], [6, "D0", 9] ]), genInstances(testData.ghoniem9, [ [5, "C7", 0], [12, "E5", 1], [78, "J3", 2], [34, "B2", 3], [9, "J9", 4], [23, "F8", 5], [56, "A1", 6], [17, "I0", 7], [98, "D4", 8], [15, "G6", 9] ]) ] } };3e:T104f,selectNeighbors = { //method to compute the accuracy of a selectNeighbors answer let accuracy = (network, node) => (response) => { let real = getNeighbors(network, node); let nrN = real.length; if (real.length == 0 && response[0].length == 0) return 1; let errors = 0; //identify number of 'false' positives for (let i=0; i { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[1]; let network = data[networkIndex]; let nodes = [i[0]]; return {inputs : [networkIndex, [nodes]], accuracy : accuracy(network, i[0])} })} else return {data : data, instances : instances.map( i=> { return {inputs : [0, [[i]]], accuracy : accuracy(data, i)} })} }; return { name : "selectNeighbors", question : "Select all the nodes directly connected to both highlighted one.", description : "Given two highlighted nodes participant selects their common neighbors.", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ { type : "vis", state : "SelectedNodes"}], instances : [ genInstances(testData.demoNetwork, [0, 1, 3, 5, 2]), genInstances(testData.lesMis, [5, 10, 16, 25, 45]), genInstances(testData.ingredients, [20, 34, 50, 70, 100, 120]), genInstances(testData.usair97,[176, 275, 9, 234, 113, 114, 271, 254, 254, 210]), genInstances(testData.ecoli, [95, 21, 76, 95, 15, 78, 13, 31, 46, 90]), genInstances(testData.weavers, [61, 3, 8, 18, 13, 23, 40, 9, 46, 63]), genInstances(testData.dblp, [2, 73, 66, 52, 69, 81, 58, 68, 5, 36]), genInstances(testData.ghoniem1, [ [5, 0], [2, 1], [8, 2], [0, 3], [1, 4], [6, 5], [9, 6], [2, 7], [1, 8], [3, 9] ]), genInstances(testData.ghoniem2, [ [11, 0], [2, 1], [13, 2], [0, 3], [1, 4], [18, 5], [9, 6], [2, 7], [19, 8], [10, 9] ]), genInstances(testData.ghoniem3, [ [12, 0], [2, 1], [17, 2], [5, 3], [1, 4], [8, 5], [9, 6], [14, 7], [17, 8], [0, 9] ]), genInstances(testData.ghoniem4, [ [32, 0], [19, 1], [17, 2], [42, 3], [28, 4], [8, 5], [1, 6], [0, 7], [8, 8], [9, 9] ]), genInstances(testData.ghoniem5, [ [8, 0], [9, 1], [27, 2], [22, 3], [2, 4], [0, 5], [11, 6], [49, 7], [1, 8], [18, 9] ]), genInstances(testData.ghoniem6, [ [18, 0], [7, 1], [45, 2], [20, 3], [12, 4], [3, 5], [19, 6], [39, 7], [9, 8], [25, 9] ]), genInstances(testData.ghoniem7, [ [0, 0], [17, 1], [25, 2], [30, 3], [45, 4], [57, 5], [34, 6], [61, 7], [79, 8], [21, 9] ]), genInstances(testData.ghoniem8, [ [50, 0], [18, 1], [20, 2], [4, 3], [52, 4], [67, 5], [30, 6], [11, 7], [79, 8], [1, 9] ]), genInstances(testData.ghoniem9, [ [0, 0], [28, 1], [42, 2], [5, 3], [12, 4], [61, 5], [39, 6], [10, 7], [7, 8], [81, 9] ]) ]} }3f:T182c,areNeighbors = { let accuracy = (network, bnode, onodes) => (response) => onodes.filter( on => areConnected(network, on, bnode)).length == onodes.length ? (response[0] === 'yes' ? 1 : 0) : (response[0] === 'no' ? 1 : 0); let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let bnode = [i[0]]; let onodes = i[1]; return {inputs : [networkIndex, [bnode,onodes]], accuracy : accuracy(network, i[0], i[1])} })} else return {data : data, instances : instances.map( i=> { let bnode = [i[0]]; let onodes = i[1]; return {inputs : [0, [bnode,onodes]], accuracy : accuracy(data, i[0],i[1])} })} }; return { name : "areNeighbors", question : "Are all orange nodes directly connected to the blue one?", description : "Given a (blue) highlighted node and a few (orange) highlighted node, are all orange nodes connected ot the blue node?", inputs : [ {type : "vis", state : "Network"}, {type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "select", options : ["select","yes","no"]}], instances : [ genInstances(testData.demoNetwork, [ [4, [0, 2, 3, 1]], [0, [4, 5, 6, 2]], [0, [4, 5, 6, 3]], [1, [2, 3, 4, 5]], [2, [0, 1, 4, 6]] ]), genInstances(testData.lesMis, [ [27, [11, 23, 25, 24, 33]], [11, [10, 3, 23, 24, 16]], [29, [23, 27, 11, 38, 12]], [55, [51, 49, 11, 16, 58]], [58, [55, 48, 27, 11, 68]], [49, [26, 11, 54, 55, 56]] ]), genInstances(testData.ingredients, [ [[256], [46, 164, 214, 21, 235]], [[9], [0, 161, 224, 41, 72]], [[239], [97, 185, 227, 229, 111]], [[178], [33, 16, 73, 238, 65]], [[141], [161, 218, 120, 212, 64]], [[141], [161, 212, 89, 180, 77]] ]), genInstances(testData.usair97, [ [77, [66, 78, 117]], // true [295, [66, 111, 117, 122, 149]], [254, [46, 50, 66, 90, 91]], [60, [39, 44, 49, 54, 61, 62]], [74, [46, 64, 73, 86, 10]], [130, [66, 93, 94, 108, 111]] ]), genInstances(testData.ghoniem1, [ [0, [0, 2, 3, 1], 0], [3, [0, 6, 7], 1], //true [0, [4, 5, 6], 2], [4, [0, 6, 7], 3], [0, [13, 3, 8], 4], // true [10, [1, 17], 5], // true [6, [0, 1, 4], 6], [18, [11, 3, 7, 15], 7], [1, [2, 8, 9], 8], [9, [4, 13, 11], 9] // true ]), genInstances(testData.ghoniem2, [ [11, [4, 5, 6, 3], 0], [2, [7, 8, 9, 10], 1], [15, [0, 6, 8, 13], 2], // true [7, [12, 13, 14, 5], 3], [13, [2, 3, 11, 18], 4], // true [12, [1, 3, 6, 17], 5], // true [4, [13, 15, 10, 11], 6], [8, [1, 4, 5, 6, 0], 7], [6, [3, 8, 9, 7], 8], // true [3, [0, 2, 5, 1], 9] ]), genInstances(testData.ghoniem3, [ [9, [14, 15, 4, 19], 0], [11, [1, 3, 18, 15], 1], // true [14, [10, 2, 15, 18], 2], // true [18, [17, 9, 15, 12], 4], [10, [1, 17, 14, 15], 5], // true [16, [0, 2, 11, 19], 6], // true [6, [7, 10, 1, 3], 7], [3, [0, 1, 18, 12], 8], [1, [6, 13, 15, 2], 9] ]), genInstances(testData.ghoniem4, [ [37, [2, 3, 19, 48], 0], // true [42, [7, 14, 16, 33], 1], // true [26, [28, 16, 37, 41], 2], [1, [2, 3, 8, 19], 3], // true [28, [13, 15, 41, 32], 4], // almost [14, [0, 15, 37, 30], 5], [32, [24, 44, 25, 39], 6], [29, [25, 31, 1, 10], 7], [1, [7, 48, 28, 31], 8], // true [42, [45, 6, 2, 49], 9] ]), genInstances(testData.ghoniem5, [ [27, [0, 4, 6, 7], 0], [31, [39, 48, 38, 45], 1], // true [36, [5, 29, 32, 2], 2], [37, [10, 14, 13, 39], 3], [15, [12, 14, 28, 46], 4], // true [33, [12, 18, 24, 29], 5], [4, [8, 37, 1, 6], 6], [19, [34, 3, 22, 27], 7], [12, [10, 41, 48, 14], 8], // true [28, [7, 12, 15, 17], 9] // true ]), genInstances(testData.ghoniem6, [ [49, [29, 19, 13, 3], 0], [16, [6, 9, 11, 12], 1], // true [9, [0, 31, 29, 23], 2], [10, [30, 13, 38, 15], 3], [4, [5, 27, 46, 37], 4], // true [6, [11, 12, 13, 15], 5], // almost [36, [19, 2, 20, 21], 6], // true [47, [29, 0, 13, 23], 7], [16, [24, 25, 27, 3], 8], // true [15, [12, 34, 47, 22], 9] ]), genInstances(testData.ghoniem7, [ [60, [9, 10, 17, 99], 0], // true [48, [82, 87, 91, 90], 1], [28, [76, 8, 81, 98], 2], // almost [47, [41, 39, 77, 29], 3], [53, [22, 24, 54, 60], 4], // true [85, [11, 2, 36, 67], 5], // true [78, [18, 47, 55, 16], 6], [56, [44, 93, 25, 3], 7], [6, [14, 22, 5, 68], 8], // true [63, [88, 89, 92, 94], 9] // true ]), genInstances(testData.ghoniem8, [ [45, [32, 17, 80, 91], 0], [98, [48, 51, 54, 57], 1], // true [4, [0, 1, 2, 33], 2], // true [22, [61, 88, 83, 56], 3], [32, [9, 12, 15, 20], 4], // true [17, [59, 5, 3, 87], 5], [65, [13, 17, 18, 19], 6], // true [28, [2, 12, 13, 19], 7], // true [30, [9, 10, 14, 17], 8], // almost [29, [59, 37, 40, 35], 9] ]), genInstances(testData.ghoniem9, [ [59, [16, 17, 18, 20], 0], // true [93, [14, 17, 18, 19], 1], // true [87, [7, 10, 11, 9], 2], // almost [51, [19, 27, 79, 40], 3], [10, [63, 56, 28, 82], 4], [2, [9, 10, 11, 14], 5], // true [32, [33, 5, 38, 75], 6], [35, [17, 19, 21, 25], 7], // true [63, [94, 45, 54, 11], 8], [3, [87, 54, 23, 82], 9] ]) ] } }40:T16af,selectCommonNeighbors = { //method to compute the accuracy of a selectNeighbors answer let accuracy = (network, nodes) => (response) => { let neighbors1 = getNeighbors(network, nodes[0]); let neighbors2 = getNeighbors(network, nodes[1]); let common = neighbors1.filter( n => neighbors2.indexOf(n) >= 0); if (common.length == 0 && response[0].length == 0) return 1; let nrN = common.length; let errors = 0; //identify number of 'false' positives for (let i=0; i { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let nodes = [i[0],i[1]]; return {inputs : [networkIndex, [nodes]], accuracy : accuracy(network, nodes)} })} else return {data : data, instances : instances.map( i=> { return {inputs : [0, [i]], accuracy : accuracy(data, i)} })} }; return { name : "selectCommonNeighbors", question : "Select all nodes directly connected to both (blue) highlighted ones.", description : "Given two highlighted nodes participant selects their commmon neighbors.", inputs : [ {type : "vis", state : "Network"}, {type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "vis", state : "SelectedNodes"}], instances : [ genInstances(testData.demoNetwork, [ [1, 4], [5, 6], [6, 2], [4, 6], [0, 1], [2, 1], [2, 0], [6, 4], [5, 1], [5, 4] ]), genInstances(testData.lesMis, [ [74, 63], [0, 11], [30, 17], [51, 11], [28, 64], [24, 16], [70, 75], [67, 62], [5, 7], [17, 22] ]), genInstances(testData.ingredients, [ [143, 156], [45, 29], [218, 41], [135, 177], [4, 205], [105, 217], [66, 55], [165, 177], [74, 33], [217, 137] ]), genInstances(testData.usair97, [ [65, 57], [128, 178], [141, 64], [218, 82], [176, 130], [207, 292], [73, 28], [265, 288], [274, 275], [310, 32] ]), genInstances(testData.ecoli, [ [24, 52], [8, 32], [11, 40], [17, 57], [55, 74], [29, 1], [88, 1], [51, 53], [8, 53], [91, 84] ]), genInstances(testData.weavers, [ [46, 28], [13, 23], [42, 36], [61, 47], [42, 3], [10, 8], [34, 37], [10, 8], [46, 62], [42, 40] ]), genInstances(testData.dblp, [ [1, 3], [41, 49], [78, 15], [15, 16], [62, 60], [45, 46], [64, 65], [55, 53], [8, 5], [12, 2] ]), genInstances(testData.ghoniem1, [ [17, 19, 0], [10, 19, 1], [19, 9, 2], [3, 10, 3], [15, 19, 4], [9, 19, 5], [6, 10, 6], [7, 11, 7], [8, 4, 8], [15, 5, 9] ]), genInstances(testData.ghoniem2, [ [18, 14, 0], [14, 5, 1], [4, 3, 2], [7, 2, 3], [17, 14, 4], [13, 4, 5], [15, 13, 6], [5, 13, 7], [7, 18, 8], [3, 18, 9] ]), genInstances(testData.ghoniem3, [ [6, 14, 0], [9, 3, 1], [3, 13, 2], [15, 4, 3], [4, 2, 4], [11, 4, 5], [15, 6, 6], [13, 14, 7], [9, 1, 8], [10, 16, 9] ]), genInstances(testData.ghoniem4, [ [46, 27, 0], [17, 38, 1], [49, 44, 2], [21, 9, 3], [7, 20, 4], [7, 1, 5], [7, 4, 6], [41, 5, 7], [34, 39, 8], [6, 46, 9] ]), genInstances(testData.ghoniem5, [ [2, 34, 0], [31, 47, 1], [11, 22, 2], [5, 19, 3], [7, 20, 4], [2, 35, 5], [17, 43, 6], [37, 15, 7], [14, 31, 8], [13, 49, 9] ]), genInstances(testData.ghoniem6, [ [16, 36, 0], [42, 8, 1], [15, 31, 2], [26, 37, 3], [46, 5, 4], [15, 49, 5], [44, 30, 6], [23, 36, 7], [15, 19, 8], [34, 31, 9] ]), genInstances(testData.ghoniem7, [ [75, 91, 0], [41, 91, 1], [7, 87, 2], [78, 32, 3], [29, 16, 4], [34, 13, 5], [11, 37, 6], [78, 31, 7], [5, 92, 8], [44, 19, 9] ]), genInstances(testData.ghoniem8, [ [27, 89, 0], [71, 20, 1], [34, 20, 2], [71, 72, 3], [89, 14, 4], [64, 40, 5], [24, 92, 6], [2, 70, 7], [61, 19, 8], [80, 92, 9] ]), genInstances(testData.ghoniem9, [ [24, 1, 0], [79, 4, 0], [69, 60, 1], [73, 32, 2], [35, 9, 3], [45, 59, 4], [9, 51, 5], [12, 58, 6], [18, 17, 7], [49, 96, 8], [75, 67, 9] ]) ]} }41:T1424,commonNeighbor = { let accuracy = (network, nodes) => (response) => { let ng1 = getNeighbors(network,nodes[0]); let ng2 = getNeighbors(network,nodes[1]); for (let i=0; i { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let nodes = [i[0],i[1]]; return {inputs : [networkIndex, [nodes]], accuracy : accuracy(network, nodes)} })} else return {data : data, instances : instances.map( i=> { let nodes = [i[0],i[1]]; return {inputs : [0, [nodes]], accuracy : accuracy(data, nodes)} })} }; return{ name : "commonNeighbor", question : "Do the two blue highlighted nodes have a common neighbour?", description : "Given two highlighted nodes, determine if they have a common neighbour", inputs : [ { type : "vis", state :"Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "select", options : ["select","yes","no"]}], instances : [ genInstances(testData.demoNetwork, [ [3, 2], [5, 1], [5, 2], [5, 3], [4, 5] ]),, genInstances(testData.lesMis, [ [7, 2], [17, 42], [9, 47], [26, 70], [74, 10], [34, 68] ]), genInstances(testData.ingredients, [ [131, 13], [14, 77], [30, 176], [48, 93], [108, 45], [144, 244] ]), genInstances(testData.usair97, [ [44, 305], [233, 93], [32, 22], [8, 10], [224, 240], [313, 260], [298, 65], [206, 173], [177, 252], [246, 0] ]), genInstances(testData.ecoli, [ [74, 68], // [42, 1], // [14, 1], // [96, 10], [47, 23], [34, 94], [21, 96], [94, 9], [48, 90], [5, 5] // ]), genInstances(testData.weavers, [ [14, 15], // [16, 63], [21, 36], // [25, 53], // [56, 7], // [37, 31], // [40, 21], // [6, 36], [4, 5], // [20, 18] // ]), genInstances(testData.dblp, [ [81, 65], [61, 65], // [27, 29], // [7, 9], // [3, 39], [66, 26], [33, 33], // [23, 71], [81, 35], [53, 39] ]), genInstances(testData.ghoniem1, [ [7, 14, 0], // [2, 8, 1], [14, 0, 2], // [17, 3, 3], [11, 6, 4], [15, 6, 5], // [13, 13, 6], [9, 7, 7], // [8, 7, 8], [5, 8, 9] ]), genInstances(testData.ghoniem2, [ [8, 11, 0], // [1, 9, 1], // [4, 4, 2], [17, 7, 3], // [6, 17, 4], [19, 5, 5], // [11, 7, 6], [16, 4, 7], // [2, 2, 8], [6, 14, 9] ]), genInstances(testData.ghoniem3, [ [9, 8, 0], [18, 8, 1], // [2, 15, 2], [1, 16, 3], [10, 18, 4], // [4, 16, 5], [8, 12, 6], // [10, 8, 7], // [8, 12, 8], [6, 5, 9] ]), genInstances(testData.ghoniem4, [ [49, 24, 0], // [3, 10, 1], [28, 43, 2], // [17, 38, 3], // [34, 43, 4], [23, 20, 5], // [7, 21, 6], [35, 21, 7], // [18, 48, 8], [8, 17, 9] // ]), genInstances(testData.ghoniem5, [ [16, 17, 0], // [12, 22, 1], // [19, 4, 2], [3, 10, 3], [31, 37, 4], // [47, 16, 5], [32, 36, 6], // [16, 21, 7], [32, 41, 8], [12, 29, 9] ]), genInstances(testData.ghoniem6, [ [20, 4, 0], [29, 14, 1], // [8, 27, 2], [39, 47, 3], // [16, 5, 4], [16, 22, 5], [45, 20, 6], [40, 27, 7], [22, 23, 8], [16, 10, 9] ]), genInstances(testData.ghoniem7, [ [26, 35, 0], // [6, 89, 1], // [92, 50, 2], [0, 77, 3], // [19, 38, 4], [55, 62, 5], [16, 12, 6], // [63, 10, 7], [99, 3, 8], [68, 64, 9] ]), genInstances(testData.ghoniem8, [ [58, 85, 0], [24, 2, 1], // [70, 41, 2], [24, 71, 3], // [86, 66, 4], // [22, 59, 5], // [81, 4, 6], [98, 16, 7], [29, 20, 8], // [29, 11, 9] ]), genInstances(testData.ghoniem9, [ [80, 37, 0], [48, 19, 1], // [30, 63, 2], [82, 25, 3], // [4, 21, 4], // [36, 68, 5], // [82, 92, 6], [69, 44, 7], // [99, 69, 8], [53, 78, 9] ]) ] } }42:T2755,namedCommonNeighbor = { //method to compute the accuracy of a selectNeighbors answer let accuracy = (network, nodes) => (response) => { let neighbors1 = getNeighbors(network, nodes[0]); let neighbors2 = getNeighbors(network, nodes[1]); let common = neighbors1.filter( n => neighbors2.indexOf(n) >= 0); let responseIndex = network.nodes.findIndex( n => n.label === response[0]); if (responseIndex < 0 || common.length == 0) return -1; //bad task if (common.indexOf(responseIndex) >= 0) return 1; else return 0; }; //instaces: 2 node indeces + 3 labels [+ network index]; the first label should be the correct answer! let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[5]; let network = data[networkIndex]; let highlightedNodes = [i[0],i[1]]; let labels = d3.shuffle([i[2],i[3],i[4]]); return {inputs : [networkIndex, [highlightedNodes], labels], accuracy : accuracy(network, highlightedNodes)} })} else return {data : data, instances : instances.map( i=> { let highlightedNodes = [i[0],i[1]]; let labels = [i[2],i[3],i[4]]; return {inputs : [0, [highlightedNodes], labels], accuracy : accuracy(data, i)} })} }; return { name : "namedCommonNeighbor", question : "Which of the three nodes is directly connected to both (blue) highlighted nodes?", description : "Given two highlighted nodes and three named nodes, participant selects which named node is a common neighbour for the higlighted ones.", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}, { type : "output"}], outputs : [ {type : "select", options : "$2"}], instances : [ //Important: first label should be the correct answer //IMPORTANT! Copy the genInstances code into rememberNeighbors! genInstances(testData.demoNetwork, [ [0, 1, "node1", "node2", "node3"], [0, 3, "node5", "node7", "node6"], [5, 1, "node1", "node3", "node3"], [4, 5, "node1", "node3", "node4"], [6, 4, "node1", "node4", "node6"] ]), genInstances(testData.lesMis, [ [72, 58, "Javert", "Prouvaire", "Mme.Burgon"], [16, 61, "Marius", "BaronessT", "Bahorel"], [60, 69, "Gavroche", "Montparnasse", "Isabeau"], [37, 33, "Valjean", "Child2", "Magnon"], [41, 59, "Marius", "Cosette", "BaronessT"], [50, 27, "Mme.Thenardier", "Geborand", "Fauchelevent"], [66, 55, "Bossuet", "Bossuet", "Favourite"], [26, 53, "Mlle.Gillenormand", "Marius", "Montparnasse"], [14, 10, "Valjean", "Gribier", "Grantaire"], [11, 5, "Myriel", "Blacheville", "Gavroche"] ]), genInstances(testData.ingredients, [ [152, 115, "saffron", "rosemary", "grape"], [28, 34, "malt", "chamomile", "rice"], [224, 218, "apricot", "sumac", "meat"], [210, 173, "currant", "octopus", "sherry"], [213, 161, "rose", "oatmeal", "olive_oil"], [0, 79, "apricot", "currant", "octopus"], [222, 232, "cumin", "chamomile", "rice"], [176, 37, "sage", "sumac", "wheat"], [7, 33, "potato", "feta_cheese", "milk"], [11, 185, "radish", "green_tea", "avocado"] ]), genInstances(testData.usair97, [ [227, 315, "San Francisco Intl", "Spokane Intl", "Seattle-Tacoma In"], [158, 195, "Lambert-St Louis Intl", "Abilene Regional", "Miami Intl"], [ 7, 283, "Chicago O'hare Intl", "Yuma Mcas/Yuma Intl", "Dallas Love Field" ], [ 47, 150, "Portland Intl", "Rio Grande Valley Intl", "Keahole-Kona Intll" ], [ 25, 175, "Seattle-Tacoma Intl", "Santa Barbara Muni", "Burbank-Glendale-Pasadena" ], [ 194, 300, "Lambert-St Louis Intl", "Johnston Atoll", "Lafayette Regional" ], [203, 130, "Chicago O'hare Intl", "Pago Pago Intl", "Cyril E King"], [32, 7, "Juneau Intl", "Chicago O'hare Intl", "Pago Pago Intl"], [15, 200, "Anchorage Intl", "Waco Regional", "Daytona Beach Intl"], [15, 4, "Anchorage Intl", "Lanai", "Gregg County"] ]), genInstances(testData.ecoli, [ [72, 49, "1", "3", "10"], [18, 17, "1", "16", "23"], [46, 23, "1", "67", "96"], [50, 45, "48", "93", "1"], [51, 9, "0", "1", "2"], [96, 91, "94", "1", "8"], [1, 42, "41", "65", "70"], [7, 1, "6", "83", "25"], [87, 31, "1", "14", "13"], [93, 62, "0", "1", "3"] ]), genInstances(testData.weavers, [ [26, 40, "UMRO", "VMWZ", "UMUV"], [62, 14, "VMVW", "WWFM", "VMZU"], [4, 12, "VMWZ", "VMVE", "UMOE"], [41, 2, "VMUU", "FMUU", "UMWW"], [14, 1, "FMOY", "UMPP", "UMWW"], [29, 1, "VMLZ", "FMBV", "VMPB"], [24, 7, "VMVE", "VMWZ", "VMZU"], [17, 57, "VMVE", "FMLL", "VMYB"], [35, 7, "VMVE", "ZYVM", "FMLZ"], [51, 14, "VMZV", "UMUV", "MVLE"] ]), genInstances(testData.dblp, [ [51, 71, "Andreas Kerren", "Bruno Pinaud", "Karsten Klein"], [76, 18, "Karsten Klein", "Bruno Pinaud", "Hsiang-Yun Wu"], [61, 50, "Romain Bourqui", "Helen C. Purchase", "Karsten Klein"], [33, 25, "Weidong Huang", "Seok-Hee Hong", "Peter Eades"], [56, 52, "Fabien Jourdan", "Nathalie Poupin", "David Auber"], [18, 3, "Karsten Klein", "Stefan Felsner", "Da Yan"], [14, 29, "Karsten Klein", "Michael Krone", "Stephen G. Kobourov"], [5, 0, "Hsiang-Yun Wu", "Florian Ganglberger", "Nicolas Swoboda"], [3, 12, "Hsiang-Yun Wu", "Stefan P. Feyer", "Bruno Pinaud"], [45, 48, "Fabrizio Montecchiani", "Bernice E. Rogowitz", "Oh-Hyun Kwon"] ]), genInstances(testData.ghoniem1, [ [0, 11, "Q", "M", "R", 0], [3, 14, "A", "B", "Q", 1], [5, 2, "N", "R", "K", 2], [14, 4, "R", "T", "B", 3], [8, 13, "A", "B", "N", 4], [4, 15, "D", "F", "C", 5], [13, 17, "B", "A", "R", 6], [5, 4, "L", "E", "K", 7], [17, 1, "Q", "I", "G", 8], [0, 3, "B", "S", "R", 9] ]), genInstances(testData.ghoniem2, [ [2, 9, "A", "K", "L", 0], [12, 2, "J", "B", "Q", 1], [6, 3, "A", "R", "K", 2], [18, 0, "E", "K", "B", 3], [9, 13, "B", "H", "M", 4], [4, 15, "D", "F", "C", 5], [10, 7, "Q", "A", "E", 6], [4, 5, "C", "F", "G", 7], [16, 2, "K", "O", "G", 8], [9, 11, "G", "S", "R", 9] ]), genInstances(testData.ghoniem3, [ [2, 9, "A", "K", "L", 0], [12, 2, "J", "B", "Q", 1], [6, 3, "A", "R", "K", 2], [18, 0, "E", "K", "B", 3], [9, 13, "B", "H", "M", 4], [4, 15, "D", "F", "C", 5], [10, 7, "Q", "A", "E", 6], [4, 5, "C", "F", "G", 7], [16, 2, "K", "O", "G", 8], [9, 11, "G", "S", "R", 9] ]), genInstances(testData.ghoniem4, [ [23, 2, "B5", "A3", "A8", 0], [33, 31, "A8", "B2", "E9", 1], [40, 4, "D9", "C9", "B8", 2], [7, 37, "B2", "E1", "B4", 3], [0, 3, "A6", "D6", "B6", 4], [29, 5, "A8", "D2", "A4", 5], [10, 25, "A4", "A0", "E2", 6], [49, 17, "A7", "F2", "E9", 7], [14, 10, "A9", "E1", "E8", 8], [37, 9, "A0", "C8", "B9", 9] ]), genInstances(testData.ghoniem5, [ [31, 2, "A0", "A3", "A8", 0], [12, 14, "A1", "B7", "E9", 1], [0, 3, "A2", "C9", "B8", 2], [7, 1, "A8", "E1", "B4", 3], [41, 2, "A4", "D6", "B6", 4], [49, 6, "A2", "D2", "A4", 5], [13, 0, "A1", "B0", "E2", 6], [49, 11, "A1", "F2", "E8", 7], [44, 11, "A4", "E1", "E5", 8], [37, 9, "A0", "C8", "B9", 9] ]), genInstances(testData.ghoniem6, [ [42, 0, "A1", "A3", "A8", 0], [9, 3, "A2", "B9", "E9", 1], [24, 1, "A0", "C9", "B8", 2], [7, 0, "A1", "E1", "B1", 3], [41, 2, "A3", "D6", "B6", 4], [49, 6, "A2", "D2", "A4", 5], [13, 1, "A0", "B0", "E2", 6], [29, 1, "A0", "F2", "E8", 7], [41, 0, "A1", "E1", "E5", 8], [0, 6, "A2", "C8", "B9", 9] ]), genInstances(testData.ghoniem7, [ [30, 0, "A4", "A3", "A8", 0], [13, 33, "C7", "B9", "E9", 1], [6, 4, "A0", "C9", "B8", 2], [24, 8, "A1", "E1", "B1", 3], [85, 6, "B1", "D6", "B6", 4], [0, 1, "E0", "D2", "A4", 5], [13, 2, "B4", "B0", "E2", 6], [51, 22, "C6", "F2", "E8", 7], [92, 5, "A6", "E1", "E5", 8], [16, 7, "B4", "C8", "B9", 9] ]), genInstances(testData.ghoniem8, [ [22, 7, "A2", "B3", "A8", 0], [0, 12, "A2", "B9", "E9", 1], [85, 1, "A0", "C9", "B8", 2], [30, 5, "A8", "E1", "B1", 3], [41, 2, "A3", "D6", "B6", 4], [78, 0, "A3", "D2", "A4", 5], [4, 0, "A2", "B0", "E2", 6], [12, 19, "B0", "F2", "E8", 7], [10, 13, "A6", "E1", "E5", 8], [82, 8, "A3", "C8", "B9", 9] ]), genInstances(testData.ghoniem9, [ [42, 0, "A1", "A3", "A8", 0], [9, 3, "A2", "B9", "E9", 1], [24, 1, "A0", "C9", "B8", 2], [7, 5, "A3", "E1", "B1", 3], [57, 6, "A2", "D6", "B6", 4], [95, 5, "A2", "D2", "A4", 5], [4, 5, "A2", "B0", "E2", 6], [50, 7, "A5", "F2", "E8", 7], [13, 7, "A2", "E1", "E5", 8], [0, 2, "A1", "C8", "B9", 9] ]) ]} }43:T1025,withinTwoHops = { let accuracy = (network, node) => (response) => { let r = response[0]; if (r == null) return 0; //get the highlighted node's direct neighbors into ret let ret = []; let ns1 = getNeighbors(network, node); ns1.map( n => ret.push(n)); //add their neighbors (if they're not already in) for (let i=0; i n == ns2[j]) < 0 && ns2[j] != node) ret.push(ns2[j]); } return 1 - Math.abs(ret.length - r)/ret.length; }; let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i => { let networkIndex = i[1]; let network = data[networkIndex]; let node = i[0]; return {inputs : [networkIndex, [[node]]], accuracy : accuracy(network, node)} })} else return {data : data, instances : instances.map( i=> ({inputs : [0, [[i]]], accuracy : accuracy(data, i)}))} }; return { name : "withinTwoHops", question : "Estimate or count how many nodes are within two hops of the blue one", description : "Given a highlighted node, approximately how many nodes are within two hops? Can be delivered with a short response time to enforce estimation.", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "integer"}], instances : [ genInstances(testData.demoNetwork, [6, 1, 4, 5, 2]), genInstances(testData.lesMis, [7, 17, 27, 32, 35, 5, 11]), genInstances(testData.ingredients, [50, 120, 34, 76, 7, 90, 30]), genInstances( testData.usair97, [225, 160, 150, 128, 230, 104, 62, 90, 6, 198] ), genInstances(testData.ecoli, [3, 46, 60, 75, 36, 12, 46, 91, 53, 77]), genInstances(testData.weavers, [58, 42, 23, 57, 6, 38, 45, 2, 60, 49]), genInstances(testData.dblp, [39, 34, 31, 21, 11, 53, 17, 21, 78, 18]), genInstances(testData.ghoniem1, [ [13, 0], [2, 1], [5, 2], [15, 3], [7, 4], [4, 5], [10, 6], [6, 7], [15, 8], [6, 9] ]), genInstances(testData.ghoniem2, [ [15, 0], [12, 1], [3, 2], [2, 3], [15, 4], [1, 5], [17, 6], [12, 7], [11, 8], [2, 9] ]), genInstances(testData.ghoniem3, [ [10, 0], [18, 1], [13, 2], [4, 3], [3, 4], [4, 5], [18, 6], [19, 7], [14, 8], [10, 9] ]), genInstances(testData.ghoniem4, [ [36, 0], [45, 1], [49, 2], [9, 3], [30, 4], [44, 5], [3, 6], [2, 7], [41, 8], [45, 9] ]), genInstances(testData.ghoniem5, [ [11, 0], [28, 1], [12, 2], [4, 3], [9, 4], [25, 5], [1, 6], [40, 7], [40, 8], [0, 9] ]), genInstances(testData.ghoniem6, [ [37, 0], [28, 1], [11, 2], [8, 3], [28, 4], [44, 5], [6, 6], [1, 7], [30, 8], [15, 9] ]), genInstances(testData.ghoniem7, [ [73, 0], [69, 1], [45, 2], [70, 3], [1, 4], [87, 5], [1, 6], [31, 7], [7, 8], [93, 9] ]), genInstances(testData.ghoniem8, [ [82, 0], [42, 1], [48, 2], [66, 3], [93, 4], [35, 5], [33, 6], [7, 7], [37, 8], [98, 9] ]), genInstances(testData.ghoniem9, [ [54, 0], [5, 1], [68, 2], [25, 3], [6, 4], [5, 5], [25, 6], [7, 7], [15, 8], [82, 9] ]) ] } }44:T13b8,findPath = { let accuracy = (network, nodes) => (responses) => { let trueLength = getPathLength(network, nodes[0], nodes[1]); if (responses[0] === "5+") return (trueLength >= 5) ? true : false; else if (parseInt(responses[0]) == trueLength) return 1; return 0; }; let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let nodes = [i[0],i[1]]; return {inputs : [networkIndex, [nodes]], accuracy : accuracy(network, nodes)} })} else return {data : data, instances : instances.map( i=> { let nodes = [i[0],i[1]]; return {inputs : [0, [nodes]], accuracy : accuracy(data, nodes)} })} }; return { name : "findPath", question : "How many hops to go from one of the blue nodes to the other?", description : "Given two highlighted nodes, determine the length of the path between them", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "select", options : ["1","2","3","4","5+"]}], instances : [ genInstances(testData.demoNetwork, [ [2, 5], [3, 5], [1, 2], [3, 0], [1, 4] ]), genInstances(testData.lesMis, [ [60, 70], [57, 21], [5, 21], [37, 5], [46, 48], [20, 40], [4, 57] ]), genInstances(testData.ingredients, [ [134, 77], [160, 146], [238, 8], [185, 30], [124, 239], [26, 111], [46, 13], [0, 90] ]), genInstances(testData.usair97, [ [30, 224], [117, 196], [275, 277], [287, 151], [101, 62], [127, 111], [197, 225], [196, 319], [291, 323], [303, 186] ]), genInstances(testData.ecoli, [ [82, 58], [89, 1], [16, 80], [35, 13], [27, 56], [83, 84], [57, 43], [2, 88], [88, 75], [6, 1] ]), genInstances(testData.weavers, [ [40, 12], [41, 15], [31, 60], [8, 56], [8, 47], [7, 42], [29, 37], [48, 1], [19, 17], [7, 23] ]), genInstances(testData.dblp, [ [35, 72], [69, 14], [19, 23], [29, 13], [8, 16], [13, 4], [53, 49], [44, 31], [32, 35], [64, 31] ]), genInstances(testData.ghoniem1, [ [7, 12, 0], [13, 18, 1], [3, 1, 2], [0, 19, 3], [17, 14, 4], [2, 3, 5], [12, 4, 6], [18, 9, 7], [11, 7, 8], [7, 18, 9] ]), genInstances(testData.ghoniem2, [ [10, 3, 0], [6, 12, 1], [3, 9, 2], [14, 7, 3], [6, 18, 4], [17, 12, 5], [14, 0, 6], [4, 12, 7], [11, 1, 8], [13, 9, 9] ]), genInstances(testData.ghoniem3, [ [12, 14, 0], [12, 14, 1], [13, 19, 2], [4, 12, 3], [4, 6, 4], [7, 0, 5], [5, 17, 6], [10, 12, 7], [14, 6, 8], [0, 1, 9] ]), genInstances(testData.ghoniem4, [ [23, 49, 0], [37, 8, 1], [11, 11, 2], [3, 23, 3], [0, 49, 4], [22, 37, 5], [12, 7, 6], [39, 24, 7], [17, 7, 8], [36, 24, 9] ]), genInstances(testData.ghoniem5, [ [31, 24, 0], [49, 15, 1], [43, 46, 2], [29, 8, 3], [8, 39, 4], [29, 16, 5], [38, 29, 6], [27, 27, 7], [7, 25, 8], [18, 34, 9] ]), genInstances(testData.ghoniem6, [ [31, 37, 0], [20, 32, 1], [0, 12, 2], [28, 13, 3], [42, 42, 4], [11, 48, 5], [27, 4, 6], [25, 16, 7], [6, 49, 8], [46, 31, 9] ]), genInstances(testData.ghoniem7, [ [79, 43, 0], [19, 79, 1], [45, 12, 2], [32, 44, 3], [36, 92, 4], [52, 36, 5], [86, 88, 6], [23, 35, 7], [22, 73, 8], [35, 35, 9] ]), genInstances(testData.ghoniem8, [ [49, 78, 0], [61, 62, 1], [19, 43, 2], [0, 99, 3], [34, 66, 4], [62, 1, 5], [76, 75, 6], [76, 24, 7], [69, 81, 8], [86, 0, 9] ]), genInstances(testData.ghoniem9, [ [28, 10, 0], [40, 69, 1], [6, 70, 2], [37, 96, 3], [36, 58, 4], [23, 54, 5], [1, 54, 6], [39, 53, 7], [32, 62, 8], [26, 83, 9] ]) ] } }45:T1a5d,comparePaths = { //method to compute the accuracy of a quickPath answer let accuracy = (network, nodes) => (response) => { let pl1 = getPathLength(network,nodes[0][0],nodes[0][1]); let pl2 = getPathLength(network,nodes[1][0],nodes[1][1]); if (pl1 == pl2) return -1; if ((pl1 < pl2 && response[0] === "blue") || (pl2 < pl1 && response[0] === "orange")) return 1; else return 0; }; //helper function to assemble instances from a dataset and //associated instances. Instances are triples (node index 1, //node index 2, network index) for multi networks or tuples for //(node 1, node 2) for regular networks let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[4]; let network = data[networkIndex]; let pair1 = [i[0],i[1]]; let pair2 = [i[2],i[3]]; return {inputs : [networkIndex, [pair1,pair2]], accuracy : accuracy(network, [pair1,pair2])} })} else return {data : data, instances : instances.map( i=> { let pair1 = [i[0],i[1]]; let pair2 = [i[2],i[3]]; return {inputs : [0, [pair1,pair2]], accuracy : accuracy(data, [pair1,pair2])} })} }; return { name : "comparePaths", question : "Which pair of nodes is connected by a shorter path?", description : "Given two pairs of nodes the participant decides which pair is closer in terms of network topology. This task is meant to be delivered with a tight time-constraint (e.g., 10 seconds) to ensure that path lengths are estimated rather than computed. Binary answer.", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "select", options : ["select","blue","orange"]}], instances : [ genInstances(testData.demoNetwork, [ [0, 1, 2, 3], [0, 1, 5, 4], [5, 2, 1, 0], [0, 1, 6, 2], [5, 3, 4, 1] ]), genInstances(testData.ingredients, [ [82, 199, 117, 39], [201, 34, 150, 250], [98, 84, 91, 117], [1, 154, 60, 87], [73, 102, 113, 249], [73, 33, 68, 25], [199, 145, 238, 117], [100, 154, 9, 41], [164, 113, 61, 247], [94, 199, 92, 30] ]), genInstances(testData.usair97, [ [243, 181, 129, 82, 0], [138, 298, 258, 56, 1], [175, 116, 1, 182, 2], [132, 160, 197, 327, 3], [84, 282, 150, 328, 4], [149, 191, 323, 223, 5], [117, 220, 211, 85, 6], [163, 329, 256, 320, 7], [310, 104, 33, 264, 8], [121, 210, 198, 268, 9] ]), genInstances(testData.ecoli, [ [0, 13, 5, 9], [8, 11, 9, 12], [0, 3, 11, 13], [3, 13, 5, 1], [2, 14, 8, 11], [3, 19, 13, 1], [18, 5, 19, 10], [1, 7, 1, 13], [3, 7, 11, 4], [6, 16, 11, 13] ]), genInstances(testData.weavers, [ [12, 19, 5, 7], [3, 7, 0, 13], [4, 14, 11, 4], [5, 17, 14, 15], [6, 5, 0, 13], [11, 0, 9, 17], [19, 5, 7, 5], [19, 9, 6, 14], [5, 17, 11, 1], [9, 12, 6, 3] ]), genInstances(testData.dblp, [ [6, 7, 8, 1], [14, 1, 6, 10], [13, 11, 14, 6], [4, 9, 5, 16], [8, 6, 6, 13], [1, 19, 0, 18], [17, 2, 15, 6], [2, 5, 4, 1], [19, 15, 14, 16], [18, 12, 19, 16] ]), genInstances(testData.ghoniem1, [ [1, 16, 2, 8, 0], [7, 16, 11, 4, 1], [5, 15, 0, 18, 2], [16, 3, 4, 18, 3], [16, 0, 9, 13, 4], [18, 2, 15, 17, 5], [9, 6, 2, 3, 6], [7, 10, 2, 12, 7], [13, 4, 11, 9, 8], [15, 12, 14, 8, 9] ]), genInstances(testData.ghoniem2, [ [12, 3, 14, 5, 0], [1, 15, 13, 8, 1], [10, 19, 14, 5, 2], [3, 14, 2, 13, 3], [11, 19, 15, 1, 4], [5, 9, 14, 18, 5], [13, 14, 7, 18, 6], [9, 3, 11, 1, 7], [8, 19, 6, 0, 8], [7, 9, 12, 19, 9] ]), genInstances(testData.ghoniem3, [ [13, 0, 12, 17, 0], [16, 1, 17, 3, 1], [18, 16, 14, 10, 2], [1, 6, 0, 12, 3], [14, 7, 13, 3, 4], [19, 3, 15, 12, 5], [14, 8, 17, 0, 6], [17, 19, 16, 3, 7], [4, 12, 14, 8, 8], [0, 2, 19, 5, 9] ]), genInstances(testData.ghoniem4, [ [16, 17, 24, 19, 0], [27, 45, 3, 32, 1], [28, 24, 30, 14, 2], [18, 17, 48, 41, 3], [16, 35, 15, 43, 4], [42, 5, 40, 36, 5], [0, 44, 47, 30, 6], [29, 36, 42, 37, 7], [37, 23, 48, 10, 8], [19, 36, 22, 32, 9] ]), genInstances(testData.ghoniem5, [ [2, 29, 23, 30, 0], [46, 34, 16, 38, 1], [28, 14, 43, 37, 2], [21, 33, 0, 27, 3], [33, 39, 28, 0, 4], [33, 34, 3, 40, 5], [17, 21, 29, 23, 6], [21, 30, 16, 29, 7], [4, 31, 5, 33, 8], [49, 47, 16, 14, 9] ]), genInstances(testData.ghoniem6, [ [38, 34, 8, 17, 0], [28, 25, 29, 39, 1], [7, 9, 24, 10, 2], [2, 9, 22, 3, 3], [27, 12, 18, 25, 4], [12, 2, 17, 37, 5], [4, 2, 20, 28, 6], [40, 29, 48, 25, 7], [20, 31, 12, 23, 8], [42, 4, 14, 33, 9] ]), genInstances(testData.ghoniem7, [ [58, 94, 83, 45, 0], [0, 91, 72, 64, 1], [46, 36, 92, 35, 2], [67, 86, 24, 17, 3], [42, 70, 87, 35, 4], [45, 33, 31, 48, 5], [10, 60, 79, 86, 6], [34, 94, 31, 10, 7], [25, 98, 31, 67, 8], [93, 41, 12, 65, 9] ]), genInstances(testData.ghoniem8, [ [19, 90, 50, 20, 0], [66, 49, 43, 76, 1], [22, 55, 25, 3, 2], [84, 0, 23, 97, 3], [89, 24, 90, 47, 4], [50, 84, 34, 26, 5], [12, 71, 34, 63, 6], [22, 63, 46, 68, 7], [19, 34, 3, 37, 8], [24, 22, 87, 31, 9] ]), genInstances(testData.ghoniem9, [ [22, 58, 6, 88, 0], [79, 93, 33, 4, 1], [81, 36, 25, 50, 2], [84, 8, 55, 52, 3], [19, 95, 35, 67, 4], [99, 96, 97, 81, 5], [44, 76, 30, 92, 6], [7, 61, 70, 68, 7], [64, 59, 75, 53, 8], [58, 18, 38, 8, 9] ]) ]} }46:T198d,rememberConnection = { //method to compute the accuracy of a quickPath answer let accuracy = (network, nodes) => (response) => { let c = areConnected(network,nodes[0],nodes[1]); if ((c && response[0] === "connected") || (!c && response[0] === "not connected")) return 1; else return 0; }; let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[2]; let network = data[networkIndex]; let ids = [i[0],network.nodes.findIndex(n => i[1] == n.label)]; let labels = [network.nodes[i[0]].label,i[1]]; return {inputs : [networkIndex, labels[0], labels[1]], accuracy : accuracy(network, ids)} })} else return {data : data, instances : instances.map( i=> { let ids = [i[0],data.nodes.findIndex(n => i[1] == n.label)]; let labels = [data.nodes[i[0]].label,i[1]]; return {inputs : [0, labels[0], labels[1]], accuracy : accuracy(data, ids)} })} }; return { name : "rememberConnection", question : "We've asked you previously whether $1 and $2 were connected. Can you remember - without the visualisation - if they were?", description : "Can participants remember their earlier answer to a namedConnection task? This task should be paired with and delivered some time after the nameConnection task (instances are paired if delivered in order).", inputs : [ { type : "vis", state : "Network"}, { type : "question"}, { type : "question"}], outputs : [ {type : "select", options : ["select","connected","not connected"]}], instances : [ //IMPORTANT! the generation of instances should be copied from the namedConnection task - but //cannot be shared with it since the genInstances methods are different genInstances(testData.demoNetwork, [ [2, "node1"], [2, "node6"], [0, "node5"], [2, "node2"], [3, "node1"] ]), genInstances(testData.lesMis, [ [55, "Mabeuf"], [74, "Gavroche"], [28, "Jondrette"], [54, "BaronessT"], [23, "Bamatabois"], [7, "Joly"], [58, "Bossuet"] ]), genInstances(testData.ingredients, [ [162, "nira"], [124, "parsnip"], [182, "tea"], [43, "cranberry"], [56, "cayenne"], [221, "cumin"], [44, "plum"] ]), genInstances(testData.usair97, [ [87, "Philadelphia Intl"], [289, "Dallas/Fort Worth Intl"], // connected, [280, "Dallas/Fort Worth Intl"], // connected, [161, "Monroe Regional"], [201, "Helena Regional"], [322, "Williamson County Regional"], [231, "Bush Field"], [307, "Houston Intercontinental"], // connected, [126, "Tallahassee Regional"], [293, "Memphis Intl"] ]), genInstances(testData.ecoli, [ [16, "11"], // connected, [25, "72"], [8, "88"], [45, "39"], // connected, [15, "53"], [89, "84"], // connected, [20, "29"], [71, "80"], [75, "1"], // connected, [50, "24"] ]), genInstances(testData.weavers, [ [30, "VMZO"], [5, "VMUU"], // connected, [37, "FMBV"], [2, "VMPO"], // connected, [12, "VMOZ"], [26, "FMBL"], // connected, [33, "YYVM"], [43, "UMOE"], // connected, [3, "FMOY"], // connected, [14, "FMLL"] // connected ]), genInstances(testData.dblp, [ [25, "Marnijati Torkel"], // connected, [27, "Peter Eades"], // connected, [78, "Bruno Pinaud"], // connected, [24, "Wulf Haubensak"], [30, "Marnijati Torkel"], // connected, [55, "Matthew O. Ward"], [45, "Giuseppe Liotta"], // connected, [26, "Kai Xu"], [66, "Takafumi Saito"], // connected, [13, "Fabien Jourdan"] ]), genInstances(testData.ghoniem1, [ [2, "A", 0], [2, "K", 1], [0, "F", 2], [1, "A", 3], [5, "C", 4], [13, "B", 5], [1, "M", 6], [10, "Q", 7], [0, "H", 8], [6, "M", 9] ]), genInstances(testData.ghoniem2, [ [2, "R", 0], [2, "H", 1], [19, "G", 2], [7, "O", 4], [13, "K", 5], [9, "D", 6], [13, "N", 7], [0, "P", 8], [3, "R", 9] ]), genInstances(testData.ghoniem3, [ [4, "A", 0], [10, "P", 1], [8, "N", 2], [7, "D", 4], [13, "K", 5], [9, "D", 6], [13, "N", 7], [4, "A", 8], [13, "T", 9] ]), genInstances(testData.ghoniem4, [ [4, "D4", 0], [10, "E1", 1], [8, "C2", 2], [7, "D5", 4], [13, "A0", 5], [9, "D1", 6], [13, "E7", 7], [4, "A3", 8], [13, "C0", 9] ]), genInstances(testData.ghoniem5, [ [4, "D4", 0], [10, "B7", 1], [18, "C2", 2], [27, "D5", 4], [48, "A5", 5], [29, "D9", 6], [20, "E7", 7], [40, "C3", 8], [3, "E0", 9] ]), genInstances(testData.ghoniem6, [ [12, "A5", 0], [6, "B3", 1], [39, "C9", 2], [8, "D1", 3], [10, "E4", 4], [7, "A2", 5], [19, "B6", 6], [5, "C0", 7], [14, "D8", 8], [11, "E3", 9] ]), genInstances(testData.ghoniem7, [ [15, "G8", 0], [15, "F1", 1], [12, "C5", 2], [52, "H2", 3], [20, "J6", 4], [10, "A7", 5], [7, "E9", 6], [37, "B2", 7], [14, "D4", 8], [6, "I3", 9] ]), genInstances(testData.ghoniem8, [ [18, "B4", 0], [9, "H7", 1], [12, "E6", 2], [7, "F9", 3], [16, "G3", 4], [55, "A8", 5], [10, "I2", 6], [83, "C1", 7], [20, "J5", 8], [6, "D0", 9] ]), genInstances(testData.ghoniem9, [ [5, "C7", 0], [12, "E5", 1], [78, "J3", 2], [34, "B2", 3], [9, "J9", 4], [23, "F8", 5], [56, "A1", 6], [17, "I0", 7], [98, "D4", 8], [15, "G6", 9] ]) ] } }47:T2570,rememberNeighbors = { //method to compute the accuracy of a rememberNeighbors answer let accuracy = (network, nodes) => (response) => (response[0][0] == nodes[0] && response[0][1] == nodes[1]) || response[0][0] == nodes[1] && response[0][1] == nodes[0]; let genInstances = (data, instances) => { if (Array.isArray(data)) return {data : data, instances : instances.map( i=> { let networkIndex = i[5]; let network = data[networkIndex]; let index = network.nodes.findIndex( n => n.label == i[2]); return {inputs : [networkIndex, [[index]]], accuracy : accuracy(network, [i[0],i[1]])} })} else return {data : data, instances : instances.map( i=> { let index = data.nodes.findIndex( n => n.label == i[2]); return {inputs : [0, [[index]]], accuracy : accuracy(data, [i[0],i[1]])} })} }; return { name : "rememberNeighbors", question : "You've previously identified the (blue) highlighted node as being connected to two other nodes (which we had shown you). Can you remember and select which those nodes were?", description : "Given their answer to an earlier namedCommonNeighbor task, can participants remember and select what the two highlighted nodes of that task were? This task should be paired with and delivered some time after the namedCommonNeighbor task (instances are paired if delivered in order).", inputs : [ { type : "vis", state : "Network"}, { type : "vis", state : "HighlightedNodes"}], outputs : [ {type : "vis", state : "SelectedNodes"}], instances : [ //Important: first label should be the correct answer //IMPORTANT! Copy the genInstances code into rememberNeighbors! genInstances(testData.demoNetwork, [ [0, 1, "node1", "node2", "node3"], [3, 5, "node2", "node3", "node4"] ]), genInstances(testData.lesMis, [ [72, 58, "Javert", "Prouvaire", "Mme.Burgon"], [16, 61, "Marius", "BaronessT", "Bahorel"], [60, 69, "Gavroche", "Montparnasse", "Isabeau"], [37, 33, "Valjean", "Child2", "Magnon"], [41, 59, "Marius", "Cosette", "BaronessT"], [50, 27, "Mme.Thenardier", "Geborand", "Fauchelevent"], [66, 55, "Bossuet", "Bossuet", "Favourite"], [26, 53, "Mlle.Gillenormand", "Marius", "Montparnasse"], [14, 10, "Valjean", "Gribier", "Grantaire"], [11, 5, "Myriel", "Blacheville", "Gavroche"] ]), genInstances(testData.ingredients, [ [152, 115, "saffron", "rosemary", "grape"], [28, 34, "malt", "chamomile", "rice"], [224, 218, "apricot", "sumac", "meat"], [210, 173, "currant", "octopus", "sherry"], [213, 161, "rose", "oatmeal", "olive_oil"], [0, 79, "apricot", "currant", "octopus"], [222, 232, "cumin", "chamomile", "rice"], [176, 37, "sage", "sumac", "wheat"], [7, 33, "potato", "feta_cheese", "milk"], [11, 185, "radish", "green_tea", "avocado"] ]), genInstances(testData.usair97, [ [227, 315, "San Francisco Intl", "Spokane Intl", "Seattle-Tacoma In"], [158, 195, "Lambert-St Louis Intl", "Abilene Regional", "Miami Intl"], [ 7, 283, "Chicago O'hare Intl", "Yuma Mcas/Yuma Intl", "Dallas Love Field" ], [ 47, 150, "Portland Intl", "Rio Grande Valley Intl", "Keahole-Kona Intll" ], [ 25, 175, "Seattle-Tacoma Intl", "Santa Barbara Muni", "Burbank-Glendale-Pasadena" ], [ 194, 300, "Lambert-St Louis Intl", "Johnston Atoll", "Lafayette Regional" ], [203, 130, "Chicago O'hare Intl", "Pago Pago Intl", "Cyril E King"], [32, 7, "Juneau Intl", "Chicago O'hare Intl", "Pago Pago Intl"], [15, 200, "Anchorage Intl", "Waco Regional", "Daytona Beach Intl"], [15, 4, "Anchorage Intl", "Lanai", "Gregg County"] ]), genInstances(testData.ecoli, [ [72, 49, "1", "3", "10"], [18, 17, "1", "16", "23"], [46, 23, "1", "67", "96"], [50, 45, "48", "93", "1"], [51, 9, "0", "1", "2"], [96, 91, "94", "1", "8"], [1, 42, "41", "65", "70"], [7, 1, "6", "83", "25"], [87, 31, "1", "14", "13"], [93, 62, "0", "1", "3"] ]), genInstances(testData.weavers, [ [26, 40, "UMRO", "VMWZ", "UMUV"], [62, 14, "VMVW", "WWFM", "VMZU"], [4, 12, "VMWZ", "VMVE", "UMOE"], [41, 2, "VMUU", "FMUU", "UMWW"], [14, 1, "FMOY", "UMPP", "UMWW"], [29, 1, "VMLZ", "FMBV", "VMPB"], [24, 7, "VMVE", "VMWZ", "VMZU"], [17, 57, "VMVE", "FMLL", "VMYB"], [35, 7, "VMVE", "ZYVM", "FMLZ"], [51, 14, "VMZV", "UMUV", "MVLE"] ]), genInstances(testData.dblp, [ [51, 71, "Andreas Kerren", "Bruno Pinaud", "Karsten Klein"], [76, 18, "Karsten Klein", "Bruno Pinaud", "Hsiang-Yun Wu"], [61, 50, "Romain Bourqui", "Helen C. Purchase", "Karsten Klein"], [33, 25, "Weidong Huang", "Seok-Hee Hong", "Peter Eades"], [56, 52, "Fabien Jourdan", "Nathalie Poupin", "David Auber"], [18, 3, "Karsten Klein", "Stefan Felsner", "Da Yan"], [14, 29, "Karsten Klein", "Michael Krone", "Stephen G. Kobourov"], [5, 0, "Hsiang-Yun Wu", "Florian Ganglberger", "Nicolas Swoboda"], [3, 12, "Hsiang-Yun Wu", "Stefan P. Feyer", "Bruno Pinaud"], [45, 48, "Fabrizio Montecchiani", "Bernice E. Rogowitz", "Oh-Hyun Kwon"] ]), genInstances(testData.ghoniem1, [ [0, 11, "Q", "M", "R", 0], [3, 14, "A", "B", "Q", 1], [5, 2, "N", "R", "K", 2], [14, 4, "R", "T", "B", 3], [8, 13, "A", "B", "N", 4], [4, 15, "D", "F", "C", 5], [13, 17, "B", "A", "R", 6], [5, 4, "L", "E", "K", 7], [17, 1, "Q", "I", "G", 8], [0, 3, "B", "S", "R", 9] ]), genInstances(testData.ghoniem2, [ [2, 9, "A", "K", "L", 0], [12, 2, "J", "B", "Q", 1], [6, 3, "A", "R", "K", 2], [18, 0, "E", "K", "B", 3], [9, 13, "B", "H", "M", 4], [4, 15, "D", "F", "C", 5], [10, 7, "Q", "A", "E", 6], [4, 5, "C", "F", "G", 7], [16, 2, "K", "O", "G", 8], [9, 11, "G", "S", "R", 9] ]), genInstances(testData.ghoniem3, [ [2, 9, "A", "K", "L", 0], [12, 2, "J", "B", "Q", 1], [6, 3, "A", "R", "K", 2], [18, 0, "E", "K", "B", 3], [9, 13, "B", "H", "M", 4], [4, 15, "D", "F", "C", 5], [10, 7, "Q", "A", "E", 6], [4, 5, "C", "F", "G", 7], [16, 2, "K", "O", "G", 8], [9, 11, "G", "S", "R", 9] ]), genInstances(testData.ghoniem4, [ [23, 2, "B5", "A3", "A8", 0], [33, 31, "A8", "B2", "E9", 1], [40, 4, "D9", "C9", "B8", 2], [7, 37, "B2", "E1", "B4", 3], [0, 3, "A6", "D6", "B6", 4], [29, 5, "A8", "D2", "A4", 5], [10, 25, "A4", "A0", "E2", 6], [49, 17, "A7", "F2", "E9", 7], [14, 10, "A9", "E1", "E8", 8], [37, 9, "A0", "C8", "B9", 9] ]), genInstances(testData.ghoniem5, [ [31, 2, "A0", "A3", "A8", 0], [12, 14, "A1", "B7", "E9", 1], [0, 3, "A2", "C9", "B8", 2], [7, 1, "A8", "E1", "B4", 3], [41, 2, "A4", "D6", "B6", 4], [49, 6, "A2", "D2", "A4", 5], [13, 0, "A1", "B0", "E2", 6], [49, 11, "A1", "F2", "E8", 7], [44, 11, "A4", "E1", "E5", 8], [37, 9, "A0", "C8", "B9", 9] ]), genInstances(testData.ghoniem6, [ [42, 0, "A1", "A3", "A8", 0], [9, 3, "A2", "B9", "E9", 1], [24, 1, "A0", "C9", "B8", 2], [7, 0, "A1", "E1", "B1", 3], [41, 2, "A3", "D6", "B6", 4], [49, 6, "A2", "D2", "A4", 5], [13, 1, "A0", "B0", "E2", 6], [29, 1, "A0", "F2", "E8", 7], [41, 0, "A1", "E1", "E5", 8], [0, 6, "A2", "C8", "B9", 9] ]), genInstances(testData.ghoniem7, [ [30, 0, "A4", "A3", "A8", 0], [13, 33, "C7", "B9", "E9", 1], [6, 4, "A0", "C9", "B8", 2], [24, 8, "A1", "E1", "B1", 3], [85, 6, "B1", "D6", "B6", 4], [0, 1, "E0", "D2", "A4", 5], [13, 2, "B4", "B0", "E2", 6], [51, 22, "C6", "F2", "E8", 7], [92, 5, "A6", "E1", "E5", 8], [16, 7, "B4", "C8", "B9", 9] ]), genInstances(testData.ghoniem8, [ [22, 7, "A2", "B3", "A8", 0], [0, 12, "A2", "B9", "E9", 1], [85, 1, "A0", "C9", "B8", 2], [30, 5, "A8", "E1", "B1", 3], [41, 2, "A3", "D6", "B6", 4], [78, 0, "A3", "D2", "A4", 5], [4, 0, "A2", "B0", "E2", 6], [12, 19, "B0", "F2", "E8", 7], [10, 13, "A6", "E1", "E5", 8], [82, 8, "A3", "C8", "B9", 9] ]), genInstances(testData.ghoniem9, [ [42, 0, "A1", "A3", "A8", 0], [9, 3, "A2", "B9", "E9", 1], [24, 1, "A0", "C9", "B8", 2], [7, 5, "A3", "E1", "B1", 3], [57, 6, "A2", "D6", "B6", 4], [95, 5, "A2", "D2", "A4", 5], [4, 5, "A2", "B0", "E2", 6], [50, 7, "A5", "F2", "E8", 7], [13, 7, "A2", "E1", "E5", 8], [0, 2, "A1", "C8", "B9", 9] ]) ] } }48:T4c7,networkTesting = ({ data : testData, tasks : { nodeCount : nodeCount, edgeCount : edgeCount, mostConnected : mostConnected, compareDegrees : compareDegrees, findNode : findNode, nodesConnected : nodesConnected, namedConnected : namedConnected, selectNeighbors : selectNeighbors, areNeighbors : areNeighbors, commonNeighbor : commonNeighbor, selectCommon : selectCommonNeighbors, namedCommon : namedCommonNeighbor, withinTwoHops : withinTwoHops, findPath : findPath, comparePaths : comparePaths, rememberConnection : rememberConnection, rememberNeighbors : rememberNeighbors, rateVis : rateVis, introduceVis : introduceVis} })49:[{"id":"1a0d6422046002f7383f41d9392b44e25cd17c09199a534a77f6392e23b1cd1cd2aab627a4ab0353bb6771ef972472907ae02256c584e4088c56d652938867a8","href":"https://static.observableusercontent.com/files/1a0d6422046002f7383f41d9392b44e25cd17c09199a534a77f6392e23b1cd1cd2aab627a4ab0353bb6771ef972472907ae02256c584e4088c56d652938867a8","name":"weaversOrdered.json","type":"application/json","lastModified":1733766850925.433,"size":7412},{"id":"566e5e744f315bb2e7f898a5426f72e997f25cf19888b7751e1ffd7021e0dbe4689b985b4e950bb71e0b60214b3aac8758a06e23823a32569b5747b94748470a","href":"https://static.observableusercontent.com/files/566e5e744f315bb2e7f898a5426f72e997f25cf19888b7751e1ffd7021e0dbe4689b985b4e950bb71e0b60214b3aac8758a06e23823a32569b5747b94748470a","name":"ghoniem9Ordered.json","type":"application/json","lastModified":1733766851459.712,"size":704721},{"id":"76c57e490de70857bf9944cd12ac594b4fdd07ff0ec4ef575083a5855829e5a0d1e46fc01870dff65f0eda5fb8bd38fa1f147c8e54657fcb2a463da0b8c69cfe","href":"https://static.observableusercontent.com/files/76c57e490de70857bf9944cd12ac594b4fdd07ff0ec4ef575083a5855829e5a0d1e46fc01870dff65f0eda5fb8bd38fa1f147c8e54657fcb2a463da0b8c69cfe","name":"ghoniem7Ordered.json","type":"application/json","lastModified":1733766851230.884,"size":135138},{"id":"1541458391d0a4124fb4b6c3fbc58e5702b75de223cd632a918b440b612a7154c3e355c874aee94008bd03feae587a1a1f573830c78f272e4d44836c49d34ab6","href":"https://static.observableusercontent.com/files/1541458391d0a4124fb4b6c3fbc58e5702b75de223cd632a918b440b612a7154c3e355c874aee94008bd03feae587a1a1f573830c78f272e4d44836c49d34ab6","name":"ghoniem2Ordered.json","type":"application/json","lastModified":1733766850927.324,"size":23095},{"id":"ccdc003f75fb491be750db95e465cb86469eabf7fc9d7939152336fa5b3efe973c6864b071536247ceb25d36821dfb468450ca2d04ff791128eec7f93a78b5c0","href":"https://static.observableusercontent.com/files/ccdc003f75fb491be750db95e465cb86469eabf7fc9d7939152336fa5b3efe973c6864b071536247ceb25d36821dfb468450ca2d04ff791128eec7f93a78b5c0","name":"ingredientsOrdered.json","type":"application/json","lastModified":1733766851143.635,"size":63878},{"id":"88705d9521dac227fb3d3bc3a91003dae351b6aaea99feafd844b518e486b909b89eaac500dfb15d3d8ad620f4e252c70bcc2af84a86b918be97480293d8ad60","href":"https://static.observableusercontent.com/files/88705d9521dac227fb3d3bc3a91003dae351b6aaea99feafd844b518e486b909b89eaac500dfb15d3d8ad620f4e252c70bcc2af84a86b918be97480293d8ad60","name":"ecoliOrdered.json","type":"application/json","lastModified":1733766850933.409,"size":11788},{"id":"4bfecd95c1fb9da1131d06baa5663c2ec34cd5e6f1364882d190ba0d5ce5681c662d52cb2f1ccb7faf23f69eed27801e9ad87633182ac84fa617bd901f79627c","href":"https://static.observableusercontent.com/files/4bfecd95c1fb9da1131d06baa5663c2ec34cd5e6f1364882d190ba0d5ce5681c662d52cb2f1ccb7faf23f69eed27801e9ad87633182ac84fa617bd901f79627c","name":"ghoniem1Ordered.json","type":"application/json","lastModified":1733766850935.496,"size":15462},{"id":"1a7dcecb73c9a4a535bca29a5cf35a77ab463c97f6b3601a54b444ff0de217bf8c7f46737ce3d847bd98f35e59eb8219d78e3cd9f2abd87544f0c45c0717af7b","href":"https://static.observableusercontent.com/files/1a7dcecb73c9a4a535bca29a5cf35a77ab463c97f6b3601a54b444ff0de217bf8c7f46737ce3d847bd98f35e59eb8219d78e3cd9f2abd87544f0c45c0717af7b","name":"ghoniem8Ordered.json","type":"application/json","lastModified":1733766851347.413,"size":348739},{"id":"17b8f0343c3476942279d996805e0753acbf8b0241465810579854190fa2d353943df045175fcb99e75df11ddffc6b8f7a520ad6b99456d0d1f5cdc4df57b7ac","href":"https://static.observableusercontent.com/files/17b8f0343c3476942279d996805e0753acbf8b0241465810579854190fa2d353943df045175fcb99e75df11ddffc6b8f7a520ad6b99456d0d1f5cdc4df57b7ac","name":"ghoniem3Ordered.json","type":"application/json","lastModified":1733766851042.461,"size":36695},{"id":"23d177bba62f4669709a16af9a8049d89b74d5cbc2d46fd30cef313fd0b58cfd15dc00973ef75eab32a5c4ecf964bf6379ab0466b122d569f6084d0fb212e961","href":"https://static.observableusercontent.com/files/23d177bba62f4669709a16af9a8049d89b74d5cbc2d46fd30cef313fd0b58cfd15dc00973ef75eab32a5c4ecf964bf6379ab0466b122d569f6084d0fb212e961","name":"lesMisOrdered.json","type":"application/json","lastModified":1733766850936.675,"size":12631},{"id":"88150999b7a8463af411da49c137f6e8db99f4f381c87513c4de1d82e42c51263d5198cdde2adf151d6e77118f8dcf2f9fbd78b6cf7fa6a4f391c1429e9fabd8","href":"https://static.observableusercontent.com/files/88150999b7a8463af411da49c137f6e8db99f4f381c87513c4de1d82e42c51263d5198cdde2adf151d6e77118f8dcf2f9fbd78b6cf7fa6a4f391c1429e9fabd8","name":"ghoniem5Ordered.json","type":"application/json","lastModified":1733766851219.167,"size":102125},{"id":"ef7be0a3eb460ae79e1e15e9cef9efd60d7ab8d84cbdd55b3b0be2cd8d2c242bf6afeff7266e3cdcc4816a6b02e5e157bb99035822f207494f9dc6be8b33ee9a","href":"https://static.observableusercontent.com/files/ef7be0a3eb460ae79e1e15e9cef9efd60d7ab8d84cbdd55b3b0be2cd8d2c242bf6afeff7266e3cdcc4816a6b02e5e157bb99035822f207494f9dc6be8b33ee9a","name":"ghoniem4Ordered.json","type":"application/json","lastModified":1733766851050.369,"size":49303},{"id":"bbfe718101a537b845be07eaa828e60bfe55cbd28aaf9266ebad85dcd35388bffa93e678ad350c139d0ea9aef32d558008db72eadc0253cfb471879440dec264","href":"https://static.observableusercontent.com/files/bbfe718101a537b845be07eaa828e60bfe55cbd28aaf9266ebad85dcd35388bffa93e678ad350c139d0ea9aef32d558008db72eadc0253cfb471879440dec264","name":"dblpOrdered.json","type":"application/json","lastModified":1733766850959.765,"size":10868},{"id":"44a96c6553a5660cb009d837c691db5a807380bbdfcbdffc7cb2d0d28e74819f3428ec67c74c17bf3e357233f019c0c73cec7a288e563a4bd133970a9065d1a3","href":"https://static.observableusercontent.com/files/44a96c6553a5660cb009d837c691db5a807380bbdfcbdffc7cb2d0d28e74819f3428ec67c74c17bf3e357233f019c0c73cec7a288e563a4bd133970a9065d1a3","name":"usair97Ordered.json","type":"application/json","lastModified":1733766851152.628,"size":68307},{"id":"3cff594da10979156902e6c904ed46036319516f564ca30879602416b0fa4cd4e57ef7b8520e1aa208155005385d59511f1c35bcda2d0b3e94cbb87691d16bcc","href":"https://static.observableusercontent.com/files/3cff594da10979156902e6c904ed46036319516f564ca30879602416b0fa4cd4e57ef7b8520e1aa208155005385d59511f1c35bcda2d0b3e94cbb87691d16bcc","name":"ghoniem6Ordered.json","type":"application/json","lastModified":1733766851314.418,"size":190159}] 31:{"id":"6faa4a81eab281e4","slug":"networktestsuite","title":"Test Suite for Network Visualisation","thumbnail":"1aea2554a24029d2b2d716de646593832e7decaeaf2b43fb21f9dded150ecfff","access_level":"public","create_time":"$D2024-12-09T17:29:21.566Z","update_time":"$D2024-12-09T18:26:24.588Z","view_time":null,"first_public_time":"$D2024-12-09T18:12:20.394Z","has_unpublished_edits":null,"fork":false,"stars":1,"starred":false,"owner":{"type":"team","id":"0eea6200fa793072","login":"tyr1981","name":"tyr1981","avatar_url":"https://avatars-next.observableusercontent.com/66f967001deee18aef0611cb3db26e44b3673352ea83d80f5c042b1d468d7b3a","tier":"starter_2024"},"version":291,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"e7e6bc615f03bb8a","login":"tyr1981","name":"tyr1981","avatar_url":"https://avatars-next.observableusercontent.com/dfb74df6c39ed24d6e79e44cfc79136ab36233a17d1e111ef6dfd91050833ec8","tier":null}],"members":[{"type":"team","id":"0eea6200fa793072","login":"tyr1981","name":"tyr1981","avatar_url":"https://avatars-next.observableusercontent.com/66f967001deee18aef0611cb3db26e44b3673352ea83d80f5c042b1d468d7b3a","tier":"starter_2024","role":"editor"},{"type":"individual","id":"e7e6bc615f03bb8a","login":"tyr1981","name":"tyr1981","avatar_url":"https://avatars-next.observableusercontent.com/dfb74df6c39ed24d6e79e44cfc79136ab36233a17d1e111ef6dfd91050833ec8","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":{"id":"2479ec79deb66c52","slug":"visunit","title":"VisUnit","owner":{"id":"0eea6200fa793072","login":"tyr1981","name":"tyr1981"}},"type":"notebook","body":{"title":"Test Suite for Network Visualisation","cells":[{"id":0,"value":"# Test Suite for Network Visualisation","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":5,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":7,"value":"

\n## Use","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":9,"value":" Use the following command to import the test-suite into your VisUnit study:\n\n import {networkTesting} from \"@tyr1981/networktestsuite\"","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":11,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":13,"value":"Browse the complete test-suite as a data object below:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":15,"value":"networkTesting","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":17,"value":"

\n## Datasets","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":19,"value":"Below are **descriptions and statistics** of the networks in the suite. These include node and edge counts, density measures, and distributions of node degrees. Further down you can **browse** th networks as node-link diagrams further down. A **discussion** around the suitability of the networks for evaluation follows. \n\nNote that the _Ghoniem et al._ datasts are collections (arrays!) of networks with similar properties.\n\nFor convience, networks have **default node layouts** (computed using D3 force layouts) and **orderings** (computed with reorder.js) attached to them. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":21,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":23,"value":"

\n**Browse** the networks using node link diagrams:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":25,"value":"viewof dataSelDatasets = Inputs.select(Object.values(networkTesting.data).map(d => d.name), {label: \"Dataset:\"})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":27,"value":"{\n updateDatasets;\n let div = document.createElement(\"div\");\n viewerDatasets.draw(div);\n return div;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":29,"value":"$36","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":31,"value":"demoDataset = ({\n nodes : [\n {label : \"node1\", x : 0.2, y : 0.3},\n {label : \"node2\", x : 0.7, y : 0.7},\n {label : \"node3\", x : 0.1, y : 0.8},\n {label : \"node4\", x : 0.7, y : 0.3},\n {label : \"node5\", x : 0.3, y : 0.5},\n {label : \"node6\", x : 0.5, y : 0.3},\n {label : \"node7\", x : 0.4, y : 0.1},\n ],\n edges : [\n { n1 : 0, n2 : 4, value : 1},\n { n1 : 0, n2 : 5, value : 1},\n { n1 : 0, n2 : 6, value : 1},\n { n1 : 4, n2 : 2, value : 1},\n { n1 : 4, n2 : 3, value : 1},\n { n1 : 1, n2 : 2, value : 1},\n { n1 : 1, n2 : 3, value : 1},\n { n1 : 1, n2 : 4, value : 1},\n { n1 : 0, n2 : 1, value : 1},\n { n1 : 0, n2 : 2, value : 1}\n ],\n name : \"demo\",\n description : \"Small network to use for training and introduction.\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":33,"value":"lesMis = {\n let ret = (await FileAttachment(\"lesMisOrdered.json\").json());\n ret.name = \"Les Mis\";\n ret.description = \"Interactions between characters in Hugo's 'Les Miserables'.\"\n return ret;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":37,"value":"ingredients = {\n let ret = (await FileAttachment(\"ingredientsOrdered.json\").json());\n ret.name = \"Ingredients\";\n ret.description = \"Cooking ingredients co-occuring in recipes (258 nodes, 1089 edges). (Okoe et al. 2018)\";\n return ret;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"usair97 = {\n let g = normCoords(await FileAttachment(\"usair97Ordered.json\").json());\n g.name = \"USAir97\";\n g.description = \"US airports sharing connectinng flights. (Okoe et al. 2018)\"\n return g;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":39,"value":"weavers = {\n let g = normCoords(await FileAttachment(\"weaversOrdered.json\").json());\n g.name = \"Weavers\";\n g.description = \"An animal social network of weaver birds (Di Giacomo et al. 2023)\"\n return g;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"ecoli = {\n let g = normCoords(await FileAttachment(\"ecoliOrdered.json\").json());\n g.nodes.map( (n,i) => n.label = \"\"+i);\n g.name = \"EColi\";\n g.description = \"Transcriptional interactions in the Escherichia coli bacterium (Di Giacomo et al. 2023)\";\n return g;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":43,"value":"dblp = {\n let g = normCoords(await FileAttachment(\"dblpOrdered.json\").json());\n g.name = \"DBLP\";\n g.description = \"Co-citation network extracted from DBLP (Di Giacomo et al. 2023)\";\n return g;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":45,"value":"ghoniem = {\n let ret = [];\n ret.push((await FileAttachment(\"ghoniem1Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem2Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem3Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem4Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem5Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem6Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem7Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem8Ordered.json\").json()).map(normCoords));\n ret.push((await FileAttachment(\"ghoniem9Ordered.json\").json()).map(normCoords));\n\n ret.map( (d,i) => d.name = \"Ghoniem\" + (i+1));\n ret.map( (d,i) => d.description = \"Random graphs with \" + \n d[0].nodes.length + \" nodes and \" + d[0].edges.length + \" edges (Ghoniem et al. 2005)\");\n return ret; \n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":47,"value":"

\n## Tasks","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":50,"value":"We included 19 tasks, predominantly used in the studies of _Ghoniem et al._, _Okoe et al._, and _Di Giacomo et al._ as well as covering the taxonomies of _Lee et al._. They are listed in the table below along with a brief description, where they were used or referenced in, and their required inputs and outputs/answers. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":52,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":55,"value":"

\n### Browse tasks and their instances\n\nMost tasks are \"instantiated\" into datasets, i.e. their inputs take values from elements of the dataset. As such, a task can be visually explored in conjunction with a datset:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":57,"value":"viewof tasksview = Inputs.select(Object.values(networkTesting.tasks).map( t => t.name), {value: findNode, label: \"Task\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":59,"value":"viewof instanceview = Inputs.select(\n selectedTask.instances == null ? [demoDataset.name] :\n selectedTask.instances.map( i => i.data.name), {value : \"lesMis\", label : \"Dataset\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":61,"value":"\nmd`__${selectedInstances == null ? 1 : selectedInstances.instances.length} instance(s)__ available for this task onto this data`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":63,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":65,"value":"$38","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":67,"value":"**nodeCount, edgeCount, and mostConnected**: Only one instance per network is possible. Their accuracy functions are non-binary: the closer the participant's approximation is to the real answer the closer the accuracy is to 1. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":69,"value":"nodeCount = {\n\n //return an accuracy function for this network+task\n let accuracy = (network) => \n (responses) => {\n if (responses[0] == null) return 0;\n else return (1 - Math.abs(responses[0] - network.nodes.length)/network.nodes.length);\n }\n \n //generate instances for this dataset (a [multi]network)\n function genInstances(d){ \n //if d is a regular network then transform it into a multi-network (i.e. array of networks)\n let mn = Array.isArray(d) ? d : [d];\n return { \n data : d, \n instances : mn.map( (n,i) => ( {inputs : [i], accuracy : accuracy(n)}))\n }\n }\n \n return {\n name : \"nodeCount\",\n question: \"Estimate or count the number of nodes in the network\",\n description: \"Estimate or count the number of nodes in a network\",\n inputs : [ { type : \"vis\", state : \"Network\"}],\n outputs : [ {type : \"integer\"}],\n //create instances in bulk for all datasets in the test suite\n instances : Object.values(testData).map(genInstances)\n }\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":71,"value":"edgeCount = {\n\n //return an accuracy function for this network+task\n let accuracy = (network) => \n (responses) => {\n if (responses[0] == null) return 0;\n else return (1 - Math.abs(responses[0] - network.edges.length)/network.edges.length)\n }\n \n //generate instances for this dataset (a [multi]network)\n function genInstances(d){ \n //if d is a regular network then transform it into a multi-network (i.e. array of networks)\n let mn = Array.isArray(d) ? d : [d];\n return { \n data : d, \n instances : mn.map( (n,i) => ( {inputs : [i], accuracy : accuracy(n)}))\n }\n }\n \n return {\n name : \"edgeCount\",\n question: \"Estimate the number of connections in the network\",\n description: \"Estimation of the number of edges in a network\",\n inputs : [ { type : \"vis\", state : \"Network\"}],\n outputs : [ {type : \"integer\"}],\n //create instances in bulk for all datasets in the test suite\n instances : Object.values(testData).map(genInstances)\n }\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":73,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":75,"value":"md `**compareDegrees** (_${compareDegrees.question}_) The benefit of this task over the *mostConnected* one is that it can be instantiated multiple times for a dataset. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":77,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":79,"value":"md `**findNode** (_${findNode.question}_) While we do not think this task is particularly realistic, it is included in most task taxonomies and evaluations so we provide the infrastructure for its testing. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":81,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":83,"value":"md `**nodesConnected** (_${nodesConnected.question}_) Argueably the most fundamental task facilitated by a network visualisation. It is included in most network evaluation studies and taxonomies. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":85,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":87,"value":"md `**namedConnected** (_${namedConnected.question}_) A potentially more realistic version of the _nodesConnected_, _findNode_, and _selectNeighbors_ tasks (and which combines perceptual elements of all three tasks). Participants need to identify whether a given node is amongst a highlighted nodes's neighbors. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":89,"value":"$3d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":91,"value":"md `**selectNeighbors** (_${selectNeighbors.question}_) While intuitive, the main disadvantage of the task is that it may take a long time for a participant to select all the neighbors (much longer than it takes to identify neighbors visually). Non-binary answer takes into account false positive and false negative selections.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":93,"value":"$3e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":95,"value":"md `**areNeighbors** (_${areNeighbors.question}_) We think this task tests similar perceptual processes to the (more popular) _selectNeighbors_ while minimising the confounding cost of providing the answer (a yes/no choice). Moreover, the task is likely to be quicker to resolve, allowing more repeats to be tested. Binary answer. `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":97,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":99,"value":"md `**selectCommonNeighbors** (_${selectCommonNeighbors.question}_) Beware of the duration of selecting nodes when interpreting results. Non-binary answer that takes into account false negative and false positive selections.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":101,"value":"$40","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":103,"value":"md `**commonNeighbor** (_${commonNeighbor.question}_) A potential alternative to the _selectCommonNeighbors_ task which reduces the confounding interaction factor. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":105,"value":"$41","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":107,"value":"md `**namedCommonNeighbor** (_${namedCommonNeighbor.question}_) A further alternative that includes label search. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":109,"value":"$42","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":111,"value":"md `**withinTwoHops** (_${withinTwoHops.question}_) Between neighborhood tasks and paths task. We think a meaningful way of delivering this task is as an estimative one, enforced by a short allowed response time.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":113,"value":"$43","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":115,"value":"md `**findPath** (_${findPath.question}_) Beware that for paths longer than two, this is a difficult and lengthy task in most real networks. Consider either adding only few repeats of this task (and allowing a long response time) or conversely enforcing estimation via a short response time and testing multiple repeats. Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":117,"value":"$44","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"md `**comparePaths** (_${comparePaths.question}_) The task is meant to be an estimative one and thus should be delivered with a tight time constraint (e.g., 10 seconds). Binary answer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":121,"value":"$45","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":123,"value":"md `**rememberConnection** (_${rememberConnection.question}_) The task would need to be delivered after the _namedConnected_ task and without access to the visualisation. Moreover, the inputs into the instances of this tasks should be in the same order as in the _nameConnected_ task (choose repeats in order rather than rely on VisUnit's repeat randomisation).`\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":125,"value":"$46","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":127,"value":"md `**rememberNeighbors** (_${rememberNeighbors.question}_) The task would need to be delivered after the _namedCommonNeighbor_ task and without access to the visualisation. Moreover, the inputs into the instances of this tasks should be in the same order as in the _nameCommonNeighbor_ task (choose repeats in order rather than rely on VisUnit's repeat randomisation).`\n\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":129,"value":"$47","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":131,"value":"md `**rateVis** (_${rateVis.question}_) To be generally used as a block task after the last showing of each viewer. Single instance per viewer.`\n\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":133,"value":"rateVis = ({\n name : \"rateVis\",\n description : \"A (single instance) viewer specific task; should be shown after last use of a viewer\", \n question : \"How would you rate the usability of the visualisation you've just used (where 1 is unusuable and 4 is very easy to use)? Explain why.\",\n outputs : [ {type : \"select\", options : [\"select\",\"1\",\"2\",\"3\",\"4\"]}, \n {type : \"long text\", label : \"why?\"}],\n});","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":135,"value":"**introduceVis:** This is a psuedo-task used to allow a visualisation to introduce itself. The way it is implemented is that each task instance is a \"step\" in the introduction process. So, if you set up your task to have 5 repeats, then you can essentially provide 5 introduction steps. You would do this by implementing the visualisation's _setIntroduction(step, context)_ method and setting different information in _context.gui.question.innerHTML_ depending on the step counter.`","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":137,"value":"introduceVis = ({\n name : \"introduceVis\",\n description : \"Lets a viewer introduce itself.\",\n inputs : [ { type : \"vis\", state : \"Introduction\"}],\n outputs : [],\n instances : \n Object.values(testData).map( d => (\n {\n data : d,\n instances : d3.range(10).map( v => ({ inputs : [v], accuracy : (outputs) => -1}))\n }))\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":139,"value":"### The test-suite","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":142,"value":"Finally, we bundle the data and tasks together into a test-suite that you can import and test your visualisation designs against.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":145,"value":"$48","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":148,"value":"testData = ({demoNetwork : demoDataset,\n lesMis : lesMis, \n ingredients : ingredients,\n usair97 : usair97,\n ecoli : ecoli,\n weavers : weavers,\n dblp : dblp,\n ghoniem1 : ghoniem[0],\n ghoniem2 : ghoniem[1],\n ghoniem3 : ghoniem[2],\n ghoniem4 : ghoniem[3],\n ghoniem5 : ghoniem[4],\n ghoniem6 : ghoniem[5],\n ghoniem7 : ghoniem[6],\n ghoniem8 : ghoniem[7],\n ghoniem9 : ghoniem[8]});","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":150,"value":"### Imports","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":152,"value":"Two Node-Link network visualisations for browsing purposes:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":154,"value":"import {NodeLinkViewer,MatrixViewer} from \"@tyr1981/network-vis\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":156,"value":"nodeLinkViewer = new NodeLinkViewer(500, 500, true)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":164,"value":"viewerDatasets = new NodeLinkViewer(600,600, () => mutable updateDatasets = true).loadData(Object.values(networkTesting.data).find( d => d.name === dataSelDatasets));","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":166,"value":"VisUnit (to browse tasks):","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":168,"value":"import {VisUnit} from \"@tyr1981/visunit\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":170,"value":"visUnit = new VisUnit();","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":172,"value":"visUnit.previewTask(document.getElementById(\"studydiv\"), \n Object.values(networkTesting.tasks).find( t => t.name == tasksview), \n nodeLinkViewer,\n Object.values(networkTesting.data).find ( d => d.name == instanceview))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":174,"value":"selectedTask = Object.values(networkTesting.tasks).find(t => t.name == tasksview)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":176,"value":"selectedInstances = \"instances\" in selectedTask ? \n selectedTask.instances.find( i => i.data.name == instanceview) :\n null;","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":178,"value":"mutable update = true;","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":180,"value":"mutable updateViewer1 = false","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":182,"value":"mutable viewer = null","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":184,"value":"mutable updateDatasets = true;","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":186,"value":"### Some useful network functions","pinned":true,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":189,"value":"nodeDegree = (network, nodeid) => \n network.edges.filter( e => e.n1 == nodeid || e.n2 == nodeid).length","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":191,"value":"maxNodeDegree = (network) => d3.max ( network.nodes.map( (n,index) => nodeDegree(network, index)))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":193,"value":"getNeighbors = (d, n) => {\n let ret = [];\n for (let i=0; i e.n2 == n) < 0) ret.push(e.n2); \n if (e.n2 == n && ret.findIndex( n => e.n1 == n) < 0) ret.push(e.n1);\n }\n return ret;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":195,"value":"getPathLength = (d, n1, n2) => {\n\n if (n1 == n2) return 0;\n \n let marked = d.nodes.map( n => false);\n let border = [n1]; marked[n1] = true;\n\n let pathLength = 0;\n \n while (true){\n //safe stop\n if (pathLength > 100) return -pathLength;\n pathLength++;\n \n let newBorder = [];\n for (let i=0; i< border.length; i++){\n let b = border[i];\n let ns = getNeighbors(d, b);\n for (let j=0; j {\n let ns = getNeighbors(d, n1);\n if (ns.indexOf(n2) >= 0) return true;\n return false;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":199,"value":"//normalise coordinates\nfunction normCoords(g){\n let mnX = d3.min(g.nodes.map( n => n.x));\n let mnY = d3.min(g.nodes.map( n => n.y));\n let mxX = d3.max(g.nodes.map( n => n.x));\n let mxY = d3.max(g.nodes.map( n => n.y));\n g.nodes.map( n=> {n.x = (n.x-mnX)/(mxX-mnX); n.y = (n.y - mnY)/(mxY - mnY)});\n return g;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":201,"value":"graphEdgeCount = (g) => g.edges.length","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":203,"value":"graphNodeCount = (g) => g.nodes.length","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":205,"value":"getNodeDist = (n1,n2) => {\n return Math.sqrt( Math.pow(n1.x - n2.x,2) + Math.pow(n1.y-n2.y,2));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W49","stdlib":"1","titling":"auto"}} 32:"$31:body"