2c:null 2d:["$","title",null,{"children":"4. Adding Automatically Generated Data: Networking Dates | Kalle Westerling | Observable"}] 32:T4fa,## Explaining the Setup In the following examples, I draw on the script that is generating the dataset that currently is the bedrock of the digital component of my dissertation. It runs on a scheduled basis in this GitHub repository to generate easily accessible and open JSON data, which can be used in any data analysis projects. In the script, I first generate a “DataFrame” containing the dataset from Google Sheets, as it has been structured (in [Step 2](${nav.ingestion.url})) and cleaned (in [Step 3](${nav.cleaning.url})). A “DataFrame” is a particular object in Python programming that can be compared to a spreadsheet. It allows for me to access columns and rows in a similar manner, to filter and remove rows and columns based on certain conditions. In the scripts below, I use **`df`** as a shorthand for the DataFrame that contains all of the data from the Google Sheet.33:Tcac,## Grouping Dates and Performers Together Considering that my sources were drawn predominantly from [digitized newspaper archives](${nav.ingestion.url}), I was faced with a dataset where I could safely assume significant gaps. A 2015 report from the Center for Research Libraries noted that “The bulk of newspapers in print and microform are, as yet, relatively untouched” (see The “‘State of the Art’: A Comparative Analysis of Newspaper Digitization to Date”). While things have likely improved over the past seven years since the publication of the report, lacking institutional resources, staffing, and the rapid deterioration of paper are major problems causing the progress to move slowly. In the light of this problem, I realized that I needed a model for how to automatically analyze and “cover” up the gaps in the _existing_ information that I had. Thus, I constructed an algorithm that would help me show the bonds between artists in the material that I did have access to. Here, I want to focus on each individual step of the date-grouping algorithm that was the result of this analysis. The algorithm was written in Python and followed these steps: 1. The algorithm first iterates over each venue in the dataset, and 2. Then iterates over all the dates in which data exists for said venue, stringing together those dates that are closer to each other than a given limit into _date groupings_. The limits that I used for my analyses were 3, 14, 31, 93, 186, and 365 days, or the equivalent of roughly a few days, a few weeks, a month, three months, six months, and a year. I found that the 14-day limit was the most optimal for my dataset. 3. For each such date grouping, a list of the performers who appeared within those groupings was created. _Here, the assumption was made that those performer had performed together, at some point during those date groupings. It was also assumed that it is likely that the performers who performed at the same venue within such a date group would have known and influenced each other. Following those assumptions, the algorithm continues:_ 4. For each date grouping (and consequently—see step 1 and 2 above—for each limit and each venue), the algorithm then went through each performer, and linked each individual to one another in the creation of clusters of those who more frequently appeared together. 5. For each edge (or _connections_ between each performer in each date group), the algorithm then continued to add the metadata to their relation, including: - In which venue they both appeared - During which dates they appeared (including all the dates in the date group) - Any names of revue/s* in which they appeared during this time period - Any names of the city/ies* in which they appeared during this time period * Only the cleaned names of revues and cities were added to the metadata here (see [Step 3](${nav.cleaning.url})). 6. Since the connections were iterated over venues and all the performers iteratively per venue, connections were also built between clusters.34:T7b4,### Test the Date-Grouping Algorithm In this section, I use a practical example to explain the algorithm described above. Let us start by imagining that we have access to three sources:

	${await FileAttachment("image@1.png").image()} Source 1	${await FileAttachment("image@2.png").image()} Source 2	${await FileAttachment("image@3.png").image()} Source 3
Publication date	${dates[0].label} ${getGroups()[0].exampleDate.format('MMMM D, YYYY')}	${dates[1].label} ${getGroups()[1].exampleDate.format('MMMM D, YYYY')}	${dates[2].label} ${getGroups()[2].exampleDate.format('MMMM D, YYYY')}
Grouped in group	Group ${getGroups()[0].group}	Group ${getGroups()[1].group}	Group ${getGroups()[2].group}

${html` `}35:T917,The table above should make clear that we have, in front of us, three separate sources. For the illustration purposes here, it does not matter which performers, venues, cities, etc. are mentioned in the sources. All that matters is _when_ a source is published and, as a consequence, whether source 1, 2, and 3 should be grouped together or not. Say that the first source above is published on **${getGroups()[0].exampleDate.format('MMMM D, YYYY')}**. Let us then assume that the second source was published on **${getGroups()[1].exampleDate.format('MMMM D, YYYY')}** (`${getGroups()[1].label}` below) and the third source was published on **${getGroups()[2].exampleDate.format('MMMM D, YYYY')}** (`${getGroups()[1].label}` below). Now, depending on the time elapsed between them, we will group them together. Our grouping is the basis for the assumption that the artists who were present in the venue during a group most likely knew of one another and their artistic work, potentially were inspired, collaborated, etc. We also assume that the groups mean that the artists were present in the venue at any given time during those days. Currently that day is set to **${settings.groupOn} days**. If you adjust the levers here below, you can see the groupings in the table above change. Given the assumptions here, the lower we keep the number of days, the more fine-tuned our assumptions will be. To be clear, although I have included in the algorithm the possibility to group dates together that are within 6 months or a year of one another, is it still reasonable to assume that artists who were at the same venue within 6–12 months of each other can be said to have inspired each other, collaborated, etc.? Below, you will find some levers that allow for you to play around with the settings for the date grouping algorithm. They are meant to help you envision the basis for the dataset in my research, but also, in general, to encourage a playful attitude towards lacking historical data. Keep a speculative question in mind as you play: How likely is it that a performer who appeared in **Source 1** would know a performer in **Source 3** and what could their relation have been? _Go ahead and toy around with the levers and the days elapsed between sources for a group to be formed!_36:T64a,{ const svg = d3.create("svg") .attr("height", 145) .attr("width", width) .attr("style", "background-color:#f8f9fa;border-radius: 1em;"); const dates = getGroups(); const drawing = svg.append("g") const sourceLabels = svg.append("g") sourceLabels.selectAll("text") .data(dates) .join("text") .text(d => d.source) .attr("x", d => d.x) .attr("y", y - 30) .attr("style", "font-weight:normal !important;") .attr("opacity", 0.3) .attr("text-anchor", "middle") const dateLabels = svg.append("g") dateLabels.selectAll("text") .data(dates) .join("text") .text(d => d.label) .attr("x", d => d.x) .attr("y", y - 10) .attr("text-anchor", "middle") // lines drawing.selectAll("line") .data(dates) .join("line") .attr("x1", d => d.x) .attr("y1", y + 20) .attr("x2", d => d.nextX ? d.nextX : d.x) .attr("y2", y + 20) .attr("stroke-width", 1) .attr("stroke", d => d.groupColor) const lineLabels = svg.append("g") lineLabels.selectAll("text") .data(dates) .join("text") .text(d=>d.nextDiff > 0 ? `Δ ${d.nextDiff} days` : null) .attr("x", d => d.nextX ? d.x + d.nextX / 4 : d.x) .attr("y", y + 45) .attr("text-anchor", "middle") .attr("font-size", 11) // dates drawing.selectAll("circle") .data(dates) .join("circle") .attr("cx", d => d.x) .attr("cy", y + 20) .attr("fill", "black") .attr("r", 10) .attr("stroke-width", "0.5px") .attr("stroke", "black") .attr("fill", d => d.groupColor) return svg.node(); }37:T889,chart = () => { const nodes = network14.nodes, links = network14.links, scale = d3.scaleOrdinal(d3.schemeCategory10), linkStrength = 0.01, chargeStrength = -5, height = 500; const scaleSize = d3.scaleLinear().domain(d3.extent(nodes, n=>n.degrees.degree)).range([1,20]); const color = (num) => scale(num); const drag = simulation => { const dragsubject = (evt) => { return simulation.find(evt.x, evt.y); }, dragstarted = (evt) => { if (!evt.active) simulation.alphaTarget(0.3).restart(); evt.subject.fx = evt.subject.x; evt.subject.fy = evt.subject.y; }, dragged = (evt) => { evt.subject.fx = evt.x; evt.subject.fy = evt.y; }, dragended = (evt) => { if (!evt.active) simulation.alphaTarget(0.025); // never stop evt.subject.fx = null; evt.subject.fy = null; } return d3.drag() .subject(dragsubject) .on("start", dragstarted) .on("drag", dragged) .on("end", dragended); } const simulation = d3.forceSimulation(nodes) .force("link", d3.forceLink(links).id(d => d.id).strength(linkStrength)) .force("charge", d3.forceManyBody().strength(chargeStrength)) .force("center", d3.forceCenter(width / 2, height / 2)); const ctx = DOM.context2d(width, height); simulation.alphaTarget(0.5); // don't stop simulation.on("tick", ticked); return d3.select(ctx.canvas).call(drag(simulation)).node(); function ticked() { ctx.clearRect(0, 0, width, height); ctx.beginPath(); links.forEach(drawLink); ctx.strokeStyle = "rgba(10,10,10,0.1)"; ctx.stroke(); //ctx.strokeStyle = "#fff"; for (const node of nodes) { ctx.beginPath(); drawNode(node); ctx.fillStyle = color(node.modularities.Louvain); ctx.fill(); ctx.stroke(); } } function drawLink(d) { ctx.moveTo(d.source.x, d.source.y); ctx.lineTo(d.target.x, d.target.y); } function drawNode(d) { ctx.fillText(d.display, d.x+10, d.y+3); ctx.moveTo(d.x + 3, d.y); ctx.arc(d.x, d.y, scaleSize(d.degrees.degree), 0, 2 * Math.PI); } }38:T407,getNodesFromLinks = ( links, groups = getFoundPerformers(), crash = true ) => { let nodes = []; links.forEach(d=>{ nodes.push(d.source.node_id); nodes.push(d.target.node_id); }); nodes = [...new Set(nodes)]; nodes = nodes.map(index=>{ let node = network14.nodes.find(n => n.node_id === index); let inSelectedLocation = groups.inSelectedLocation.includes(index); let inIntersection = groups.intersection.includes(index); let inCompareLocation = groups.inCompareLocation.includes(index); let group = undefined; if (inIntersection) { group = 2 } else if (inSelectedLocation) { group = 1 } else if (inCompareLocation) { group = 3 } else { let msg = `No group could be determined for ${index}.`; if (crash) throw new Error(msg); console.warn(msg, node); } return { index, node, group, inSelectedLocation, inCompareLocation, inIntersection } }); return nodes }39:T6fe,showGroupedData = () => { const ret = { links: [], nodes: [], } if (!compareLocation) { // console.log('no compare location') ret.links = network14.links.filter(d=>Object.keys(d.coLocated).includes(selectedLocation)) } else { // console.log('has compare location') ret.links = [...network14.links.filter(d=>Object.keys(d.coLocated).includes(selectedLocation)), ...network14.links.filter(d=>Object.keys(d.coLocated).includes(compareLocation))] } ret.links = ret.links.map(d=>{ return { weight: d.weights.dateGroups, source: d.source.node_id, target: d.target.node_id } }) const setupNode = (id) => { let inSelected = getPerformersInLocation(selectedLocation).has(id); let inCompare = getPerformersInLocation(compareLocation).has(id); let group = networkHelpers.getGroup(inSelected, inCompare); let display = networkHelpers.findNode(id).display; return { id, group, display, /* inSelected, inCompare, */ }; } // setup nodes from links ret.links.forEach(d=>{ if (ret.nodes.find(c=>c.id === d.source)) { // return } else { // do source let id = d.source; let node = setupNode(id); ret.nodes.push(node) } if (ret.nodes.find(c=>c.id === d.target)) { // return } else { // do target let id = d.target; let node = setupNode(id); ret.nodes.push(node) } }) return groupedGraph(ret, { height: venueComparationheight, color: networkHelpers.color, strokeWidth: d => Math.sqrt(d.weight), groupX: [(200), (width / 2), (width - 200)], onHover: (evt, d) => tooltip.show(evt, d), onLeave: (evt, d) => tooltip.hide(evt, d), groupColorNames: [selectedLocation, "Both", compareLocation] }) }3a:T471,function getGroups() { const getColor = (group, lighter) => { switch (group) { case 1: return '#fabed4'; case 2: return '#ffd8b1'; case 3: return '#fffac8'; } } const getGroup = (prevDiff, prevGroup) => { if (prevDiff <= settings.groupOn) return prevGroup return prevGroup + 1; } const getX = (value) => {return 50+value*10}; // ;xScale(value) console.log(dates); dates.forEach((d, ix) => { d.x = getX(d.value); }) dates.forEach((d, ix) => { const previous = dates[ix-1]; const next = dates[ix+1]; d.prevX = previous ? previous.x : undefined; d.nextX = next ? next.x : undefined; d.prevValue = previous ? previous.value : undefined; d.nextValue = next ? next.value : undefined; d.prevDiff = previous ? Math.abs(previous.value - d.value) : undefined; d.prevGroup = previous && previous.group ? previous.group : 1; d.nextDiff = next ? Math.abs(next.value - d.value) : undefined; d.group = previous ? getGroup(d.prevDiff, d.prevGroup) : 1; d.groupColor = getColor(d.group); }) return dates; }3b:[{"id":"c6c3caccb4e53539d9474ae4c69c3b35baeed40ff7550103b93afd055503c8d24a2175852a2960dccba0e6e2e92f64a524063ac38fb83936ebd3efdd1e9ea9ea","href":"https://static.observableusercontent.com/files/c6c3caccb4e53539d9474ae4c69c3b35baeed40ff7550103b93afd055503c8d24a2175852a2960dccba0e6e2e92f64a524063ac38fb83936ebd3efdd1e9ea9ea","name":"100x160.png","type":"image/png","lastModified":1641474555912.782,"size":276},{"id":"f2bd59b2a67879897764df7d6e5ed7e58819d9a535008c274c1d802077118ca9c02f2ffdcdf25d128b34c8b7ca3f21f3f9f737ab7f24e842c4ee0680ff8f60fd","href":"https://static.observableusercontent.com/files/f2bd59b2a67879897764df7d6e5ed7e58819d9a535008c274c1d802077118ca9c02f2ffdcdf25d128b34c8b7ca3f21f3f9f737ab7f24e842c4ee0680ff8f60fd","name":"image@1.png","type":"image/png","lastModified":1641474808387.689,"size":6032},{"id":"8bc72ecd4edf8955effb88fff1560e1d72b53c371127ab6e6a9301048042fa03e38d5242de1de8766550d9d69f747198ab2f048770e6fffcf2a20c86026ea038","href":"https://static.observableusercontent.com/files/8bc72ecd4edf8955effb88fff1560e1d72b53c371127ab6e6a9301048042fa03e38d5242de1de8766550d9d69f747198ab2f048770e6fffcf2a20c86026ea038","name":"image@2.png","type":"image/png","lastModified":1641474815973.639,"size":6781},{"id":"49a28300b04d8b0aa337277e8574357883c259f443454bcab70f33db89cba9f7451fb46b3c82afc603c24e432702b5cada996f0d09acf0e5391e99dca522ae9d","href":"https://static.observableusercontent.com/files/49a28300b04d8b0aa337277e8574357883c259f443454bcab70f33db89cba9f7451fb46b3c82afc603c24e432702b5cada996f0d09acf0e5391e99dca522ae9d","name":"image@3.png","type":"image/png","lastModified":1641474819496.617,"size":7258}] 30:{"id":"b374d753f207ada8","slug":"networking-dates","title":"4. Adding Automatically Generated Data: Networking Dates","thumbnail":"fa5bcc12545eb7b748dc8453fab7bf4beef04a0cb471e8f6285dbe20f49c3cca","access_level":"public","create_time":"$D2022-01-04T17:51:41.182Z","update_time":"$D2022-01-17T19:08:21.617Z","view_time":null,"first_public_time":"$D2022-01-05T11:49:58.695Z","has_unpublished_edits":null,"fork":false,"stars":3,"starred":false,"owner":{"type":"team","id":"32362545e9f3330d","login":"kallewesterling","name":"Kalle Westerling","avatar_url":"https://avatars-next.observableusercontent.com/f759a45dfbde6f2da561a90bcc8852bd0efd30d6a12ae3031f4d0bedc2464b65","tier":"starter_2024"},"version":2147,"publish_version":2147,"role":null,"authored":false,"authors":[{"type":"individual","id":"a4098a3e48d12fd5","login":"kallewesterling","name":"Kalle Westerling","avatar_url":"https://avatars-next.observableusercontent.com/f759a45dfbde6f2da561a90bcc8852bd0efd30d6a12ae3031f4d0bedc2464b65","tier":null}],"members":[{"type":"individual","id":"a4098a3e48d12fd5","login":"kallewesterling","name":"Kalle Westerling","avatar_url":"https://avatars-next.observableusercontent.com/f759a45dfbde6f2da561a90bcc8852bd0efd30d6a12ae3031f4d0bedc2464b65","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":{"id":"f56ba7f1433fe336","slug":"dissertation-data","title":"Dissertation data","owner":{"id":"32362545e9f3330d","login":"kallewesterling","name":"Kalle Westerling"}},"type":"notebook","body":{"titling":"auto","title":"4. Adding Automatically Generated Data: Networking Dates","cells":[{"id":0,"value":"# 4. Adding Automatically Generated Data: Networking Dates\n\n

The biggest component of my research was made up by the network analysis of the dataset’s performers and their performances together. I have detailed the network further in my dissertation’s text and will focus here mainly on my technological and algorithmic approach to the data, detailing _what_ and _how_ I transformed and visualized (in order to finally analyze) the data.

\n\nIn the figure below, each _node_ (or dot) represents one performer and a line connecting two performers, an _edge_, means that they appeared together in a venue within any given 14-day period at some point between 1930 and 1940. It is important to remember that this figure filters out any performers without a name and any performers who have no connections with other performers involved in the drag scene at the time. The size of the node represents the number of other performers that any given performer is connected to.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":85,"value":"chart()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1512,"value":"Sometimes, visualizations such as this one are called a “hairball” due to their inability to make it easier for us to read data or even identify pattern. Despite the coloring and the connections in the network above, the visualization can hardly be recognized as legible.\n\nTo make the visualization more legible, one can apply a community-detecting algorithm which, depending on the relationship between nodes, assigns them to certain groups. These groups can then assist in emulating gravitational forces inside and between the different groups, which in turn makes such “hairballs” more legible to the human eye.\n\nBut in order to do so, I needed to perform some data processing. This rather lengthy part of this site concerns the processing of the data to make it more legible.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":13,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":16,"value":"$32","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":22,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":26,"value":"$33","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1422,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1426,"value":"$34","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1761,"value":"$35","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1786,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1794,"value":"// solution of wide settings from https://observablehq.com/collection/@cobus/quick-demos\n\nviewof settings = Inputs.form({\n date2: Object.assign(Inputs.range([12,100], {\n value: 12,\n step: 1,\n label: md`**Second** source published (days after source 1)`\n }), {style: '--label-width: 300px'}),\n \n date3: Object.assign(Inputs.range([12,100], {\n value: 15,\n step: 1,\n label: md`**Third** source published (days after source 2)`\n }), {style: '--label-width: 300px'}),\n \n groupOn: Object.assign(Inputs.select([3, 14, 31, 93, 186, 365], {\n label: 'Group on days',\n value: 14\n }), {style: '--label-width: 300px;padding-top:0.25em;'}),\n \n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1790,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1440,"value":"In the following graphic, the sources above are represented one dot each. You can see that, **as a result of the algorithm taking your settings above into consideration, Source 1, Source 2, and Source 3 would be ${new Set(getGroups().map(d=>d.group)).size > 1 ? `divided into ` + new Set(getGroups().map(d=>d.group)).size : 'combined into one'} group${new Set(getGroups().map(d=>d.group)).size > 1 ? 's' : ''}.**","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1446,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1777,"value":"Now, imagine iterating this process over each and every one of the ${numberOfSources} sources in the dataset. What number of grouped dates do we end up with? Let us have a look at the resulting data.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":894,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":890,"value":"## Resulting Data\n\n...","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1628,"value":"Inputs.table(displayResults)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1567,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1559,"value":"### Visualizing the Result\n\nIn the simplified graph below, you can see the result of the date grouping.\n\nThe graph allows you to select two venues from the dataset to compare. The connections between each performer means that they were grouped together in _at least_ one date grouping. I have ensured that the performers are color-coded to display which venue they performed at (or whether they performed in both). I have also started by selecting the two nightclubs that are the subject of one of my dissertation's chapters: Echo Tavern and Club Richman, but you should feel very welcome to change them around and see how the relationship between the clubs change.\n\nIn this simplified graph, the thickness of the connections between the performers signifies the number of date groups that the two performers share.\n\n_Go ahead and choose different venues to compare from the dropdown menus below:_","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":931,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":275,"value":"viewof selectedLocation = Inputs.select(getLocations(), {label: 'Visualize venue', value: 'Echo Tavern (Albany, NY)'})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":371,"value":"viewof compareLocation = Inputs.select([null].concat(getLocations()), {label: 'Compare with', value: 'Club Richman (New York, NY)'}) // set value to none here...","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1533,"value":"

","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":498,"value":"The algorithm found **${getFoundPerformers().inSelectedLocation.length}** performers in the venue selected (${selectedLocationSimple}) and **${getFoundPerformers().inCompareLocation.length}** performers in the comparison venue (${compareLocationSimple}).\n\n${getFoundPerformers().intersection.length ? md`Between all of them, **${getFoundPerformers().intersection.length}** were present at both venues.` : `None of the performers at ${selectedLocation} ever performed together with any performer from ${compareLocation} in the dataset. That is why there are no connections between the two groups below. This also implies that none of the performers in the dataset who performed at ${selectedLocationSimple} seem to have performed at ${compareLocationSimple}.`}","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":670,"value":"swatches(\n [\n [color(selectedLocation), selectedLocation], \n [color(\"Both\"), \"Both\"], \n [color(compareLocation), compareLocation]\n ]\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1285,"value":"html`${showGroupedData().node()}

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":829,"value":"The settings below adjusts the visualization. The height can be adjusted to see more nodes. Feel free to play around with any of the settings. Every time you make a change the network above will reload.","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":833,"value":"viewof venueComparationheight = Object.assign(Inputs.range([100, 1000], { label: \"Adjust height of visualization\", step: 50, value: 200 }), {\n oninput: event => event.isTrusted && event.stopImmediatePropagation(),\n onchange: event => event.currentTarget.dispatchEvent(new Event(\"input\"))\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1882,"value":"If you choose ${chooseTwo(\"Echo Tavern (Albany, NY)\", \"Dwan's Recreation Gardens (Benton Harbor, MI)\")}, you can see that there are no relationship between the two venues. That is because no one who performed at venue 1 also performed at venue 2. Compare to ${chooseTwo(\"Echo Tavern (Albany, NY)\", \"Earle Club (Baltimore, MD)\")} where you can see Bobbie La Marr's importance (as the point in-between the two venues).\n\nYou can also note the connections inside each venue. At times, there are clear clusters _within_ the venue, where some group(s) of performers never really appeared with someone else.\n\nTODO: Finish this...","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1517,"value":"smallNav({\n prev: {url: nav.cleaning.url},\n next: {url: nav.network1.url}\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":34,"value":"scriptDisclaimer","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1466,"value":"ga()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1620,"value":"##### GroupedData","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1923,"value":"chooseTwo = (alt1, alt2, text) => {\n function setVals(evt) {\n viewof selectedLocation.value = alt1;\n viewof selectedLocation.dispatchEvent(new CustomEvent(\"input\", { bubbles: true }));\n \n viewof compareLocation.value = alt2;\n viewof compareLocation.dispatchEvent(new CustomEvent(\"input\", { bubbles: true }));\n }\n\n text = text ? text : `${alt1} and ${alt2}`;\n const ret = html`${text}`\n \n ret.addEventListener('click', setVals);\n \n return ret;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1625,"value":"displayResults = Object.entries(exampleGroupData)\n .map(d=>{\n const [venue, data] = d;\n const groupA = Object.values(data['grouped-by-3-days']);\n const groupB = Object.values(data['grouped-by-14-days']);\n const aDates = groupA.map(d=>d.dates);\n const bDates = groupB.map(d=>d.dates);\n const sameDates = JSON.stringify(aDates) === JSON.stringify(bDates);\n return {\n venue,\n 'grouped-by-3-days': groupA,\n 'grouped-by-14-days': groupB,\n sameDates\n }})\n .filter(d=>d.sameDates === false)\n .map(d=>{\n // console.log(d['grouped-by-3-days'].map(d=>d.dates).map(a=>a.length).join(', '))\n return {\n \"Venue\": d.venue,\n \"Dates △ 3 days\": d['grouped-by-3-days'].map(d=>d.dates)\n }\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1489,"value":"##### Mostly visual additions","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":2021,"value":"tooltip = ({\n style: html``,\n show: (evt, d) => {\n d3.selectAll('.nodes circle').attr('opacity', node => node === d ? 1 : 0.3)\n d3.select('#tooltip').text(d.display).attr(\"style\", `top: ${evt.offsetY + 5}px; left: ${evt.offsetX + 5}px;`)\n },\n hide: (evt, d) => {\n d3.selectAll('.nodes circle').attr('opacity', 1);\n d3.select('#tooltip').attr(\"style\", \"display:none;\");\n }\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":39,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2024,"value":"tooltip.style","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1843,"value":"fullWidth","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":309,"value":"","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1243,"value":"basicStyling","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1831,"value":"","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1871,"value":"tachyons","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1495,"value":"##### Utils","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":496,"value":"getFoundPerformers = (\n location1 = selectedLocation,\n location2 = compareLocation\n) => {\n return {\n inSelectedLocation: [...getPerformersInLocation(location1)],\n inCompareLocation: [...getPerformersInLocation(location2)],\n intersection: getPerformersInIntersection(location1, location2)\n };\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":548,"value":"getPerformersInIntersection = (\n location1 = selectedLocation,\n location2 = compareLocation\n) => {\n return [...getPerformersInLocation(location1)].filter(x => getPerformersInLocation(location2).has(x));\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":515,"value":"getPerformersInLocation = (location = selectedLocation) => {\n return new Set([...network14.links.filter(d=>Object.keys(d.coLocated).includes(location)).map(n=>n.source.node_id), ...network14.links.filter(d=>Object.keys(d.coLocated).includes(location)).map(n=>n.target.node_id)])\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":943,"value":"selectedLocationSimple = selectedLocation.split(' (')[0]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":950,"value":"compareLocationSimple = compareLocation ? compareLocation.split(' (')[0] : ''","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1355,"value":"swatches = (colors) => {\n return html`${colors.map(c =>`\n

\n ${c[1] ? c[1] : ''}\n

`)}`;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":428,"value":"##### Network helpers","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":422,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":680,"value":"color = d3.scaleOrdinal([selectedLocation, \"Both\", compareLocation], d3.schemeCategory10)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2083,"value":"networkHelpers = ({\n findNode: (id) => {\n const node = network14.nodes.find(n=>n.node_id === id);\n if (!node)\n throw new Error(`Node ${id} could not be found.`)\n return node;\n },\n getGroup: (inSelected, inCompare) => {\n if (inSelected && inCompare) {\n return 1\n } else if (inSelected) {\n return 0\n } else if (inCompare) {\n return 2\n }\n },\n color: d3.scaleOrdinal([selectedLocation, \"Both\", compareLocation], d3.schemeCategory10),\n \n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1172,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1103,"value":"focalPoints = (\n location1 = selectedLocation,\n location2 = compareLocation\n) => {\n let hasCompare = getPerformersInLocation(location2).size;\n let hasIntersection = getPerformersInIntersection(location1, location2).size;\n \n let d = {\n group1: {\n x: undefined,\n color: color(0)\n },\n group2: {\n x: undefined,\n color: color(1)\n },\n group3: {\n x: undefined,\n color: color(2)\n }\n }\n\n let unit = width / 6;\n \n if (hasCompare === undefined) {\n d.group1.x = width / 4;\n d.group2.x = 200;\n d.group3.x = 200;\n } else if (hasIntersection === undefined) {\n d.group1.x = unit * 0.01;\n d.group2.x = unit * 3;\n d.group3.x = unit * 5;\n console.log(d.group1.x, d.group2.x, d.group3.x)\n } else {\n console.warn('here!')\n }\n\n return d;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1049,"value":"##### Imports","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":44,"value":"import { network14, exampleGroupData, getNetworkLocations as getLocations, nav, navigate, scriptDisclaimer, fullWidth, smallNav, tachyons, moment, ga } from '@kallewesterling/drag-dataset'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":672,"value":"import {Swatches} from '@d3/color-legend'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1168,"value":"import {groupedGraph, basicStyling} from '@kallewesterling/groupedgraph'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1780,"value":"import {numberOfSources} from '@kallewesterling/2-data-ingestion-and-tagging'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1449,"value":"##### Algorithm tester","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"","database":"$undefined"},{"id":1462,"value":"y = 70","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1457,"value":"dates = [\n {\n label: 'date',\n source: 'Source 1',\n value: 1,\n exampleDate: moment('1930-01-01', \"YYYY-MM-DD\"),\n y\n },\n {\n label: `date + ${settings.date2} days`,\n source: 'Source 2',\n value: settings.date2,\n exampleDate: moment('1930-01-01', \"YYYY-MM-DD\").add(settings.date2, 'days'),\n y\n },\n {\n label: `date + ${settings.date2 + settings.date3} days`,\n source: 'Source 3',\n value: settings.date2 + settings.date3,\n exampleDate: moment('1930-01-01', \"YYYY-MM-DD\").add(settings.date2 + settings.date3, 'days'),\n y\n }\n ];","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1455,"value":"extent = d3.extent(dates, d => d.value)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1453,"value":"xScale = d3.scaleLinear().domain(extent).nice().range(1, width)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1469,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W3b","stdlib":"1"}} 31:"$30:body"