2c:null 2d:["$","title",null,{"children":"Correlation Filtering | Boris Vishnevsky | Observable"}] 32:T40e,md`In the world of correlations the most famous is the [Pearson correlation coefficient](https://en.wikipedia.org/wiki/Pearson_correlation_coefficient). It is well-known that the Pearson correlation coefficient works reliably as statistical measure of linear dependence between two variables when applied to normally-distributed data. Under normality, one of the advantages of the Pearson product-moment correlation is its interpretability. For a geometrical interpretation of the correlation coefficent we usually attempt to draw a line of best linear square fit through the data of two variables. The value of the Pearson correlation coefficient indicates how far away the data points are to this line, and its sign indicates the direction of the linear relationship between variables. It is interesting that there are at least [thirteen ways to interpret the Pearson correlation coefficient](https://www.stat.berkeley.edu/~rabbee/correlation.pdf). We will not discuss those interpretaions here, but advise to read the cited article. `33:T67d,md` Using a slider to produce graphs of correlation matrix (see approach #3 from [the list above](#points) allows us to discover the interesting patterns and form hypotheses that could be semantically and statistically verified at the next steps of analysis. The goal of the following paragraph is to show heterogeneity of information that we can get by delving into correlation matrix. Let start from considering first the strongest positive corrrelations: - In the correlation range [.8-1] there is only one strong positive correlation link between variables *Rad* and *Tax*. This link indicates that the accessibility to radial highways is strongly correlated with tax payments that measures the cost of public services. This fact does not contradict the common sense. - At the correlation range [.72,.9] the second component of the graph consists from two links [*Age* - *Nox*] and [*Nox* - *Indus*]. It seems highly probable that the level of hazard is strongly correlated with both: the proportion of the aged houses and proportion of non-retail business occupying space in the town. - At the correlation range [.7,1] we see link added between variables [*Tax* - *Indus*], it make's one connected component from five previously considered variables: [*Rad* *Tax* *Indus* *Nox* *Age*]. - At the correlation range [.68,1] we see added new link [*RM*-*Medv*] which is a separate component. Again, positive correlation between "average number of rooms" and "Median value of owner-occupied homes" is pretty easy to explain. - At the correlation range [.65,1] we see a new component [*ZN*-*DIS*] together with extending the biggest correlation component. `34:T513,md` There is also set of strong negative correlations between variables that can be easily interpreted. - At the correlation range [-.76-1] [*DIS*-*NOX*] - whereby longer distances to five Boston employment centers then lesser level of nitric oxide health hazard - At the correlation range [-.72,-1] we can see the two components. Firs the added link is [*DIS*-*AGE*] - longer distances to five Boston employment centers implies less aged houses. Second, the new component [*LSTAT*-*MEDV*] stated that median price of the house negatively correlated with %of lower status of the population - At the correlation range [-.7,-1] we see added link [*DIS*-*INDUS*] - the distance to five Boston employment centers is negatively correlated with the proportion of non-retail business acres per town. In other words bigger distance to Boston employment centers provided less industry-noise, heavy traffic, and unpleasant visual effects - At the correlation range [-.59,-1] [*LSTAT*-*RM*] new link indicate that quantity of rooms is negatively correlated with increasing the % of lower status of the population - At the correlation range [-.51,-1] we see [*AGE*-*ZN*], the proportion of owner-occupied units built prior to 1940 negatively correlated with the proportion of big sized residential land. `35:T573,chart = (Links, Nodes, size = 400) => vl .layer( vl .markRule() .data(Links) .encode( vl .x() .fieldQ('x1') .axis({ labels: false, ticks: false, grid: false, tickExtra: false, domain: false }) .title(''), vl .y() .fieldQ('y1') .axis({ labels: false, ticks: false, grid: false, tickExtra: false, domain: false }) .title(''), vl.x2().fieldQ('x2'), vl.y2().fieldQ('y2') // vl.color().value('#000'), // vl // .color() // .fieldQ('cor') // .scale({ domain: [0, 1], }) ), vl .markCircle() .data(Nodes) .encode( vl.x().fieldQ('x'), vl.y().fieldQ('y'), vl.text().fieldN('label'), vl.color().value('#000'), vl.size().value(50) ), vl .markText() .data(Nodes) .encode( vl.x().fieldQ('textX'), vl.y().fieldQ('textY'), vl.text().fieldN('label') ) ) .config({ view: { stroke: 'transparent' } }) .width(size) .height(size) .render()36:[{"id":"195679337d9a1e2afb141a6a9aca89c36bd42be9364bff6d560a1a34cccfa6561e403de9dcee42ae0e14b317da46135b5fb4168e469867fdf4d19a9bac5bed66","href":"https://static.observableusercontent.com/files/195679337d9a1e2afb141a6a9aca89c36bd42be9364bff6d560a1a34cccfa6561e403de9dcee42ae0e14b317da46135b5fb4168e469867fdf4d19a9bac5bed66","name":"BostonVars@2.csv","type":"text/csv","lastModified":1609093462817.302,"size":759},{"id":"e8e81ce7a05b2968367f9862736bb4f251f1ce7d41298c0afbe93ab97c931b022621c871e9d517a2b0fce5a02ed783adc733e7f0e4b6076a21f0e27c8912b867","href":"https://static.observableusercontent.com/files/e8e81ce7a05b2968367f9862736bb4f251f1ce7d41298c0afbe93ab97c931b022621c871e9d517a2b0fce5a02ed783adc733e7f0e4b6076a21f0e27c8912b867","name":"untitled (16).png","type":"image/png","lastModified":1609093462817.302,"size":248648}] 30:{"id":"d736d3ec05856b6c","slug":"correlation-filtering","title":"Correlation Filtering","thumbnail":"13acdb0cab5681336e5f66091e37675f9106d999f7ac488288c5b5e93df123c5","access_level":"public","create_time":"$D2020-12-27T18:24:22.817Z","update_time":"$D2021-01-04T23:35:50.587Z","view_time":null,"first_public_time":"$D2020-12-27T18:26:14.155Z","has_unpublished_edits":null,"fork":true,"stars":26,"starred":false,"owner":{"type":"team","id":"50fe3bcda3819f34","login":"borisvish","name":"Boris Vishnevsky","avatar_url":"https://avatars-next.observableusercontent.com/0d895f263623b3c8ceb79a9aba991a99936e2e85ae4ef045fd69fb829177aa37","tier":"starter_2024","bio":""},"version":2477,"publish_version":2477,"role":null,"authored":false,"authors":[{"type":"individual","id":"55d1b7de9593d593","login":"borisvish","name":"Boris Vishnevsky","avatar_url":"https://avatars-next.observableusercontent.com/0d895f263623b3c8ceb79a9aba991a99936e2e85ae4ef045fd69fb829177aa37","tier":null,"bio":""}],"members":[{"type":"individual","id":"55d1b7de9593d593","login":"borisvish","name":"Boris Vishnevsky","avatar_url":"https://avatars-next.observableusercontent.com/0d895f263623b3c8ceb79a9aba991a99936e2e85ae4ef045fd69fb829177aa37","tier":null,"role":"editor"}],"comments":[{"id":"5f8111665df4a00f","reply_to_id":null,"content":"FYI: See https://observablehq.com/@mariodelgadosr/dow-jones-industrial-average-correlation-analysis-with-ar use of Datalib's summary function to create a data profile of the data to identify/isolate all numeric columns in a dataset with both measures and dimensions.","create_time":"$D2022-02-07T10:13:55.689Z","update_time":null,"resolved":false,"user":{"type":"individual","id":"1d3988d482a8ec0f","login":"mariodelgadosr","name":"Mario Delgado ","avatar_url":"https://avatars-next.observableusercontent.com/2d2982ad09c4ca9dfd6570810ae00301e0db6ef798c4f7025ea03d4325d4807d","tier":null}}],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":9,"collection":{"id":"6ad523ac4dcb4b45","slug":"instagram-shares","title":"Observable's Instagram Shares","owner":{"id":"f35c755083683fe5","login":"observablehq","name":"Observable"}},"type":"notebook","body":{"titling":"auto","title":"Correlation Filtering","cells":[{"id":0,"value":"md`# Correlation Filtering`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2076,"value":"md`By [Boris Vishnevsky](https://observablehq.com/@borisvish/) and [Yuri Vishnevsky](https://observablehq.com/@yurivish/)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1090,"value":"md` The goal of this notebook is to present a simple technique for [exploratory data analysis](https://en.wikipedia.org/wiki/Exploratory_data_analysis) of a correlation matrix. *Correlation Filtering*, presented here, is a data mining tool that allows a human to efficiently extract and observe useful [information](https://en.wikipedia.org/wiki/Correlation_does_not_imply_causation) contained in a correlation matrix.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2066,"value":"FileAttachment(\"untitled (16).png\")\n .image()\n .then(im => {\n im.style.maxWidth = '475px';\n return im;\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1062,"value":"$32","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1082,"value":"md`There are a few approaches to analyzing the structure of a matrix of pairwise correlations. Most important of them are based on a transformation matrix using a few [principal components]() to simplify and clean the matrix structure. [Seriation](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2527533/) based on simultaneous direct reordering rows and columns of the correlation matrix to make the underlying structure clearly visible. We name the approach that we presented here *Correlation Filtering*. As far as we know this technique first appears in the area of applied [biological research](https://yandex.ru/search/?text=%D0%BA%D0%BE%D1%80%D1%80%D0%B5%D0%BB%D1%8F%D1%86%D0%B8%D0%BE%D0%BD%D0%BD%D1%8B%D0%B5%20%D0%BF%D0%BB%D0%B5%D1%8F%D0%B4%D1%8B%20%D0%A2%D0%B5%D1%80%D0%B5%D0%BD%D1%82%D1%8C%D0%B5%D0%B2%D0%B0&lr=102798) at the end of 1950s, and is still used, mostly in that area.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1581,"value":"md`The idea is to allow the analyst to concentrate on the inter-connectivity of the correlated variables defined in the small interval of the values of correlation, then shift the interval and analyze the new correlation structure. This process repeats until the full correlation structure is researched. *Correlation Filtering* works equally well with Pearson and many other types of correlation.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1463,"value":"md` We may start analysis from reviewing the full picture of the correlation graph. The full correlation graph reflects the correlation matrix, with variables represented by nodes and correlations represented by edges. While the picture looks beautiful because of graph symmetry it does not give any insight about data. This is because we look at all correlations at once. The full correlation graph is determined by interval of correlations in range [-1,1]. `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2087,"value":"chart(links(filterCor(-1., 1.)), Nodes)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1590,"value":"md`For explanatory analysis we may be interested in the filtering out some correlations by using a smaller range of correlation values. We may be interested in looking to strongly correlated variables with correlations in the intervals [.8,1] and/or [-.8,-1]. What variables would be involved at those levels of correlation? What would the inter-connectivity structure look like? \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1600,"value":"md`Another use case would be to explore weakly (linearly) dependent variables corresponded to correlations in the interval [-.2,.2]. It could be just statistical noise. But in our experience the structure of weak correlations often is important to consider, especially when it's centred around one variable. The structure of weak correlations sometimes more stable than structure of strong correlations. Using as a tool of trade the sliding window of correlations may allow us to better understand the dataset at hand. *Correlation Filtering* allows us to solve many tasks of explanatory data analysis by filtering out correlations not embedded in the chosen correlation range.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1478,"value":"md` To illustrate the use of *Correlation Filtering* we chose the Boston Housing dataset. For us it is a toy example that provids a diverse correlation matrix based on the real life dataset [published in 1978](https://deepblue.lib.umich.edu/bitstream/handle/2027.42/22636/0000186.pdf?sequence=1). The Boston Housing dataset includes 506 rows for the 14 following variables:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1617,"value":"html`\n

\nCRIM:\tper capita crime rate by town\n
\nZN:\tproportion of residential land zoned for lots over 25000 sq.ft.\n
\nINDUS:\tproportion of non-retail business acres per town.\n
\nCHAS:\tCharles River dummy variable (1 if tract bounds river; 0 otherwise)\n
\nNOX:\tnitric oxides concentration (parts per 10 million)\n
\nRM:\taverage number of rooms per dwelling\n
\nAGE:\tproportion of owner-occupied units built prior to 1940\n
\nDIS:\tweighted distances to five Boston employment centres\n
\nRAD:\tindex of accessibility to radial highways\n
\nTAX:\tfull-value property-tax rate per $10,000\n
\nPTRATIO:\tpupil-teacher ratio by town\n
\nBLACK:\t1000(Bk - 0.63)^2 where Bk is the proportion of the Black population by town\n
\nLSTAT:\t% lower status of the population\n
\nMEDV:\tMedian value of owner-occupied homes in $1000s\n

\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1638,"value":"md`Let start by viewing the heatmap of the full correlation matrix. The picture below shows full correlation structure. You can inspect individual correlation values by hovering over individual cells.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":103,"value":"vl\n .markRect()\n .data(fullCor)\n .encode(\n vl.x().fieldN('key1'),\n vl.y().fieldN('key2'),\n vl.color().fieldQ('cor'),\n vl.tooltip().fieldQ('cor')\n )\n .render()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1646,"value":"md`The corresponding full graph of correlation structure is shown above.\n\nLet's make our analysis interactive by adding slider that would allow us dynamically define a range (window) of correlations inside the the maximal correlation range of [-1,1]. Correlations enclosed in the window are presented as links on the graph. Changing the window size and shifting the window along whole correlation range allow to see different correlation structures. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2287,"value":"points = md`Here are a few ways to use slider for analysis: \n1. Define a sliding window on a range of correlations and move it to review the graph structure and it's changes\n2. Review the set of the charts corresponding to the set of the non-overlapping ranges\n3. Fix one side of the slider and move the other side. This way we can review the cumulative picture of the correlation structure\n4. Consider separately and/or together the positive and negative correlations\n5. Consider separately and/or together the strong and weak correlations \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":528,"value":"viewof thresholds = rangeSlider({\n min: -1,\n max: 1,\n value: [-1, 1],\n title: 'High and Low Thresholds for Correlations',\n format: stepFormat,\n theme: themes.Flat\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1230,"value":"chart(Links, Nodes)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1635,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2120,"value":"md`At this stage of explanatory analysis we may assume that three sets of initially measured variables could be represented in the future analysis by the following three aggregated factors:\n- 1) [*ZN*-*DIS*] - a recreational factor. It linking together the proportion of big residential land positively correlated with increasing weighted distance to five Boston employment centres. \n- 2) [*RM*-*Medv*] - a home price factor, linking together median price of the homes and quantity of the house rooms \n- 3) [*Rad* *Tax* *Indus* *Nox* *Age*] - a closeness-to-jobs factor defined by positively correlated five variables.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1724,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2126,"value":"md` \nIt is worth emphasizing that all information that we just discovered is available and contained in the full correlation matrix of the Boston Housing dataset. *Correlation Filtering* is used here as a mining tool that allows a human to efficiently extract and observe the useful analytical information.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2049,"value":"md` Another example of analysis using *Correlation Filtering* is following the analytical approach #2 from [the list above](#points), which involves reviewing multiple charts corresponding to the set of the non-overlapping correlation ranges. `","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1282,"value":"html`

${mini}`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2317,"value":"md`This grid of charts shows the decomposition of the correlation matrix of the Boston Housing dataset into ten non-overlapping correlation slices. In pictures #5 and #6 we can see the dummy variable CHAS that captures the amenities of riverside locations. CHAS is the root node of the weak correlations between variables. Discovering this fact may help us to decompose the original Boston housing dataset to two more homogenous parts: one dataset of the areas bounded to the Charles River and another dataset for areas that are not bounded to the Charles River.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2352,"value":"md`In conclusion, *Correlation Filtering* is useful tool for exploratory data analysis and hypothesis generation. The story of this tool has just began. :-)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1481,"value":"md``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2148,"value":"md`---\n\n### Appendix: Code`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"data = newData","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":16,"value":"dl = require('datalib')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":32,"value":"flds = Object.keys(data[0])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":44,"value":"cor = {\n let out = [];\n for (let i = 0; i < flds.length; i++) {\n for (let j = i + 1; j < flds.length; j++) {\n let obj = {\n key1: flds[i],\n key2: flds[j],\n cor: Math.min(1, Math.max(dl.cor(data, flds[i], flds[j]), -1))\n };\n out.push(obj);\n }\n }\n return out;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1847,"value":"fullCor = {\n let out = [];\n for (let i = 0; i < flds.length; i++) {\n for (let j = 0; j < flds.length; j++) {\n let obj = {\n key1: flds[i],\n key2: flds[j],\n cor: Math.min(1, Math.max(dl.cor(data, flds[i], flds[j]), -1))\n };\n out.push(obj);\n }\n }\n return out;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":634,"value":"TreshHoldInterval = format(HThreshold - LThreshold)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":644,"value":"format = d3.format('.2')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":252,"value":"Nodes = {\n let out = [];\n let angle = (2 * Math.PI) / flds.length;\n for (let i = 0; i < flds.length; i++) {\n let angle_i = angle * i;\n let obj = {\n label: flds[i],\n angle: angle_i,\n x: CircleRadius * Math.cos(angle_i),\n y: CircleRadius * Math.sin(angle_i),\n textX: (CircleRadius + textOffset) * Math.cos(angle_i),\n textY: (CircleRadius + textOffset) * Math.sin(angle_i)\n };\n out.push(obj);\n }\n return out;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":662,"value":"function GetNode(sLabel) {\n return Nodes.find(x => x.label === sLabel);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":480,"value":"filteredCor = filterCor(LThreshold, HThreshold)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1260,"value":"filterCor = (lo, hi) => cor.filter(x => x.cor >= lo && x.cor <= hi)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1233,"value":"Links = links(filteredCor)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":727,"value":"links = cor => cor.map(d => {\n let node1 = GetNode(d.key1);\n let node2 = GetNode(d.key2);\n return { x1: node1.x, y1: node1.y, x2: node2.x, y2: node2.y, cor: d.cor };\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1488,"value":"DataFields","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":738,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1309,"value":"stepFormat = d3.format('.2f')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1540,"value":"filteredCor","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1266,"value":"mini = {\n let step = 0.2;\n return Promise.all(\n d3.range(-1, 1, step).map(\n async lo =>\n html`

${await chart(\n links(filterCor(lo, lo + step)),\n Nodes,\n 125\n )}

\n ${stepFormat(lo)} to ${stepFormat(lo + step)}

`\n )\n );\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1192,"value":"textOffset = 50 - CircleRadius","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":209,"value":"CircleRadius = 45","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":533,"value":"LThreshold = thresholds[0]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":534,"value":"HThreshold = thresholds[1]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":153,"value":"import { text, slider, radio, button } from \"@jashkenas/inputs\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":642,"value":"d3 = require('d3@6')","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":526,"value":"import { rangeSlider, themes } from '@mootari/range-slider'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":872,"value":"import { datasets } from \"@tmcw/rdatasets\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":904,"value":"newData = datasets.Boston.data.then(data => data.map(parseObject))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":957,"value":"parseObject = obj =>\n Object.fromEntries(\n Object.entries(obj)\n .filter(([k, v]) => k != '')\n .map(([k, v]) => [k, +v])\n )","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":874,"value":"boston = datasets.Boston","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":924,"value":"boston.data","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":885,"value":"boston.raw","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1486,"value":"DataFields = attachment","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1167,"value":"attachment = FileAttachment(\"BostonVars@2.csv\").csv({ typed: true })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1351,"value":"md`Reference for Boston Housing Data fields [Names & Descriptions](https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":11,"value":"import { vl } from \"@vega/vega-lite-api\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1285,"value":"html`