2e:null 2f:["$","title",null,{"children":"Concentration of Measure | Toph Tucker | Observable"}] 34:T5ee,md`${width > 640 ? html`

${getIllo()}

` : ""} # Concentration of Measure

Addendum to [There’s Plenty of Room in the Corners](https://observablehq.com/@tophtucker/theres-plenty-of-room-in-the-corners)

In high dimensions, the volume of an ${tex`n`}-sphere becomes overwhelmingly concentrated very close to the surface. When I wrote about hyperspheres [in 2018](https://observablehq.com/@tophtucker/theres-plenty-of-room-in-the-corners) I didn’t address this, because I mostly learned about it from several people’s wonderful responses. In particular, [Dirk Brockmann](https://twitter.com/DirkBrockmann) wrote to me, in a great PDF: > “Say you have a 3D sphere of radius ${tex`R = 1m`} and you take a shell of thickness ${tex`1cm`} which is 1% of the radius. This shell contains about 3% of the volume of the original sphere. But if you have a 100D sphere, the shell contains already 64% of the [volume].” We can crudely illustrate this by tweaking a graphic from my 2018 essay, which shows slices of the first dimension horizontally, the second vertically, the third horizontally again, the fourth vertically again, and so on, to illustrate how little of an ${tex`n`}-cube lies inside the inscribed ${tex`n`}-sphere in higher dimensions. We can color the sampled points by how far they are from the origin: red for the center, blue for the surface.`35:T45e,// There’s some code left over from when I was smoothly // interpolating between dimensions. Sorry! hyperChart = { const w = Math.min(width, 640), h = w, dim = Math.floor(dimension); const context = DOM.context2d(w, h, 1); var r = radius; if (dim % 2 == 0) { var w0 = d3.interpolate(w, w / r / 3)(dimension % 1); } else { var w0 = w; } var dx = d3.range(3).map(() => 0); for (var i = Math.ceil(dim / 2) - 1; i >= 0; i--) { if (i == Math.ceil(dim / 2) - 1) { dx[i] = w0 / r / 2.5; } else { dx[i] = dx[i + 1] / r / 2.5; } } var dr = dx[0] / 3; context.clearRect(0, 0, w, h); context.fillStyle = "#000000"; context.translate(w / 2, h / 2); data.forEach(d => { context.fillStyle = distance(d) <= r ? color(distance(d) / r) : "#fff"; context.beginPath(); context.arc( d[0] * dx[0] + (d[2] || 0) * dx[1] + (d[4] || 0) * dx[2], (d[1] || 0) * dx[0] + (d[3] || 0) * dx[1] + (d[5] || 0) * dx[2], dr, 0, 2 * Math.PI ); context.fill(); }); context.translate(-w / 2, -h / 2); yield context.canvas; }36:T8d4,{ const w = Math.min(width, 640); const h = 200; const top = 20, bottom = 30; const r = (h - top - bottom) / d3.max(groupedPoints, d => d[1].length) / 2; const m = { top, right: Math.max(r, 15), bottom, left: Math.max(r, 15) }; const x = d3 .scaleLinear() .domain(d3.extent(volumeOverRadius, d => d[0])) .range([m.left, w - m.right]); const y = d3 .scaleLinear() .domain(d3.extent(volumeOverRadius, d => d[1])) .range([h - m.bottom, m.top]); const xAxis = g => g .attr("transform", `translate(0,${h - m.bottom})`) .call( d3 .axisBottom(x) .ticks(w / 80) .tickSizeOuter(0) .tickFormat(d3.format(".0%")) ) .call(g => g .append("text") .attr("x", w) .attr("y", m.bottom - 4) .attr("fill", "currentColor") .attr("text-anchor", "end") .text("Distance from origin as percentage of radius") ); const area = d3 .area() .x(d => x(d[0])) .y0(() => y(0)) .y1(d => y(d[1])); const gradient = DOM.uid(); return svg``; }37:T66d,md`For me this stuff is mostly just fun to think about. The applications feel kind of philosophical, like “If personalities are high dimensional, then almost nobody is normal.” The two communities for which it’s more than trivia seem to be physicists and machine learning researchers. In Dirk’s PDF, he noted that concentration of measure becomes important in statistical physics where the dimension is the number of particles in a box, i.e. something ridiculous like ${tex`10^{23}`}. I don’t know anything about that, but it does remind me of the sense of wonder I got as a kid from realizing that the nature of normal life depends on the number of dimensions we live in. Like when someone told me that if gravity operated in one dimension, it wouldn’t get any weaker as you got farther from a mass. Or when Brian Greene said on his NOVA special that gravity may be so much weaker than the other forces because it’s leaking out into more dimensions. I’d like to try an illustration where, layer by layer, one (hexagonal?) shell of radius is added, with transitions, in several dimensions side-by-side, and you can see how in higher dimensions every increment in radius adds way more, in more directions. I couldn’t have written this without Dirk’s concise and readable PDF. And thanks to Adam Strandberg for [tweeting](https://twitter.com/The_Lagrangian/status/1347692040306487307) about my old essay, prompting me to finally revisit my draft of an addendum! And thanks to Rob and Katherine, who I think stood over my shoulder and gently coaxed me forward at like 2 a.m. for my first draft of the top graphic years ago.`38:Tb02,md`--- ## Further reading Marc Khoury, [Counterintuitive Properties of High Dimensional Space](https://marckhoury.github.io/blog/counterintuitive-properties-of-high-dimensional-space/)

> “Given the rather unexpected properties of high dimensional cubes and spheres, I hope that you’ll agree that the following are somewhat more accurate pictorial representations. Notice that the corners of the cube are much further away from the center than are the sides. The ${tex`d`}-sphere is drawn so that it contains almost no volume but still has radius 1.” John D. Cook, [The empty middle: why no one is average](https://www.johndcook.com/blog/2016/02/20/the-empty-middle-no-one-is-average/) >“In 1945, a Cleveland newspaper held a contest to find the woman whose measurements were closest to average. This average was based on a study of 15,000 women by Dr. Robert Dickinson and embodied in a statue called Norma by Abram Belskie. Out of 3,864 contestants, no one was average on all nine factors, and fewer than 40 were close to average on five factors.” John D. Cook, [Nearly all the area in a high-dimensional sphere is near the equator](https://www.johndcook.com/blog/2017/07/13/concentration_of_measure/) >“Nearly all the area of a high-dimensional sphere is near the equator. And by symmetry, it doesn’t matter which equator you take. Draw any great circle and nearly all of the area will be near that circle. This is the canonical example of ‘concentration of measure.’” Sander Dieleman, [Musings on typicality](https://benanne.github.io/2020/09/01/typicality.html) >“For unimodal high-dimensional distributions, such as the multivariate Gaussian, the mode (i.e. the most likely value) usually isn’t part of the typical set. More generally, individual samples from high-dimensional (and potentially multimodal) distributions that have an unusually high likelihood are not typical, so we wouldn’t expect to see them when sampling. This can seem paradoxical, because they are by definition very ‘likely’ samples — it’s just that there are so few of them! Think about how surprising it would be to randomly sample the zero vector (or something very close to it) from a 100-dimensional standard Gaussian distribution. This has some important implications: if we want to learn more about what a high-dimensional distribution looks like, studying the most likely samples is usually a bad idea. If we want to obtain a good quality sample from a distribution, subject to constraints, we should not be trying to find the single most likely one. Yet in machine learning, these are things that we do on a regular basis.” ${tweet("1010681473161093121")}`39:[{"id":"ec9c64fdc6dc026d85508c0119010fd569025c72686449cadedc9ff8169dfea51738d5d35ee8cbc1e7ca8ec11a1cb0046456c678a6506f75df76d41370a75222","href":"https://static.observableusercontent.com/files/ec9c64fdc6dc026d85508c0119010fd569025c72686449cadedc9ff8169dfea51738d5d35ee8cbc1e7ca8ec11a1cb0046456c678a6506f75df76d41370a75222","name":"Dirk Brockmann - N-sphere.pdf","type":"application/pdf","lastModified":1610683408168.489,"size":62951},{"id":"bc8ee3eed7605826a269c12d8af63e7a1dacd998af1ade2295117eed5a948f99f0e54425816f7a286798307daf01d0225db6aec388a093a4983ca3116b3be68d","href":"https://static.observableusercontent.com/files/bc8ee3eed7605826a269c12d8af63e7a1dacd998af1ade2295117eed5a948f99f0e54425816f7a286798307daf01d0225db6aec388a093a4983ca3116b3be68d","name":"image.png","type":"image/png","lastModified":1610863201410.514,"size":17245}] 32:{"id":"0156e7b347ca5ffd","slug":"concentration-of-measure","title":"Concentration of Measure","thumbnail":"5557526720f92d9330da01e8b923b23856a87d5cf772dae18b181487f799d9f4","access_level":"public","create_time":"$D2018-07-28T20:08:24.788Z","update_time":"$D2022-01-26T03:10:43.871Z","view_time":null,"first_public_time":"$D2021-01-17T06:23:51.699Z","has_unpublished_edits":null,"fork":false,"stars":16,"starred":false,"owner":{"type":"team","id":"97290e936c339d8e","login":"tophtucker","name":"Toph Tucker","avatar_url":"https://avatars-next.observableusercontent.com/3706c31a8f9c59bb716c022c04b9c073144cd1368b78ac85f49139be6c93c34e","tier":"pro_2026"},"version":1275,"publish_version":1275,"role":null,"authored":false,"authors":[{"type":"individual","id":"0bb2842b01e0c652","login":"tophtucker","name":"Toph Tucker","avatar_url":"https://avatars-next.observableusercontent.com/3706c31a8f9c59bb716c022c04b9c073144cd1368b78ac85f49139be6c93c34e","tier":null}],"members":[{"type":"individual","id":"0bb2842b01e0c652","login":"tophtucker","name":"Toph Tucker","avatar_url":"https://avatars-next.observableusercontent.com/3706c31a8f9c59bb716c022c04b9c073144cd1368b78ac85f49139be6c93c34e","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":2,"collection":{"id":"ee48d2d87f751f74","slug":"sketches","title":"Sketches","owner":{"id":"97290e936c339d8e","login":"tophtucker","name":"Toph Tucker"}},"type":"notebook","body":{"title":"Concentration of Measure","cells":[{"id":0,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":268,"value":"legend({\n color: d3.scaleDiverging([0, 0.5, 1], color),\n title: \"Distance from origin as percentage of radius\",\n tickFormat: d3.format(\".0%\")\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":278,"value":"md`As the number of dimensions increases, more of the points are bluer, meaning closer to the surface of the hypersphere. (You can increase the radius to show more points, which becomes very slow to render in higher dimensions.)`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"slider({title: \"Dimensions\", min: 1, max: 6, step: 1, view: viewof dimension})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":43,"value":"slider({\n title: \"Radius\",\n min: 2,\n max: 6,\n step: 1,\n view: viewof radius\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":9,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":761,"value":"md`These points are just samples from the ${tex`n`}-sphere, so it’s not perfectly representative. Below, you can see the points stacked according to their distance from the origin, with the unevenness resulting from the sampling; the shaded area shows the true distribution of volume:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":597,"value":"slider({\n title: \"Dimensions\",\n min: 1,\n max: 6,\n step: 1,\n view: viewof dimension\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":84,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":845,"value":"md`In 1D, the distribution of volume as radius increases is uniform: there are two points at ${tex`r = 1`}, and two at ${tex`r = 2`}, and so on. In 2D, the share of volume increases linearly as radius increases: ${tex`2\\pi`} at ${tex`r = 1`}, and ${tex`4\\pi`} at ${tex`r = 2`}, and in general ${tex`2\\pi r`} at ${tex`r`}: that is, the equation for the circumference of a circle, which you can think of as the amount you add to the volume (i.e. area) for each increment in radius. In 3D, it’s ${tex`4\\pi r^2`}, the equation of the surface area of a sphere.\n\nDrag the slider to choose a range of radii; the table shows what percentage of the volume is contained within that slice for different dimensions:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":851,"value":"viewof range = rangeInput({ min: 0, max: 1, step: 0.01, value: [0.7, 1] })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":880,"value":"table = html`\n\n \n \n \n \n\n\n ${tableData.map(\n ([dim, vol]) => html`\n \n \n `\n )}\n\n

Dimensions	\n \n Percentage of volume in\n ${circleRange}\n \n
${dim}	${format(vol)}

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":94,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":100,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":224,"value":"md`---\n\n## Appendix`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":525,"value":"// Slider with title and numeric value, to be used with synchronized views\nslider = ({ min, max, step, view, title }) =>\n html`

${title}

\n ${view.bind(html``)}\n ${view.bind(html``)}\n `","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":603,"value":"viewof radius = new View(2)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":516,"value":"viewof dimension = new View(1)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1071,"value":"md`### Figure 1: Dots`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":171,"value":"w = Math.min(640, width)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":464,"value":"color = d3.interpolateRdBu","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":48,"value":"data = getPoints(dimension, radius)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1072,"value":"md`### Figure 2: Distribution`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":343,"value":"volumeOverRadius = [\n [0, 0],\n ...d3\n .pairs(d3.range(0, radius + 0.11, 0.1))\n .map(([r0, r1]) => [\n r0 / radius,\n (volume(r1, dimension) - volume(r0, dimension)) /\n volume(radius, dimension)\n ])\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":374,"value":"groupedPoints = d3.groups(\n data.map(d => distance(d) / radius).filter(d => d <= 1),\n d => d\n)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1074,"value":"md`### Figure 3: Table with range slider`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":895,"value":"// For when you may get very close to 0 and 100%, and the distinction remains important, this shows percentages with just enough precision to distinguish them from those extremes\nformat = n => {\n const f = d3.format(\".0%\")(n);\n const p = n * 100;\n const closenessTo1 = Math.abs(Math.floor(Math.log10(100 - p)));\n const closenessTo0 = Math.abs(Math.floor(Math.log10(p)));\n return n === 0 || n === 1\n ? f\n : f === \"100%\"\n ? p.toFixed(Math.min(closenessTo1, 100)) + \"%\"\n : f === \"0%\"\n ? p.toFixed(Math.min(closenessTo0, 100)) + \"%\"\n : f;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":863,"value":"r = d => d * 20","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":858,"value":"arc = d3.arc()({\n innerRadius: r(range[0]),\n outerRadius: r(range[1]),\n startAngle: 0,\n endAngle: 2 * Math.PI\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":861,"value":"circleRange = svg``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":874,"value":"tableData = [...d3.range(1, 10), ...d3.range(10, 101, 10)].map(dim => [\n dim,\n (volume(range[1], dim) - volume(range[0], dim)) / volume(1, dim)\n])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":552,"value":"html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1259,"value":"// Header illustration\ngetIllo = () => {\n const w = width + 28,\n h = 250;\n const context = DOM.context2d(w, h, 1);\n const getRandomColor = () => `hsl(${Math.random() * 360},50%,50%)`;\n\n context.canvas.style.margin = \"0 -14px 0 -14px\";\n context.canvas.style.position = \"relative\";\n context.canvas.style.zIndex = \"-1\";\n context.canvas.style.imageRendering = \"pixelated\";\n context.globalCompositeOperation = \"xor\";\n\n const data = d3.range(3).map(d => ({\n cx: d3.randomUniform(w * 0.7, w * 0.9)(),\n cy: d3.randomUniform(h * 0.2, h * 0.8)(),\n r: (Math.random() * h) / 6,\n c: getRandomColor(),\n t0: 0\n }));\n\n let dr = 4;\n d3.timer(t => {\n if (t > 2000) return true;\n data.forEach(d => {\n if (t < d.t0) return;\n context.strokeStyle = d.c;\n context.beginPath();\n context.arc(d.cx, d.cy, d.r, 0, Math.PI * 2);\n context.stroke();\n d.r += dr;\n dr *= 0.95;\n });\n });\n\n return context.canvas;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":749,"value":"md`### Imports`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":11,"value":"import {distance, getPoints, volume} from \"@tophtucker/theres-plenty-of-room-in-the-corners\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":333,"value":"import { View } from \"@mbostock/synchronized-views\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":111,"value":"import {tweet} from \"@mbostock/tweet\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":266,"value":"import { legend, ramp } from \"@d3/color-legend@704\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":849,"value":"import { rangeInput } from \"@mootari/range-slider\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"d3 = require(\"d3\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W39","stdlib":"1","titling":"auto"}} 33:"$32:body"