2d:null 2e:["$","title",null,{"children":"Zoom strategies for huge scatterplots with three.js | Benjamin Schmidt | Observable"}] 33:T859,md`# Zoom strategies for huge scatterplots with three.js What I call 'deep' scatterplots are plots with thousands or millions points, each with significant metadata that require zooming or other similar interactions to fully explore. I previously used canvas to visualize [15 million books in the Hathi Trust digital library](http://creatingdata.us/techne/deep_scatterplots/), and wrote up there some of my general strategies especially around tiling and labeling. This notebook is about an issue I didn't explicitly address there, but that becomes more pressing with deep scatterplots; the tradeoff between semantic or geometric zoom. I'll explore it initially with some data that comes from a [UMAP embedding of a word2vec model](https://benschmidt.org/profCloud) with 138,839 points, colorized by sentiment. This is a nice in-between size: it's more than canvas can hold, but about 1% the size of the data I want to work with ultimately. One issue with canvas is that most browsers stutter when rendering more than 10,000 points. I've found @grantcuster's notebooks about three.js and webgl especially useful here, and have borrowed much of his code. ## Balancing semantic and geometric zooms Some of Mike Bostock's old visualizations have d3 users comfortable with the notion that scatter plots can use ["semantic zoom"](https://bl.ocks.org/mbostock/3680957), where points stay the same size when you zoom in, and ["geometric zoom"](https://bl.ocks.org/mbostock/3680999), where points grow as you zoom in. For scatterplots with tens of thousands of points, both of these methods have shortcomings. You can see these shortcomings by switching the zoom strategy below. At high zoom levels, geometric zoom usually leaves you with heavy overplotting of points at high zoom levels; while semantic zoom strategies are usually overcrowded at the overview level if the point size is large enough to support any interactions. Below this chart you can turn on either of those to see how they work. But by default, I do something I call 'balanced zoom' that's generally better for deep scatters, but that seems to not always be obvious. `34:T9b0,md`Fortunately, there's a relatively straightforward way to balance between these two imperatives that I have ticked by default above. In semantic zoom, any given point has size p; in geometric zoom, any given point has size \`p*k\`, where k is the d3 zoom level. To smoothly interpolate between this, we just need to remember that d3-zoom operates on a logarithmic scale; that is, the same amount of mouse wheeling produces zoom of 2, then 4, then 8, etc. So the geometric zoom formula is actually \`size = p*(exp(z))\`, where \`z == log(k)\`, the zoom parameter in a linear domain. From here, we can create a tradeoff parameter \`g\` that ranges between zero and one. The size of any given point is then **\`p*(exp(z*g))\`**. When \`g\` is zero, this reduces simply to \`p\` (semantic zoom); when \`g\` is one, it's geometric zoom. But for values in between, it represents the share of zoom level that should translate into also enlarging points. When using charts with high levels of zoom, almost any intermediate value seems preferable to 0 or 1; I generally just use 0.5, but YMMV. (In theory, this function isn't actually bounded by 0 and 1: you could also set it below zero or above one to switch the direction of growth so points get ever smaller as you zoom in. In practice, I don't think this would ever be useful.) I think this smooth interpolation general looks better than the most widely used alternatives for making deep zoom plots legible (periodically stepping up sizes, as in mapping applications; tricks involving opacity; and bounding point size within a minimum and maximum so that they grow until they're 20 points or so wide). But it's also compatible with any of them. ### A 2-d three.js camera trick. In SVG or canvas, this calculation is relatively straightforward. In pure WebGL, I think it can be done with a shader. In three.js, though, updating the size of points may be a pain compared to moving the camera. But there's a trick that avoids updating any of the underlying geometry buffers: it's possible to change this parameter with camera updates alone, because three.js exposes two parameters that correspond to geometric and semantic zoom: the camera height, and the field-of-view angle. The math here is a little tricky, but inverting one of @grantcuster's trigonometric functions, we can set both parameters in tandem. Using using his field-of-view visualization below, you can see the way that z height and field of view trade off. ` 35:T4bc,md`## Illustration key (*This text is by Grant Custer*) The illustration above is a cutaway side-view of the three.js camera and scene. The horizontal view represents the z axis and the vertical is the y. The black dot represents the camera. The green line shows the height and poistion of the group of dots. The dots are centered around the position \`x: 0, y: 0, z: 0\`. The dotted line is the z position you adjusted above -- the distance between the camera and the points in the z-axis. The purple lines show the field of view angle. What they contain determines what is visible in the visualization. The blue line represents the height of the visualization itself, which is useful for thinking about scale. The dark gray rectangle represents the space between the near and far plane. Anything outside of that space will not be rendered. Note that the planes are defined relative to the position of the camera.`36:T8c5,function drawCamera(canvas_inputs) { let ctx = d3.select("#fov_image").node().getContext('2d') let cx = width/2; let cy = canvas_height/2; ctx.clearRect(0, 0, width, canvas_height) let scaled_z = canvas_inputs.z * canvas_scale; let z_padding = canvas_z_padding * canvas_scale; let y_adjust = -canvas_inputs.y * canvas_scale; // Near and far threshold ctx.beginPath(); ctx.moveTo(width - z_padding - scaled_z + canvas_inputs.near_plane * canvas_scale, 0) ctx.lineTo(width - z_padding - scaled_z + canvas_inputs.far_plane * canvas_scale, 0) ctx.lineTo(width - z_padding - scaled_z + canvas_inputs.far_plane * canvas_scale, canvas_height) ctx.lineTo(width - z_padding - scaled_z + canvas_inputs.near_plane * canvas_scale, canvas_height) ctx.closePath(); ctx.fillStyle = "#ddd"; ctx.fill(); // Z line ctx.beginPath(); ctx.moveTo(width - z_padding, cy + y_adjust) ctx.lineTo(width - z_padding - scaled_z, cy + y_adjust); ctx.setLineDash([4, 4]); ctx.lineWidth = 2; ctx.stroke(); let half_fov = canvas_inputs.fov/2; let half_fov_radians = toRadians(half_fov); let fov_height = 2* Math.tan(half_fov_radians) * canvas_inputs.z; // Field of view triangle ctx.setLineDash([]); ctx.beginPath(); ctx.moveTo(width - z_padding - scaled_z, cy + y_adjust) ctx.lineTo(width - z_padding, cy - (fov_height / 2 * canvas_scale) + y_adjust); ctx.lineTo(width - z_padding, cy + (fov_height / 2 * canvas_scale) + y_adjust); ctx.closePath(); ctx.strokeStyle = "purple"; ctx.lineWidth = 3; ctx.stroke(); // Camera circle ctx.beginPath(); ctx.arc(width - z_padding - scaled_z, cy + y_adjust, 9, 0, Math.PI * 2, true); ctx.lineWidth = 0; ctx.fillStyle = "black"; ctx.fill(); // Screen indicator ctx.beginPath(); ctx.moveTo(width - z_padding, cy - viz_height / 2 * canvas_scale); ctx.lineTo(width - z_padding, cy + viz_height / 2 * canvas_scale); ctx.lineWidth = 4; ctx.strokeStyle = "blue"; ctx.stroke(); // Dots indicator ctx.beginPath(); ctx.moveTo(width - z_padding, cy - points_radius * canvas_scale); ctx.lineTo(width - z_padding, cy + points_radius * canvas_scale); ctx.lineWidth = 6; ctx.strokeStyle = "green"; ctx.stroke(); }37:[] 31:{"id":"9910cfe7991166da","slug":"zoom-strategies-for-huge-scatterplots-with-three-js","title":"Zoom strategies for huge scatterplots with three.js","thumbnail":"681453442fc50a189cb3fd391a946e700153d767018ff244fdcdaf926cbba1c0","access_level":"public","create_time":"$D2019-12-18T18:28:34.331Z","update_time":"$D2021-03-05T16:52:46.237Z","view_time":null,"first_public_time":"$D2019-12-31T19:15:36.392Z","has_unpublished_edits":null,"fork":{"id":"a0610390eee907fa","slug":"understanding-scale-and-the-three-js-perspective-camera","title":"Understanding scale and the three.js perspective camera","owner":{"id":"bf2c1edd2a33aa1b","login":"grantcuster","name":"Grant Custer"}},"stars":52,"starred":false,"owner":{"type":"team","id":"18bbeceb36c8f564","login":"bmschmidt","name":"Benjamin Schmidt","avatar_url":"https://avatars-next.observableusercontent.com/addc2adb5ec95cc72a64820e4a08488bad5d48d95a895eef85a45b136ed66792","tier":"starter_2024"},"version":2284,"publish_version":2284,"role":null,"authored":false,"authors":[{"type":"individual","id":"33354ca265b21f01","login":"bmschmidt","name":"Benjamin Schmidt","avatar_url":"https://avatars-next.observableusercontent.com/addc2adb5ec95cc72a64820e4a08488bad5d48d95a895eef85a45b136ed66792","tier":null}],"members":[{"type":"individual","id":"33354ca265b21f01","login":"bmschmidt","name":"Benjamin Schmidt","avatar_url":"https://avatars-next.observableusercontent.com/addc2adb5ec95cc72a64820e4a08488bad5d48d95a895eef85a45b136ed66792","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":32,"collection":{"id":"129de12642a17c31","slug":"deep-scatterplots","title":"Deep Scatterplots","owner":{"id":"18bbeceb36c8f564","login":"bmschmidt","name":"Benjamin Schmidt"}},"type":"notebook","body":{"titling":"auto","title":"Zoom strategies for huge scatterplots with three.js","cells":[{"id":0,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1963,"value":"function jump(scale = 3700) {\n // A function to continually choose and zoom to a new random word.\n let dat = random_point()\n\n d3.select(\"#viztxt2\").style(\"color\", colorscale(+dat.sentiment)).text(`Focused on: '${dat.word}'`)\n const [x, y] = [+dat.x, -dat.y];\n canvas.transition().duration(3500).call(\n // zoom out on the new point quickly\n zoomBehavior.transform,\n d3.zoomIdentity.translate(width / 2, viz_height / 2).scale(width/95).translate(-x, -y)\n ).transition().duration(7500).call(\n //zoom in slowly\n zoomBehavior.transform,\n d3.zoomIdentity.translate(width / 2, viz_height / 2).scale(scale).translate(-x, -y)\n )\n .on(\"end\", function() {jump()})\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2101,"value":"plot_legend = {\n // This handles starting the jump process.\n [replay, replay2] //trigger replay on button clicks.\n jump()\n return html`

Preparing zoom

` \n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2250,"value":"viewof replay = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1372,"value":"html`

`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2269,"value":"viewof replay2 = html``","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1604,"value":"md` ## Parameterizing the semantic-geometric tradeoff`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1586,"value":"viewof binary = radio({options:[\"Semantic zoom\", \"Geometric zoom\", \"Balanced zoom\"], value:\"Balanced zoom\", title: \"Choose zoom strategy. By default, the 'balanced' version.\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":915,"value":"viewof point_size = slider({min:0.1, max:10, step:0.1, value:1.33, description: \"Base point size (may be affected by zoom\", title: \"Point Size\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1241,"value":"viewof tradeoff = slider({title:\"Geometric zoom share\", value: 0.5, min: 0, format: '.01%', max: 1, step:0.01, description: \"Balance between semantic and geometric zooming. Represents a share of the zoom that goes into the new data. 0 is semantic zooming, where the size is constant in pixels: 100% makes the size grow linearly with zoom: and 50% is the geometric mean (I think)? of the two sizes.\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1320,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2279,"value":"md`# Remaining questions\n\nI think this is a good general strategy, but there are few other remaining issues. The choice of initial parameters still requires hand-tuning. My instinct is that at most zoom levels, the right strategy will have about 1/4 to 1/2 of the screen filled with dots and 1/2 to 3/4 empty; but building that in as a default is a little tricky. Perhaps it should be done simply as a function of the number of points in the scatterplot and the number of pixels on the screen.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1177,"value":"get_camera_z = function(zoom_level) {\n let v\n switch (binary) {\n case \"Semantic zoom\": v = 0; break;\n case \"Geometric zoom\": v = 1; break;\n case \"Balanced zoom\": v = tradeoff; break;\n }\n return viz_height/(Math.exp(zoom_level*v)\n)}\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1755,"value":"zoomBehavior = {\n // adapted from https://observablehq.com/@grantcuster/using-three-js-for-2d-data-visualization\n let d3_zoom = d3.zoom()\n .scaleExtent([10, 400000])\n //.extent([[-30, -30], [30, 30]])\n //.translateExtent([[0, 0], [width, viz_height]])\n .extent([[0, 0], [width, viz_height]])\n \t.on('zoom', () => {\n const e = d3.event.transform\n zoomEvent(e);\n });\n return d3_zoom;\n}\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1978,"value":"canvas = d3.select(\"#gl1\").call(zoomBehavior)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1821,"value":"html`

`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1152,"value":"viewof zoom1 = slider({min: 0, max: 15, step: .25, value:3.5, title:\"Base Zoom level\", description: \"Change this to adjust the zooming distance. The number you adjust here is the log of the d3 zoom value (transform.k)\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2239,"value":"d3.zoomIdentity","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1943,"value":"function setUpZoom() {\n // This probably should just be called directly, not wrapped as a function.\n let initial_scale = zoom1;\n var initial_transform = d3.zoomIdentity\n .translate(width/2, viz_height/2)\n .scale(initial_scale); \n zoomBehavior.transform(d3.select(\"#gl1\"), initial_transform);\n camera.position.set(0, 0, get_camera_z(zoom1));\n \n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1957,"value":"setUpZoom()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1384,"value":"zoom = zoom1// + (animation ==\"True\" ? zoom_modifier : .4)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":390,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1597,"value":"md`The zoom modifier just uses the time to continuously zoom in and out smoothly.`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1539,"value":"zoom_modifier = 0//Math.sin(now / 2000)* 2.3 + 1.2","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":726,"value":"viewof year_size = slider({\n min: 1, \n max: 200, \n value: 200,\n step: 1,\n description: \"Number of years to show\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2225,"value":"random_point = function() {\n let r = Math.floor(Math.random()*data.length)\n // Pick a random word weighted by zipfian frequency.\n r = Math.round(Math.exp(Math.random()*Math.log(data.length)))\n return data[r]\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":812,"value":"md`## Near Plane calculations\n\nIgnore this: it will be the subject of my next notebook in this series, and I'm too lazy to remove it yet.\n\nFirst Year: __${(near_year)}__. Last year: __${far_year}__.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1041,"value":"far_year = near_year + year_size","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":723,"value":"viewof near_year = slider({\n min: 1600, \n max: 2010, \n step: 1, \n description: \"First year\"\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":918,"value":"md`Point settings:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":923,"value":"viewof sizeAttenuation = html``","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":755,"value":"md`## Using these settings in three.js\n\nI set field of view, but the updating zoom here means I set more variables before render than Custer.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1507,"value":"camera = new THREE.PerspectiveCamera(90, width / viz_height, 1, 1000);","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1350,"value":"renderer = {\n const renderer = new THREE.WebGLRenderer({antialias: true, canvas: d3.select(\"#gl1\").node() });\n renderer.setSize(width, viz_height);\n renderer.setPixelRatio(devicePixelRatio);\n return renderer\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1739,"value":"function zoomEvent(zoom) {\n let canvas_inputs = {}\n const scale = zoom.k;\n camera.position.x = -(zoom.x - width/2) / scale;\n camera.position.y = (zoom.y - viz_height/2) / scale;\n camera.position.z = get_camera_z(Math.log(scale));\n camera.fov = needed_fov(viz_height, camera.position.z, scale)\n \n // future compatibility\n camera.near_plane = 1//camera.position.z * 2 // + near_year/1000\n camera.far_plane = 10000//camera.position.z / 2//camera.position.z + far_year/1000\n \n camera.updateProjectionMatrix()\n renderer.render(scene, camera);\n\n canvas_inputs.k = scale\n canvas_inputs.x = camera.position.x\n canvas_inputs.y = camera.position.y\n canvas_inputs.z = camera.position.z\n canvas_inputs.fov = camera.fov\n canvas_inputs.near_plane = camera.near_plane\n canvas_inputs.far_plane = camera.far_plane\n \n drawCamera(canvas_inputs)\n \n return canvas_inputs\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1904,"value":"needed_fov(850, 26, 200)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":801,"value":"pointsMaterial = {\n let point_size_adjuster = 1\n if (binary == \"Semantic zoom\") {\n point_size_adjuster = 2\n } else if (binary == \"Geometric zoom\") {\n point_size_adjuster = 0.25 \n } else if (binary == \"Balanced zoom\") {\n point_size_adjuster = 0.85 \n }\n return new THREE.PointsMaterial({\n size: point_size * point_size_adjuster,\n sizeAttenuation: sizeAttenuation,\n vertexColors: THREE.VertexColors,\n map: circle_sprite,\n transparent: true\n})\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":820,"value":"md`## How to think about scale\n\nThe original visualization here calculates a D3 scale from zoom and width parameters. Here I want the inverse; how do we calculate zoom and field-of-view parameters given a d3-zoom value?`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1156,"value":"function needed_fov(viz_height, camera_z, scale) {\n const fov_height = viz_height/scale\n const half_fov_radians = Math.atan(fov_height/(2 * camera_z))\n const half_fov = toDegrees(half_fov_radians)\n return half_fov * 2\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":862,"value":"md`### Illustration code`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":292,"value":"canvas_z_fit = 1000","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":294,"value":"canvas_z_padding = 40","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":301,"value":"canvas_scale = width/(canvas_z_fit + canvas_z_padding * 2)\n","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":855,"value":"mutable canvas_height = 400 * canvas_scale","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":34,"value":"$36","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":873,"value":"md`### Visualization code`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":877,"value":"viz_height = width * 0.7","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":880,"value":"points_radius = 30","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"scene = {\n const scene = new THREE.Scene();\n scene.background = new THREE.Color(0x2f2f2f);\n scene.add(points);\n return scene;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":127,"value":"points = {\n let pointsGeometry = new THREE.BufferGeometry();\n const buffer = new ArrayBuffer(data.length * 3 * 4);\n const color_buffer = new ArrayBuffer(data.length * 3 * 4);\n const positions = new Float32Array(buffer)\n const colors = new Float32Array(color_buffer);\n\n data.forEach((datum, i) => {\n positions.set(datum.position, i * 3)\n const c = d3.color(colorscale(+datum.sentiment))\n colors.set([c.r/255, c.g/255, c.b/255], i*3)\n // colors.set([0, 0, 1], i*3)\n })\n let sizes = data.map(d => d.size)\n sizes = Float32Array.from(sizes)\n pointsGeometry.setAttribute('color', new THREE.BufferAttribute(colors, 3));\n pointsGeometry.setAttribute('size', new THREE.BufferAttribute( sizes, 1 ))\n pointsGeometry.setAttribute('position', new THREE.BufferAttribute( positions, 3))\n return new THREE.Points(pointsGeometry, pointsMaterial);\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1671,"value":"colorscale = d3.scaleLinear().domain([-0.15, 0, 0.15]).range([d3.interpolatePuOr(0), d3.interpolatePuOr(.5), d3.interpolatePuOr(1)]).clamp(true)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":133,"value":"circle_sprite= new THREE.TextureLoader().load(\n \"https://blog.fastforwardlabs.com/images/2018/02/circle-1518727951930.png\"\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":953,"value":"d3 = require('d3@5')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1084,"value":"function return_tile(depth, i, j) { return d3.tsv(`https://creatingdata.us/data/scatter/hathi/tiles/${depth}/${i}/${j}.tsv`).catch(err => Promise.resolve([])).then(d => d.map(e => {return {position: [+e.x, +e.y, -e.date/1000], ix: +e.ix, name: e.title, data: e, group: e.lc1.slice(1)}}))}\n ","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1089,"value":"function children(depth, i, j) {\n return [[depth * 2, i*2, j*2],[depth * 2, i*2+1, j*2],[depth * 2, i*2, j*2+1],[depth * 2, i*2+1, j*2+1]]\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1046,"value":"function load_points(n, start = [1, 0, 0], depth=0) { \n //if (depth >= 257) {\n // Debugging regress loop.\n // return \"YOWZA\" \n //}\n const requests = []\n return return_tile(...start).catch(err => []).then(\n data => {\n const last_point = data.slice(-1)[0]\n // Break condition; only up to the nth point in the data.\n if (last_point.ix > n) {\n return data.filter(d => (+d.ix <= n)) \n }\n // If unbroken, add points from all children tiles as well using this same function.\n const kids = children(...start).map(ix => load_points(n, ix, depth+1))\n return Promise.all(kids).then(kidders => {\n return [].concat(...kidders, data);\n })\n })\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":951,"value":"old_data = load_points(500).then(p => {p.sort((a, b) => a.group == ''); return p})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1139,"value":"import {slider, radio} from \"@jashkenas/inputs\"\n","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1304,"value":"data = d3.tsv(\"https://benschmidt.org/profCloud/d.tsv\", function(row, i) {\n row.position = [+row.x, +row.y, -1.900]\n row.group = row.word.length\n row.ix = i\n row.size = 1/(i + 1)\n return row\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":1020,"value":"accent = d3.scaleOrdinal(d3.schemeAccent)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":89,"value":"function toRadians (angle) {\n return angle * (Math.PI / 180);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":92,"value":"function toDegrees (angle) {\n return angle * (180 / Math.PI);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":14,"value":"THREE = require(\"three@0.111.0/build/three.min.js\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W37","stdlib":"1"}} 32:"$31:body"