23:null 24:["$","title",null,{"children":"WebSocket-powered notebook editing tools | Bryan Gin-ge Chen | Observable"}] 29:T15da,md`# WebSocket-powered notebook editing tools *Inspired by [@mootari](/@mootari)'s posts in [this forum thread](https://talk.observablehq.com/t/backup-method/1628/13). (Use of [ObservableHQ's logo](https://observablehq.com/@observablehq/logo) does not imply any endorsement of the use of this notebook on their part!)* ## Intro and setup If you've had to make a bunch of repetitive edits to a notebook, you may have wondered if there's a way to control the editor with a script or macro. In fact, an Observable notebook can't access the Observable editor app since the live code is run from an iframe hosted on a different domain (\`your_username.observableusercontent.com\` versus \`observablehq.com\`) and the [same-origin policy](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy) gets in the way. ~~However, a notebook _can_ communicate with the Observable servers in the same way that the editor does (at least for now), namely by opening a [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)~~. **(WebSocket access has been restricted and this notebook no longer works. This is left up as a hint at what might be possible in a future with true notebook meta-programming.)** This greatly enhances our editing power, as I hope to demonstrate here! You can use this notebook to perform some simple code-driven edits to any notebooks that you have edit access to. After loading your notebook (unfortunately a bit involved), you can: - revert to an older version, - pin or unpin all cells (as well as reset the pin state to that in a given version), - perform a regex-based search and replace on the text of the cells, - copy cells into it from another notebook, or - visually reorder the cells. *Let's get started...* ### Step 1: enter a token The Observable WebSocket server will only accept a connection if it opens with a "hello" message with a valid session token attached. So the first thing you'll need to do in order to get this notebook to "work" is to get such a token. Here's one way: - open your browser's cookie store (in the "Application" tab of Chrome dev tools, or the "Storage" tab of Firefox's dev tools), - find the value under the key \`T\` in the \`observablehq.com\` cookie, **or** - If you're using Firefox and you don't see it there, look in the Network tab of the dev tools and find any of the (relatively recent) GET requests to \`api.observablehq.com\`; the "Cookies" tab for that request should have a \`T\` key. > ☠️ **WARNING**! DO NOT SHARE THIS TOKEN WITH ANYONE YOU DO NOT TRUST! It can be used to impersonate you on the Observable site! Before continuing further, I HIGHLY recommend reading over the rest of this notebook to make sure you understand what it does. For instance, you may want to make sure that this token is not being uploaded somewhere else and that the edit commands truly do what you expect. - copy+paste the token into the input below. ⚠️ This token will expire eventually, and you'll have to replace it manually when that happens. `2a:T86f,md`### Step 2: enter notebook data from Observable API The next thing you'll need to do to use this notebook is to specify a notebook to edit / observe (you'll need to have edit privileges for that notebook). For instance, if you fork this notebook, you can even use this notebook to edit itself! In an earlier prototype, there was a form here that let you enter the notebook ID (aka the hex string in the URL before publishing). This plus the token above is sufficient to open an edit WebSocket connection. However, to do any real editing, we'll need information about the state of the notebook (specifically, we need the version number of the notebook as well as the internal cell "node id"s, the source code ("value") of the cells, and pinning states of all of the cells). Without this information, about all we can do is listen to edit events that are triggered by editing in other tabs / windows. That's kinda interesting too, but we'll be a bit more ambitious here. The easiest way to get all that info is to read it from the data returned by the Observable API when the notebook is loaded. Unfortunately, we can't fetch from the API with code running here due to CORS restrictions, so to proceed you'll have to do one of the following (options 2 and 3 from [the "Getting the source" list in this notebook](https://observablehq.com/@bryangingechen/exporting-notebooks-with-their-source#getSource)): - copy and paste the API response string into the text area below and hit "Submit", or - download a notebook object using a script and upload it using the form below. (The other options from that list will not work because we need to use notebook data which is synced with what is on Observable's servers.) ⚠️ This notebook does not support editing notebook versions which occur before a notebook "revert" event. These events occur when you've shared a notebook, edited it, and then clicked the "Revert" link in the top-right menu to go back to the shared version. Enter the API response data for a version of the notebook after the revert to get things to work. When you've done so, the title of the notebook should appear beneath the form.`2b:T4b1,md`${sock.socket ? `The socket is currently ${sock.socket.readyState === 0 ? '\`connecting\`' : (sock.socket.readyState === 1 ? '\`open\`' : (sock.socket.readyState === 2) ? '\`closing\`' : '\`closed\`')} and the last non-"\`pong\`" message received from the server was of type: \`${sock.msgHistory.length ? sock.msgHistory.slice(-1)[0].type : '(no messages yet)'}\`.` : ''} Note that every 40 seconds, the async generator object sends a \`ping\` to the server to keep the socket connection alive, and the server automatically responds with \`pong\`. However, we filter out these events to avoid constantly refreshing the cells in this notebook. You can check the console for a full log. (If you're using Chrome, you can also check the websocket frames in the dev tools network tab by clicking the WS filter – you should see two "edit" websockets, one used by the Observable editor, and one arising from this cell.) ⚠️ If you are getting "error" messages, try reconnecting by clicking the "Submit" button on either of the forms above. `2c:Teb1,viewof revert = { const revertform = html``; const goBack = revertform.querySelector('#goBack'); goBack.textContent = revertform.value = revertform.version.value = currentVersion; const saveEventSpan = revertform.querySelector('#saveEventSpan'); saveEventSpan.textContent = JSON.stringify(saveEvents[currentVersion-nbInitVersion-1]) ; revertform.version.addEventListener("input", event => { event.stopPropagation(); goBack.textContent = revertform.version.value; saveEventSpan.textContent = revertform.version.value > 0 ? JSON.stringify(saveEvents[revertform.version.value-nbInitVersion-1]) : '(initial version)'; }) revertform.version.addEventListener("change", event => { revertform.version.value = Math.min(Math.max(revertform.version.value, nbInitVersion), currentVersion); revertform.value = revertform.version.value; revertform.dispatchEvent(new CustomEvent('input')); }); revertform.addEventListener("submit", event => { event.preventDefault(); const version = revertform.version.value; if (version >= currentVersion) return; const events = []; let k = currentVersion; const oldState = stateHistory[version - nbInitVersion].state; const oldIDs = new Set(oldState.map(({id}) => id)); const ids = new Set(nbCurrState.map(({id}) => id)); // better approach: compute a diff and then send those events like the editor does // remove the cells that are missing for (const {id} of nbCurrState) { if (!oldIDs.has(id)) { events.push({version:++k, type:"remove_node", node_id:id}); } } // work from the end of oldState to the beginning const nbCurrCopy = nbCurrState.filter(({id}) => oldIDs.has(id)); let new_next_node_id = null; for (let j=oldState.length-1; j>=0; j--) { const {id:node_id, value:new_node_value, pinned:new_node_pinned} = oldState[j]; if (ids.has(node_id)) { // was found, need to move and / or modify (seems unsafe, maybe I should do this in a different loop?) const {value, pinned} = nbCurrCopy.find(({id}) => id === node_id); const curr = nbCurrCopy.pop(); if (curr.id !== node_id) { nbCurrCopy.push(curr); events.push({version:++k, type:"move_node", node_id, new_next_node_id}); const loc = nbCurrCopy.findIndex(({id}) => id === node_id); nbCurrCopy.splice(loc,1); // remove that node } // modify the nodes that are present in both and pin / unpin if (value !== new_node_value) events.push({version:++k, type:"modify_node", node_id, new_node_value}); if (pinned !== new_node_pinned) events.push({version:++k, type:pinned ? 'unpin_node' : 'pin_node', node_id}); } else { // not found, need to insert new node events.push({version:++k, type:"insert_node", node_id, new_node_value, new_node_pinned, new_next_node_id}); } new_next_node_id = node_id; // console.log(nbCurrCopy.map(({id}) => id)); } // console.log(events); submitSaveMessage(token, form, {type:"save", events}, currentVersion); }); return revertform; }2d:T6df,searchSec = md`### Search and replace These forms allow you to perform a search and replace on the cell source strings in the notebook. This is performed in two steps: - **search**: First, the notebook state array is [filtered](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter) using a combination of two functions: - The first text area lets you enter a [RegExp](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp) to [match](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/match) against each cell's source string. (If empty, then no cells are matched.) - The second text area takes [the body of a function](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Function) of \`i\`, the position of each cell in the notebook array. The function should return a boolean which is also used to filter the notebook state array. (If empty, no filtering is applied to the cell position indices.) - **replace**: The "Replace with" text boxs lets you enter a "replacement" string which is used in [\`String.prototype.replace()\`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace) to change the strings matched by the first "search" text area below. (If it's empty, then the matched strings are replaced by the empty string, i.e. they are deleted.) You now have the power to: - replace all / some occurrences of a certain term with another, - add code comments to the beginning of all cells matching a pattern, - mass-remove all cells in a certain range (or all odd-numbered cells, etc.), - change the levels of certain markdown headers, and - much, much more! `2e:Ta6d,viewof replace = { const replaceform = html``; const update = () => { replaceform.value = search.map(({value,id,pinned}) => ({id, value:value.replace( new RegExp(searchInput, 'm' + (replaceform.global.checked ? 'g' : '')), replaceform.replace.value), pinned})) .filter(({value}) => !replaceform.deleteEmpty.checked || value !== ""); replaceform.dispatchEvent(new CustomEvent('input')); }; replaceform.replace.placeholder = 'input is passed to String.prototype.replace (so you can refer to capture groups in the search RegExp)' replaceform.replace.value = (this && this.replace && this.replace.value) || ""; replaceform.replace.addEventListener("input", update); replaceform.global.checked = (this && this.global && this.global.checked) || true; replaceform.global.addEventListener("change", update); replaceform.deleteEmpty.checked = (this && this.deleteEmpty && this.deleteEmpty.checked) || true; replaceform.deleteEmpty.addEventListener("change", update); replaceform.addEventListener("submit", event => { event.preventDefault(); const events = []; let k = currentVersion; for (let j=0; j applySaveEvent(acc, ev), nbCurrState)); // console.log(events); submitSaveMessage(token, form, {type:"save", events}, currentVersion); }); replaceform.value = search.map( ({value,id,pinned}) => ({id, value:value.replace( new RegExp(searchInput, 'm' + (replaceform.global.checked ? 'g' : '')), replaceform.replace.value), pinned})) .filter(({value}) => !replaceform.deleteEmpty.checked || value !== ""); return replaceform; }2f:T422,viewof radioReplace = { replace; const prevValue = this && this.value; const form = html`

iframe shows live preview of:
matched cells in current version of notebook
matched cells with replacements applied${viewof replace.deleteEmpty.checked ? ` (empty cells are removed)` : ''}
full notebook with replacements applied${viewof replace.deleteEmpty.checked ? ` (empty cells are removed)` : ''} `; form.value = form.choice.value; form.addEventListener('change', () => { form.value = form.choice.value; form.dispatchEvent(new CustomEvent('input')); }); return form; }30:T46b,viewof nbToCopy = { switch (radioCopyGet) { case 'JSON': { return jsonInput({initialUrl:"https://gist.githubusercontent.com/bryangingechen/80a83b909d4cea9b4ba4d81637e5d548/raw/66b2e790d61e5f68ce892421a0d5e09bf9bcf77c/scratch.json"}); } case 'form': { return submit({description:"API Response string:", textArea:true, placeholder:"Paste response string here", parse:JSON.parse}); } case 'API': { // https://observablehq.com/@mbostock/notebook-visualizer const {text} = submit({ placeholder:"Input notebook URL or name here" }); text.addEventListener("input", () => { let v = text.value, m; if (m = /\.js(\?|$)/i.exec(v)) v = v.slice(0, m.index); if (m = /^https:\/\/(api\.|beta\.|)observablehq\.com\//i.exec(v)) v = v.slice(m[0].length); if (v !== text.value) text.value = v; }); text.addEventListener("paste", () => { Promises.delay(50).then(() => { text.form.dispatchEvent(new CustomEvent("submit", { cancelable: true })); }); }); return text.form; } } }31:T816,viewof copyPosition = { const copyform = html`
Copy selected cells (${nbCurrState.length} cells in the notebook)
Note that the selection will persist after copying, so the updated preview may show two copies of the selected cells.

`; const copyPos = copyform.querySelector('#copyPos'); copyform.value = copyform.position.valueAsNumber = nbCurrState.length; copyPos.textContent = (copyform.value < nbCurrState.length) ? `before cell ${copyform.value}` : 'at the end of the notebook'; copyform.position.addEventListener("input", event => { event.stopPropagation(); copyform.value = copyform.position.valueAsNumber; copyPos.textContent = (copyform.value < nbCurrState.length) ? `before cell ${copyform.value}` : 'at the end of the notebook'; }) copyform.position.addEventListener("change", event => { copyform.position.value = Math.min(Math.max(copyform.position.value, 0), nbCurrState.length); copyform.value = copyform.position.valueAsNumber; copyform.dispatchEvent(new CustomEvent('input')); }); copyform.addEventListener("submit", event => { event.preventDefault(); const position = copyform.position.value; const new_next_node_id = position < nbCurrState.length ? nbCurrState[position].id : null; const events = []; let k = currentVersion; for (let j = 0; j < cellsToCopy.length; j++) { const {value:new_node_value, pinned:new_node_pinned} = cellsToCopy[j]; const version = ++k; events.push({version, type:"insert_node", node_id:version, // give the copied cells new "node_id"s new_node_value, new_node_pinned, new_next_node_id}); } // console.log(events); submitSaveMessage(token, form, {type:"save", events}, currentVersion); }); return copyform; }32:T521,reorder = { const revertform = html``; revertform.addEventListener("submit", event => { event.preventDefault(); const events = []; let k = currentVersion; // this is going to contain the cells which haven't been checked yet const nbCurrCopy = nbCurrState.map(({id}) => id); let new_next_node_id = null; // start from end of desired ordering and move forwards // worst case, move last cell to beginning? this moves almost every cell, but we only need one... for (let j = order.length-1; j >= 0; j--) { const {id:node_id, value:new_node_value, pinned:new_node_pinned} = order[j].cell; const id = nbCurrCopy.pop(); if (id !== node_id) { nbCurrCopy.push(id); events.push({version:++k, type:"move_node", node_id, new_next_node_id}); const loc = nbCurrCopy.findIndex(id => id === node_id); nbCurrCopy.splice(loc,1); // remove that node to the end } new_next_node_id = node_id; // console.log(nbCurrCopy.slice()); } // console.log(events); submitSaveMessage(token, form, {type:"save", events}, currentVersion); }); return revertform; }33:T85a,async function* messagesFromEditSocket(token, form, cancel) { const gen = Generators.queue(notify => { const {id, version} = form; // Create WebSocket connection. const socket = new WebSocket(`wss://ws.observablehq.com/document/${id}/edit`); console.log(socket); const msgHistory = []; const object = {socket, msgHistory}; socket.onopen = () => { // The server requires a "hello" message with a session token, // as well as the local version number of the notebook. console.log('read socket opened'); socket.send(`{"type":"hello","token":"${token}","version":${version}}`); // The server acknowledges the client's "hello" by returning "hello". // The server then sends a "save" message with an array "events" // which contains all changes between the local version number and the server's version // and if there are comments, it sends a "comments" message, containing the comment data. notify(object); }; socket.onmessage = (m) => { console.log(m); const parsed = JSON.parse(m.data); // only add non-pong messages to history if (parsed.type !== 'pong') { msgHistory.push(parsed); notify(object); } else console.log('received pong', m); } socket.onerror = (e) => notify(Promise.reject(new Error("socket error", e))); // The Observable editor sends a "ping" to the server every 30 seconds or so to keep the socket open, // and the server immediately responds with a "pong". // If the server does not receive a "ping" or other message within 50 seconds or so, the socket gets closed. // We can make do here with a ping every 40 seconds. const ping = () => { socket.send('{"type":"ping"}'); console.log('sent ping'); }; const pingInterval = setInterval(ping, 40000); cancel.then(() => socket.close()); // On close socket.onclose = () => { console.log('read socket closed'); clearInterval(pingInterval); notify(object); }; return () => socket.close(); }); for await (const obj of gen) { yield obj; } }34:Tb62,function applySaveEvent(state, event) { const {version, type, node_id, new_node_value, new_node_pinned, new_next_node_id} = event; state = state.slice(); // make a copy (to be absolutely safe we should make a deep copy, but this is probably fine...) switch (type) { case "pin_node": { const index = state.findIndex(({id}) => id === node_id); const newObj = Object.assign({}, state[index]); newObj.pinned = true; state[index] = newObj; break; } case "unpin_node": { const index = state.findIndex(({id}) => id === node_id); const newObj = Object.assign({}, state[index]); newObj.pinned = false; state[index] = newObj; break; } case "modify_node": { const index = state.findIndex(({id}) => id === node_id); const newObj = Object.assign({}, state[index]); newObj.value = new_node_value; state.splice(index, 1, newObj); break; } case "insert_node": { const index = new_next_node_id === null ? state.length : state.findIndex(({id}) => id === new_next_node_id); state.splice(index, 0, {id:node_id, value:new_node_value, pinned:new_node_pinned}); break; } case "remove_node": { const index = state.findIndex(({id}) => id === node_id); state.splice(index, 1); break; } case "move_node": { const index = state.findIndex(({id}) => id === node_id); const obj = state[index]; state.splice(index, 1); const newNextIndex = new_next_node_id === null ? state.length : state.findIndex(({id}) => id === new_next_node_id); state.splice(newNextIndex, 0, obj); break; } case "modify_title": { // {version:555, type:"modify_title", new_node_value:"scratch" } // console.log('title modified', new_node_value); break; } case "revert": { // These events seem to occur only when you click "Revert" in the menu // when the message "This notebook has been edited since sharing. View changes?" is visible // So merging an older version doesn't cause these. // {version: 726, type: "revert", node_id: 723} // node_id seems to be the version to revert to... // If you open a notebook in another window and you revert to a version // that's older than the one at the beginning of that window's editor session has, // it says: // "Oops, someone reverted this notebook! Please reload?" // So I think it may be OK to throw a message here? console.log(`revert occured: {type:${type}, version:${version}, node_id:${node_id}}`); throw new Error(`Revert event occurred at version ${version}. Please input a version of the notebook after all revert events.`); } default: { debugger; throw new Error(`unsupported operation ${type} at version ${version}. Event: ${JSON.stringify(event)}`); } } return state; }35:T542,function submitSaveMessage(token, form, message, currVersion) { const {type, events} = message; if (type !== 'save') { console.log('not a save event'); return false; } const versions = events.map(({version}) => version); const minVersions = Math.min(...versions); if (minVersions !== currVersion+1) { console.log('minimum version must be current version + 1'); return false; } if (versions.some((d,i) => d - minVersions !== i)) { console.log('versions must increase one by one'); return false; } // open a new socket so that it's easier to keep the current state in sync const {id} = form; const socket = new WebSocket(`wss://ws.observablehq.com/document/${id}/edit`); // say hello and send message socket.onopen = () => { console.log('write socket opened'); socket.send(`{"type":"hello","token":"${token}","version":${currVersion}}`); socket.send(JSON.stringify(message)); console.log('message sent', message); }; // close on confirmation or failure socket.onmessage = (event) => { console.log('write socket received msg:', event); if (JSON.parse(event.data).type === 'saveconfirm' || JSON.parse(event.data).type === 'error') { socket.close(); } }; // bye socket.onclose = (event) => { console.log('write socket closed'); }; return true; }36:T4c0,d = `M12.1450213,20.7196596 C11.0175263,20.7196596 10.0411956,20.4623004 9.216,19.9475745 C8.39080438,19.4328485 7.75761923,18.7343023 7.31642553,17.8519149 C6.87523184,16.9695275 6.55251166,16.0340475 6.34825532,15.0454468 C6.14399898,14.0568461 6.04187234,12.990644 6.04187234,11.8468085 C6.04187234,10.9971021 6.09497819,10.1841741 6.20119149,9.408 C6.30740479,8.63182591 6.50348793,7.84340826 6.78944681,7.0427234 C7.07540569,6.24203855 7.44306158,5.54757741 7.89242553,4.95931915 C8.34178948,4.37106089 8.93003892,3.89310822 9.65719149,3.52544681 C10.3843441,3.1577854 11.2136124,2.97395745 12.1450213,2.97395745 C13.2725163,2.97395745 14.2488469,3.23131658 15.0740426,3.74604255 C15.8992382,4.26076853 16.5324233,4.95931474 16.973617,5.84170213 C17.4148107,6.72408952 17.7375309,7.65956953 17.9417872,8.64817021 C18.1460436,9.6367709 18.2481702,10.702973 18.2481702,11.8468085 C18.2481702,12.6965149 18.1950644,13.5094429 18.0888511,14.285617 C17.9826378,15.0617911 17.7824696,15.8502088 17.4883404,16.6508936 C17.1942113,17.4515785 16.8265554,18.1460396 16.3853617,18.7342979 C15.944168,19.3225561 15.3600036,19.8005088 14.6328511,20.1681702 C13.9056985,20.5358316 13.0764302,20.7196596 12.1450213,20.7196596`37:[] 27:{"id":"36d0bf5317d11dbf","slug":"websocket-powered-notebook-editing-tools","title":"WebSocket-powered notebook editing tools","thumbnail":"a2dbf7f11707f661ac10d20d53f02c7ec2fca180e8b19da27ab34746c8d62585","access_level":"public","create_time":"$D2019-02-18T19:15:19.965Z","update_time":"$D2019-09-10T02:24:06.374Z","view_time":null,"first_public_time":"$D2019-03-13T13:38:06.401Z","fork":true,"stars":11,"starred":false,"owner":{"type":"team","id":"d2fd23101beb2a61","login":"bryangingechen","name":"Bryan Gin-ge Chen","avatar_url":"https://avatars-next.observableusercontent.com/73f0dccc0b7fc5c845e6a52a58e01a2ae432cc529aaada0d10ab602137e603c0","tier":"starter_2024"},"version":2411,"publish_version":2411,"role":null,"authored":false,"authors":[{"type":"individual","id":"5c13a1a27cf52564","login":"bryangingechen","name":"Bryan Gin-ge Chen","avatar_url":"https://avatars-next.observableusercontent.com/73f0dccc0b7fc5c845e6a52a58e01a2ae432cc529aaada0d10ab602137e603c0","tier":null}],"members":[{"type":"individual","id":"5c13a1a27cf52564","login":"bryangingechen","name":"Bryan Gin-ge Chen","avatar_url":"https://avatars-next.observableusercontent.com/73f0dccc0b7fc5c845e6a52a58e01a2ae432cc529aaada0d10ab602137e603c0","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"subscribed":null,"forks":8,"collection":{"id":"938fbc4744e2d8f6","slug":"observable","title":"Observable","owner":{"id":"d2fd23101beb2a61","login":"bryangingechen","name":"Bryan Gin-ge Chen"}},"type":"notebook","body":{"title":"WebSocket-powered notebook editing tools","cells":[{"id":0,"value":"$29","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":37,"value":"viewof token = submit({description:\"token:\", placeholder:\"Paste token from cookie here\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":289,"value":"$2a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1990,"value":"viewof radioAPI = {\n const form = html`

Get API response from: \n JSON file\n\n JSON pasted into form\n`;\n form.value = form.choice.value;\n form.addEventListener('change', () => {\n form.value = form.choice.value;\n form.dispatchEvent(new CustomEvent('input'));\n });\n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":269,"value":"viewof form = radioAPI === 'form' ? submit({description:\"API Response string:\", textArea:true, placeholder:\"Paste response string here\", parse:JSON.parse}) : jsonInput({})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":952,"value":"md`The title of the notebook whose API data has been pasted above is:\n\n> **\"${form.title}\"**,\n\nand the earliest version loaded here is:\n\n> version **${form.version}**.\n\nYou can inspect the properties yourself by examining \\`form\\`:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1092,"value":"form","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":242,"value":"md`(In fact, the only API response fields that this notebook makes use of are \"\\`id\\`\", \"\\`version\\`\", \"\\`title\\`\", and \"\\`nodes\\`\".)\n\n## Socket info and notebook save / state history\n\nThe following cell \\`sock\\` is an async generator which yields an object containing the actual WebSocket object \\`socket\\` and an array \\`msgHistory\\` which contain the parsed responses from the Observable server.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":94,"value":"sock = messagesFromEditSocket(token, form, invalidation)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1827,"value":"$2b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":419,"value":"md`The API response above contains an array of notebook cell objects corresponding to the notebook state at version ${nbInitVersion}:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":551,"value":"nbInitVersion = form.version","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":329,"value":"nbInitState = form.nodes","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":428,"value":"md`If you've edited the opened notebook since opening it, the \\`currentVersion\\` will be larger:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":194,"value":"currentVersion = saveEvents.reduce(\n (acc, {version}) => \n Math.max(acc,version), \n nbInitVersion)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":244,"value":"md`This cell, \\`saveEvents\\`, collects the \\`save\\` events from \\`sock.msgHistory\\` - this is essentially an array of all edits made to the opened notebook since \\`nbInitVersion\\`.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":217,"value":"saveEvents = sock.msgHistory.filter(({type, events}) => type === 'save')\n .reduce((acc, {events}) => {\n acc.push(...events);\n return acc;\n}, [])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":430,"value":"md`The following cell, \\`stateHistory\\`, computes an array with all notebook states between \\`nbInitVersion\\` and \\`currentVersion\\` by applying the save events in the above array to \\`nbInitState\\`. The cell afterwards, \\`nbCurrState\\`, contains the current state of the notebook.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":571,"value":"// with the conventions here, saveEvents[n-1] applied to stateHistory[n-1].state yields stateHistory[n].state\n// stateHistory[n].version is nvInitVersion + n\n// TODO: use 'this' to make this more efficient\nstateHistory = saveEvents.reduce((acc, ev) => {\n try {\n const state = applySaveEvent(acc[acc.length-1].state, ev);\n acc.push({version:ev.version, state});\n } catch (e) {\n // bad event; we should disconnect (and disable editing)\n sock.socket.close();\n throw e;\n }\n return acc;\n }, [{version:nbInitVersion, state:nbInitState}])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":336,"value":"nbCurrState = stateHistory[stateHistory.length-1].state","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":686,"value":"md`## Controls for simple programmatic editing\n\nHere's a list of the tools in this notebook. I recommend opening the loaded notebook in another tab for live feedback and also having your browser's console open (on this page) to check for errors:\n\n1. [Revert to version](#revertSec)\n2. [Pin / unpin all cells / Restore pin state](#pinSec)\n3. [Search and replace](#searchSec)\n4. [Copy Cells](#copyFromSec)\n5. [Reorder notebook cells visually](#reorderSec)\n\n⚠️ Since every version of an Observable notebook is saved, all edits made here or elsewhere are reversible by navigating to the \\`https://observablehq.com/d/XXX@version\\` url, comparing to the present version, comparing in reverse, and then clicking \"Merge\". For this reason I **HIGHLY RECOMMEND that you keep track of a \"good\" version number that you can revert to** if things go wrong. \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2041,"value":"md`For convenience, the current version is **${currentVersion}**.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2047,"value":"md`### Notes:\n- Buttons are disabled below if the listening socket is closed. To restart the connection you can click the \"Submit\" buttons on either the \"[token](#token)\" or the \"[API request](#form)\" forms above. \n- I haven't implemented title changes in any of the controls below, so changes to the markdown in the first cell of a notebook will not be reflected in the title of that notebook until you edit it using the Observable editor in another window or tab.\n- If your cursor sits in a cell in another tab, you may notice that that cell can't be deleted programmatically (it will be immediately recreated as a blank cell). Just click outside that cell to fix this.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1940,"value":"revertSec = md`### Revert to version\n\nThe following form lets you revert a notebook to any version in the \\`stateHistory\\`. It's safer to navigate to the desired version with the @version URL and click \"Compare to latest\", \"Compare reverse\" and finally click the Merge button, but much slower and much less exciting!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":512,"value":"$2c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":796,"value":"stateHistory[revert-nbInitVersion].state","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1513,"value":"md`Here we use the [Observable parser](https://github.com/observablehq/parser) to [generate Observable \"modules\" from our notebook state arrays](https://observablehq.com/@bryangingechen/from-raw-notebook-source-to-observable-runtime-v1-modules) which can then be fed into the [runtime](https://github.com/observablehq/runtime) to embed a \"live\" copy of the state at the selected version. You can click the pink regions below unpinned cells to show their source. (You can't edit the source below, sorry!)\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1396,"value":"viewof revertPreview = {\n const version = parseInt(revert);\n const nb = Object.assign(Object.assign({}, form), {nodes:stateHistory[version - nbInitVersion].state, version});\n const div = await embedWithSourceIframe(nb, 600);\n div.value = nb;\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1980,"value":"DOM.download(new Blob([JSON.stringify(revertPreview)], {type:'application/json'}), `notebook@${revert}.json`, `Download notebook@${revert}.json`)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":694,"value":"pinSec = md`### Pin / unpin all cells / Restore pin state\nThe buttons that follow allow you to pin / unpin all cells of the notebook (a frequently requested feature on the forums) or restore the pin state from any version in the \\`stateHistory\\`.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":703,"value":"{\n const button = html`\n

`;\n pinform.addEventListener(\"submit\", event => {\n event.preventDefault();\n const version = revert;\n if (version >= currentVersion) return;\n const oldState = stateHistory[version - nbInitVersion].state;\n const events = [];\n let k = currentVersion;\n for (let j=0; j oldid === id && oldpinned !== pinned))\n events.push({type:pinned ? 'unpin_node' : 'pin_node', node_id:id, version:++k});\n }\n // console.log(events);\n submitSaveMessage(token, form, {type:\"save\", events}, currentVersion);\n });\n return pinform;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":726,"value":"$2d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":744,"value":"viewof searchInput = searchBox(`Search version ${currentVersion} for:`, this && this.value || \"\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":984,"value":"viewof searchIndexFilter = filterBox('searchIndexFilter', (this && this.querySelector('textarea').value) || \"\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":765,"value":"md`Click outside the above input boxes to update this array of matching cells:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1868,"value":"search = searchInput !== \"\" ? nbCurrState.filter(\n ({value}, i) => value.match(new RegExp(searchInput, 'm')) && (!searchIndexFilter || searchIndexFilter(i))) : []","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":740,"value":"$2e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":773,"value":"{\n replace;\n return md`Preview the result on the source of the matching cells${viewof replace.deleteEmpty.checked ? ` (cells whose \\`value\\` is the empty string \\`\"\"\\` have been removed)` : ''} (non-matching cells are untouched and not shown below):`\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":776,"value":"replace","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2090,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1797,"value":"viewof replacePreview = {\n const matched = new Set(search.map(({id}) => id));\n const changed = new Map(replace.map(({id}, i) => [id, i]));\n let nodes;\n switch (radioReplace) {\n case 'match': {\n nodes = search;\n break;\n } \n case 'replace': {\n nodes = replace;\n break;\n }\n case 'full': {\n nodes = nbCurrState.map(\n node => \n (matched.has(node.id)) ?\n (changed.has(node.id) ? replace[changed.get(node.id)] : null) :\n node\n ).filter(node => node !== null);\n }\n }\n const nb = Object.assign(Object.assign({}, form), {nodes, version:currentVersion+1});\n const div = await embedWithSourceIframe(nb, 600);\n div.value = nb;\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1985,"value":"{\n let filename;\n switch (radioReplace) {\n case 'match': {\n filename = `matched_cells@${currentVersion}.json`;\n break;\n } \n case 'replace': {\n filename = `replaced_cells@${currentVersion}.json`;\n break;\n }\n case 'full': {\n filename = `notebook_replaced@${currentVersion}.json`;\n }\n }\n return DOM.download(new Blob([JSON.stringify(replacePreview)], {type:'application/json'}), filename, `Download ${filename}`);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1845,"value":"copyFromSec = md`### Copy cells \n\nIn this section you can select cells from another notebook and insert them in the loaded notebook. You may input the source notebook in a few ways:\n- As above, you can upload or paste a JSON file containing a notebook object. Here I've made it's also OK to input just the \"\\`nodes\\`\" array.\n- You can also enter a link to a notebook to use [notebook source generated from the runtime V1 module served by Observable's API](https://observablehq.com/@bryangingechen/fromv1). As described in the linked notebook, the \"decompilation\" isn't perfect, and cells using \\`import...with\\` will have to be fixed by hand.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1934,"value":"viewof radioCopyGet = {\n const form = html`

Get source notebook from: \n JSON file\n\n JSON pasted into form\n\n decompile\n`;\n form.value = form.choice.value;\n form.addEventListener('change', () => {\n form.value = form.choice.value;\n form.dispatchEvent(new CustomEvent('input'));\n });\n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1930,"value":"$30","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2254,"value":"copySource = {\n const getScript = async (nbName) => {\n const url = nbName.match(/\\//) ? nbName : 'd/'+nbName;\n const module = await importModule(`https://api.observablehq.com/${url}.js`);\n return fromV1(module);\n }\n return typeof nbToCopy === 'string' ? await getScript(nbToCopy) :\n(nbToCopy.nodes ? nbToCopy.nodes : nbToCopy);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2234,"value":"md`These two text forms behave like their analogues in the above [Search and replace](#searchSec) tool.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1918,"value":"viewof copySearchInput = searchBox('Select cells from source matching:', this && this.value || \"\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1922,"value":"viewof copyIndexFilter = filterBox('copyIndexFilter', (this && this.querySelector('textarea').value) || \"\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1976,"value":"cellsToCopy = copySearchInput !== \"\" ? copySource.filter(\n ({value}, i) => value.match(new RegExp(copySearchInput, 'm')) && (!copyIndexFilter || copyIndexFilter(i))) : []","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2238,"value":"viewof radioCopyPreview = {\n const form = html`iframe shows live preview of:
\n \n source notebook
\n \n cells selected from source
\n \n full notebook with copied cells\n`;\n form.value = form.choice.value;\n form.addEventListener('change', () => {\n form.value = form.choice.value;\n form.dispatchEvent(new CustomEvent('input'));\n });\n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2264,"value":"$31","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2250,"value":"viewof copyPreview = {\n let nodes;\n switch (radioCopyPreview) {\n case 'source': {\n nodes = copySource;\n break;\n } \n case 'select': {\n nodes = cellsToCopy;\n break;\n }\n case 'full': {\n nodes = nbCurrState.slice();\n nodes.splice(copyPosition, 0, ...cellsToCopy.map(\n (node,i) => ({\n id:currentVersion+i+1,\n pinned:node.pinned, value:node.value})\n ));\n }\n }\n const nb = Object.assign(Object.assign({}, form), {nodes, version:currentVersion+1});\n const div = await embedWithSourceIframe(nb, 600);\n div.value = nb;\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2316,"value":"{\n let filename;\n switch (radioCopyPreview) {\n case 'source': {\n filename = `source.json`;\n break;\n } \n case 'select': {\n filename = `selected.json`;\n break;\n }\n case 'full': {\n filename = `notebook_inserted@${currentVersion}.json`;\n }\n }\n return DOM.download(new Blob([JSON.stringify(copyPreview)], {type:'application/json'}), filename, `Download ${filename}`);\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1153,"value":"reorderSec = md`### Reorder notebook cells visually\n\nInspired by [@fil](/@fil)'s notebook \"[Drag to re-arrange](/@fil/drag-to-re-arrange)\".\n\nDrag the boxes below (representing notebook cells) until you're happy with the order. Then click the \"Save reordering\" button to submit your changes.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1163,"value":"viewof order","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1186,"value":"viewof resetOrder = html`\n

`;\n const inputEl = textArea ? form.area : form.text;\n inputEl.value = value;\n inputEl.placeholder = placeholder;\n inputEl.addEventListener(\"input\", event => event.stopPropagation());\n \n form.addEventListener(\"submit\", event => {\n form.value = parse ? parse(inputEl.value) : inputEl.value;\n form.dispatchEvent(new CustomEvent(\"input\"));\n event.preventDefault();\n });\n return form;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1852,"value":"function searchBox(str, prevValue) {\n const div = html`${str}
\n \n }`;\n const text = div.querySelector('textarea');\n text.value = prevValue;\n text.placeholder = 'empty input means no array index filtering, non-empty input is passed to Function constructor with argument \"i\" and used in .filter() on the array of cells matching the search RegExp';\n div.value = text.value !== \"\" ? Function('i', text.value) : () => true;\n text.onchange = () => {\n div.value = Function('i', text.value);\n div.dispatchEvent(new CustomEvent('input'));\n };\n return div;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1218,"value":"md`This import is a modification of [@fil](/@fil)'s [\"Drag to re-arrange\"](https://beta.observablehq.com/@fil/drag-to-re-arrange) notebook:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1159,"value":"import {viewof order} with {dragdata as data} from '@bryangingechen/drag-to-re-arrange-vertical-edition'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1161,"value":"dragdata = {\n resetOrder;\n return nbCurrState.map(\n ({value, id,pinned}) => ({data:JSON.stringify(value).slice(0,200), cell:{value,id,pinned}}));\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1394,"value":"import {toV1, utoa, importModule, observable} from '@bryangingechen/from-raw-notebook-source-to-observable-runtime-v1-modules'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1448,"value":"import {fromV1} from '@bryangingechen/fromv1'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1470,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1473,"value":"newd = (scale) => d.replace(/\\d+\\.\\d+/g,s => (parseFloat(s) - 12)*scale +12)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1729,"value":"import {embedWithSourceIframe} from '@bryangingechen/exporting-notebooks-with-their-source'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":1932,"value":"import {jsonInput} from \"@mbostock/file-input@157\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"},{"id":2379,"value":"import {getCurrentPinnedName} from '@bryangingechen/version-pinning-for-notebooks@467'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined"}],"files":"$W37","stdlib":"1","titling":"auto"}} 28:"$27:body"