28:null 29:["$","title",null,{"children":"How the Observable Runtime works | Tom Larkworthy | Observable"}] 2e:T5d6,cell_map_diagram = mermaid` stateDiagram-v2 observablehq.com --> user user --> cell: codemirror user --> FileAttachment: upload cell --> user: LSP cell --> user: highlight state ___.static.observableusercontent.com { cell --> variable: parse + compile variable --> cell: decompile cell --> module: import _ from _ variable --> main_module: define FileAttachment --> stdlib: part of module --> runtime: defines state static { main_module --> runtime runtime --> main_module: access-runtime stdlib --> runtime runtime --> inspector } inspector --> DOM: inspect } DOM --> observablehq.com: iframe `2f:T455,md`## The \`api.observablehq.com\` endpoints Observable has no public write API, but the endpoints the site itself uses are reachable and the whole read side answers **unauthenticated**. Measured 2026-09-26 against \`@tomlarkworthy/module-map\`; none of it is contractual, so check rather than trust. | endpoint | returns | |---|---| | \`/@user/slug.js?v=4\` | the notebook compiled to an ES module — \`export default function define(runtime, observer)\` | | \`/d/{documentId}.js?v=4\` | the same module, addressed by document id | | \`/document/@user/slug\` | the document as JSON — \`nodes\`, \`version\`, \`id\`, \`files\`, \`resolutions\` | | \`/api/import/@user/slug\` | \`401\`. Internal, not for you | Both open endpoints answer \`cache-control: public, max-age=30, stale-while-revalidate=30\`, so a push takes about half a minute to become visible — long enough to make you think the push failed. Only the \`.js\` endpoints carry \`access-control-allow-origin: *\`; \`/document/…\` does not, so you can import a notebook from anywhere in the browser but you cannot read its pin list there.`30:T781,md`### \`v\` selects a compiler, it is not a cache buster \`v\` chooses the **output format**, and the three that work are three different artifacts. Same notebook, same minute (2026-09-26, \`@tomlarkworthy/module-map\`): | \`v\` | exports | dependencies | bytes | |---|---|---|---| | absent | \`notebook = {id, modules:[…]}\` — the notebook as **data** | every imported module **inlined** | 95527 | | \`3\` | \`function define(runtime, observer)\` | **static** \`import define1 from "/@user/dep.js?v=3&resolutions=…"\` | 42332 | | \`4\` | the same \`define\` | **dynamic** \`import(…)\`, inline in the import cell | 43623 | | \`5\` | \`{"errors":[]}\` | HTTP \`400\` | 13 | \`v=4\` is the one to run. Observable's runtime is lazy — a cell nobody observes never runs — and only the dynamic form lets an unobserved *import* stay unloaded too. The unversioned form is the one to **read**, and it is not a commented version of the others. It is a description rather than a program: a provenance header, then each module as a literal, each variable as \`{name, inputs, value}\` with \`value\` a real function. \`\`\`js // URL: https://observablehq.com/@tomlarkworthy/module-map // Title: Module map // Version: 1424 // Runtime version: 1 const m0 = { id: "db42ae70222a8b08@1424", variables: [ { inputs: ["md"], value: (function(md){return( md\`# Module map\` )}) }, \`\`\` Nothing is imported because everything is already there — \`modules\` held all 10 for module-map: the host, then \`@tomlarkworthy/lopepage-urls\`, \`@tomlarkworthy/flow-queue\`, \`@tomlarkworthy/observablejs-toolchain\` and six more, each with its cells. That makes it the right endpoint for analysing a notebook and the wrong one for loading it: it is over twice the size, and an inlined module is identified by bare slug (\`@tomlarkworthy/flow-queue\`) where the host carries \`docId@version\`, so the dependency pins are gone.`31:T684,md`### \`resolutions\` re-resolves the whole closure, from the importer's point of view A notebook pins its imports per document, and the pin list is readable without auth: \`\`\`sh curl -s https://api.observablehq.com/document/@tomlarkworthy/module-map | jq '.resolutions' # [{specifier: "@mootari/access-runtime", value: "e1c39d41e8e944b0@939"}, …] \`\`\` A value with \`@version\` is a hard pin; without one it tracks latest. Appending \`&resolutions=@\` to a \`.js\` request asks for **that document's** view of the graph, which rewrites the import URLs inside the module you get back — and, because the rewritten URLs carry the same parameter, on down the whole closure. Measured 2026-09-26: all 7 of module-map's import cells, asked for as itself and then as \`@tomlarkworthy/exporter-3\` (\`c5bd58fe29172a81@17355\`): \`\`\` import("/@tomlarkworthy/flow-queue.js?v=4&resolutions=db42ae70222a8b08@1424") as itself import("/@tomlarkworthy/flow-queue.js?v=4&resolutions=c5bd58fe29172a81@17355") as exporter-3 \`\`\` So **resolution context belongs to the importer, not to the root**. Fixing a pin in the notebook you are looking at does nothing for the notebooks it imports, and one stale pin anywhere on the path re-breaks everything below it. On new.observablehq.com the viewed notebook has no resolution context at all, so only the imported notebooks' maps are ever consulted. If you re-emit one of these URLs yourself, emit the pin **verbatim**: [es-module-shims](https://github.com/guybedford/es-module-shims) keys its registry on the URL string, so a percent-encoded \`%40\` is a different module and you get two copies of it.`32:T521,md`### There is no write API, but there is a WebSocket Observable has had [an open feature request for a write API since 2021](https://talk.observablehq.com/t/api-for-notebooks/1584). The editor itself uses a persistent socket, and it works from Node: \`\`\` wss://ws.observablehq.com/document/{documentId}/edit Origin: https://observablehq.com Cookie: I={jwt}; T={token} -> {"type":"hello","token":"{T}","version":{currentVersion},"next":true} \`\`\` \`version\` must be the document's **current** version from the REST API — \`0\` answers 404. Then \`save\` events carry \`modify_node\`, \`insert_node\` or \`remove_node\`, and \`node_id\` for an insert **must equal the event version** or the server answers 400. Two limits found the hard way. The save endpoint 404s after about three back-to-back writes, so pace them and retry — a timed-out save is not applied, which is what makes resend safe rather than a duplicate-insert risk. And the socket goes quiet after ~57 saves whatever the pacing: measured 2026-08-14 pushing 165 cells, two runs died at 57 and 58 operations, the second on a cell the first had already written. Retire the socket every 40 saves. \`insert_node\` with \`new_node_mode: "md"\` answers 400. A prose cell has to go up as mode \`js\` whose value is \`\` md\`…\` \`\`.`33:T566,md`### \`/api/import\` — the third compiled format \`observablehq.com/api/import/@user/slug\` is where the new site gets the notebooks it **imports**, and it is neither of the \`api.observablehq.com\` forms. Measured 2026-09-26 on a cold load of \`@tomlarkworthy/module-map\`: **147 requests, 18 of them to \`/api/import/\`, and zero module fetches from \`api.observablehq.com\`.** So the rule that imported notebooks get the legacy \`?v=4\` compile is no longer true — it was, and it is worth re-checking rather than believing either version. The format is a hybrid. It exports \`define(runtime)\` with **no observer parameter** and uses only \`main.define(…)\`, so nothing in it is displayable. Variable names are the legacy spaced ones while the function names carry the kit \`$\` spelling — one cell, both spellings live at once: \`\`\`js main.define("initial main_mutable", …) main.define("mutator main_mutable", …) // legacy emits "mutable main_mutable" here function mutable$main_mutable(){…} function viewof$cellMapModule(Inputs){…} \`\`\` Anonymous cells are named \`cell \` — the **document** node id, so unlike an index it is stable across edits. Its own imports are emitted as bare relative specifiers, \`import("./flow-queue")\`, carrying no version: \`resolutions\` is **inert** here, and passing one returns a byte-identical file.`34:T583,md`## Notebook Kit, and why a notebook has two behaviours [\`new.observablehq.com\`](https://new.observablehq.com) runs [notebook-kit](https://github.com/observablehq/notebook-kit); \`observablehq.com\` still runs the legacy runtime. The difference that matters most is not which site you are on — it is that on the new site **the same notebook behaves differently depending on whether it is being viewed or imported**: | | viewed on new.observablehq.com | imported by another notebook | |---|---|---| | served as | server-rendered into the worker frame | \`observablehq.com/api/import/@user/slug\` | | compiler | notebook-kit | the hybrid form below | | cell names | \`viewof$x\` | \`viewof x\` | | \`FileAttachment\` | one runtime-level builtin | one per module | So \`@tomlarkworthy/editor-5\` open as a page and \`editor-5\` imported by \`svg-lens\` are two materially different environments in the same browser tab. Any bug report about the new platform has to say which one it is in. Two things that look like they should tell them apart, and do not. \`isOnObservableCom()\` matches \`observableusercontent.com\`, and *both* sites render the notebook inside an iframe on that host. And the viewed notebook's source is never fetched over the network at all — it arrives server-rendered in the page payload — so there is no request to intercept when the bug is in the notebook you are looking at.`35:T540,md`### \`require\` is gone, and you cannot put it back from a cell \`require\` is a builtin in the legacy stdlib — classic Observable *and* the lopecode bootloader — and is absent from notebook-kit's. A cell compiled with \`["require"]\` in its dependency array raises \`RuntimeError: require is not defined\` on the new site and nowhere else. Measured 2026-09-08 across our corpus: 32 modules depend on it, and it is the only legacy-only builtin any of them actually use (\`resolve\`, \`SQLite\`, \`SQLiteDatabaseClient\` and \`flare\` are all unreferenced). Setting \`window.require\` from a cell does not fix it, and the reason is worth knowing generally. The runtime's \`module_resolve\` does fall back to \`globalThis[name]\` — that is how \`Element\`, \`Response\` and \`DecompressionStream\` resolve without being builtins — but it runs at **module-define time**, and when the name is missing it caches a placeholder \`Variable\` in \`_scope\` that is never resolved again. Any global you set from a cell body is already too late. The remedy has to be at the cell: shadow the name with a module-local cell, or drop the dependency. These builtins exist on **both** platforms, so a \`require\` of any of them can simply be deleted: \`_ aq Arrow d3 dot DuckDBClient htl html Inputs L md mermaid Plot svg tex topojson vl width\`.`36:T468,md`### A viewed notebook is not lazy any more Everything above about laziness — a cell nobody observes never runs — still holds for the runtime, but it stops describing what you see on the new site. \`define()\` builds each cell's output root, its \`display\`/\`view\` shadows **and its observer** in one constructor call, and a variable with an observer is reachable from birth. So every cell in a viewed notebook computes whether or not it is on screen. Measured 2026-09-26 on \`@tomlarkworthy/module-map\`, whose document is 57 nodes, reading the live runtime before touching the page: \`\`\` kitHeads 57 headsWithObserver 57 headsReachable 57 headsComputed 55 …after scrolling to the bottom: kitHeads 57, headsReachable 57 (unchanged) \`\`\` One head per authored cell, all 57 already reachable. Note the tell used to count them: \`v._shadow instanceof Map\` picks out exactly one variable per authored cell, which no name test does. Rendering is still lazy even though computation is not, so a headless probe that never scrolls sees fewer *cells* than a human while the values are all there regardless.`37:[] 2c:{"id":"f0fa91a52ef17a93","slug":"observable-notes","title":"How the Observable Runtime works","thumbnail":"dab1604ccf4a760060379630da0876da27b79509b738f8d5c300c9a9a320e38a","access_level":"public","create_time":"$D2024-11-21T07:09:17.680Z","update_time":"$D2026-09-26T12:20:22.191Z","view_time":null,"first_public_time":"$D2024-11-23T10:09:28.680Z","has_unpublished_edits":null,"fork":false,"stars":11,"starred":false,"owner":{"type":"team","id":"7db5ed2b0697d645","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":"starter_2024"},"version":1310,"publish_version":null,"role":null,"authored":false,"authors":[{"type":"individual","id":"5215f6ec4a999d40","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":null}],"members":[{"type":"individual","id":"5215f6ec4a999d40","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":null,"role":"editor"},{"type":"team","id":"7db5ed2b0697d645","login":"tomlarkworthy","name":"Tom Larkworthy","avatar_url":"https://avatars-next.observableusercontent.com/47327a8bc1966f2186dcb3ebf4b7ee6e4e7ab9a5c2a07405aff57200ea778f71","tier":"starter_2024","role":"editor"}],"comments":[],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":{"id":"cf72f19f55f3a048","slug":"lopebook","title":"lopecode","owner":{"id":"7db5ed2b0697d645","login":"tomlarkworthy","name":"Tom Larkworthy"}},"type":"notebook","body":{"title":"How the Observable Runtime works","cells":[{"id":0,"value":"# How the Observable Runtime works\n## Living documentation","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"title_cell","database":"$undefined"},{"id":5,"value":"A lot of Observable is open source and MIT/ISC licensed. Additional tools have been built by the community\n\n1st party\n- [Observable JS parser](https://github.com/observablehq/parser)\n- [Lezer Grammar for Observable JS](https://github.com/observablehq/lezer)\n- [Observable Runtime](https://github.com/observablehq/runtime)\n- [Standard Library](https://observablehq.com/documentation/misc/standard-library)\n- [Inspector](https://github.com/observablehq/inspector)\n\n3rd party\n- [Access the runtime](https://observablehq.com/@mootari/access-runtime)\n- [The Unofficial Observable Compiler](https://github.com/asg017/unofficial-observablehq-compiler/tree/beta)\n- [Userspace Observable compiler and decompiler](https://observablehq.com/@tomlarkworthy/observablejs-toolchain)\n- [Notebook Distiller](https://observablehq.com/@tmcw/notebook-distiller)\n- [Single File Exporter](https://observablehq.com/@tomlarkworthy/exporter)","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":24,"value":"$2e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":325,"value":"## How Observable works\n\nIn this section I try to explain the main concepts, surfacing the runtime concepts as live variables you can explore inline.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":355,"value":"### the sandbox","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":358,"value":"The main security mechanism on [observablehq.com](https://Observablehq.com) is to run all user javascript inside a sandboxed iframe. Each user has their own subdomain, so local storage of user code is isolated from one another and from the main webpage. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":376,"value":"iframe_location = document.location // we are inside the sandbox, so the document is the iframe","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":935,"value":"A drawback of the sandbox approach is several Web APIs don't work (sharedArrayBuffer, Bluetooth) and default forms submission will redirect the inner page.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":338,"value":"### The runtime","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":215,"value":"At the core is the [Observable runtime](https://github.com/observablehq/runtime), which hot-reloads and schedules recompilation of the dependancy graph. Its the engine of reactivity, implemented in Javascript, executing javascript snippets in units called `variables`.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":348,"value":"The runtime runs *inside* the sandbox, so we can use [[mootari]()](https://observablehq.com/@mootari)'s hack to get a reference to the runtime from within an Observable notebook. ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":342,"value":"runtime_reference = toObject(runtime)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":340,"value":"### Variables","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":231,"value":"The runtime holds a set of variables.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":384,"value":"runtime_variables = [...runtime._variables].map(toObject)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":226,"value":"Variables are scoped to a module. All notebooks come with a \"builtin\" module which is defined by the stdlib","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":397,"value":"builtin_module_ref = [...runtime._modules][0][1]","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":411,"value":"builtin_module = toObject(builtin_module_ref)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":250,"value":"builtin_variables = toMap(\n ...[...runtime._variables].filter((v) => v._module === builtin_module_ref)\n)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":268,"value":"Two variables clash if they are named the same and belong to the same scope. However, when you import a module, you only expose a few variables to another module, so it is possible to have variables of the same name but in different modules.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1048,"value":"### Variable Definitions\n\nVariable updates are scheduled when inputs become available. The runtime abstracts over the difference between async and syncronous execution via its scheduler. A variable can emit multiple values per update when defined as a generator.\n\nWhen definitions are called, the `this` value bound to the prior state, allowing cells to reduce over executions i.e. chain state forward.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1049,"value":"viewof count_button = Inputs.button(\"count\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1050,"value":"{\n count_button;\n return this + 1 || 0; // Add 1 to prior state (this)\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1051,"value":"Observable does not enforce any kind of dataflow programming purity. Inputs are passed by reference, so definitions can mutate objects outside of the dataflow paradigm. Furthermore, definitions can reference globals like the window, change the DOM and perform arbitrary side effects outside the dataflow graph.\n\nWhile functional reactive purist may find this unattractive, being ordinary Javascript functions reduces integration friction when importing existing Javascript libraries.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1052,"value":"### Glitch free Observable Reactivity Semantics","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1053,"value":"Observable builds a dataflow dependancy graph between variables. When a variable is marked dirty, it is scheduled for recomputation next tick, as long as its inputs are not dirty. This batched computation avoids common pitfuls with reactivty such as glitching.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1054,"value":"mermaid`graph TD\nA-->D\nA-->C\nB-->D[\"D 🕣\"]\nC-->E\nD-->E\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1055,"value":"In syncronous reactive systems without batching, an update to `A` will chain to C and D and then trigger E twice -- a so called \"glitch\" which can have unwanted side effects. In Observable, an update to `A` automatically marks A, C, D and E as dirty. C recomputes quickly and updates, followed by D and E after the asynchronous process in D completes. Thus E remains in the dirty state until D has completed and updates only once.\n\nIf B and A update temporally close together, D still emits a value once, but internally the async computation is ran twice with overlap. E then only updates once.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1056,"value":"Inputs.button(\"a\", {\n reduce: () => viewof a.dispatchEvent(new Event(\"input\"))\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1057,"value":"Inputs.button(\"b\", {\n reduce: () => viewof b.dispatchEvent(new Event(\"input\"))\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1058,"value":"viewof a = Inputs.input(0)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1059,"value":"viewof b = Inputs.input(0)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1060,"value":"c = {\n a, b;\n return this + 1 || 0;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1061,"value":"d = {\n a, b;\n await new Promise((r) => setTimeout(r, 1000));\n return this + 1 || 0;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1062,"value":"e = {\n c, d;\n debugger;\n return this + 1 || 0;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1046,"value":"viewof module = thisModule()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1087,"value":"vars = new Map([\n [\"a\", await lookupVariable(\"a\", module)],\n [\"b\", await lookupVariable(\"b\", module)],\n [\"c\", await lookupVariable(\"c\", module)],\n [\"d\", await lookupVariable(\"d\", module)],\n [\"e\", await lookupVariable(\"e\", module)]\n])","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1092,"value":"#### Cycles\n\nYou can get fake cycles by using a viewof and \"posting upstream\" programatically to the DAG","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1119,"value":"cycles = Inputs.button(\"do a cycle\", {\n reduce: () => {\n viewof cycle.value = 0;\n viewof cycle.dispatchEvent(new Event(\"input\"));\n }\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1095,"value":"viewof cycle = Inputs.input()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1123,"value":"cycle","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1097,"value":"increment = {\n viewof cycle.value = cycle + 1;\n viewof cycle.dispatchEvent(new Event(\"input\"));\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1133,"value":"#### Glitch Free cycles (with caveats)\n\nIf you use programmatic trigger, do we still get glitch free operation? i.e. do simultaneous triggers get merged into a single dataflow sweep? What about double triggers?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1160,"value":"Inputs.button(\"n-multi trigger DAG\", {\n reduce: () => {\n viewof left.value = 0;\n viewof right.value = 0;\n for (let i = 0; i < n; i++) {\n debugger;\n viewof left.dispatchEvent(new Event(\"input\"));\n viewof right.dispatchEvent(new Event(\"input\"));\n }\n }\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1167,"value":"viewof n = Inputs.range([1, 10], { label: \"n\", step: 1, value: 1 })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1144,"value":"viewof left = Inputs.input()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1146,"value":"viewof right = Inputs.input()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1150,"value":"left_right = {\n left, right;\n return this + 1 || 0;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1165,"value":"The above experiment shows triggers on seperate dataflow parents are merged in the same update batch (glitch free!). However, repeated triggers lead to multiple batches up to a max of 2, which is super weird and unexpected.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1063,"value":"#### Responsiveness\n\nObservable prioritises user-responsiveness. Dataflow decendants of high frequency streams will miss updates when they exceed the animation frame rate (typically 60fps). Its best to view the Observable dataflow graph as monotonically converging to the latest state rather than a stream processing engine.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1064,"value":"Inputs.button(\"burst 100\", {\n reduce: async () => {\n Array.from({ length: 100 }).reduce(\n (promise) =>\n promise.then(async () => {\n await new Promise((r) => setTimeout(r, 0));\n viewof burst.dispatchEvent(new Event(\"input\"));\n }),\n Promise.resolve()\n );\n }\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1065,"value":"viewof burst = Inputs.input(0)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1066,"value":"burst_decendant = {\n burst;\n return this + 1 || 0;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1207,"value":"### Determining the state of a variable\n\nCan you tell if a variable is actively pending, computed a value or never computed?","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1247,"value":"state_experiment = new Runtime().module()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1260,"value":"observerFactory = (name) => ({\n fulfilled: () => (states[name] = \"fulfilled\"),\n rejected: () => (states[name] = \"rejected\"),\n pending: () => (states[name] = \"pending\")\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1264,"value":"state_experiment_resolved_variable = state_experiment\n .variable(observerFactory(\"resolved_variable\"))\n .define(\"resolved_variable\", [], () => 1)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1289,"value":"state_experiment_undefined_variable = state_experiment\n .variable(observerFactory(\"undefined_variable\"))\n .define(\"undefined_variable\", [], () => undefined)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1266,"value":"state_experiment_hung_variable = state_experiment\n .variable(observerFactory(\"hung_variable\"))\n .define(\"hung_variable\", [], () => new Promise(() => {}))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1271,"value":"state_experiment_error_variable = state_experiment\n .variable(observerFactory(\"error_variable\"))\n .define(\"error_variable\", [], () => {\n throw \"error\";\n })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1273,"value":"state_experiment_generator_variable = state_experiment\n .variable(observerFactory(\"generator_variable\"))\n .define(\"generator_variable\", [], function* () {\n let i = 0;\n while (true) yield i++;\n })","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1275,"value":"state_experiment_async_generator_variable = state_experiment\n .variable(observerFactory(\"async_generator_variable\"))\n .define(\"async_generator_variable\", [], async function* () {\n let i = 0;\n while (true) {\n await new Promise((r) => setTimeout(r, 1000));\n yield i++;\n }\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1252,"value":"states = ({})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1278,"value":"experiment_variables = [\n state_experiment_resolved_variable,\n state_experiment_undefined_variable,\n state_experiment_hung_variable,\n state_experiment_error_variable,\n state_experiment_generator_variable,\n state_experiment_async_generator_variable\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1280,"value":"Inputs.table(experiment_variables.map(toObject))","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1284,"value":"new Map(\n await Promise.all(\n experiment_variables.map(async (v) => [\n v._name,\n await getPromiseState(v._promise)\n ])\n )\n)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":335,"value":"### Cells","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":337,"value":"You program cells in notebooks using ObservableJS. They are compiled to the runtime definitions. There are few major types of cell.\n- Javascript cells\n - with some special handling for async, generator subtypes\n- viewof, mutables\n- Tagged template cells (e.g. markdown, html blocks)\n- import statements","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":468,"value":"#### Cell -> variables","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":470,"value":"Cells are mapped to one or more runtime variables. A 3rd party tool can group them for a given module. Most cells in this notebook are anonymous cells without names, e.g. markdown blocks.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":475,"value":"main_cells = (await cellMap()).get(main)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":576,"value":"Each cell is mapped to several variables, for instance, a mutable cell has three variables, the initial value, the non-reactive value, and the reactive value. Cells recompute when the *inputs* change.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":583,"value":"viewof cell = Inputs.select(main_cells, {\n label: \"select a cell to expand\",\n format: (c) => c.name\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1234,"value":"Inputs.table([cell])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1230,"value":"cell_variables = cell.variables","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":588,"value":"cell_variables_table = Inputs.table(cell_variables.map(toObject), {\n columns: [\"_name\", \"_inputs\", \"_definition\"],\n format: {\n _definition: (d) =>\n Inputs.textarea({ value: d.toString(), disabled: true }),\n _inputs: (d) => d.map((i) => i._name)\n },\n width: {\n _definition: \"50%\"\n }\n})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":945,"value":"The definition of a variable is the code that is executes when it recomputes. The arguments to that function are the dependancies that need to be computed first. A cell specifies its dependencies in the inputs array.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":624,"value":"mutable example_mutable = 0","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":630,"value":"Each imported variable exposes creates at least one variable to cross between modules.\n```js\nimport {a, b, c as d} from \"@...\"\n```\n\nImporting viewofs and mutables are a bit more complex because both the container and the data channels are imported even if you only import one of them. `with` syntax is even more complicated as a seperate runtime is spawned.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":451,"value":"#### Parsing","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":516,"value":"The first stage of transforming ObserrvableJS source code to the runtime variable representation is parsing. The [observablehq/parser](https://github.com/observablehq/parser) does this. Its a wrapper around [acorn](https://github.com/acornjs/acorn). \n\nAs the examples below show, where Observable goes beyond typical javascript is often expressed in the custom top level \"cell\" AST node. The other thing it does is figure out external references, which is what drives the dependancy resolution.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":522,"value":"viewof_ast = parser.parseCell(\"viewof foo = 'bar'\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":524,"value":"mutable_ast = parser.parseCell(\"mutable foo = 'bar'\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":531,"value":"async_ast = parser.parseCell(\"foo = { await ''}\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":528,"value":"generator_ast = parser.parseCell(\"foo = { yield 'async'}\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":534,"value":"fileattachment_ast = parser.parseCell(\"foo = FileAttachment('filepath')\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":544,"value":"viewof_ref_ast = parser.parseCell(\"foo = viewof bar\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":638,"value":"import_ast = parser.parseCell(\"import {foo} from 'blah'\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":554,"value":"Inputs.table(\n [\n Object.assign(viewof_ast, { name: \"viewof_ast\" }),\n Object.assign(mutable_ast, { name: \"mutable_ast\" }),\n Object.assign(async_ast, { name: \"async_ast\" }),\n Object.assign(generator_ast, { name: \"generator_ast\" }),\n Object.assign(fileattachment_ast, { name: \"fileattachment_ast\" }),\n Object.assign(viewof_ref_ast, { name: \"viewof_ref_ast\" }),\n Object.assign(import_ast, { name: \"import_ast\" })\n ],\n {\n columns: [\n \"name\",\n \"id\",\n \"async\",\n \"generator\",\n \"references\",\n \"fileAttachments\"\n ],\n width: {\n async: \"10%\",\n generator: \"10%\"\n },\n format: {\n id: (id) => JSON.stringify(id),\n references: (r) => JSON.stringify(r),\n fileAttachments: (r) => [...r.keys()]\n }\n }\n)","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":663,"value":"### Compiling","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":666,"value":"The Observable compiler is not open source. However, there is the community [unofficial-observablehq-compiler](https://github.com/asg017/unofficial-observablehq-compiler/tree/beta) and the [userspace decompiler/compiler](https://observablehq.com/@tomlarkworthy/observablejs-toolchain).","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":671,"value":"import { compile } from \"@tomlarkworthy/observablejs-toolchain\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":674,"value":"compiled_variables = compile(\"mutable example = 12\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":680,"value":"## Running your own runtime","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":687,"value":"To execute a notebook outside of [observablehq.com](https://observablehq.com) you need to instantiate a runtime (this is what the [notebook export feature](https://observablehq.com/documentation/embeds/advanced) does). First grab import the runtime from npm:","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":700,"value":"import { Runtime } from \"@tomlarkworthy/observable-runtime-v6\"","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":706,"value":"embedded_runtime = new Runtime()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":796,"value":"Create a main module","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":714,"value":"embedded_main = embedded_runtime.module()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":800,"value":"Create and define variables, we will use the definitions that we compiled earlier","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":717,"value":"embedded_variables = compiled_variables.map((v) =>\n embedded_main\n .variable({})\n .define(v._name, v._inputs, toFunction(v._definition))\n)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":743,"value":"toFunction = (definition) => {\n let _fn;\n eval(`_fn = ${definition}`);\n return _fn;\n}","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":808,"value":"The runtime will just run automatically, but as the variables are asyncronously processed, it can take a little while before we can observe the value directly in `_value', so we can observe the promise instead.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":723,"value":"{\n await new Promise((r) => setTimeout(r, 1000));\n return embedded_variables[0]._promise;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":827,"value":"### Value change notification with Observers","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":845,"value":"When a variable is setup, you can set a callback for value changes. Supply functions for handling for `pending`, `fulfilled` and `rejected` ([observers](https://github.com/observablehq/runtime?tab=readme-ov-file#observers) docs). Here we create a variable on a timer and send it result to a Generator function.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":855,"value":"Generators.observe((notify) => {\n // Generator.observe is a notebook userspace concept to make emitting values simpler\n embedded_main\n .variable({\n fulfilled: (value) => {\n // Observer.fulfilled\n notify(value); // pipe the variable notification up to userspace\n }\n })\n .define(async function* () {\n // zero input async generator\n // this is pure JS, normally the output a compile step\n // is used here\n let i = 0;\n while (true) {\n await new Promise((resolve) => setTimeout(resolve, 1000));\n yield i++;\n }\n });\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":896,"value":"### Rendering values with Inspector","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":830,"value":"To display cell values reactively on a _webpage_, the values need to be converted to DOM and kept up to date reactively. Observable provides the `Inspector` for this purpose, which implements the Observer interface ","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":905,"value":"

","pinned":true,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"cell_out","database":"$undefined"},{"id":1242,"value":"import { Inspector } from \"@tomlarkworthy/inspector\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":911,"value":"inspector = new Inspector(cell_out) // connect Inspector to our DOM node","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":902,"value":"Generators.observe((notify) => {\n // Generator.observe is a notebook userspace concept to make emitting values simpler\n embedded_main.variable(inspector).define(async function* () {\n // zero input async generator\n // this is pure JS, normally the output a compile step\n // is used here\n let i = 0;\n while (true) {\n await new Promise((resolve) => setTimeout(resolve, 1000));\n yield i++;\n }\n });\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":966,"value":"The Observer used in notebooks is not the same as the distributed Inspector, there is interesting metadata in the notebook's version. You can get a reference to it via the variable `_observer` reference. You can discover the type of cell by looking at the mode.","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":973,"value":"notebook_observer = main._scope.get(\"title_cell\")._observer","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":995,"value":"notebook_observer.mode","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":984,"value":"Furthermore, you can relate the variable to the rendered DOM node in the notebook","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":991,"value":"main._scope.get(\"title_cell\")._observer._node.innerHTML","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":1189,"value":"## Views\n\n`viewof` macro expands to two variables. The 1st is a EventSource (typically a HTML element), this is the UI control, and the events it emits are the \"data\". The 2nd variable source is a `Generators.input()` which registers to the event source and pipes the data as the value to the reactive variable.\n\nYou don't really *need* views. You can create the equivalent manually:-","pinned":false,"hidden":false,"mode":"md","author":"$undefined","summary":"$undefined","output":"","database":"$undefined"},{"id":1194,"value":"control = html`