2f:["$","title",null,{"children":"Introduction to Asynchronous Iteration | Observable | Observable"}] 34:T4be,md`# Introduction to Asynchronous Iteration

The Persistence of Memory (1931) — Image: [Salvador Dali](https://en.wikipedia.org/wiki/The_Persistence_of_Memory)

Observable now supports [asynchronous iteration](http://2ality.com/2016/10/asynchronous-iteration.html), an exciting new feature of ES2018 that was [finalized last month](http://2ality.com/2017/02/ecmascript-2018.html) by TC39! You’ll need a [very recent browser](http://kangax.github.io/compat-table/es2016plus/#test-Asynchronous_Iterators_for-await-of_loops) to run this notebook, such as Chrome 63+, Firefox 57+ or Safari Technology Preview 48. [Asynchronous programming](/@observablehq/introduction-to-promises) (or *async*, for short) allows the browser to do multiple things at the same time, such as to download files and perform complex calculations and respond fluidly to user interaction. Async iteration is a simple extension of this concept: rather than a single async value, you have *multiple* async values and you want to step through them one at a time. For instance, say you have several files you want to load into your notebook:`35:[{"id":"e2a5a0ac299b53fc1016e082c7c0732d50d0dc9388d8b99adadeee40d6d1c856991c15698dcdf582a6827b411ba6d470bf7a6d28426fb8bc1cd7cc28f17199bd","href":"https://static.observableusercontent.com/files/e2a5a0ac299b53fc1016e082c7c0732d50d0dc9388d8b99adadeee40d6d1c856991c15698dcdf582a6827b411ba6d470bf7a6d28426fb8bc1cd7cc28f17199bd","name":"dali.jpg","type":"image/jpeg","lastModified":1572461324260.177,"size":179013},{"id":"6d317b07c3b0665873744a7699c11427fc6c80980fca0577db2e773a044faa75be62c8c7f68f7b3389bde6e40408e44e7b539fc125a04543fabb3655a8be7272","href":"https://static.observableusercontent.com/files/6d317b07c3b0665873744a7699c11427fc6c80980fca0577db2e773a044faa75be62c8c7f68f7b3389bde6e40408e44e7b539fc125a04543fabb3655a8be7272","name":"2014-acs5-B01003-state-45.json","type":"application/octet-stream","lastModified":1572462173085.986,"size":33097},{"id":"e31c9708a456119f4bbbb8f341db462d2b286bd9c4f8ea8b91c1e5a47cd7c84368c22655076f58553974bafe96d51bb760239487134801d663a5fbb9ef52a25d","href":"https://static.observableusercontent.com/files/e31c9708a456119f4bbbb8f341db462d2b286bd9c4f8ea8b91c1e5a47cd7c84368c22655076f58553974bafe96d51bb760239487134801d663a5fbb9ef52a25d","name":"2014-acs5-B01003-state-01.json","type":"application/octet-stream","lastModified":1572462172893.3,"size":35460},{"id":"e5b3b36161b5b1ed9a4368989da86a9b93b7ada70503212a8608dce2ddf8472dc1ea491082c55fe23ea1da56bbaf6102b466047b082ed71926faea2530bbd824","href":"https://static.observableusercontent.com/files/e5b3b36161b5b1ed9a4368989da86a9b93b7ada70503212a8608dce2ddf8472dc1ea491082c55fe23ea1da56bbaf6102b466047b082ed71926faea2530bbd824","name":"2014-acs5-B01003-state-04.json","type":"application/octet-stream","lastModified":1572462172955.221,"size":45795}] 32:{"id":"417ace5e790eb84c","slug":"introduction-to-asynchronous-iteration","title":"Introduction to Asynchronous Iteration","thumbnail":"5fd801fe3afd6c9ca0fd57e8419bbdd11e99635a08799ba45f5183c189bfb817","access_level":"public","create_time":"$D2018-01-19T05:26:39.661Z","update_time":"$D2021-06-15T05:10:32.580Z","view_time":null,"first_public_time":"$D2018-02-09T19:17:09.233Z","has_unpublished_edits":null,"fork":false,"stars":67,"starred":false,"owner":{"type":"team","id":"f35c755083683fe5","login":"observablehq","name":"Observable","avatar_url":"https://avatars-next.observableusercontent.com/424c6c1f7794371c9915a160519bdfb34cf34e50069fcd6def38a1a04d4e9702","tier":"enterprise_2024"},"version":514,"publish_version":514,"role":null,"authored":false,"authors":[],"members":[{"type":"individual","id":"074c414ad1d825f5","login":"mbostock","name":"Mike Bostock","avatar_url":"https://avatars-next.observableusercontent.com/7dbba99ced40bd5ab8013c99b1a11da61231bce321ba1148a9d16ad0f121550a","tier":null,"role":"editor"},{"type":"team","id":"f35c755083683fe5","login":"observablehq","name":"Observable","avatar_url":"https://avatars-next.observableusercontent.com/424c6c1f7794371c9915a160519bdfb34cf34e50069fcd6def38a1a04d4e9702","tier":"enterprise_2024","role":"viewer"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":36,"collection":null,"type":"notebook","body":{"title":"Introduction to Asynchronous Iteration","cells":[{"id":0,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":251,"value":"files = [\n FileAttachment(\"2014-acs5-B01003-state-01.json\"),\n FileAttachment(\"2014-acs5-B01003-state-04.json\"),\n FileAttachment(\"2014-acs5-B01003-state-45.json\")\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":258,"value":"md`These files contain population estimates by county for Alabama, Arizona and South Carolina; the exact meaning isn’t important for this notebook. You can load these files in parallel using [*array*.map](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array/map) and the [FileAttachment API](/@observablehq/file-attachments):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":270,"value":"promises = files.map(file => file.json())","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":422,"value":"datasets = Promise.all(promises)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":297,"value":"md`You can wait for these fetches to finish using [Promise.all](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise/all). Either define the all-promise in a separate cell for implicit await, as above, or use an explicit \\`await\\`, as below. Once the files are loaded, you can use normal, *synchronous* iteration—a for loop—to iterate over them and do something, like compute a sum.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":293,"value":"{\n let total = 0;\n const datasets = await Promise.all(files.map(file => file.json()));\n for (const dataset of datasets) {\n for (let i = 1; i < dataset.length; ++i) {\n total += +dataset[i][0];\n }\n }\n return total;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":274,"value":"md`But what if you don’t want to wait until *all* the data is loaded? What if you want to [show intermediate results](/@mbostock/showing-progress) as the files load? Observable uses [generators](https://developer.mozilla.org/docs/Web/JavaScript/Guide/Iterators_and_Generators) to allow cells to yield a value that changes over time. Wait for each file to load, then yield the incremental sum:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":277,"value":"{\n let total = 0;\n for (const file of files) {\n const dataset = await file.json();\n for (let i = 1; i < dataset.length; ++i) {\n total += +dataset[i][0];\n yield total;\n }\n }\n return total;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":349,"value":"md`By using \\`await\\` and \\`yield\\` in the same cell, you’ve written an asynchronous generator. Congrats! 🎉 But wait, there’s more! \n\nThe cell above loads the files serially; it waits for the sum to be computed for the previous file before starting to fetch for the next file. If desired, you can instead start all the fetches simultaneously, and then step through the files one-by-one to compute the sum. As long as the server can handle the load, it’s typically faster to download multiple files concurrently. And unlike Promise.all, we don’t have to wait for all the files to download—just the next file in the list.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":341,"value":"{\n let total = 0;\n for (const promise of files.map(file => file.json())) {\n const dataset = await promise;\n for (let i = 1; i < dataset.length; ++i) {\n total += +dataset[i][0];\n yield total;\n }\n }\n return total;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":368,"value":"md`This pattern—iterating over an array of promises, and waiting for each one to resolve sequentially—can be expressed more succinctly using the new for-await-of loop:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":289,"value":"{\n let total = 0;\n for await (const dataset of files.map(file => file.json())) {\n for (let i = 1; i < dataset.length; ++i) {\n total += +dataset[i][0];\n yield total;\n }\n }\n return total;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":254,"value":"md`So, async iteration is a way to *consume* (iterate over) async values, while an async generator is a way to *produce* (yield) async values. The async generators above are implicit—they are cells that both \\`await\\` and \\`yield\\`. But you can also make an explicit async generator function in Observable using vanilla JavaScript:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":4,"value":"async function* foo() {\n for (let i = 1; i <= 1000; ++i) {\n await Promises.delay(100);\n yield i;\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":377,"value":"md`Calling *foo* returns an async generator that yields an incrementing number every 100 milliseconds. It starts at 1 and goes up to 1000. You can show these numbers in a notebook cell by simply returning the generator:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":373,"value":"foo()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":67,"value":"md`Alternatively, you can read the yielded values “by hand” using the iterator protocol:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":72,"value":"{\n const generator = foo();\n const iterator = generator[Symbol.asyncIterator]();\n while (true) {\n const {done, value} = await iterator.next();\n if (done) return;\n yield value;\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":217,"value":"md`Observable already supported async generators in a sense: you can also define a (synchronous) generator cell that yields promises. Observable waits until the previous promise resolves before pulling the next one from the generator.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":207,"value":"function* altFoo() {\n for (let i = 0; i < 1000; ++i) {\n yield Promises.delay(100, i);\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":446,"value":"altFoo()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":486,"value":"md`And even a synchronous generator cell in Observable is implicitly async: Observable only pulls one value per animation frame from the generator, giving you fluid sixty frames per second by default.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":481,"value":"function* fastFoo() {\n for (let i = 0; i < 1000; ++i) {\n yield i;\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":482,"value":"fastFoo()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":219,"value":"md`But async generators are a lot more flexible that normal generators: you can both \\`yield\\` and \\`await\\`! (Also, a normal generator must know synchronously whether it’s done, whereas an asynchronous generator can decide whenever it wants to return.) For example, async generators make it very easy to express animations as a sequence of transitions between keyframes: first yield the DOM element you want to display in the notebook, and then repeatedly await for each transition to finish before starting the next.`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":181,"value":"{\n const w = Math.min(640, width);\n const h = 320;\n const r = 20;\n const t = 1500;\n const svg = d3.select(DOM.svg(w, h));\n const circle = svg.append(\"circle\").attr(\"r\", r).attr(\"cx\", w / 4).attr(\"cy\", h / 4);\n while (true) {\n yield svg.node();\n await circle.transition().duration(t).attr(\"cy\", h * 3 / 4).end();\n await circle.transition().duration(t).attr(\"cx\", w * 3 / 4).end();\n await circle.transition().duration(t).attr(\"cy\", h * 1 / 4).end();\n await circle.transition().duration(t).attr(\"cx\", w * 1 / 4).end();\n }\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"},{"id":184,"value":"d3 = require(\"d3@5\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W35","stdlib":"1","titling":"auto"}} 33:"$32:body" 2e:E{"digest":"NEXT_REDIRECT;replace;/@observablehq/introduction-to-asynchronous-iteration;307;"}