45:["$","title",null,{"children":"NOAA Fisheries Marine Recreational Information Program Survey Data | Observable Ambassadors | Observable"}] 4d:[{"id":"9e8f0167022d0a2156a20c30f7f00a81fada137618fc77bad3f534507653e085b78c4a45d8839dff49c29c377bc9ce6e6f1daf78fa4497c60164da7f64737230","href":"https://static.observableusercontent.com/files/9e8f0167022d0a2156a20c30f7f00a81fada137618fc77bad3f534507653e085b78c4a45d8839dff49c29c377bc9ce6e6f1daf78fa4497c60164da7f64737230","name":"ps_2020_csv.zip","type":"application/zip","lastModified":1619557564146.409,"size":10430250}] 49:{"id":"75b23352f0227831","slug":"noaa-fisheries-mrip-survey","title":"NOAA Fisheries Marine Recreational Information Program Survey Data","thumbnail":"edf830e895814520f24c4232a89c9bac6466aa210a7a0f2bba2f85b33a5ae105","access_level":"public","create_time":"$D2021-04-27T20:31:30.172Z","update_time":"$D2021-10-01T13:07:49.408Z","view_time":null,"first_public_time":"$D2021-04-29T15:08:57.487Z","has_unpublished_edits":null,"fork":false,"stars":2,"starred":false,"owner":{"type":"team","id":"11bfeb572ddbe905","login":"ambassadors","name":"Observable Ambassadors","avatar_url":"https://avatars-next.observableusercontent.com/93bf798c058a2bf8d3bb90c0754f774b6278b8b16748784f520b07a8800ef772","tier":"education_2026","bio":"The Ambassador program is made up of a group of inclusive, collaborative, and curious minds in the Observable community. ","followed_by_you":false,"follows_you":false},"version":864,"publish_version":864,"role":null,"authored":false,"authors":[{"type":"individual","id":"a7fa3027921865cf","login":"aaronkyle","name":"Aaron Kyle Dennis","avatar_url":"https://avatars-next.observableusercontent.com/eb6d0b32ccc6c8febbf105ce4d2a938819bcbd1c6a2c108d3a9c01745d3590f2","tier":null,"bio":"Social development practitioner. Trying to understand our world.","followed_by_you":false,"follows_you":false}],"members":[{"type":"team","id":"11bfeb572ddbe905","login":"ambassadors","name":"Observable Ambassadors","avatar_url":"https://avatars-next.observableusercontent.com/93bf798c058a2bf8d3bb90c0754f774b6278b8b16748784f520b07a8800ef772","tier":"education_2026","role":"editor"}],"comments":[{"id":"c81e5d6897efcada","reply_to_id":null,"content":"Thank you @fil!","create_time":"$D2021-04-28T13:38:47.897Z","update_time":null,"resolved":false,"user":{"type":"individual","id":"a7fa3027921865cf","login":"aaronkyle","name":"Aaron Kyle Dennis","avatar_url":"https://avatars-next.observableusercontent.com/eb6d0b32ccc6c8febbf105ce4d2a938819bcbd1c6a2c108d3a9c01745d3590f2","tier":null}}],"custom_thumbnail":true,"trashed":false,"lapsed":null,"subscribed":null,"forks":3,"collection":null,"type":"notebook","body":{"titling":"auto","title":"NOAA Fisheries Marine Recreational Information Program Survey Data","cells":[{"id":0,"value":"md`# NOAA Fisheries Marine Recreational Information Program Survey Data`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":180,"value":"md`\n\n**What's This?**\n\nThis is a work-in-progress welcome and intro to getting data in JS for our friends at NOAA and others interested in using Observable and ready to roll with data.\n\nPS - **Feel free to edit and elaborate!** If you have ideas for how to render data, we're happy to have you!\n\n_Some thoughts for next steps:_\n\n1. mess with data using tidy.js to make it more familiar wiht R folks\n2. chart some stuff; look for time-series visualization ideas, etc.\n3. think more about mapping options\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":782,"value":"md`\n**Without these people, I could not have accomplished this write-up:**
\n[@chrispahm](https://observablehq.com/@chrispahm)
\n[@tomlarkworthy](https://observablehq.com/@tomlarkworthy)
\n[@fil](https://observablehq.com/@fil)
\n**Thank you!**\n\n----\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":2,"value":"md`\n## [Recreational Fishing Data](https://www.fisheries.noaa.gov/recreational-fishing-data/recreational-fishing-data-downloads)\n\n_The Marine Recreational Information Program encourages its partners and stakeholders to download its recreational fishing data and catch and effort estimates._\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":274,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":186,"value":"md`\nLet's dive into a dataset! We'll start with the Marine Recreational Information Program (MRIP) Survey Data, described on the NOAA Fisheries as follows:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":182,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":7,"value":"md`\n### [MRIP Survey Data](http://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/) (SAS, [CSV](https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/))\n\n* **TRIP_YYYYW.sas7bdat:** Trip-level data (analogous to MRFSS i1 dataset) and variables required for estimation. Contains one record per angler trip interview (identified by id_code).\n* **CATCH_YYYYW.sas7bdat:** Catch-level data and variables required for estimation. Contains one record per species for each angler trip interview.\n* **SIZE_YYYYW.sas7bdat:** Fish-level length and weight data and variables required for estimation. Contains one record per fish caught and measured or weighed by interviewer.\n\nMore information about each file type can be found in the [MRIP Read Me](https://media.fisheries.noaa.gov/2020-04/mrip_datasets_and_template_programs_read_me.doc). Variable formats and descriptions can be found in [MRIP_Survey_Variables](https://media.fisheries.noaa.gov/2020-04/mrip-survey-variables.xls) (.xls).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":190,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":189,"value":"md`\n[CSV](https://en.wikipedia.org/wiki/Comma-separated_values) is a [clear text](https://en.wikipedia.org/wiki/Plaintext) file type. I love it: simple, original, classic. Loading CSV data into Observable is a breeze for one file at a time using [File Attachments](https://observablehq.com/@observablehq/file-attachments).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":195,"value":"md`\nHowever the NOAA Fisheries site throws us a bit of a curve ball by supplying CSV data in [.ZIP](https://en.wikipedia.org/wiki/ZIP_(file_format)) bundles. Have a look at their [CSV link](https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/); it'll show something like this:\n\n\n\\`\\`\\`\n[PARENTDIR]\tParent Directory\t \t-\t \n[ ]\tps_1981_1989_csv.zip\t2018-08-23 08:14\t29M\t \n[ ]\tps_1990_1994_csv.zip\t2018-08-22 16:22\t32M\t \n[ ]\tps_1995_1999_csv.zip\t2018-08-22 14:50\t37M\t \n[ ]\tps_2000_2004_csv.zip\t2018-08-22 13:11\t46M\t \n[ ]\tps_2005_2009_csv.zip\t2018-08-22 12:15\t41M\t \n[ ]\tps_2010_2014_csv.zip\t2018-08-22 08:36\t40M\t \n[ ]\tps_2015_2016_csv.zip\t2020-04-20 10:26\t15M\t \n[ ]\tps_2017_csv.zip\t2020-04-20 10:26\t7.7M\t \n[ ]\tps_2018_csv.zip\t2019-04-30 07:52\t7.7M\t \n[ ]\tps_2019_csv.zip\t2020-04-20 10:26\t7.9M\t \n[ ]\tps_2020_csv.zip\t2021-04-16 13:15\t9.9M\t \n[ ]\tps_2021_preliminary_..> 2021-04-26 09:16\t383K\t \n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":198,"value":"md`\nAnd what is stored inside each ZIP? Let's download one and peek quickly the contents, e.g., looking inside \\`ps_2020_csv.zip\\`, here's what we get:\n\n\\`ps_2020_csv\\catch_20201.csv\\`
\n\\`ps_2020_csv\\catch_20202.csv\\`
\n\\`ps_2020_csv\\catch_20203.csv\\`
\n\\`ps_2020_csv\\catch_20204.csv\\`
\n\\`ps_2020_csv\\catch_20205.csv\\`
\n\\`ps_2020_csv\\catch_20206.csv\\`
\n\\`ps_2020_csv\\size_20201.csv\\`
\n\\`ps_2020_csv\\size_20202.csv\\`
\n\\`ps_2020_csv\\size_20203.csv\\`
\n\\`ps_2020_csv\\size_20204.csv\\`
\n\\`ps_2020_csv\\size_20205.csv\\`
\n\\`ps_2020_csv\\size_20206.csv\\`
\n\\`ps_2020_csv\\trip_20201.csv\\`
\n\\`ps_2020_csv\\trip_20202.csv\\`
\n\\`ps_2020_csv\\trip_20203.csv\\`
\n\\`ps_2020_csv\\trip_20204.csv\\`
\n\\`ps_2020_csv\\trip_20205.csv\\`
\n\\`ps_2020_csv\\trip_20206.csv\\`
\n\n... 18 individual CSV files!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":206,"value":"md`\nSince this data set comprises _a lot_ of files, let's look into methods for loading an entire ZIP archive.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":473,"value":"md`\nTo figure out how to achieve direct-access to ZIPs, I googled for 'access a zip using JavaScrpit', and quickly was referred to [JSZip](https://stuk.github.io/jszip/). \n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":151,"value":"md`\nI immediately search Observable for JSZip examples. The first notebook I encountered was from [@mbostock](https://observablehq.com/@mbostock), which shows us how to read information from a zip archive File Attachement: https://observablehq.com/@mbostock/hello-jszip\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":19,"value":"// We'll require in JSZip, which is the package for manipulating zip archives\njszip = require('jszip@3.6.0/dist/jszip.js')","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"archive = FileAttachment(\"ps_2020_csv.zip\")\n .arrayBuffer()\n .then(buffer => jszip.loadAsync(buffer))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":166,"value":"archive.files","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":69,"value":"Object.keys(archive.files)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":527,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":418,"value":"md`\nMike's approach is great, but since we have several ZIP files each hosted online, I wanted to find a more 'direct' route to accessing data (as in, not requiring us to download and re-upload the data). Being new to JavaScript I wasn't immediately understanding how to adapt this code to point to a URL rather than a file attachment. I decided to try another of the notebooks on JSZip available on Observable, and found that [@fil](https://observablehq.com/@fil/jszip) gives us the pattern in [this notebook](https://observablehq.com/@fil/jszip):`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":145,"value":"// We'll require in JSZip, which is the package for manipulating zip archives.\n// Since we already have JSZip loaded above, I am being a bit silly to require it again under a different name, but doing so because it follow's @fil's notebook more closely (for learning purposes)\nzip = require(\"jszip@3.6.0/dist/jszip.js\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":94,"value":"// Here @fil gifts us the sequence of processing actions needed to return contents of a zip file. It works simialrly to Mike's version, with Fil's notebook demonstrating how to couple it with a url fetch, which we need in this case.\nzipreader = (fileDescriptor, { type = \"string\" } = {}) =>\n fileDescriptor\n .arrayBuffer()\n .then(d => zip().loadAsync(d))\n .then(\n ({ files }) =>\n new Map(Object.keys(files).map(f => [f, files[f].async(type)]))\n )","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":462,"value":"import { Text } from '@observablehq/inputs'","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":83,"value":"// We'll need to point our zipreader function to a url, so let's separate the value here\nviewof url = Text({\n value:\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2021_preliminary_csv.zip\"\n})","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":81,"value":"// Now to load the data into our notebook\n// Using a CORS proxy is sometimes necessary [not possible w/ CORS-anywhere due to abuse; let's also go easy on Observablehq]\nA = fetch(\"https://observable-cors.glitch.me/\" + url).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":478,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":435,"value":"md`\nAs an alternative to the \\`fetch\\` method used by @fil, which required a CORS proxy (that he set up separately), [@tomlarkworthy](https://observablehq.com/@tomlarkworthy) offers us [fetchp](https://observablehq.com/@tomlarkworthy/fetchp), which takes care of everything in one fell swoop:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":160,"value":"import { fetchp } from \"@tomlarkworthy/fetchp\"","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":162,"value":"refs = fetchp(url).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":632,"value":"Promise.all(\n Array.from(refs, ([key, content]) => content.then(d => [key, d3.csvParse(d)]))\n).then(all => new Map(all))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":481,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":450,"value":"md`\nNice! We've found 2 (or 3, depending on how one is counting) viable aproaches to parsing a ZIP file! Now we're ready to delve into the data.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":490,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":447,"value":"md`\nOnce a ZIP archive is loaded, to select a CSV dataset from within it is a simple matter of selecting our target file and then instructing our notebook to interpret these data in the correct format. Here's how:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":428,"value":"// First, we'll require in D3.js, which we'll use to read in our data in CSV format below.\nd3 = require(\"d3@6\")","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":125,"value":"// And here, we specify the file name that we're interested in:\ncatch_20211 = A.get(\"catch_20211.csv\").then(d3.csvParse)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":659,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":177,"value":"md`\nGreat, so at this point we can read and work with any CSV file in a ZIP archive that is hosted in its native habitat (pun intended: i mean to say that we can access a file directly from its source).\n\nBut let's not stop here! Rather than loading each individually, let's instead load in _all_ the files within an archive.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":703,"value":"md`\nBefore going too much further, let's first have a look back at the structure of the archive we've fetched above. We're going to need this information to be able to manipulate the data.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":122,"value":"A","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":707,"value":"md`\nThe archive is a JavaScript [map](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map). We wish to determine the value of each of the items listed in this map.\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":716,"value":"// JS Maps have methods, one of which is keys that returns an iterator object of for each Map element\n// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/keys\n// Adding .keys() to our map returns that iterator\nA.keys()","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":708,"value":"// To actually access the key values, we'll have to render the iterator values out as data. We can do this by transforming it into an array:\nArray.from(A.keys())","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":712,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":738,"value":"md`\nSo now onto the challenge of parsing all files archived in our ZIP.\n\nWe have some options an alternatives for now to do this:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":493,"value":"// We'll call this the 'traditional' approach (thanks @chrispahm)!\nasync function parse_archive_index(files) {\n const results_array = [];\n for (let index = 0; index < files.length; ++index) {\n const result = await A.get(files[index]).then(d3.csvParse); // whoops! we linked this to a static file\n results_array.push(result);\n }\n return results_array;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":503,"value":"parse_archive_index(Array.from(A.keys()))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":498,"value":"// We'll call this the 'streamlined' approach (thanks @fil and @chrispahm)!\nasync function parse_archive(files) {\n const results_array = [];\n for (const [file] of files) {\n const result = await files.get(file).then(d3.csvParse); // here I was able to replace 'A' with 'files' and it works!\n results_array.push(result);\n }\n return results_array;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":216,"value":"parse_archive(A)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":496,"value":"// We'll call this the 'modern' approach (thanks @chrispahm)!\nasync function parse_archive_ES6(files) {\n const promises = files.map((file) => A.get(file).then(d3.csvParse)); // whoops! we linked this to a static file\n const result = await Promise.all(promises);\n return result;\n}","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":848,"value":"Array.from(A.keys())","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":511,"value":"parse_archive_ES6(Array.from(A.keys()))","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":759,"value":"md`\nExcellent: With this infrastrcture in place, all that's left to do is to identify which zip archive we'd like to work with. In the addendum below, we'll defined named cells for each ZIP archive within the MRIP Survey Data. From there, all we need to do is run our preferred function on the data and we're ready to roll!\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":762,"value":"parse_archive(ps_2021_preliminary)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":531,"value":"md`\n---\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":755,"value":"md`\n### Addendum\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":771,"value":"md`\nBecause these data sets get to be rather large rather quickly, we have to be judicious. Loading in this full data set all at once crashes our CORS proxies (and is generally far too much information to process in a normal browser session).\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":774,"value":"md`\nIn case we need it later, here's an array of the contents of the MRIP Survey Data:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":692,"value":"archives = [\n \"ps_1981_1989_csv.zip\",\n \"ps_1990_1994_csv.zip\",\n \"ps_1995_1999_csv.zip\",\n \"ps_2000_2004_csv.zip\",\n \"ps_2005_2009_csv.zip\",\n \"ps_2010_2014_csv.zip\",\n \"ps_2015_2016_csv.zip\",\n \"ps_2017_csv.zip\",\n \"ps_2018_csv.zip\",\n \"ps_2019_csv.zip\",\n \"ps_2020_csv.zip\",\n \"ps_2021_preliminary_csv.zip\"\n]","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":777,"value":"md`\nAnd here are links to each data source:\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":594,"value":"ps_1981_1989 = fetch(\n \"https://observable-cors.glitch.me/\" +\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_1981_1989_csv.zip\"\n).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":817,"value":"parse_archive(ps_1981_1989)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":532,"value":"/*ps_1981_1989_fetchp = fetchp(\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_1981_1989_csv.zip\").then(zipreader)*/","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":596,"value":"// ps_1990_1994 = fetch(\"https://observable-cors.glitch.me/\" + \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_1990_1994_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":534,"value":"/*ps_1990_1994_fetchp = fetchp(\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_1990_1994_csv.zip\").then(zipreader)*/","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":598,"value":"/*ps_1995_1999 = fetch(\n \"https://observable-cors.glitch.me/\" + \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_1995_1999_csv.zip\").then(zipreader)\n */","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":536,"value":"/*ps_1995_1999_fetchp = fetchp(\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_1995_1999_csv.zip\"\n).then(zipreader)*/","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":600,"value":"// ps_2000_2004 = fetch(\"https://observable-cors.glitch.me/\" + \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2000_2004_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":538,"value":"// ps_2000_2004_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2000_2004_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":602,"value":"// ps_2005_2009 = fetch(\n// \"https://observable-cors.glitch.me/\" +\n// \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2005_2009_csv.zip\"\n// ).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":540,"value":"// ps_2005_2009_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2005_2009_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":604,"value":"// ps_2010_2014 = fetch(\n// \"https://observable-cors.glitch.me/\" +\n// \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2010_2014_csv.zip\"\n// ).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":542,"value":"// ps_2010_2014_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2010_2014_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":606,"value":"// ps_2015_2016 = fetch(\n// \"https://observable-cors.glitch.me/\" +\n// \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2015_2016_csv.zip\"\n// ).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":550,"value":"// ps_2015_2016_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2015_2016_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":608,"value":"// ps_2017 = fetch(\n// \"https://observable-cors.glitch.me/\" +\n// \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2017_csv.zip\"\n// ).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":562,"value":"// ps_2017_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2017_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":610,"value":"// ps_2018 = fetch(\"https://observable-cors.glitch.me/\" + \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2018_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":565,"value":"// ps_2018_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2018_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":612,"value":"// ps_2019 = fetch(\"https://observable-cors.glitch.me/\" + \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2019_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":566,"value":"// ps_2019_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2019_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":614,"value":"ps_2020 = fetch(\n \"https://observable-cors.glitch.me/\" +\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2020_csv.zip\"\n).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":568,"value":"ps_2020_fetchp = fetchp(\"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2020_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":807,"value":"parse_archive(ps_2020)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":616,"value":"ps_2021_preliminary = fetch(\"https://observable-cors.glitch.me/\" + \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2021_preliminary_csv.zip\").then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":574,"value":"ps_2021_preliminary_fetchp = fetchp(\n \"https://www.st.nmfs.noaa.gov/st1/recreational/MRIP_Survey_Data/CSV/ps_2021_preliminary_csv.zip\"\n ).then(zipreader)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":809,"value":"parse_archive(ps_2021_preliminary)","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":138,"value":"md`\n\n----\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W4d","stdlib":"1"}} 44:null