The avatar for @elmisback
Edward Misback
Live programming @ University of Washington
Public
Edited
Nov 22, 2023
Insert cell
# Atlas Obscura scraping
Insert cell
user = Inputs.text({value:'kevbot'})
Insert cell
user_places = {
var regex = /AtlasObscura\.user_profile\s*=\s*({[\s\S]*?\}\}\]\}\});/;

// Find the match
var match = text.match(regex);

// Initialize obj as null
var obj = null;

// Check if there was a match and parse the JSON
if (match && match[1]) {
try {
// The JSON object string is in match[1]
obj = JSON.parse(match[1]);
} catch (e) {
console.error("Parsing error:", e);
}
}
return obj.user.places
}
Insert cell
url = `https://www.atlasobscura.com/users/${user.value}?view=passport`
Insert cell
fetch(url) // rip, CORS cross origin resource sharing (avoids cross-site scripting attacks)
Insert cell
import {fetchp} from '@tomlarkworthy/fetchp' // NOTE notebook must be published/link-shared for this to run
Insert cell
fetchp
Insert cell
response = fetchp(url)
Insert cell
response.status
Insert cell
text = response.text() // empty?
Insert cell
baz = 42
Insert cell
html`${text}`
Insert cell
places = ['australia', 'japan', 'germany']
Insert cell
(await fetchp(`https://www.atlasobscura.com/things-to-do/australia`)).status
Insert cell
result = Promise.all(places.map(async p => ((await fetchp(`https://www.atlasobscura.com/things-to-do/${p}`)).text())))
Insert cell

One platform to build and deploy the best data apps

Experiment and prototype by building visualizations in live JavaScript notebooks. Collaborate with your team and decide which concepts to build out.
Use Observable Framework to build data apps locally. Use data loaders to build in any language or library, including Python, SQL, and R.
Seamlessly deploy to Observable. Test before you ship, use automatic deploy-on-commit, and ensure your projects are always up-to-date.
Learn more