The avatar for @enjalot
Ian Johnson
pixel flipper, data sifter. trying to see what I can while I'm here
Published
Edited
Jun 24, 2020
Fork of Working with Wikipedia Data
Importers
1 star
6
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
fips = d3.nest()
.key(d => d.fips)
.rollup(leaves => leaves)
.entries(departments)
.map(d => {
return {
fips: d.key,
departments: d.value,
sworn: d3.sum(d.value, e => e.sworn)
}
})
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
countsByState = (await FileAttachment("csllea08at06.csv").text()).split("\r\n")
Insert cell
countsByState[13]
Insert cell
Insert cell
Insert cell
Insert cell
// downloaded the result of this and uploaded as a File Attachment below
// fetch(`https://en.wikipedia.org/w/api.php?format=json&origin=*&action=parse&prop=text&page=List_of_United_States_state_and_local_law_enforcement_agencies&section=1`).then(response => response.json())
Insert cell
text = FileAttachment("List_of_United_States_state_and_local_law_enforcement_agencies.json").json()
Insert cell
parsed = text.parse.text["*"]
Insert cell
Insert cell
Insert cell
rows = wikitable.querySelectorAll("li")
Insert cell
list = Array.from(rows, r => r.textContent.trim())
Insert cell
regions = list.map(d => d.replace("List of law enforcement agencies in ", ""))
Insert cell
Insert cell
viewof data = tsvInput({})
Insert cell
data[5]
Insert cell
processed_departments = data.map(d => {
if(d.AGCYNAME == 34) {
// There seems to be a formatting errors in the original source file for this row
// (only seems to affect this department)
// console.log("bad row", d)
let r = {
name: d.ADDR1,
sworn: d.AGCYNAME,
type: agencyTypes[d.AGCYTYPE],
state: d.ZIP,
county: d.FIPS,
city: d.STATE,
ftciv: d.PTSWORN,
ptciv: d.Q5A,
ptsworn: d.PTCIV,
fips: "32003"
}
// console.log("good row", r)
return r
}
return {
name: d.AGCYNAME,
sworn: d.Q6_TOT,
type: agencyTypes[d.AGCYTYPE],
state: d.STATE,
county: d.COUNTY,
city: d.CITY,
ftciv: d.FTCIV,
ptciv: d.PTCIV,
ptsworn: d.PTSWORN,
zip: d.ZIP,
fips: d.FIPS
}
}).sort( (a,b) => b.sworn - a.sworn )
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell
Insert cell

One platform to build and deploy the best data apps

Experiment and prototype by building visualizations in live JavaScript notebooks. Collaborate with your team and decide which concepts to build out.
Use Observable Framework to build data apps locally. Use data loaders to build in any language or library, including Python, SQL, and R.
Seamlessly deploy to Observable. Test before you ship, use automatic deploy-on-commit, and ensure your projects are always up-to-date.
Learn more