2f:T4a2,Plot.plot({ marginLeft: 75, y: { domain: ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'], label: null, tickSize: 0, paddingInner: 0.03 }, x: { tickSize: 0, //specify the domain explicitly, otherwise there are missing hours domain: d3.ticks(...d3.extent(dow_hour_description_clean, d => d.HOUR), 24), label: 'Hour of the day', paddingInner: 0.03 // instead of inset since it's a band scale }, color: { // had to define the color space with a narrower range as the chart above since the "lowest" color values did not have enough contrast with the white background scheme: select_color, reverse: toggle, //range: d3.schemeRdPu[3], //interpolate: "rgb", //legend: 'ramp', legend: true, //domain: d3.extent(dow_hour_description_clean.filter(aq.escape( d => d.OFFENSE_DESCRIPTION == type_selector )), d => d.count), label: 'Number of incidents reported' }, marks: [ Plot.cell(dow_hour_description_clean.filter(aq.escape( d => d.OFFENSE_DESCRIPTION == type_selector )), { x: 'HOUR', y: 'DAY_OF_WEEK', fill: 'count', title: 'count' }) ] })30:T41e,"zoomed in map of boston" // Plot.plot({ // inset: -1500, // insetRight: -1000, // insetTop: -250, // width, // height: 440, // projection: { // type: 'reflect-y', // domain: boston_city_council_districts_current // }, // color: { // scheme: select_color, // reverse: toggle, // legend: true, // label: 'Number of incidents reported' // }, // marks: [ // Plot.geo(boston_city_council_districts_current, {fill: '#f2f2f2', opacity: 0.55}), //stroke: 'grey', strokeOpacity: 0.75, strokeWidth: 1}),, // Plot.dot(lat_long_count, Plot.hexbin({title: 'count', fill: 'count', r: 'count'},{ // x: 'Long', // y: 'Lat', // symbol: 'circle', // title: 'count', // clip: 'frame' // })), // //Plot.frame(), // // Plot.text(boston_city_council_districts_current.features, // // Plot.centroid({ // // text: (d) => d.properties.Councillor, // // textAnchor: "middle", // // stroke: "white", // // fill: "black" // // })) // ] // })31:[{"id":"164900f1911c2a882cb38dbef5d80c14ce67838dc2625a6e9dcb470335ef2be3a5a64f7eb272d561d7df78a6e6d7102fff1b783ebdcde3348326c1e125944af4","href":"https://static.observableusercontent.com/files/164900f1911c2a882cb38dbef5d80c14ce67838dc2625a6e9dcb470335ef2be3a5a64f7eb272d561d7df78a6e6d7102fff1b783ebdcde3348326c1e125944af4","name":"tmp7_f32p54.csv","type":"text/csv","lastModified":1643326880746.716,"size":12970344},{"id":"6e6fb005140e6d622d579cdfe48f7e507735e8893de2b0822d0ce985afbc87439a81431c9afaffbb239cae8acad8397a2b4e2ddebbdbef5e8deb627817ba97d1","href":"https://static.observableusercontent.com/files/6e6fb005140e6d622d579cdfe48f7e507735e8893de2b0822d0ce985afbc87439a81431c9afaffbb239cae8acad8397a2b4e2ddebbdbef5e8deb627817ba97d1","name":"City_Council_Districts_-_Current.geojson","type":"application/geo+json","lastModified":1678591892513.968,"size":399177},{"id":"c30ddb20dc761b164da26258200c50078eaa78a0cf90f4bd12ce7801d2c9474d9f1d1ebe0c79f91e7c970832169f0944d4bb129f62a7d68305ca457a2e8b9c88","href":"https://static.observableusercontent.com/files/c30ddb20dc761b164da26258200c50078eaa78a0cf90f4bd12ce7801d2c9474d9f1d1ebe0c79f91e7c970832169f0944d4bb129f62a7d68305ca457a2e8b9c88","name":"census_data_for_2022_redistricting.xlsx","type":"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet","lastModified":1679345619665.19,"size":226762}] 2d:{"title":"Boston Crime Incident Reports (2022)","cells":[{"id":0,"value":"# Boston Crime Incident Reports (2022)\n\nThis notebook documents an exploration of the data on the city of Boston crime incident reports for the year of 2022.\n\nBelow you can read a description of the dataset from the [boston.gov](https://data.boston.gov/dataset/crime-incident-reports-august-2015-to-date-source-new-system) website: \n\n*Crime incident reports are provided by Boston Police Department (BPD) to document the initial details surrounding an incident to which BPD officers respond. This is a dataset containing records from the new crime incident report system, which includes a reduced set of fields focused on capturing the type of incident as well as when and where it occurred. Records in the new system begin in June of 2015.*\n\n
","pinned":false,"hidden":false,"mode":"md"},{"id":952,"value":"md `## When do incidents take place throughout the year?`","pinned":false,"hidden":false,"mode":"ojs"},{"id":778,"value":"###
\nWe can use a heat map to display the number of incidents reported by day of the week and by hour. When looking at the distribution, there are a few clusters become apparent: \n- Three clusters of most incidents reported are: right at 12am, between 4pm and 6pm, and at 12pm\n- Least incidents reported are between the hours of 4am and 6am.","pinned":false,"hidden":false,"mode":"md"},{"id":1233,"value":"Plot.plot({\n marginLeft: 75,\n y: {\n domain: ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday'],\n label: null,\n tickSize: 0,\n paddingInner: 0.03\n },\n x: {\n tickSize: 0,\n label: 'Hour of the day',\n paddingInner: 0.03\n },\n color: {\n scheme: select_color,\n reverse: toggle,\n legend: true,\n label: 'Number of incidents reported'\n },\n marks: [\n Plot.cell(dow_hour_clean.objects(), {\n x: 'HOUR',\n y: 'DAY_OF_WEEK',\n fill: 'count',\n title: 'count'\n })\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":925,"value":"As the description suggests, the dataset also contains various categories of types of incidents, so by filtering by the type of offense we can see that, for instance, most auto theft incident reports take place during the late hours of the day (after 5pm) and the least between 3am and 6am-a similar pattern than the overall distribution. Feel free to look through the various offense types by clicking on the drop-down menu.","pinned":false,"hidden":false,"mode":"md"},{"id":824,"value":"viewof type_selector = Inputs.select(clean_obj.map(d => d.OFFENSE_DESCRIPTION), {label: \"Select a type of offense\", unique: true, sort: true, value: 'AUTO THEFT'})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1249,"value":"$2f","pinned":false,"hidden":false,"mode":"ojs"},{"id":811,"value":"If we slightly shift our view to see the same metric--number of incidents--by hour of the day and by week number, we see that with the exception of week 0, the data is evenly distributed throughout the year.","pinned":false,"hidden":false,"mode":"md"},{"id":1219,"value":"Plot.plot({\n marginBottom: 50,\n marks: [\n Plot.cell(week_hour_clean.objects(), {\n x: 'week_num',\n y: 'hour',\n fill: 'count',\n title: d => `Week: ${d.week_num}\\n Hour: ${d.hour}\\n ${d.count} incidents reported`\n }),\n ],\n y: {\n tickSize: 0,\n ticks: d3.ticks(...d3.extent(week_hour_clean.objects(), d => d.hour), 12),\n padding: 0\n },\n x: {\n tickSize: 0,\n ticks: d3.ticks(...d3.extent(week_hour_clean.objects(), d => d.week_num), 24),\n label: 'Week number',\n labelOffset: 32.5,\n padding: 0\n },\n height: 350,\n color: {\n scheme: select_color,\n reverse: toggle,\n legend: true,\n label: 'Number of incidents reported'\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":958,"value":"

\n## Where are incidents reported?","pinned":false,"hidden":false,"mode":"md"},{"id":1036,"value":"The map below displays a dot for every location where an incident was reported.\n

","pinned":false,"hidden":false,"mode":"md"},{"id":1046,"value":"boston_outline = Plot.plot({\n width,\n projection: {\n type: 'reflect-y',\n domain: boston_city_council_districts_current\n },\n color: {\n scheme: 'RdPu'\n },\n marks: [\n //Plot.frame(),\n Plot.geo(boston_city_council_districts_current, {fill: '#f2f2f2'}), //{fill: d3.interpolateMagma(0.75), stroke: 'white'}),\n Plot.dot(lat_long_count, {\n x: 'Long',\n y: 'Lat',\n r: 0.15\n })\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1731,"value":"Some of the major avenues become evident as well as some areas where, at first glance, there are too many events reported (some areas like the airport and bodies of water have less incident reports as well).\n\nTo get a sense of where more incidents are reported, we can use the size of each circle to represent the number of incidents reported. Here's a \"zoomed in\" view of the downtown Boston area.","pinned":false,"hidden":false,"mode":"md"},{"id":1083,"value":"Plot.plot({\n inset: -1500,\n insetRight: -1000,\n insetTop: -250,\n width,\n height: 440,\n projection: {\n type: 'reflect-y',\n domain: boston_city_council_districts_current\n },\n marks: [\n Plot.geo(boston_city_council_districts_current, {fill: '#f2f2f2'}),\n Plot.dot(lat_long_count, {\n x: 'Long',\n y: 'Lat',\n r: 'count',\n clip: 'frame',\n fill: true\n }),\n //Plot.frame(),\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1735,"value":"After zooming in we can get some sense of the distribution, however, the map suffers from overplotting: points are placed on top of each other, so it is challenging to get a clear sense of the clusters. \n\nOne technique we can use to alleviate overplotting is called density plotting. By using this technique, we can bin the data spatially and in conjunction with size and color, we can get a better view of where concentrations of incidents take place.","pinned":false,"hidden":false,"mode":"md"},{"id":960,"value":"$30","pinned":false,"hidden":false,"mode":"ojs"},{"id":1755,"value":"viewof bin_selector = Inputs.select([\"circle\", \"hexagon\"], {label: \"Select the shape\", value: 'circle'})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1746,"value":"Plot.plot({\n inset: -1500,\n insetRight: -1000,\n insetTop: -250,\n width,\n height: 440,\n projection: {\n type: 'reflect-y',\n domain: boston_city_council_districts_current\n },\n color: {\n scheme: select_color,\n reverse: toggle,\n legend: true,\n label: 'Number of incidents reported'\n },\n marks: [\n Plot.geo(boston_city_council_districts_current, {fill: '#f2f2f2', opacity: 0.4, stroke: 'grey', strokeOpacity: 0.4, strokeWidth: 1}),,\n Plot.dot(lat_long_count, Plot.hexbin({title: 'count', fill: 'count', r: 'count'},{\n x: 'Long',\n y: 'Lat',\n symbol: bin_selector,\n title: 'count',\n clip: 'frame'\n })),\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1694,"value":"md `Yet another way to look at this spatial distribution is to estimate the density of the data. Here's a brief explanation from the documentation of the density mark: *\"the estimated density of two-dimensional point clouds. Contours guide the eye towards the local peaks of concentration of the data, much like a topographic map does with elevation. This is especially useful given overplotting in dense datasets\"*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1772,"value":"Below is a zoomed in view of the downton Boston, Back Bay and East Boston areas.","pinned":false,"hidden":false,"mode":"md"},{"id":869,"value":"Plot.plot({\n inset: -1500,\n insetRight: -1000,\n insetTop: -250,\n width,\n height: 440,\n //height: 600, // used to generate website image\n projection: {\n type: 'reflect-y',\n domain: boston_city_council_districts_current\n },\n color: {\n scheme: select_color,\n reverse: toggle,\n //legend: true\n },\n marks: [\n Plot.geo(boston_city_council_districts_current, {stroke: 'grey', strokeOpacity: 0.75, strokeWidth: 1}),\n Plot.density(lat_long_count, {\n x: 'Long',\n y: 'Lat',\n fill: 'density',\n opacity: 0.25,\n thresholds: 15,\n bandwidth: 7.5,\n clip: 'frame'\n }),\n //Plot.frame()\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1776,"value":"But what if we want to look at different type of incident reports like we did in our exploration of the temporal distribution? ","pinned":false,"hidden":false,"mode":"md"},{"id":1007,"value":"Inputs.bind(Inputs.select(clean_obj.map(d => d.OFFENSE_DESCRIPTION), {label: \"Select a type of offense\", unique: true, sort: true, value: 'AUTO THEFT'}), viewof type_selector)","pinned":false,"hidden":false,"mode":"ojs"},{"id":1027,"value":"viewof show_heatmap = Inputs.toggle({label: \"Show Heatmap\", value: true})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1676,"value":"viewof show_dots = Inputs.toggle({label: \"Show Dots\", value: true})","pinned":false,"hidden":false,"mode":"ojs"},{"id":989,"value":"Plot.plot({\n width,\n projection: {\n type: 'reflect-y',\n domain: boston_city_council_districts_current\n },\n color: {\n scheme: select_color,\n reverse: toggle,\n },\n marks: [\n Plot.graticule(),\n Plot.geo(boston_city_council_districts_current, {fill: '#f2f2f2'}),\n show_heatmap ? Plot.density(geo_filtered_type, {\n x: 'Long',\n y: 'Lat',\n fill: 'density',\n opacity: 0.25,\n thresholds: 15,\n bandwidth: 10,\n }) : null,\n show_dots ? Plot.dot(geo_filtered_type, {\n x: 'Long',\n y: 'Lat',\n fill: 'white',\n stroke: 'black',\n //opacity: 0.5,\n r: 3,\n }) : null,\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1698,"value":"raw_boston_population = FileAttachment(\"census_data_for_2022_redistricting.xlsx\").xlsx()","pinned":true,"hidden":false,"mode":"ojs"},{"id":1701,"value":"boston_population = raw_boston_population.sheet(\"2020\", {headers: true})","pinned":true,"hidden":false,"mode":"ojs"},{"id":1704,"value":"aq.from(boston_population)\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":5,"value":"

\n## Appendix\n
\nBelow you can see a high level description of the data querying, processing and cleanup for the notebook.\n
","pinned":false,"hidden":false,"mode":"md"},{"id":1488,"value":"md `### *Data querying*`","pinned":false,"hidden":false,"mode":"ojs"},{"id":657,"value":"// the url to fetch the csv file\nraw_url = 'https://data.boston.gov/dataset/6220d948-eae2-4e4b-8723-2dc8e67722a3/resource/313e56df-6d77-49d2-9c49-ee411f10cf58/download/tmpdfeo3qy2.csv'","pinned":true,"hidden":false,"mode":"ojs"},{"id":659,"value":"// using d3.csv() to parse the file and get an array of objects\ncsv_fetch = d3.csv(raw_url)","pinned":true,"hidden":false,"mode":"ojs"},{"id":124,"value":"// import arquero for data manipulation\nimport { aq, op } from '@uwdata/arquero'","pinned":true,"hidden":false,"mode":"ojs"},{"id":1533,"value":"### Main dataset, clean\n
\nThe goal of this table is to have the data I need with the values being the correct type (i.e. dates as js date objects) as well as preparing the data for further segmentation: creating time intervals and new categories, etc.","pinned":false,"hidden":false,"mode":"md"},{"id":661,"value":"// the main table where the output data is clean. The goal is to get a table that is generic enough to derive other tables as needed\n\nviewof clean_data = aq.from(csv_fetch) // viewof shows the table view, but assigns the table value so I can create new tables downstream\n .derive({\n date: aq.escape( d => d3.utcDay(op.parse_date(d['OCCURRED_ON_DATE'])) ), // parse the date to convert into js date objects\n hour: d => +d.HOUR, // coerce the hour value to an integer\n Lat: d => +d.Lat, // coerce both latitutde and longitude into integers\n Long: d => +d.Long\n })\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":822,"value":"// this is an instance of the main table as an array of objects. Some manipulations such as new Maps are not possible from a table format, so having both is useful\n\nclean_obj = clean_data.objects()","pinned":true,"hidden":false,"mode":"ojs"},{"id":850,"value":"// data table for seeing number of incidents reported by hour and day of the week\nviewof dow_hour_clean = clean_data\n .derive({HOUR: d => +d.HOUR})\n .groupby('HOUR', 'DAY_OF_WEEK')\n .count()\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":703,"value":"// data table for seeing number of incidents reported by hour, day of the week and type (i.e. offense description)\nviewof dow_hour_description_clean = clean_data\n .derive({HOUR: d => +d.HOUR})\n .groupby('HOUR', 'DAY_OF_WEEK', 'OFFENSE_DESCRIPTION')\n .count()\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":802,"value":"viewof week_hour_clean = clean_data\n .derive({\n week_num: d => op.utcweek(d.date),\n hour: d => +d.HOUR\n })\n .groupby('week_num', 'hour')\n .count()\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":784,"value":"viewof year_date_clean = clean_data\n .groupby('date')\n .count()\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":889,"value":"viewof lat_long_count = clean_data\n .groupby('Lat', 'Long')\n .count()\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":993,"value":"viewof lat_long_count_type = clean_data\n .groupby('Lat', 'Long', 'OFFENSE_DESCRIPTION')\n .count()\n .view()","pinned":true,"hidden":false,"mode":"ojs"},{"id":991,"value":"geo_filtered_type = lat_long_count_type.filter(aq.escape( d => d.OFFENSE_DESCRIPTION == type_selector ))","pinned":true,"hidden":false,"mode":"ojs"},{"id":1543,"value":"### GeoJSON Data","pinned":false,"hidden":false,"mode":"md"},{"id":865,"value":"// Found this file on the boston.gov site that represents the city of Boston by district\n\nboston_city_council_districts_current = FileAttachment(\"City_Council_Districts_-_Current.geojson\").json()","pinned":true,"hidden":false,"mode":"ojs"},{"id":873,"value":"import {rewind} from \"@fil/rewind\"","pinned":true,"hidden":false,"mode":"ojs"},{"id":1564,"value":"viewof select_color = Inputs.select(color_schemes, {label: \"Select a color scheme\", value: 'magma'})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1634,"value":"md `### Color`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1578,"value":"viewof toggle = Inputs.toggle({label: \"reverse\", value: true})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1637,"value":"color_schemes = [\"BuGn\", \"BuPu\", \"GnBu\", \"OrRd\", \"PuBu\", \"PuBuGn\", \"PuRd\", \"RdPu\", \"YlGn\", \"YlGnBu\", \"YlOrBr\", \"YlOrRd\", \"cividis\", \"inferno\", \"magma\", \"plasma\", \"viridis\", \"cubehelix\", \"warm\", \"cool\"]","pinned":false,"hidden":false,"mode":"ojs"},{"id":9,"value":"import {vl} from '@vega/vega-lite-api'","pinned":false,"hidden":false,"mode":"ojs"},{"id":1597,"value":"md `### Images Only`","pinned":false,"hidden":false,"mode":"ojs"},{"id":1604,"value":"viewof width_range = Inputs.range([300, 1000], {label: \"width\", step: 10})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1608,"value":"viewof height_range = Inputs.range([100, 1000], {label: \"height\", step: 10})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1600,"value":"Plot.plot({\n width: width_range,\n height: height_range,\n marks: [\n Plot.cell(week_hour_clean.objects(), {\n x: 'week_num',\n y: 'hour',\n fill: 'count',\n title: d => `${d.count} incidents reported`\n }),\n ],\n y: {\n axis: null,\n padding: 0\n },\n x: {\n axis: null,\n padding: 0\n },\n color: {\n scheme: select_color,\n reverse: toggle\n }\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1617,"value":"viewof inset_form = Inputs.form({\n inset: Inputs.range([-5000, 100], {label: \"Inset\", step: 50}),\n insetRight: Inputs.range([-500, 100], {label: \"Inset Right\", step: 10, value: 0}),\n insetLeft: Inputs.range([-500, 100], {label: \"Inset Left\", step: 10, value: 0}),\n insetBottom: Inputs.range([-500, 100], {label: \"Inset Bottom\", step: 10, value: 0}),\n insetTop: Inputs.range([-500, 100], {label: \"Inset Top\", step: 10, value: 0}),\n})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1628,"value":"viewof frame = Inputs.toggle({label: \"Frame\", value: true})","pinned":false,"hidden":false,"mode":"ojs"},{"id":1615,"value":"Plot.plot({\n inset: inset_form.inset,\n insetRight: inset_form.insetRight,\n insetTop: inset_form.insetTop,\n // insetBottom: inset_form.insetBottom,\n // insetLeft: inset_form.insetLeft,\n width: width_range,\n height: height_range,\n projection: {\n type: 'reflect-y',\n domain: boston_city_council_districts_current\n },\n color: {\n scheme: select_color,\n reverse: toggle\n },\n marks: [\n Plot.geo(boston_city_council_districts_current, {stroke: 'grey', strokeOpacity: 0.75, strokeWidth: 1}),\n Plot.density(lat_long_count, {\n x: 'Long',\n y: 'Lat',\n fill: 'density',\n opacity: 0.25,\n thresholds: 15,\n clip: 'frame'\n }),\n frame ? Plot.frame({strokeWidth: 0.5}) : null\n ]\n})","pinned":false,"hidden":false,"mode":"ojs"}],"files":"$W31","stdlib":"1","titling":"auto"}