Comment 1 [REUSED FROM ASSIGNMENT 1]: This cell simply connects the notebook to the d3.js library. This library is full of lots of visualizations that we can use in future assignments when we begin mapping. By typing in d3 = require("d3@5"), you make that connection. You honestly don’t even need to type it out traditionally, as there’s documentation online for ObservableHQ that has such command lines and their functions typed out already, ready for you to copy and paste.

Comment 2 [REUSED FROM ASSIGNMENT 1]: In this cell, we complete a very similar procedure as the last, where we import the topojson-client library into our notebook assignment. This library is able to read TopoJSON files, so it must be loaded into ObservableHQ from in some of the first cells in our projects. As I mentioned, this is similar to cell 2, where both utilizes a require() function in order to load external libraries such as this, and because we loaded the library using this function, we now have the TopoJSON functions available to use within this particular notebook. Once again, you can expect to find this command on online documentation for ObservableHQ if you ever lose it

Comment 3 [REUSED FROM ASSIGNMENT 1]: Next, ObservableHQ allows you to upload files to manipulate on a web browser. On the righthand side of the page, there’s a list of vertically oriented icons, at the very top, there’s a paperclip icon. Click on this, and then simply click, or drag and drop your .csv and TopoJSON files into the notebook. Then, you must call upon your TopoJSON file attachment in this cell via the following command line: FileAttachment(“file_name_here”).json() ... This process turns it into a JavaScript object. Also, be sure not to have the name of your file or any of its attributes lead with numbers, I learned this the hard way. Now that you have it attached to the file into the notebook, you may access the files via commands like this and others, as will be demonstrated below. This same cell is how you access the shapefile through observable, and you can click on the arrow to the left of the "Object" command line, which then allows you to observe the nested structure of the file, where you can find all of the attributes associated with, in my case, census tracts. However, this applies to any shapefile, be it counties, states, etc. Underneath the nested structure, look to the left side at the purple-colored text which reads, "type" "arcs" "transform" "objects" which appear to be vertically oriented in a line. Under "objects" (yes, the lowercase one, not Object with a capital O), you'll notice the object collection. For me, it's named "tract2020," but for you, it will certainly have another name. Type this name in to the file attachment area, as initially mentioned above in the first paragraph. Under the array, you'll see the chain of various vertices that form the polygons in your shapefile.

Comment 4 [REUSED FROM ASSIGNMENT 1]: This line transforms raw data from cell 4 into a feature collected to be used in the following assignments, all thanks to the topojson-library that we imported back in cell 3. After inserting the necessary text (in my case, tract2020_features = topojson.feature(tract2020, tract2020.objects.tract2020), but be warned, the name of your objects may be different than what your shapefile attributes show. For example, after my blunder in naming the TopoJSON “2020tracts” instead of “tract2020” (leading numbers do not work in ObservableHQ), I had to redo all of my data cleaning via QGIS and Mapshaper since the wrong name traced all the way back to the beginning, even before I changed the CRS to WGS84. I ended up having to redo all of my data because something deep inside one of the file format names would not change. Just a fair warning, be careful with this. Anyway, once loaded in, you can drop down the arrow next to Object and see the giant nested structure of this cell area. There’s tons of arrows and various attributes, coordinates, and various other information they tell about your dataset. It’s easy to get lost, but follow each arrow slowly and sequentially, and it’ll start to make some sense. Thankfully, my longitude and latitude coordinates did load in, even when the name wasn’t working. If you don’t change your .prj file to WGS84 before importing into ObservableHQ, this is where you’ll run into trouble. If it’s a 6 digit coordinate set, it’s possible that it’s UTM or some other projection that doesn’t work on web mapping. You must see expected coordinates (for instance, mine range in the lower -90 and lower 40 degrees, which is absolutely expected for Iowa. Use context clues to guide your problem-solving process. If you see a decimal point following two digits and then a handful of numbers after, chances are you imported your data correctly. Observable is officially able to read the TopoJSON at this point.

Comment 5 [REUSED FROM ASSIGNMENT 1]: In this cell, we're connecting the TopoJSON feature collection of polygons to the attributes listed in the .csv file. Mine is called "population.csv," but obviously, yours will be named something else, as will your attributes. The three I'm using here are GEOID (my join between the .csv and TopoJSON shapefile), Black population, and Total population for 2020 census tracts in Iowa. Add a + in front of the names of your attributes to convert it into a number in order to do math operations on it, but only in square brackets area. In order to do math operations, all attributes involved in an operation MUST have a + leading in front of the name with no spaces, like in my example ([[ ... +Black/+Total]]) All of this is done via the d3 library, as indicated toward the start with the "d3.csvParse command line. If run correctly, you'll see "csv_data = drop down arrow Array(n), n representing the number of polygons you have. In my case, there were 896 census tracts in Iowa in 2020. Each following array are individual polygons. In addition, we create and normalize new variables when we use a formula on existing columns/attributes within my .csv file. In my case, I wanted a percentage of the black population per census tract in Iowa in 2020, so I divided the black population by the total population. We never want to map raw counts, so this step is crucial for anything to make sense following this step and in assignments 2 and 3; otherwise, larger census tracts with higher populations will always have higher raw counts, and wouldn’t meaningfully tell us anything about the population make up. This is how we compute a percentage. We can also divide population by area to calculate density, but that’s not what I decided to do with my data. Normalization adjusts the values so that all polygons are compared on the same scale, an even playing field, so to speak. We need our new attribute to reflect accurate rates in the upcoming web map, so this step is essential to get right in this assignment, which I believe I have.

Comment 6 [ALL NEW INSTRUCTIONS GOING FORWARD UNLESS OTHERWISE STATED]: In this cell, we load the excel file I uploaded in assignment 1 called population.csv. This step converts the columns to the necessary data type using the command “d3.autoType,” which then restructures the dataset into a simplified format for the sake of quick classification (and eventually, mapping). After going through the data, each row goes through a process of transforming into objects that contain two fields: GEOID (a unique number that’ll allow the .csv to be joined to the census tracts I downloaded in assignment 3) and pctPopBlack (a variable I derived by dividing +Black and +Total to come up with a numeric percentage for each census tract). Effectively, this step is recalling and restructuring only the necessary information needed for the purpose of classification, and eventually mapping (this includes normalizing the data for consistent use throughout this notebook). The output consists of an array of one JavaScript object for each geographic unit (census tract), which, as I mentioned before, consists of the GEOID and normalized percentage. With this established structure, we can now easily get things from the D3 library, like histograms and classification functions/schemes.

Comment 7: This cell simply acts as an interactive slider bar with a range of bins determined by Inputs.range([n, m] … ). Because we have the minimum and maximum set as [0, 20], we’re able to set the number of bins to any number included in that area. I have mine set to 10, as a reasonable, mid-way point compromise. viewof allows you to see the slider and its current value, allowing you to see how the data you’re looking at is distributed across the histogram. Of course, everyone's data will be different, but in the case of mine, it is very clearly right skewed, with the tail tapering off to the right having far fewer values than the bin on the far left. This indicates a relatively low percentage of black population across the census tracts of Iowa. Subsequently, I believe natural breaks or quantiles will be the best way to represent my data when mapping it in assignment 3. We've also learned that logarithmic classification can be useful for highly right skewed data, so perhaps I can give that a try as well.

This code takes from the .csv file that I previously loaded in (csv_data) and then extracts specific columns of my choosing, done via the command: Array.from(csv_data.values(), => d[2]), which I believe looks through each object inside of my .csv dataset and then uses a mapping function (d => d[2]), which each correspond to a column that I’m trying to retrieve. 2 in this case is the third value I established in a prior cell, since 0 was the starting value there, 2 ends up acting as the third value here. The third value was the one derived from +Black/+Total to obtain the popPctBlack attribute.

Comment 11: This cell forms an unclassed, continuous color scale utilizing the d3.scaleLinear(…) command, which enables it to map the entire range of the normalized variable along a smooth gradient, not organized into discrete categories. However, this is not visible in the chart, and the following disclaimer explains why: PLEASE NOTE: Unfortunately, due to the massive size of my dataset (896 individual census tracts and therefore, 896 individual values), it is not possible to see the color change throughout my dataset on observable ASIDE FROM the showScaleGrouping command line, which we’ll get to in a bit. As far as I can tell from that, though, my dataset and the commands/functions are working as intended. This problem persists with all of the different classification types, so please be aware of that fact when looking through each of them. I have not found a way to expand the chart to show all 896 values. In order to ensure each work, I changed the colors to be random for each, which makes differentiating between the classification methods a bit easier in the showScaleGrouping section toward the end of the assignment. Moving on, the domain gets established through d3.extent(pctBlack), which makes certain that the scale extends from the min to max values in the dataset. The range is showcased via the color values changing from blue to green, min to max, except again, you cannot actually see the smooth gradient due to the sheer number of values I have in this dataset. Despite all of the blue squares looking like they’re the same shade of blue, they are supposed to be their own unique colors. It would be much easier to see if I had a smaller dataset, but since I’ll be mapping this in assignment 3 anyway, it didn’t really make much sense to go looking for a smaller dataset I’d only use in this one instance. The chart orders the values via their visual relationship from lowest to highest (min to max), which should allow you to see how the color gradient transitions across the distribution of your data.

Comment 13: In this cell, we’re creating a quantile classification scheme using the d3.scaleQuantile() command. This divides up the data into classes that contain an equal number of observations, and by putting pctBlack into the domain, the scale through the d3 library automatically computes the breakpoints that’re needed to divvy up the dataset into three quantiles. Within the range, three colors are assigned to each class (in this case, in the .range ["#f1eef6", "#d7b5d8", "#df65b0", "#dd1c77", "#980043"] section of code). Each quantile is assigned to one color, in order. Again, because my dataset is so large, you can only see the change from the first lightest to second lightest shade of pink/purple, but I assure you, there's darker 3 more classes, each getting darker as you go. You can see the size of the bins in the showScaleGrouping cell, later on in this assignment. Even though there’s an equal number of values in each bin, the width of the bars, which demonstrate the range of values each bin falls into, vary quite a lot. Note later on, in the cell with showScaleGrouping, the pink bin has far less value range than that of the green bin. It can be a bit misleading in certain situations, and I think this may be one of them. I believe quantiles can certainly be useful in cases where you want to highlight the differences in distribution when there’s skewed data, though.

Comment 14: Refer to comment 12 for explanation. Not how quantile has replaced the position of linear in the cell that comment 12 is attached to.

Comment 20: This cell creates a graphic comparing the various classification scales I’ve created (minus unclassed) for pctBlack. Having the results displayed together gives us a more clear understanding of their distribution and which might be most appropriate for mapping. This allows us to see the quantile, natural breaks, manual threshold, Jenks, and equal interval side by side, with their respective colors. We can see what distributions probably don’t work well for mapping, and based on how quantile looks, I’m skeptical as to if it’ll fit in during assignment 3. Still, this also allows us another chance to see the data distribution and where the clusters and outliers occur. In this case, there’s a few points on the right end of the spectrum, acting as another way of visualizing the right skewed tail from the histogram earlier on in the assignment. Classification choice matters, and this graphic demonstrates that. We also have the quantize.copy().nice() function, which adds a new version of the quantize scale that includes a niced domain, which effectively means that d3 slightly expands/manipulates the domain to round numbers that are a bit easier to deal with in boundaries.

Comment 21: This cell has a function called numericSort, and it does what the name implies. It sorts numeric values from smallest to largest. I believe it’s acting as a consistency checker, of sorts. The data we’re working with works best when it’s ordered (especially unclassed), so having this function running helps us ensure that’s always the case. I believe the default is an ascending order, based on the histogram and the showScaleGrouping cell.

Comment 22: After doing some digging online (unless it was mentioned in the assignment videos and I missed it), require("simple-statistics ...") is actually enabling a library which has the ckmeans function in it. Think of this, and the surrounding cells/functions as a backbone structure for the computations completed throughout this assignment. Indeed, both showScaleGrouping and chart functions also require this imported tool library in order to property visualize the dataset I'm using in this notebook.

Comment 18: As a reminder, because my dataset is so large with so many unique values (896), you can't always see the class changes in these chart tabs on observable. In the showScaleGrouping cell below, you'll finally be able to see the true distributions of my data across all of the classifications used so far, but you'll have to wait until then. If you use a smaller dataset, you will be able to see changes in your data in the chart cells.

Comment 16: This cell enables the natural breaks computed in the previous cell into a classification scale that can be used via the function d3.scaleThreshold(…) The domain is established by linking to the list of break points determined by the ckmeans command, I believe. The .range function assigns specific colors for each class. For this, and all other classification schemes, I simply went to https://colorbrewer2.org and used the sequential color schemes that fit my needs. Be sure to select the number of data classes to the number you want in order to get a series of colors that change hue, brightness, or saturation in the way you desire. In the 5-class box, select HEX for the most user-friendly set of color codes to copy and paste into your project. Remember, the number of these you paste in must match with the number of classes you want to include on the observable notebook.

Comment 12: This cell contains code that will require nothing more than a word or two replaced between each classification type, so there's no need to go over it again. In this case, linear = unclassed according to the documentation and what we've learned in lecture, so that will subsequently be replaced by other, more familiar classification terms as we continue through this assignment.

Comment 8: This cell creates the histogram that was mentioned in the previous comment. The histogram, as per the name on the bottom right, is a visualization of the variable I normalized in Assignment 1, pctPopBlack. Plot.binX(…) creates bins along the x-axis, conforming to the number of bins chosen by the slider in the previous cell. Plot.rectY(…), as you might have guessed, creates the height for the corresponding bin values or observations in the designated interval (which obviously change as you change the number of bins on the slider widget in the previous cell).

Comment 10 [REUSED FROM ASSIGNMENT 1]: This cell maps the IDs to variables and/or attribute arrays, as indicated toward the end of the last cell comment. I’m not mapping this cartographically just yet, but it’s the first step toward doing that. Effectively, this is how you join the attributes to the polygons. You may click the arrows to reveal the array of nested information regarding the IDs and what numbers are associated with them (raw counts first, for black populations in each tract, then black percentage in each tract immediately after that). I even gave it a title for the upcoming attributes, just to keep my ducks in a row so to speak. Let me show you: data = Object.assign(new Map(csv_data), {title: ["Black Pop", "Black Pct"]}) is a fairly intuitive command, you can see how the new map is going to use the csv_data that we imported and normalized in previous steps.

Comment 15: The purpose of this cell is to compute the natural breaks (Jenks) classification via ckmeans(pctBlack, 5), which does the clustering to locate groups of values that help lower in-class variance while maximizing the differences between the classes. The 5 in (pctBlack, 5) specifies the number of classes. I chose 5 classes for every single classification scheme because that’s generally a fair number to use. It’s right in the middle of the road, not too few, not too many. 3 or 7 also would’ve been okay, but with 3, you lose some detail, and with 7, there might be a bit too much (at least when comparing to other classification histograms in showScaleGrouping, as you’ll see a bit later). Honestly, you don’t need to know a ton about this line of code, it’ll almost always be identical any time you want to make a Jenks classification scheme. Just know that using my dataset, pctBlack, this code then finds statistically meaningful places to distribute the data into the bins. This method can be useful for data that have uneven clustering, spacing, outliers, etc., because the algorithm/method tends to conform to the data you input instead of applying some arbitrary division, like quantile. Use on a case-by-case basis, as not all data may be wise to use with Jenks.

Comment 16: This cell enables the natural breaks computed in the previous cell into a classification scale that can be used via the function d3.scaleThreshold(…) The domain is established by linking to the list of break points determined by the ckmeans command, I believe. The .range function assigns specific colors for each class. For this, and all other classification schemes, I simply went to https://colorbrewer2.org and used the sequential color schemes that fit my needs. Be sure to select the number of data classes to the number you want in order to get a series of colors that change hue, brightness, or saturation in the way you desire. In the 5-class box, select HEX for the most user-friendly set of color codes to copy and paste into your project. Remember, the number of these you paste in must match with the number of classes you want to include on the observable notebook.

Comment 17: This cell is responsible for making a quantize classification, or the term we’re more used to in ArcGIS Pro, equal interval classification, using d3.scaleQuantize(…). As the name suggests, the domain is set to the min and max values of pctBlack, and then using the information, determines where splits are to be located, creating five evenly sized intervals, each bin represented by one color. Equal interval classification makes it so that each of the classes contains the same width of values along the range of data in your dataset. Again, like the other classes, the classes are based on the range and the color specified in it, which I once again got from ColorBrewer, moving from light red to an orange-ish maroon. This particular classification method is useful for showing consistent interval widths, and I believe can be used when wanting to show comparisons of magnitude across your dataset in question (not to compare magnitude across multiple datasets of different values, you’d have to find some way to normalize before doing that).

Comment 19: This cell creates a 5-class manual threshold scale utilizing d3.scaleThreshold(…), in that class boundaries come from the natural breaks that I computed in that part of the assignment. Initially, I was going to go with the [0.05, 0.1, 0.2, …] format that was put into the original forked assignment before I changed anything, but after rewatching the assignment videos again, I just decided to do it this way, similar to what’s done in the instructions. So, the naturalbreaks5 array creates 4 breakpoints which divide up the data into 5 statistically meaningful groups, exactly the same as the Jenks classification did. The major difference here is that Threshold means these are MANUAL breaks. You can change the range to be whatever you want in the .domain and .range sections. Do what makes sense to you, but I figured since natural breaks already did the job for me and since it was demonstrated how you can put that information through to manual breaks, I decided to go off of that for my own data. I made this chart on a blue color scheme as opposed to the green I used for Jenks, just to help differentiate them a bit on the showScaleGrouping cell.