31:T1daf,# Using Bluebikes data in R Below is a quick guide to loading the monthly Bluebikes system ridership data into R and doing some basic analysis. R is a statistical analysis software tool. If you've never used it before, you can [download it here](https://www.r-project.org/) and learn the basics [with the swirl package](https://swirlstats.com/students.html). You might also want to download [RStudio](https://posit.co/download/rstudio-desktop/), an integrated development environment for R. You can download the Bluebikes data for each month going back to January 2015 [from the Bluebikes website](https://s3.amazonaws.com/hubway-data/index.html). For this example, we're downloading the data for January and February in 2023 and 2024. ## Working with the data First, we'll unzip our downloaded files and load the resulting CSV files into R: jan23 <- read.csv('202301-bluebikes-tripdata.csv') feb23 <- read.csv('202302-bluebikes-tripdata.csv') jan24 <- read.csv('202401-bluebikes-tripdata.csv') feb24 <- read.csv('202402-bluebikes-tripdata.csv') The data format varies slightly from year to year, so let's check the structure of each dataframe we just loaded (here I'm using `lapply` to iterate the same `colnames` function – which will give us the column names – for each dataframe in the list): lapply(list(jan23,feb23,jan24,feb24), colnames) I can see that both of the 2023 datasets have the same structure, with 14 columns, but the 2024 dataset has different column names and only 13 columns. To make apples-to-apples comparisons across both years, we'll need to address those differences later, but for now, I'm going to `rbind` the two dataframes from each year into a single dataframe, then combine them into a list, and then use the `nrow` function to get the number of rows for each year: r23<-rbind(jan23,feb23) r24<-rbind(jan24,feb24) r <- list(r23,r24) names(r) <- c('winter23','winter24') lapply(r, nrow) That last `nrow()` command should return 293,315 rows for 2023 and 398,646 rows for 2024, with each row representing a unique Bluebike trip. ### Formatting variables For an accurate analysis of data, it's important to take a look at the structure of the data you've downloaded and get the variables into a consistent format. Let's look at the variables for each trip in our winter 2023 data, using the `str()` function: str(r$winter23) This will show you the variables for each trip – the time and date the bike left the dock, when it was returned, the starting and ending station names and their longitude/latitude coordinates, and a unique bike ID number. Now, let's do the same for the 2024 dataframe: str(r$winter24) Note that the 2024 dataset has dropped some variables, like the `tripduration` column (which was redundant, since we already knew the start and end times for each ride), and gained another variable, the `rideable_type`, which indicates whether it was an electric bike or a regular bike. Some of the variable names have also changed, even though they still represent the same thing. Since we want to compare 2023 with 2024 ridership, let's get these dataframes into a more consistent format (for my purposes right now, I'm not concerned that they don't have exactly the same dimensions, but I do want to be able to compare the date and location columns). colnames(r$winter24) <- c("ride_id","rideable_type","starttime","endtime" , "start.station.name", "start.station.id", "end.station.name","end.station.id", "start_lat", "start_lng", "end_lat", "end_lng" ,"usertype" ) colnames(r$winter23) <- c("tripduration","starttime","endtime", "start.station.id", "start.station.name" , "start_lat", "start_lng", "end.station.id","end.station.name", "end_lat", "end_lng", "bikeid", "usertype","zip" ) ### Formatting dates Note also that the `starttime` and `endtime` columns default to factors, with the format YYYY-MM-DD HH:MM:SS.XXXX. I'd like to convert these to datetime objects in R, so I'm going to split them up with a substring command, then convert the left side to dates. I'll also take a substring of the time to create a new hour-of-the-day factor, and do this for each of the dataframes I'm working with: cleandates <- function(x) { x$date <- as.Date(substr(x$starttime,0,10), format = "%Y-%m-%d") x$hour <- as.factor(substr(x$starttime, 12, 13)) return(x) } r <- lapply(r, cleandates) Let's now check to see if it worked: lapply(r, summary) Take note of the new `date` and `hour` columns for each year; in both cases, you should see the most common hours for bike rides are 17 and 16 (corresponding to check-out times between 4 and 6 p.m.) and the `date` factors should range from Jan. 1 to the end of February, with a median date that's just past the beginning of February. ### A warning about the latitude/longitude numbers Let's take a look at the subset of our winter 2024 bike trips that used electric bikes, started at the Chinatown T station (station ID D32019) and ended at Washington and Rutland (station ID 32006): subset(r$winter24, end.station.id=="C32006" & start.station.id == "D32019" & rideable_type == "electric_bike") This gives us 22 electric bike trips between these two specific stations in January 2024. But note how, even though the starting and ending stations are identical for all these rows, the `start_lat`, `end_lat`, `start_lng` and `end_lng` vary slightly. In this subset, the `start_lat` ranges from 42.35189 to 42.3526 (the distance between these two latitudes is about 264 feet, and some of the variation could be attributed to people parking bikes at different ends of Bluebike dock, and some is probably due to GPS signal variability). In other words: the latitude and longitude data are imprecise. But that's OK, because every Bluebikes trip ends at a station, and we know where the stations are located. The [Bluebikes data website](https://bluebikes.com/system-data) also includes a downloadable index of stations that includes the station ID, number of docks, municipality, and latitude/longitude of every station in the system. If you need to use latitude/longitude data – to plot trips [on a map](https://observablehq.com/@vigorousnorth/electric-bluebikes-by-the-numbers#cell-464), for instance – I recommend ignoring the data from individual trips, and instead use the `merge()` function in R to cross-reference station IDs from individual trips with the longitudes/latitudes from the station data (I'll share code showing how to do that here at a future date). ## Analyzing the data By now, you'll have a good basic framework for analyzing Bluebikes trips over time: you can compare the volume of bike trips across different dates, you can subset data for specific stations or bike types, and you can do basic spatial analysis. With some effort, then, you should be able to answer questions like: 1. Does the average length of a Bluebike trip increase in the warmer summer months? 2. How much more distance does the typical electric Bluebike trip cover, compared to the classic bike trips? 3. What are the most common destinations for Bluebikes trips that originate from the Grove Hall Library station at 41 Geneva Avenue in Boston? 4. Which stations saw the biggest gains in trip activity between 2023 and 2024? ...and many more. For my part, I'm also continuing to probe these data in new ways, and as I do, I'll plan to update this page with additional R code snippets and examples from my projects. 32:[] 2f:{"title":"Using Bluebikes data in R","cells":[{"id":0,"value":"$31","pinned":false,"hidden":false,"mode":"md"}],"files":"$W32","stdlib":"1","titling":"auto"}