Approaches for working with data that changes over time

Mike Freeman

Ian Johnson

Observable

What’s different? Analyzing Time-Series Forecast Performance

What caused this? An analysis adventure

Discovering Date Patterns

Just because it’s Time, it doesn’t have to be a Line

Correlation over time

Analyzing Time Series Data Interactive Workshop

EIA Electricity Data

NOAA Hourly Weather Data

IEEE VIS 2021 Demo

Three Simple, Flexible Tools: Color, Faceting, and Order

Analyzing Time Series Data

Interesting that the first "high" point is colored wrongly, since the moving average is only defined after. You could extrapolate here by repeating the first value, or use a symmetric / centered window.

Good points, and good suggestions for how to address. Thanks! 

I think with symmetric you'd run into the same problem, but on the right-hand-side for 7/31. Assuming that we're more interested in recent history than older history (esp if we imagine this as something that continues to update w/ new data), I'd rather the issue affect the oldest data rather than the newest. 

Your idea to apply the first moving average value to classify the first 6 data points is a good option. Essentially assuming that "ave of first 7 points is applicable to classify those points as 'high' or 'normal', which seems like a good assumption. 

Or, could calculate the moving average as the average of what is available (ie - average over only 2,3,4,5,6 days respectively for the first 6 data points). Upside: if trend was changing fast, it might capture that. Downside is this puts more weight on the earliest points and still leaves a question of what to do with the first point. Therefore, I think the repeat option above is preferable: simpler and likely less error-prone (unless trend changing fast). 

Alternatively could show those first 6 left-hand-side values in grey, instead of orange/blue, to designate a "undefined" sort of status and acknowledging that there isn't enough data for a 7-day moving average yet. 

I think in this case, where the data is static and I can see that it would work well, I would implement the "repeat" option that you suggested. If this was for a dashboard / diagnostic chart for somebody who works at the energy company or for myself, somebody who is going to look at the chart over and over with data for different customers / time periods and wants to prioritizing avoid misclassification, I'd lean towards the grey option. But, if it was a chart that the energy company was going to send to every customer as part of a report on their energy consumption, then I feel like there is an argument to be made for either: repeat (for simplicity) or grey (for avoiding misclassification). 

What do you think?

Alternatively, we could use the array-blur strategy instead of the rolling-window; array-blur is a fast approximation of a gaussian kernel; it does not lose any data point, and does not "vanish" at the ends of the series. Here's an (unpublished) notebook where I've applied this strategy https://observablehq.com/d/d1a59bb628f72118

data.csv

justbecause1 (3).png

justbecause2 (3).png

justbecause3 (5).png

3-points.png

1-line.png

2-bar.png

1-line@1.png