Time-Series Anomalies and Change Points

A monitoring system that flags every holiday surge is comparing a holiday against an ordinary weekday. Official statistical agencies settled this long before anomaly detection had a name. They made the calendar a component of the expected value instead of a source of surprise.

Eurostat's seasonal adjustment guidelines, 2024 edition, define the effect exactly: “Calendar effects arise from annual differences in the number of working or trading days in a month or a quarter, or the dates or days of public holidays.” The guidelines require those effects to be filtered out before short-term movements are interpreted. They add that “a normal working day adjustment implicitly adjusts for moving national holiday effects”. Easter still needs separate treatment, because it moves between months.

The US Census Bureau builds the same reasoning into software. Its X-13ARIMA-SEATS documentation defines the category: “Holidays whose effects move between or among months (or quarters) from year to year are called moving holidays”. The program ships with regressors for Easter, Labor Day and Thanksgiving built in.

Temporal anomalies require context across time, season, operating state and recent history. A large residual is meaningful only when the expected trajectory was credible. In production statistics the credible trajectory already has the calendar inside it.

The order and duration of observations can carry the anomaly signal. So can the defects in the labels used to score it. Wu and Keogh examined the standard anomaly-detection benchmarks built by Yahoo, Numenta, NASA and Pei's lab, and found that most individual series carry one or more of four specific defects. Their introduction names them: “These flaws are triviality, unrealistic anomaly density, mislabeled ground truth and run-to-failure bias.” They argue that the ideal number of anomalies in a test series is one. An independent group summarised the same four defects and answered them with TSB-UAD, a replacement benchmark of 13,766 series.

The detection unit determines the model, the labels, and the alert-matching rule. A point anomaly is one timestamp with an unusual value or residual. A contextual anomaly is unusual for its season, state, or entity history. A subsequence anomaly is a short shape or motif that differs from expected temporal behavior. A collective anomaly is a set of individually plausible points forming an unusual episode. A change point is a shift in the generating regime — in mean, variance, frequency, dependency, or parameters.

The last of those runs operationally at national scale. NOAA's National Centers for Environmental Information applies change-point detection to the Global Historical Climatology Network monthly dataset: about 26,000 temperature stations in version 4, against 7,200 in v2 and v3. Its Pairwise Homogenization Algorithm “systematically evaluates each time series of monthly average surface air temperature to identify cases with abrupt shifts”, and the agency records that “shifts in monthly temperature series are detected through automated pairwise comparisons of the station series using the algorithm described in Menne and Williams (2009)”. Three properties are worth copying. The regime changes are physical and named: station moves, instrument swaps, changes in the time of observation. They are detected against correlated neighbouring stations, not against a global baseline. And the correction is published for others to use. NASA's Goddard Institute for Space Studies states that “Beginning in June 2019, with the release of GISTEMP v4, we are using the adjusted monthly mean data of NOAA/NCEI's Global Historical Climatology Network (GHCN) version 4”. The change point is not an alert here. It is a correction another agency can build on.

One approach scores deviations around a baseline. Another searches for persistent regime transitions. A third conditions the whole notion of normal on operating state.

A residual anomaly detector compares observed values with expected values or distributions. It supports point and interval scores, and it depends on forecast quality. It can react to temporary spikes, and it needs seasonal context.

A change-point method partitions time where statistical behavior appears to shift. It targets persistent transitions, and it needs minimum-segment and penalty choices. It can detect mean or distribution changes, but it may locate the change only after a delay.

A state-aware model conditions normal behavior on operating mode or external context. It reduces false alerts across regimes, and it requires reliable state information. It can hide novel state transitions, and it needs a rule for unknown states.

The three are not ranked. They answer different questions.

The protocol should reflect how alerts become incidents and responses. Define the event ontology: point, interval, subsequence, collective episode, or regime change. Build a contextual baseline covering season, calendar, operating state, entity history and exposure. Preserve time order, with chronological training, validation and test periods and realistic label delay. Match alerts to events with tolerance windows, merge rules, minimum duration and duplicate handling. Measure operations as event recall, precision, time to detect, alert burden and missed-event severity. Then monitor baseline drift, seasonality, missingness and state coverage after deployment.

Steps 2 and 3 have a working national implementation. A Farrington-type seasonal baseline runs every week over a database of more than 3,300 distinct organisms. Noufaily and colleagues revised it, stating their objective as being “to improve the performance of the England and Wales large scale multiple statistical surveillance system for infectious disease outbreaks with a view to reducing the number of false reports, while retaining good power to detect genuine outbreaks”. The baseline is built from the same calendar position in earlier years: the original method fits its GLM to “w timepoints centred around the timepoint located 1, 2, . . . , b years before t0”. The revision then deliberately excludes the most recent 26 weeks before the monitored week. Salmon and colleagues at the Robert Koch Institute, who reimplemented the method in R, record the reason: “it is better to exclude the last 26 weeks before t0 from the baseline in order to avoid reducing sensitivity when an outbreak has started recently”. Their implementation notes that “the newly implemented function farringtonFlexible supports the use of this original method as well as of the improved method built on suggestions made by Noufaily et al. (2012) for improving the specificity without reducing the sensitivity”. An outbreak already under way must not be allowed to inflate its own expected value.

Step 5 is where the cost of skipping this shows. The Joint Commission's Sentinel Event Alert on medical device alarm safety in hospitals, dated 8 April 2013, states that “It is estimated that between 85 and 99 percent of alarm signals do not require clinical intervention”. Its Sentinel Event database held 98 alarm-related events between January 2009 and June 2012, and the alert records the outcome: “Of the 98 reported events, 80 resulted in death, 13 in permanent loss of function, and five in unexpected additional care or extended stay.” The AHRQ Patient Safety Network reaches the same range from an independent direction: “Research has shown that 80%–99% of ECG monitor alarms are false or clinically insignificant.” Alert burden and missed-event severity are not soft metrics reported after the F1 score. They are the quantities the system is judged on.

The detector needs somebody to say what a normal trajectory is, and what counts as an incident. Without both, scores cannot be evaluated against the workflow they are meant to support. Eurostat and the Census Bureau answer the first question with named calendar regressors. The England and Wales surveillance system answers the second with a baseline that refuses to look at the last 26 weeks.

Separate short deviations from persistent changes, and contextualize every alert. A strong system tells operators when evidence changed, how long it persisted, and which baseline was violated. A weak evaluation protocol will tell you a random score is state of the art.