A 100-year-old algorithm is beating modern SOTA anomaly detection
The core issue lies in the evaluation process. Most researchers rely heavily on the TSB-AD-M benchmark to claim "State-of-the-Art" (SOTA) status. But after running some tests on these datasets, the results are honestly embarrassing for the modern deep learning crowd: a century-old statistical method, Statistical Process Control (SPC), can outperform or even match these complex neural architectures.
In several test cases, particularly with ECG traces, simple SPC yields perfect results. When a method developed a hundred years ago can achieve flawless detection on a benchmark that supposedly requires cutting-edge AI, it suggests the benchmark itself is fundamentally broken. It’s too trivial.
The Illusion of Progress
If we look closely at datasets like the "TAO" traces, the anomalies are so blatant that they don't require a complex LLM agent or a deep transformer-based architecture to identify. They are statistical outliers that a simple control chart can catch instantly. This creates a massive gap between academic "progress" and real-world utility.
This leads to a serious problem in the AI workflow for time series:
- Benchmark Saturation: We are chasing incremental gains on datasets that are essentially "solved" by basic statistics.
- Complexity Bias: There is a tendency to favor complex, parameter-heavy models because they look more impressive in papers, even if they are overkill for the task.
- Lack of Real-World Robustness: Just because a model performs well on a TSB-AD benchmark doesn't mean it will survive the noise and non-stationarity of actual industrial sensor data.
Moving Toward Harder Problems
The community needs a massive reality check. To move past this "illusionary progress," we need to shift our focus toward much more challenging, non-trivial TSAD problems. We shouldn't be celebrating a 1% improvement on a trivial ECG dataset. Instead, we should be looking at high-stakes, complex environments like:
- Smart Manufacturing: Where sensor noise and shifting baselines are constant.
- Fuel Cell Monitoring: Involving complex chemical and thermal interactions.
- Biological Systems: Such as specialized tuna tracking or complex physiological signals.
- Sled Dog Kinematics: Highly dynamic and unpredictable movement patterns.
If our "advanced" models can't provide a significant advantage over a simple statistical process control chart, then we aren't actually advancing the field—we're just making the math more expensive. It is time for the TSAD community to stop relying on easy wins and start building benchmarks that actually demand the intelligence we claim to be creating.