fev-bench
Compare forecasting models on realistic tasks, including forecasts that use additional explanatory variables.
Explore the original evidence
Explore the original evaluation, its methods, and reported findings.
Open evalCoverage
100 forecasting tasks span seven domains; 46 include covariates. Authors: Oleksandr Shchur, Abdul Fatir Ansari, Caner Turkmen, Lorenzo Stella, Nick Erickson, Pablo Guerron, Michael Bohlke-Schneider and Yuyang Wang.
Methodology
The fev evaluation library supports task-level comparisons, pairwise win rates and skill scores with bootstrap confidence intervals. Skill scores measure error reduction relative to a fixed baseline; they answer a different question from win rates. Paper revision v5 is dated November 4, 2026; it is not a benchmark edition number. Task selection, error metric, compared models and training-data overlap affect conclusions. Timestral release use does not establish independent replication.