Data comes from a combination of internal evaluations and external reports. Aider Polyglot, SimpleBench, and VPCT are each collected from external benchmark leaderboards. GPQA Diamond and FrontierMath are run internally by Epoch. See the FAQs of our Benchmarking Hub for more information on how these evaluations are run.

The selected benchmarks cover only a subset of relevant skills which models have improved at. Notably, we do not yet track benchmarks for domains like robotics or biology, both of which appear to have improved notably over the past year.

Conversely, strong benchmark scores do not guarantee that models will generalize perfectly to real-world scenarios. For example, scoring 100% on GPQA Diamond does not immediately imply that a model can replace a human scientist; models can be overfit to benchmarks, and benchmarks typically do not capture all aspects of real-world work.