METR finds frontier AI task-completion horizons doubling about every seven months

METR’s original study combined 170 software and research tasks and modelled agent success against the amount of time an expert human would need to complete each task. The resulting 50% horizon provided a common human-time scale for comparing model-agent systems across generations.

The peer-reviewed NeurIPS version retained the long-run trend and reported an approximately 110-minute horizon for o3. METR’s TH1.1 update expanded the suite to 228 tasks; its stitched long-run estimate was about 196.5 days per doubling, close to the original 195.8-day estimate. Faster post-2023 and post-2024 slopes were more sensitive to task composition, model selection, and modelling assumptions.

A time horizon is a human-time difficulty measure, not the amount of wall-clock time an agent can work without supervision.

The benchmark mostly uses self-contained, automatically gradable software and research tasks. Results depend on the model, scaffold, tools, prompts, and resource limits. Human baselines are also noisy; in TH1.1, only five of the 31 tasks estimated at eight hours or more had measured human baselines. BRIDGE recovered a similar exponential trend with a different statistical model, but its calibration still partly uses METR’s human-time annotations.