International
The Morning Dev

Only the news worth knowing

‹
 

Research2 sources

METR updates its time horizon plot to add judgment-heavy tasks, finding some AI capabilities still grow exponentially but models lag at designing AI evaluations (Grace Huckins/MIT Technology Review)

Techmeme ·

METR updates its time horizon plot to add judgment-heavy tasks, finding some AI capabilities still grow exponentially but models lag at designing AI evaluations (Grace Huckins/MIT Technology Review)

The Brief · written from 2 sources

  • METR updated its AI time horizon plot to include judgment-heavy tasks.
  • Some AI capabilities continue to improve on an exponential trajectory.
  • Models are weaker at judgment-heavy tasks than at coding, including designing AI evaluations.

Why it matters

The update suggests AI capability gains are uneven: coding-style tasks scale faster than judgment-heavy work, so developers should not assume benchmark gains on coding transfer to tasks requiring judgment.

Who’s affected

  • Developers relying on AI time horizon benchmarks
  • Teams building or running AI evaluations

Grace Huckins / MIT Technology Review : METR updates its time horizon plot to add judgment-heavy tasks, finding some AI capabilities still grow exponentially but models lag at designing AI evaluations — A new report from the AI evaluation nonprofit METR finds that some AI…

Timeline

  1. METR published an update to its time horizon plot adding judgment-heavy tasks

  2. Artificial intelligence – MIT Technology Review

    The most misunderstood graph in AI just got a major update

  3. Techmeme

    METR updates its time horizon plot to add judgment-heavy tasks, finding some AI capabilities still grow exponentially but models lag at designing AI evaluations (Grace Huckins/MIT Technology Review)

Full coverage · 2 sources

How do you feel about this?

0 Comments

    METR updates its time horizon plot to add judgment-heavy tasks, finding some AI capabilities still grow exponentially but models lag at designing AI evaluations (Grace Huckins/MIT Technology Review) – The Morning Dev