Research
The most misunderstood graph in AI just got a major update
MIT Tech Review · By Grace Huckins ·
The Brief
- An AI evaluation nonprofit, METR, published a report finding that some AI capabilities continue to improve along an exponential trajectory.
- The report indicates that models are not as good at tasks requiring judgment as they are at coding.
- The report is an update to a widely discussed graph of AI capability trends.
Why it matters
The report gives a measured, updated picture of how AI capabilities are changing over time, which helps readers separate real trend data from hype. The gap between coding and judgment-based tasks matters for anyone judging where AI can be relied on.
Who’s affected
- AI researchers and evaluators
- people following trends in AI capabilities
- #ai
- #metr
- #ai evaluation
- #ai capabilities
- #benchmarks
A new report from the AI evaluation nonprofit METR finds that some AI capabilities are continuing to improve along an exponential trajectory—but it also indicates that models aren’t quite as good at tasks that require judgment as they are at coding. The report is an update to the…