📅July 17, 2025

A recent study by METR (Model Evaluation & Testing Research) suggests that the productivity boost promised by AI coding tools isn't universal. Researchers conducted controlled experiments with developers working on real-world tasks using tools like Copilot.

The findings were nuanced: AI tools helped novice developers complete simple tasks faster. However, for experienced engineers working with large, legacy codebases, productivity actually declined. Many found themselves spending more time reviewing and correcting AI-suggested code than writing it themselves.

The report warns companies to approach AI adoption with context-awareness. Blanket deployments may not yield the expected productivity dividends, especially in complex environments like enterprise software or large open-source projects.

The study recommends using AI selectively: for boilerplate code, documentation generation, or prototyping, while relying on traditional methods for complex logic.