AI hype backlash video

A new YouTube video is pushing back on the idea that artificial intelligence will soon replace software developers, arguing the claim outruns the evidence. (youtube.com) The uploader says teams should judge coding assistants by concrete delivery measures, including bug-fix turnaround, test coverage, and how quickly new engineers can get productive in a codebase. The video centers on day-to-day software work rather than headline claims about “10x” productivity. (youtube.com) That argument lands as coding tools are getting better at bounded tasks, not whole jobs. OpenAI said in August 2024 that SWE-bench Verified measures whether models can resolve real software issues, and the benchmark covers 500 human-validated problems drawn from GitHub repositories. (openai.com) Anthropic said Claude 3.7 Sonnet reached state-of-the-art results on SWE-bench Verified, while OpenAI said its Codex research preview can write features, fix bugs, and propose pull requests inside a cloud sandbox. Both companies describe those systems as tools for software work, not as a finished substitute for an engineering team. (anthropic.com) (openai.com) The harder question for managers is not whether a model can solve a benchmark, but whether a team ships better software after adopting one. DORA, the DevOps Research and Assessment program published by Google Cloud, says software delivery should be tracked with outcome measures such as deployment frequency, lead time for changes, change failure rate, and failed-deployment recovery time. (dora.dev) DORA’s 2024 report, based on responses from more than 39,000 professionals, said artificial intelligence is having a broad impact on software development but also reported reductions in software delivery performance and uncertain effects on product performance. That is closer to the video’s case for measurement than to blanket claims that developers are becoming obsolete. (research.google) (dora.dev) The strongest pro-artificial-intelligence evidence still tends to come from narrower studies. Microsoft Research said developers using GitHub Copilot finished a JavaScript HTTP server task 55.8% faster in a controlled experiment, and GitHub said a study with Accenture found gains in developer satisfaction and perceived ability to stay in flow. (microsoft.com) (github.blog) Those results do not settle what happens in a large production codebase with legacy systems, reviews, outages, and compliance work. GitHub’s own documentation now includes onboarding plans and setup guides for Copilot, reflecting that teams still need documentation, environment setup, and human review around the tools. (docs.github.com) (github.blog) Research on testing points in the same direction. A 2024 paper on artificial-intelligence assistants for test development found generated tests could match the quality of original tests, but the work focused on test generation, not on replacing the engineers who decide what should be tested and what failures matter. (arxiv.org) The thread running through the backlash video is simple: if a tool really helps, the evidence should show up in fewer escaped bugs, faster fixes, broader test coverage, and shorter ramp-up for new hires. Until those numbers move in production, the case against developers looks more like marketing than measurement. (youtube.com)