Developers Verify AI
- A UCSD study observed experienced developers treating AI agents as prototyping aids rather than production-ready coders. - Senior devs reportedly plan work, review diffs carefully, and read generated changes before merging code. - The study suggests experienced engineers focus on verification and oversight when integrating agent outputs into production. (x.com)
A UC San Diego–led paper finds experienced software developers treat AI coding agents as prototyping aids and insist on verifying outputs before merging code. (arxiv.org) The study used 13 field observations and a qualitative survey of 99 professional developers to analyze agent use in 2025. (arxiv.org) Authors Ruanqianqian (Lisa) Huang, Avery Reyna, Sorin Lerner, Haijun Xia and Brian Hempel submitted the paper to arXiv on December 16, 2025. (arxiv.org) Survey respondents reported between 3 and 41 years of professional experience; 50 of 99 said they drove architectural requirements themselves. (arxiv.org) Respondents said they modify agent-generated code about half the time (mean 3.0 on a 1–5 scale) and described always reading agent output before steering changes. (arxiv.org) Field observations documented control strategies including detailed prompting, external plan files, and stepwise execution of agent actions rather than one-shot autonomy. (arxiv.org) The paper reports agents were most useful for small, well-defined tasks such as scaffolding, boilerplate and tests, and less useful for complex domain logic or integrating legacy code. (arxiv.org) The authors contrast their findings with social-media claims of fully autonomous agent swarms and call for improved agentic interfaces and clearer guidelines for production use. (arxiv.org)
