ASI-Evolve discovers 105 new architectures
- Researchers behind ASI-Evolve posted a paper and code showing an agentic system discovered 105 linear-attention architectures that beat the human-designed DeltaNet baseline. - The paper says the search ran 1,773 rounds, generated 1,350 candidates, and its best architecture improved benchmark scores by 0.97 points. - The release adds evidence for closed-loop AI research across models, data, and training methods. (arxiv.org)
Neural architecture search is the work of finding a better model blueprint — the wiring diagram that decides how an artificial intelligence system processes information. A new paper says ASI-Evolve automated that search and found 105 linear-attention architectures that outperform the human-designed DeltaNet baseline. (arxiv.org) The paper, posted to arXiv on March 31, 2026, describes ASI-Evolve as a closed-loop research system that cycles through four steps: learn, design, experiment, and analyze. Its codebase is public on GitHub under the GAIR-NLP organization. (arxiv.org) (github.com) In the architecture experiment, the system ran 1,773 exploration rounds and generated 1,350 candidate designs. The authors say 105 of those candidates beat prior state of the art for linear attention, and the top result scored 0.97 points above DeltaNet. (arxiv.org) Linear attention is a family of models built to handle long sequences with lower compute costs than standard transformer attention. The paper compares ASI-Evolve’s best gain with recent hand-designed progress and says the 0.97-point improvement was about three times larger. (arxiv.org) The system is not a single model searching blindly. The authors describe three agents — a Researcher, an Engineer, and an Analyzer — plus two memory stores that keep past papers, prior ideas, code, results, and lessons from earlier failed runs. (arxiv.org) (github.com) That same loop was also tested on two other parts of the artificial intelligence stack: pretraining data curation and reinforcement learning algorithm design. The paper reports an average 3.96-point improvement for data curation, more than 18 points on MMLU, and up to 12.5 points over GRPO on AMC32 in reinforcement learning. (arxiv.org) The authors also include smaller transfer experiments outside core model-building. In biomedicine, they report a 6.94 AUROC gain on a drug-target interaction task aimed at cold-start generalization, where the system must handle molecules or proteins it has not effectively seen before. (arxiv.org) The architecture story has a wrinkle: an earlier project from overlapping researchers, ASI-Arch, reported 106 novel linear-attention architectures after 1,773 autonomous experiments and roughly 20,000 GPU hours. The newer ASI-Evolve paper presents the architecture result as one piece of a broader framework spanning models, data, and learning algorithms. (arxiv.org 1) (arxiv.org 2) The paper’s claims now turn on replication. The code is public, but the reported gains come from the authors’ own evaluation pipeline, and the strongest test will be whether outside researchers can rerun the search and recover the same edge over DeltaNet. (github.com) (arxiv.org)
