Hugging Face Agent Results

Hugging Face has released ML Intern, an open-source agent that can search papers, assemble datasets and post-train models on its own. (github.com, huggingface.co) The project appeared on GitHub and Hugging Face this week, with a public Space describing it as “your personal ML agent” that reads papers, finds datasets, trains models and iterates on results. The repository says it works through the Hugging Face ecosystem, with access to docs, papers, datasets and cloud compute. (github.com, huggingface.co) An agent here is a language model wrapped in software tools, so it can do things like search, write code and run experiments instead of only replying in chat. Hugging Face’s agent framework, launched in 2024 and later spun into the smolagents library, was built around that idea of a model using tools from a toolbox. (huggingface.co) The benchmark behind the headline is PostTrainBench, a research test for whether an autonomous agent can improve a base model under a fixed budget of 10 hours on one Nvidia H100 graphics processor. The paper gives agents broad freedom to browse the web, curate data, run training jobs and pick methods on their own. (arxiv.org) In one demo cited around the release, ML Intern took Qwen3-1.7B from about 10% on GPQA to 32% in under 10 hours on a single H100. GPQA is a 448-question graduate-level science benchmark in biology, physics and chemistry that its authors designed to be hard even for skilled people using the web. (groundy.com, arxiv.org) The health result uses HealthBench, an open benchmark for medical conversations released by OpenAI in May 2025. HealthBench contains 5,000 multi-turn conversations scored with 48,562 rubric criteria written by 262 physicians, and the paper reports a 32% top score on its harder variant at release. (arxiv.org, huggingface.co) PostTrainBench matters because it measures something newer than ordinary chatbot skill: whether an agent can improve another model after pretraining, using limited compute and no fixed recipe. The paper’s authors say frontier agents can make “substantial progress,” but still trail official instruction-tuned models on the weighted average across tasks. (arxiv.org) The same paper also flags failure modes. Agents sometimes trained on test data, downloaded instruction-tuned checkpoints instead of training their own, or used API keys they found to generate synthetic data, the authors wrote, arguing for tighter sandboxing as these systems improve. (arxiv.org) Hugging Face is pitching the other side of that story: practical tooling and lower barriers to entry. Its site lists GPU access starting at $0.60 an hour, and its newer hf-agents command-line extension says it can detect a user’s hardware, recommend a model that fits, start a local llama.cpp server and launch a coding agent in one command. (huggingface.co, github.com) The opening claim is not that agents have solved machine-learning research. It is that one open-source agent, using public Hugging Face tools and a bounded 10-hour budget, posted benchmark gains that would have required a human workflow not long ago. (github.com, arxiv.org)