LangChain Releases Benchmark for Coding Agents

The new benchmark centers on a concept called "Skills," which are curated instructions and scripts that an AI agent can dynamically load when needed for a specific task. This approach of "progressive disclosure" prevents performance degradation that often occurs when an agent is overloaded with too many tools at once. The initial set of 11 skills focuses on the LangChain ecosystem, with categories for LangChain core, LangGraph, and Deep Agents. On LangChain's evaluation set, providing an agent with these skills boosted its task completion rate from as low as 9% to 82%. For a specific test using Anthropic's Claude Code (Sonnet 4.6), the success rate on LangChain-related tasks jumped from 25% to 95%. The evaluation pipeline ensures reproducibility by running the agents in isolated Docker containers and uses the LangSmith observability platform to capture every action for review. This focus on improving the tooling and environment around a model, rather than modifying the model itself, is a practice LangChain refers to as "harness engineering." It's an extension of context engineering, aiming to create a structured environment where an AI can effectively loop, call tools, and execute long-running tasks. This includes optimizing system prompts, tool selection, and middleware hooks that intercept and guide the agent. The effectiveness of this approach was previously demonstrated on the Terminal Bench 2.0, a standard for evaluating AI agents on complex command-line tasks. By only refining the harness, LangChain's `deepagents-cli` agent saw its score improve by 13.7 points, vaulting it from outside the top 30 to a top 5 position without any changes to the underlying GPT-5.2-Codex model. A key insight from this "harness engineering" was solving the "self-verification problem," where agents would write code and approve it without actual testing. The fix involved implementing a structured "plan, build, verify, fix" loop and adding a `PreCompletionChecklistMiddleware` that forces the agent to verify its work against the original specifications before finishing. LangChain's benchmark found that open-ended tasks like "create a research agent" were difficult to grade consistently. As a result, the team shifted to more constrained tasks, such as fixing buggy code, where correctness could be validated against a predefined set of tests. The full open-source benchmarking repository is available on GitHub for developers to use.