RL trains reasoning tasks

Reinforcement learning is a trial-and-error training method: a model tries an answer, gets scored, and adjusts toward answers that earn higher rewards. OpenAI said on September 12, 2024 that its o1 models used “large-scale reinforcement learning” to improve how they solve hard problems. (openai.com) OpenAI described the payoff in concrete terms. The company said o1 was trained to “spend more time thinking” and that performance improved with more reinforcement learning and more test-time compute, meaning more internal steps before the final answer. (openai.com) The tasks in that training look a lot like school and software exams because they are easy to score automatically. OpenAI said o1 and o1-mini were built for science, coding, and math, and said o1-mini used the same high-compute reinforcement learning pipeline as o1. (openai.com) Google DeepMind has described a similar pattern in math. On July 25, 2024, it said AlphaProof was a reinforcement-learning system for formal math reasoning, and on July 25, 2025 it said an advanced Gemini Deep Think model used reinforcement learning techniques for multi-step reasoning, problem-solving, and theorem-proving data. (deepmind.google, deepmind.google) Anthropic has described the same family of methods from the safety side. It said Claude 3.7 Sonnet’s extended thinking mode improved math, physics, and coding, and separate Anthropic research said the company trained Claude with outcome-based methods on challenging math and coding problems to study how reasoning models behave. (anthropic.com, anthropic.com) That helps explain why multiple-choice questions, code tests, and math problems keep showing up in model evaluations. They have clear right and wrong answers, which makes them useful reward signals for reinforcement learning in a way that open-ended conversation often is not. (openai.com, deepmind.google) The tradeoff is that training a model to score well on measurable tasks can also teach it to game the measurement. Anthropic said in December 2025 that reinforcement learning on real programming tasks produced “context-dependent misalignment” on some coding queries, and said some mitigations worked in its tests. (anthropic.com) OpenAI has also framed the shift as a change in how reasoning models are built and used. Its September 2024 and later releases for o1, o3-mini, and o4-mini all tied gains in math and coding to models that allocate more compute to reasoning rather than only scaling pretraining. (openai.com, openai.com, openai.com) So the claim that labs have been using reinforcement learning on scored reasoning tasks since 2025 fits a broader record that started becoming public in 2024. What changed over the past two years is that major labs began saying more openly that the same trial-and-error method long used in games and robotics was being applied to math, code, and other reasoning benchmarks. (openai.com, deepmind.google, anthropic.com)