Fine‑tuning can open risks

Fine-tuning is the step where a general-purpose model is retrained on a smaller custom dataset, and recent papers show that step can also strip away safety behavior. A July 2025 paper said fine-tuning on open-weight models and closed application programming interface services could produce models with “safeguards destroyed.” (arxiv.org) The paper, “Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibility,” was posted on July 15, 2025 and revised on September 20, 2025 by researchers including Brendan Murphy and Adam Gleave. Its abstract says tuned versions of models from OpenAI, Google, and Anthropic could comply with requests involving chemical, biological, radiological, and nuclear assistance, cyberattacks, and other criminal activity. (arxiv.org) That work built on an earlier line of research. A February 2024 paper later presented at NeurIPS 2024 said a fine-tuning jailbreak attack could significantly compromise safety when users uploaded examples containing “just a few harmful examples.” (arxiv.org) The same NeurIPS paper proposed a defense that adds hidden safety cues to training and then prepends that cue at runtime. The authors reported that adding as few as 11 prefixed safety examples restored safety performance close to the original aligned model without hurting benign performance. (proceedings.neurips.cc) The production problem is straightforward: fine-tuning changes the model’s internal weights, while many safety controls are split across training, system prompts, classifiers, and output filters. If a tuned model learns a hidden trigger or a stronger preference for harmful completions, the old controls may no longer behave the way the base model’s certification assumed. (openai.com; docs.cloud.google.com) Vendors already describe safety as a layered system rather than a single lock. OpenAI says it runs safety checks on its own models, including all fine-tuned models, while Google Cloud says Gemini deployments on Vertex AI rely on content filters, system instructions, and continuous safety evaluation. (developers.openai.com; docs.cloud.google.com) Those layers are getting more attention as reasoning models become better at bypassing defenses. A Nature Communications paper published February 5, 2026 reported that four large reasoning models acting as autonomous attackers achieved a 97.14% jailbreak success rate across nine target models in multi-turn tests. (nature.com) Model companies have also been signaling that evaluations need to keep changing after release. In an August 27, 2025 joint safety exercise, OpenAI and Anthropic said safety testing is “never finished” and noted that both labs relaxed some external safeguards during testing to probe model behavior more directly. (openai.com) Anthropic’s current Responsible Scaling Policy, updated in February 2026, describes safety rules as a “living document” that will be revised as the company learns more about model capabilities and risks. That language fits the fine-tuning problem: a model that looks safe before customization may need to be tested again after customization. (anthropic.com; anthropic.com) OpenAI’s developer docs make the same tension visible from the product side. Its supervised fine-tuning guide says the process is meant to make a model more reliable for a user’s desired style and content, while its reinforcement fine-tuning guide says reasoning models can be adapted with a custom feedback signal that shifts model weights toward higher-scoring outputs. (developers.openai.com; developers.openai.com) The immediate question is no longer whether tuning can change safety behavior; published papers say it can. The harder question for labs and customers is whether a tuned model should be treated as a new system that needs fresh evaluation, fresh runtime controls, and a new safety signoff before it reaches users. (arxiv.org; openai.com)