Model secrecy climbs
Recent reporting says labs are withholding more model details, the Foundation Model Transparency Index fell from 58 to 40, and the debate over secrecy versus transparency is intensifying after a high-profile safety retreat was described in the New Yorker. (x.com) (x.com) (x.com)
Artificial intelligence labs are disclosing less about how their biggest models are built, tested, and monitored than they did a year earlier. (crfm.stanford.edu) Stanford’s 2025 Foundation Model Transparency Index said the average score for major developers fell to 40 out of 100, down from 58 in 2024. The project assessed 13 companies, including OpenAI, Google, Meta, Anthropic, Amazon, xAI, Mistral, DeepSeek, and Alibaba. (crfm.stanford.edu) The index said companies were least transparent about training data, training compute, and what happens after deployment, including usage and impact. Seven companies submitted their own transparency reports, while the Stanford team compiled reports for six others. (crfm.stanford.edu) A foundation model is the base system underneath tools like chatbots, coding assistants, and image generators. Transparency in this context means basic documentation: what data went in, how much computing power was used, what the model can do, what it fails at, and what safeguards were added before release. (crfm.stanford.edu) Stanford’s earlier scores had moved in the opposite direction. The first index in October 2023 put the average at 37, and the May 2024 update said the average had risen to 58 after developers were asked to file detailed self-assessments. (crfm.stanford.edu 1) (crfm.stanford.edu 2) The 2025 index also changed its yardstick. Stanford said it revised the indicators to reflect shifts in how foundation models are developed and deployed, so the drop captures both less disclosure and a tougher, updated measurement standard. (crfm.stanford.edu) The pressure for more disclosure has been rising outside the labs. California’s Assembly Bill 2013, signed on September 28, 2024, requires developers to post documentation about the data used to train generative artificial intelligence systems made available to Californians, with the requirement taking effect by January 1, 2026. (legiscan.com) (calmatters.digitaldemocracy.org) California lawmakers also moved beyond training-data summaries. Senate Bill 53, chaptered on September 29, 2025, requires large frontier developers to publish a frontier artificial intelligence framework describing how they assess and mitigate catastrophic risks. (legiscan.com) (calmatters.digitaldemocracy.org) Labs say there are reasons to hold details back. OpenAI publishes system cards for major releases, including GPT-5 updates, and Anthropic says its model system cards document capabilities, safety evaluations, and deployment decisions; both companies frame those reports as part of responsible release practice. (openai.com) (anthropic.com) Stanford’s index shows that publishing a system card is not the same as full disclosure. In the 2025 ranking, xAI and Midjourney tied for the lowest score at 14, while even companies that released open-weight models, including DeepSeek and Alibaba, scored 32 and 26. (crfm.stanford.edu) The fight is now over how much secrecy a lab can claim in the name of safety, competition, or national security before outside researchers, customers, and regulators lose the ability to check what those systems are doing. Stanford’s latest numbers show the industry is moving toward less visibility, not more. (crfm.stanford.edu)
