MLflow adds hard budget controls

A lot of artificial intelligence bills do not explode because one model call is expensive. They explode because a loop, retry storm, or busy agent makes thousands of calls before anyone notices. (mlflow.org) MLflow sits in the middle of those calls with what it calls an artificial intelligence gateway, which is a single front door that routes requests to model providers, holds the keys, and tracks usage in one place. MLflow says teams use that gateway instead of wiring every app directly to OpenAI, Anthropic, Amazon Bedrock, and other providers one by one. (mlflow.org) The new piece is a budget policy. In MLflow’s docs, a budget policy is a dollar threshold tied to a recurring window, and that window can reset daily, weekly, or monthly. (mlflow.org) When that threshold is crossed, MLflow now gives you two choices. “Alert” sends a webhook and lets traffic keep flowing, while “Reject” blocks later requests and returns Hypertext Transfer Protocol 429, the standard “too many requests” response. (mlflow.org) That means the gateway can act like a circuit breaker for spending. The request that pushes costs over the line still goes through, but the next requests can be stopped automatically instead of waiting for a human to wake up and shut things down. (mlflow.org) MLflow already supports webhooks for gateway events, and the documented event here is `budget_policy.exceeded`. A team can point that webhook at an internal service, or at tools like Slack and PagerDuty through their usual webhook flows, so the same over-budget event can page an engineer or post in a channel. (mlflow.org) This lands in a part of the stack that has been getting crowded fast. MLflow says its gateway gives one OpenAI-compatible endpoint for more than 50 large language model providers, which means one runaway app can chew through spend across several back-end vendors unless there is a shared limit above them. (mlflow.org) The timing also fits MLflow’s broader pitch. On its main site and GitHub page, MLflow describes itself as an open-source platform for agents, large language models, and machine learning models, with cost control and governance now sitting next to tracing, evaluation, and monitoring instead of being a separate finance tool. (mlflow.org) (github.com) So this is not a new model and not a faster inference engine. It is a hard stop in the plumbing: set a dollar cap, pick a reset window, wire the alert where your team already works, and let the gateway cut off traffic before a bad afternoon turns into a five-figure invoice. (mlflow.org)