ClearML ships AI application gateway
- ClearML has rolled out its AI Application Gateway as an enterprise networking layer for exposing internal AI workloads — including LLM endpoints — safely outside cluster networks. - The telling detail is scope: it supports both HTTP/S and raw TCP routing, plus static routes with load balancing and session stickiness. - This matters because enterprise AI teams want one policy and access layer across Jupyter, VS Code, SSH, Gradio, Streamlit, and model APIs.
AI infrastructure is the part nobody wants to think about until a model actually has to be reachable. That is where things get messy fast. A demo running inside a cluster is easy. A production endpoint that outside users can hit securely — without punching ugly holes through Kubernetes, VPCs, or internal networks — is the hard part. ClearML’s AI Application Gateway is basically its answer to that problem, and by April 2026 the company was positioning it as a core layer for production model serving and interactive AI apps. (clear.ml) ### What is this thing, exactly? It is an enterprise gateway that sits in front of ClearML-managed workloads and gives them externally reachable routes. ClearML describes it as a way to provide secure, authenticated access to jobs and applications running on compute nodes from outside the workload network. Those routes are SSL-secured, tied to ClearML role-based access control, and can handle both HTTP/S traffic and raw TCP. (clear.ml) ### Why is that a real problem? Because AI workloads are not neat, fixed web apps. A Jupyter session, a VS Code instance, a vLLM deployment, or a Gradio app may spin up on demand, live on a random node, use a temporary address, and disappear later. Without a dedicated routing layer, teams end up hand-editing reverse proxies, exposing SSH, or juggling ingress rules every time a session m(clear.ml)ward to expose safely at scale. (clear.ml) ### What changed now? The important shift is not just “ClearML has a gateway.” The company spent late 2025 and early 2026 turning it into a more explicit enterprise control layer around serving and access. Its documentation now treats the AI Application Gateway as an enterprise feature with dedicated deployment guides for Kubernetes and Docker Compose, and ClearML published a separate April 14, 2026 post focused specifically on securing production model serving with it. (clear.ml) ### What can it sit in front of? Quite a lot. ClearML lists JupyterLab, VS Code, SSH Session UI, Gradio launcher, Streamlit launcher, Deploy vLLM, embedding model deployment, llama.cpp deployment, containerized app launcher, and an LLM UI among the applications that use the gateway. That matters because it turns the product into one access plane for both developer tools and inference endpoints, instead of a point solution for only one category. (clear.ml) ### What makes it more than a simple proxy? The useful part is the control surface around routing. Admins can monitor routers, test gateway functionality, manage access tokens, and create static routes that stay fixed even when the backend service instance changes. Those static routes can also run in load-balancing mode, with session context preserved so repeat requests keep landing on (clear.ml)tarts to look like an application access layer, not just a tunnel. (clear.ml) ### Why mention tokens and RBAC so much? Because that is where the governance story lives. ClearML has been adding more visibility around token management, including the ability to list active tokens, label them, and revoke them quickly. Combined with RBAC-bound access and expiring credentials, the pitch is straightforward: let teams expose AI services without giving up centralized control over who can reach what. (clear.ml) ### Is this only for LLMs? No — and that is actually the point. LLM endpoints are the headline-friendly use case, but the gateway is really a networking and policy layer for any AI workload that needs outside access. Think of it like a front door and badge reader for a whole AI building, not just one model API. If your company runs notebo(clear.ml)oc paths. (clear.ml) ### So what is the bottom line? ClearML is trying to make AI deployment feel less like bespoke DevOps and more like managed infrastructure. The AI Application Gateway matters because it turns “how do we expose this safely?” into a product feature with routing, identity, token controls, and observability built in. For enterprise AI teams, that is not flashy — but it is exactly the layer that decides whether pilots stay pilots or become real production systems. (clear.ml)
