Google Cloud RAG private‑connect blueprint

Google Cloud has published a new reference architecture for a problem that keeps slowing down enterprise AI projects. Companies want retrieval-augmented generation, or RAG, because it lets a model answer questions from a company’s own documents instead of guessing from training data. They also want managed services, because nobody is eager to run every model, vector store, and ingestion pipeline by hand. But the moment sensitive records enter the picture, the same question appears: does any of this traffic ever touch the public internet? Google’s answer is a blueprint built around private IP networking from end to end. The architecture, published in Google Cloud’s blog and Architecture Center in early March 2026, is meant for RAG systems where communications “must use private IP addresses and must not traverse the internet.” It splits the deployment into a routing project, a Shared VPC host project, and separate service projects for data ingestion, serving, and frontend systems. That sounds like cloud diagram boilerplate until you see what it is trying to solve. The point is not just to run RAG on Google Cloud. The point is to keep the whole path in-band, even when the application still depends on managed Google services. (cloud.google.com) That matters because RAG is really two systems stitched together. One pipeline pulls in raw documents, chunks them, embeds them, and writes them into a datastore. Another serves live user queries, retrieves relevant context, and sends that context to a model for inference. In a normal deployment, pieces of that path often cross public endpoints, even if the data is encrypted. Google’s design tries to close those gaps with networking primitives rather than application promises. The external network connects into Google Cloud through Cloud Interconnect or Cloud VPN. Network Connectivity Center ties the hybrid links and VPCs together. Private Service Connect exposes internal endpoints for managed services. Shared VPC keeps the projects on one controlled network fabric. (cloud.google.com) The most concrete example is document ingestion. In Google’s reference design, data engineers on an on-premises network do not upload source files to a public Cloud Storage address. They hit a Private Service Connect endpoint in a routing VPC, backed by private DNS, which forwards traffic to a regional Cloud Storage endpoint. That is a small detail, but it is the kind that compliance teams care about, because it turns “we trust TLS over the internet” into “the route never leaves private connectivity in the first place.” The same pattern now extends to Vertex AI access. Google’s recent Vertex AI networking docs describe Private Service Connect options for reaching Generative AI on Vertex AI and other Vertex services through internal IPs instead of external addresses. (docs.cloud.google.com) Private transport alone is not enough, so the blueprint layers on service perimeters. Google positions VPC Service Controls as the backstop against data exfiltration. Its Vertex AI documentation is unusually blunt on this point: private access methods such as Private Service Connect and Private Google Access do not, by themselves, remove public internet accessibility to the APIs. If an enterprise wants to explicitly block that path, it needs a VPC Service Controls perimeter around Vertex AI and related services. In other words, the new architecture is not just about hiding endpoints. It is about pairing private paths with policy that says the data cannot wander off anyway. (docs.cloud.google.com) The frontend side shows the same logic. Google’s design places an Application Load Balancer in the frontend project and pairs it with Cloud Armor for traffic filtering and abuse protection. Internally, the Shared VPC can host an internal Application Load Balancer so services communicate behind a single private address. Externally exposed entry points, where they exist at all, get pushed behind Google’s proxy and security stack instead of talking directly to backend services. That leaves the serving subsystem free to focus on RAG itself: retrieve from the datastore, combine the results with the prompt, and call the model without breaking the network boundary the rest of the system was built to enforce. (cloud.google.com)