Zero‑cost self‑hosted AI stack

A new open-source blueprint argues an AI app can reach production without paying an application vendor, by running models and retrieval software on hardware you already control. (x.com) The stack pairs Ollama for running local language models, LangGraph for agent orchestration, LlamaIndex for retrieval-augmented generation, and either Qdrant or ChromaDB for vector storage. Ollama says its software runs models locally and offline, while LangGraph describes itself as a framework for long-running, stateful agents. (x.com) (docs.ollama.com) (docs.langchain.com) Retrieval-augmented generation, or RAG, is the pattern that lets a model look up your documents before it answers, instead of relying only on what was in training. LlamaIndex’s documentation describes it as an open-source framework for building applications over private data, with integrations for vector stores including Qdrant and Chroma. (docs.llamaindex.ai) (github.com 1) (github.com 2) A vector database is the part that stores numerical fingerprints of text so the system can find the closest match to a question. Qdrant calls itself a vector search engine and database, and Chroma says its software can run locally with built-in retrieval features. (github.com) (docs.trychroma.com 1) (docs.trychroma.com 2) The pitch is not that artificial intelligence becomes free in every sense. The software licenses can be zero-cost, but teams still pay for compute, storage, electricity, maintenance, and the engineering work needed to run the system themselves. (docs.ollama.com) (github.com) (docs.trychroma.com) That trade-off has become more relevant as companies test agents and document search without wanting to commit early to a managed platform. Chroma markets both local and cloud versions from the same codebase, and LangGraph’s documentation positions its open-source framework alongside a separate commercial platform. (trychroma.com) (docs.langchain.com) (github.com) The open-source route also changes where the risk sits. Instead of depending on one vendor’s application programming interface prices, model availability, or data policies, a team takes on model serving, updates, observability, and uptime inside its own environment. (docs.ollama.com) (docs.langchain.com) (docs.trychroma.com) That makes the blueprint most useful as a test bed. A team can swap models, try different retrieval settings, and decide whether the workload belongs on a self-hosted stack or on a managed cloud product after it has real usage data. (docs.ollama.com) (docs.llamaindex.ai) (docs.trychroma.com) In that sense, the “zero-cost” claim is really about software choice at the start of the project. The bill does not disappear, but the architecture can stay portable while a company figures out what it actually needs. (x.com) (trychroma.com)