Vector DB & hybrid retrieval notes
Podcasts and walkthroughs this week stressed running real bakeoffs on vector DBs—Weaviate praised for multi‑tenancy and hybrid search, Pinecone for low‑latency filtering—and warned that most real‑world RAG failures are recall problems, not model hallucinations. Engineers also showed latency wins by using in‑memory indexes and sharding, and flagged egress/location costs as major hidden expenses. (YouTube links referenced in media briefing) (x.com/devopscube/status/2035204783728968069)
Weaviate implements a “one shard per tenant” physical partitioning that isolates tenant data and lets operators deactivate or replicate individual tenant shards for performance control. (weaviate.io) Weaviate’s hybrid-search implementation runs BM25 and dense vector queries in parallel and fuses normalized scores using strategies such as relativeScoreFusion and rankedFusion to produce a single ranked result set. (docs.weaviate.io) A recent single-stage filtering benchmark showed Pinecone’s filtered queries dropping median latency from ~79 ms (unfiltered) to ~51.6 ms with a 1% selective filter, and throughput gains at higher thread counts as filtering reduced search work. (yudhiesh.github.io) Pinecone’s product documentation highlights real-time indexing, tiered storage, and caching across storage mediums as key architectural levers for maintaining low-latency filtered queries in production. (pinecone.io) Multiple diagnostics guides and practitioner posts identify poor recall, bad chunking, stale indexes and query-drift as the dominant production failure modes in RAG pipelines rather than generator-only hallucinations, with concrete checklists for measuring Recall@K and index health. (dev.to) In-memory indexes (FAISS-style) and careful sharding are repeatedly recommended to hit sub-10 ms query targets; open-source Milvus documents in-memory index types for accelerated search, while production write-ups show unsharded platforms spiking from ~120 ms p95 to multiple seconds at exabyte scale if partitions aren’t applied. (blog.milvus.io) Engineering cost analyses flag cloud networking and index rebuilds as major hidden line items: outbound egress in AWS starts around $0.09/GB on initial tiers, NAT-gateway and cross‑AZ charges add cents/GB, and vendor comparisons estimate index-rebuild compute in the ballpark of $12–$40 per 10M vectors for common rebuild scenarios. (nops.io)
