Hyperspace P2P cache cuts redundant inference
HyperspaceAI’s Varun Mathur posted that a P2P distributed cache using response caching, KV prefix caching, and intelligent routing can eliminate roughly 70–90% of redundant inference compute across teams. If reproducible, that approach changes cost and capacity planning for large on‑prem or cross‑org inference deployments. (x.com)
Hyperspace published a product page describing a peer-to-peer distributed inference cache with the tagline “Compute once. Cache globally,” including a visual demo that distinguishes cached hits from peer-computed misses. (cache.hyper.space)) The hyperspace-node repository advertises a libp2p/IPFS-based network and publicly states a multi‑million‑node footprint, listing “2,000,000+ agents and counting” as the claimed scale of the P2P inference network. (github.com)) A recent academic submission, “Harvest: Opportunistic Peer‑to‑Peer GPU Caching for LLM Inference” (arXiv, submitted Jan 30, 2026), models P2P GPU caching as a response to KV‑cache growth and shows the approach can trade off GPU memory pressure against network/PCIe latency. (arxiv.org)) vLLM’s design docs describe automatic prefix caching that stores and reuses KV cache blocks for common prompt prefixes, noting that prefix caching avoids redundant prefilling work and materially improves throughput for repeated inputs. (docs.vllm.ai)) Benchmarks from a Ranvier Systems kv‑cache prefix‑routing study report 4–7× improvements in cache utilization and up to ~80% reductions in P99 tail latency for workloads around 13B models under moderate concurrency. (github.com)) Operational writeups and projects implementing KV‑cache–aware schedulers (llm‑d / gateway‑API approaches) describe scoring and routing systems that combine cache‑affinity, SLA, and load to steer requests toward servers already holding relevant KV state. (developers.redhat.com)) Hyperspace’s public code and project messaging include an “agi” repo and multiple node tooling repos that the team positions as components for decentralized autoresearch and for sharing inference results and KV state across peers. (github.com))
