On-Device AI Goes Production-Grade

RunAnywhere, founded by Shubham Malhotra and Sanchit Monga, is a Y Combinator W26-backed company aiming to solve the operational challenges of deploying AI at scale. The founders identified that while running a model on a single device is simple, managing it across thousands of varied devices in an enterprise setting presents significant hurdles in observability, performance, and version control. The platform's core is a unified SDK and a centralized control plane. This combination allows enterprises to deploy, update, and monitor multimodal AI applications across diverse hardware and operating systems. It supports native SDKs for Swift (iOS/macOS), Kotlin (Android), React Native, Flutter, and a web SDK, designed to abstract away the complexity of different inference backends like Core ML or ONNX. A key feature is its hybrid routing capability, which allows for intelligent workload distribution between the device and the cloud. An organization can set policies to first attempt inference locally for low latency and privacy, but automatically fall back to a cloud-based model if the device has insufficient power, is too old, or the model's confidence score is low. RunAnywhere supports a range of models, including LLMs like Llama and Mistral, as well as speech-to-text models like Whisper. This enables on-device features such as private chat, real-time transcription, and full voice assistant pipelines (STT→LLM→TTS) that can function entirely offline, a critical feature for journalists in low-connectivity areas. This technology enters a market where newsrooms are increasingly adopting AI for tasks like transcription and converting articles into video formats to expand their reach. The global market for AI video generators is projected to grow from $534 million in 2024 to $2.56 billion by 2032, highlighting a major shift in content production workflows. The move to on-device AI addresses several enterprise adoption barriers, including data privacy, latency, and the high cost of cloud inference for repetitive tasks. By providing a vendor-agnostic operational layer, the platform aims to prevent vendor lock-in and reduce the internal engineering effort required to manage fragmented device ecosystems.