Microsoft releases Foundry Local for on‑device AI

On-device artificial intelligence means the model runs on your laptop or phone instead of a remote data center. Microsoft said on April 9 that Foundry Local is now generally available as its package for building those apps. (devblogs.microsoft.com) Microsoft’s blog said Foundry Local runs natively on Windows, macOS, Android, phones, laptops, and desktops, and also supports on-premises and distributed deployments validated on Azure Local. The company tied the release to Microsoft Foundry, its broader platform for cloud models, agents, and fine-tuning. (devblogs.microsoft.com) The software is meant to handle the plumbing developers usually have to wire up by hand: model downloads, hardware selection, and inference, which is the step where a trained model generates an answer. Microsoft’s GitHub repository says the package includes native software development kits for C#, JavaScript, Python, and Rust, plus a command line tool for local testing. (github.com) Microsoft said the runtime automatically uses graphics processing units and neural processing units when they are available, then falls back to the central processing unit when they are not. The same repository says the package is about 20 megabytes and is built on ONNX Runtime, Microsoft’s inference engine for running trained models efficiently. (github.com) The company is pitching privacy, offline use, and cost control. Microsoft’s Windows documentation says Foundry Local offers privacy, customization, and cost benefits over cloud-based alternatives, while the GitHub release notes say user data stays on the device and apps can work without a network connection. (learn.microsoft.com) (github.com) Microsoft’s current documentation says developers can use Foundry Local through a command line interface, software development kit, or a local REST application programming interface. The same docs list tutorials for chat assistants, tool calling, speech transcription, note taking, and document summarization. (learn.microsoft.com 1) (learn.microsoft.com 2) The release also fits Microsoft’s larger push to spread artificial intelligence workloads across cloud servers, company-owned hardware, and consumer devices. A Microsoft Build session page says Windows artificial intelligence tools are expanding beyond neural processing units to support graphics processing units and central processing units, with Foundry Local used to run open-source models locally. (build.microsoft.com) Foundry Local itself is not brand new. Microsoft introduced it in preview about 10 months ago for Windows and macOS, then said this month that the product had reached general availability. (devblogs.microsoft.com 1) (devblogs.microsoft.com 2) For developers, the immediate change is that Microsoft now has a named, documented runtime for shipping artificial intelligence features directly inside apps instead of routing every request back to the cloud. The company’s latest quickstart shows that process starting with local installation, model discovery, and direct chat-completions calls from an app. (learn.microsoft.com)