On-device semantic search
A bilingual, fully on-device semantic search approach was described using NLContextualEmbedding, Foundation Models and mean-centering to avoid server calls and preserve privacy. The write-up emphasises local embeddings and architecture choices that keep search queries and context native to the device rather than routed to external services. The implementation is presented as privacy-preserving by design for personal search scenarios. (x.com)
Semantic search is a way to find meaning, not just matching words, by turning text into number lists and comparing which lists sit closest together. Apple’s Natural Language framework can build those vectors on the device, without sending text to a server. (developer.apple.com) Apple’s `NLContextualEmbedding` class generates dense vectors for natural-language text and changes those vectors based on surrounding words, which is why “bank” in a finance note and “bank” in a river note can land differently. Apple says the class is available across iPhone, iPad, Mac, Apple Watch, Apple TV and Vision Pro platforms, with language-specific or multilingual models depending on the pair. (developer.apple.com) The on-device approach described in this case keeps both the query and the indexed text local, then compares embeddings inside the app instead of calling an external application programming interface. Apple’s documentation says developers can request embedding assets onto the device and check whether those assets are already available locally. (developer.apple.com) That design fits a broader Apple push toward local artificial intelligence features. Apple’s Foundation Models framework gives developers access to the on-device language model behind Apple Intelligence, and Apple says it can work without internet connectivity once the required features are enabled on supported devices. (developer.apple.com 1) (developer.apple.com 2) In plain terms, the privacy claim here comes from where the math runs. If an app can turn your notes, files or messages into vectors on your phone and search them there, the raw text does not have to leave the device for every query. (developer.apple.com 1) (developer.apple.com 2) The bilingual piece is harder than it sounds, because two languages often produce vector spaces that are close in meaning but not perfectly lined up. Research on cross-lingual embeddings has long used steps like length normalization and mean-centering to make those spaces more comparable before nearest-neighbor search. (aaai.org) (aclanthology.org) Mean-centering is a simple adjustment: subtract the average vector so the cloud of points is recentered around zero before comparing distances. In bilingual search, that can reduce language-specific bias in the geometry and improve whether a Spanish query lands near an English document about the same thing. (aaai.org) (arxiv.org) Apple’s own docs draw a distinction between older `NLEmbedding` tools and the newer contextual version. `NLEmbedding` supports direct similarity and nearest-neighbor lookups for words and sentences, while `NLContextualEmbedding` produces context-sensitive vectors that developers can use when they need deeper language understanding and custom retrieval logic. (developer.apple.com 1) (developer.apple.com 2) That leaves tradeoffs. On-device search can avoid per-query cloud costs and preserve local privacy, but developers still have to manage model assets, device compatibility, memory limits and language coverage, especially when one model supports English and Spanish but another language pair may require separate models. (developer.apple.com) (developer.apple.com) The result is a search stack that behaves more like a private index than a web service: your device computes the meaning, stores the vectors and answers the query where the data already lives. That is the core architectural shift in this write-up, not a new search engine on a remote server. (developer.apple.com) (developer.apple.com)
