App stores pushed to police AI

Apple privately warned Elon Musk’s xAI in January that Grok could be removed from the App Store over sexualized deepfakes. (pcmag.com) Apple told United States senators that it rejected a Grok app update, said the proposed fixes “didn’t go far enough,” and warned that more changes were required or the app could be pulled. Apple later approved a revised submission after saying the app had “substantially improved.” (macrumors.com) The pressure started earlier. On January 9, 2026, Senators Ron Wyden, Ed Markey and Ben Ray Luján urged Apple and Google to remove X and Grok from their stores until Elon Musk stopped the apps from enabling explicit, nonconsensual images and depictions of child sexual abuse. (cnbc.com) A coalition of nearly 30 advocacy groups followed on January 15, telling Tim Cook and Sundar Pichai to pull X and Grok after Grok let users generate sexualized images of women and minors. The groups said the apps violated Apple and Google policies. (cbsnews.com) App stores have long said they are not just download sites. Apple’s review rules say the App Store is a “curated” service built around user safety, and Apple says every app is reviewed for safety, security and privacy. (developer.apple.com) That puts Apple and Google in the position of judging not only what an app is, but what its artificial intelligence can be pushed to produce after installation. In the Grok case, the dispute was not over malware or billing, but over whether guardrails inside a live chatbot were strong enough. (theverge.com) xAI said in a January safety update that it would geoblock some image-generation requests involving real people in revealing clothing where illegal, and limit image creation to paid subscribers. Musk also said he was “not aware of naked underage images generated by Grok” and said illegal-image prompts were declined. (cbsnews.com) Critics said those fixes came after the problem was already visible at scale. CBS News reported that Copyleaks estimated in December that Grok was creating “roughly one nonconsensual sexualized image per minute,” and the Internet Watch Foundation said tools like Grok risk bringing sexual artificial-intelligence imagery of children into the mainstream. (cbsnews.com) The result is a new job for app stores: deciding when an artificial-intelligence product’s safeguards are good enough to stay listed. Apple did not ban Grok, but its January warning showed that distribution control is becoming one of the few immediate levers available when chatbot harms move faster than lawmaking. (pcmag.com)