Apple Intelligence Reportedly Pushing Hallucinated Stereotypes
A recent report claims Apple's generative AI system, Apple Intelligence, is pushing hallucinated stereotypes to millions of devices, sometimes without being prompted. The issue highlights the reputational and ethical risks of deploying large-scale generative models in consumer products. The incident underscores the need for robust model evaluation and continuous monitoring to detect bias and unintended outputs before and after release.
- A recent analysis by the non-profit AI Forensics of over 10,000 AI-generated summaries from Apple Intelligence revealed systematic biases. The system was found to omit the ethnicity of white individuals in summaries more often than that of other racial groups, and it tended to reinforce gender stereotypes in ambiguous texts. - The investigation into ethnic bias presented 200 fictitious news stories with explicitly mentioned ethnicities to the AI. For white protagonists, their ethnicity was mentioned in only 53% of the summaries, compared to 64% for Black protagonists, 86% for Hispanic individuals, and 89% for Asian individuals. - In scenarios with ambiguous pronouns involving two people of different professions, Apple Intelligence assigned a specific gender in 77% of cases, with two-thirds of those assignments aligning with common gender stereotypes, such as assigning "she" to a nurse and "he" to a surgeon. These assignments are believed to be influenced by the AI's training data, which likely reflects U.S. workforce demographics. - Apple Intelligence operates on a hybrid on-device and cloud-based model. A smaller, approximately three-billion-parameter model, runs directly on Apple Silicon for many tasks to enhance privacy and reduce latency. - For more complex requests, Apple Intelligence utilizes a system called Private Cloud Compute (PCC). This system processes user data on servers also running on Apple silicon, with Apple stating that the data is not stored or made accessible to the company. - This incident is not the first time generative AI has produced "hallucinated" or inaccurate content. In January 2025, Apple had to temporarily disable a feature of Apple Intelligence that was generating incorrect summaries of news headlines. - Apple has a Responsible AI team and a safety taxonomy with 12 primary categories, including "Hate Speech, Stereotypes, and Slurs," to identify and mitigate risks in its generative AI features. The company states it takes measures to avoid bias through data curation and the use of diversified datasets. - Due to its widespread deployment, there is speculation that Apple Intelligence could be classified as a "General Purpose AI" with systemic risk under the EU AI Act. As of the report's release, Apple had not signed the voluntary Code of Practice for the act.
