User Privacy Expectations for AI Tools Are Evolving
As AI tools like ChatGPT become more integrated into daily life, user expectations for data privacy are shifting. Users are increasingly requesting granular controls, such as separate toggles for storing text and media, amid growing scrutiny of how third-party partners handle sensitive information.
- General consumer trust in companies' use of AI is low, with 70% of Americans reporting little to no confidence that companies will use AI responsibly and 57% of global consumers viewing AI as a significant threat to their privacy. - Many major consumer AI tools, including OpenAI's ChatGPT and Anthropic's Claude, use customer data to train their models by default; users must proactively navigate to their data or privacy settings to opt out. - The principle of "purpose limitation," central to regulations like GDPR, is being applied to AI, requiring companies to define a specific, lawful purpose for any data collected and prohibiting them from repurposing that data for other uses, like training new models, without fresh consent. - A significant emerging risk is "inferred data," where AI systems predict sensitive user attributes such as health conditions or political beliefs from seemingly unrelated information, creating privacy harms beyond what users explicitly share. - Regulatory frameworks are becoming more stringent, with the EU AI Act prohibiting certain practices like the untargeted scraping of facial images from the internet and placing strict requirements on "high-risk" AI systems used in areas like recruitment or credit scoring. - The Federal Trade Commission (FTC) has taken enforcement action against firms that secretly use customer data for model training, in some cases requiring them to delete entire algorithms and models built with unlawfully obtained data. - When Apple introduced its App Tracking Transparency feature, which asks users for permission before an app can track them across other apps and websites, an estimated 80% to 90% of users chose to opt out, demonstrating a strong consumer preference for privacy when controls are made simple and accessible. - The involvement of third parties complicates data privacy, as AI companies often share user information with external services for functions like cloud hosting, analytics, and payment processing, expanding the potential for data misuse.
