Can AI Agents Truly Ensure Your Privacy?
At this year’s OpenAI DevDay, CEO Sam Altman introduced the new AI agent, Dots, emphasizing the company’s commitment to establishing a new benchmark for privacy in advanced AI. Throughout the event, OpenAI subtly criticized its main competitor for not adequately protecting user data. Just months prior, that competitor had launched a supposedly more secure alternative to its previous model, with its CEO vowing it was designed with privacy and security in mind.
In a landscape where personal data is often mishandled and cyber threats are rampant, AI developers are urging users to share even more information with their systems. Their latest tactic involves outshining competitors by assuring users that their data will remain confidential. The real challenge lies in whether these companies can deliver on their promises.
The head of product at Meta Superintelligence Labs stated that the aim with their new AI was to create a safe and user-friendly version of its predecessor. User data is stored on a secure virtual machine, described as an isolated Linux environment equipped with essential computing resources. The company claims that significant effort went into designing Muse to operate more securely. They acknowledged that while mistakes can still occur, the built-in safety measures should minimize their frequency and impact.
Despite Muse’s initial success on app charts and a rapid increase in daily active users, concerns about its data handling have emerged. While user data is kept separate, the company plans to implement a feature to cryptographically ensure that it cannot access data in the virtual machine. However, the company still retains access to this data. A security researcher recently uncovered a vulnerability that could have allowed unauthorized control of Muse, which has since been addressed, but it raised alarms about potential external threats. Reports indicated that serious security issues arose just before launch, including one that could have exposed internal databases.
Moreover, Muse appears to collect and share user data extensively. By default, it allows the company to use user inputs for model training, although opting out is an option. Some users have reported that Muse accessed their private messages without consent, and others found it sharing personal information unexpectedly. The platform reportedly creates detailed profiles of users’ contacts, which, while not surprising given the company’s history, undermines claims of a strong focus on privacy.
OpenAI capitalized on these concerns when unveiling Dots. During the event, a product lead highlighted the company’s dedication to creating a trustworthy and secure assistant, showcasing features that allow users to set specific rules for their Dots, such as spending limits. OpenAI’s leadership expressed confidence in their approach, contrasting it with their competitor’s challenges, particularly regarding user trust.
Altman reinforced the privacy message at DevDay, presenting a framework for businesses that includes robust data control options and a zero data retention policy, ensuring no data is stored on OpenAI’s servers. So far, Dots has not faced significant privacy controversies, although its availability is limited to higher-tier subscriptions, likely resulting in a smaller user base.
Despite these assurances, potential users may still hesitate to share personal information, especially given the extensive data requirements of AI agents. For example, one user felt uneasy providing bank details when prompted by the bot.
Not all companies are making privacy commitments. One firm faced backlash for overly broad terms of service that allowed extensive data access, prompting them to revise their policies. Currently, AI developers seem to be relying on a three-pronged strategy to encourage public adoption of AI agents: ensuring usefulness, presenting a friendly and approachable interface, and making privacy promises—while hoping they can uphold them.
