Signal President Calls Out Agentic AI As Having 'Profound' Security and Privacy Issues

Signal President Meredith Whittaker warned at SXSW that agentic AI poses significant privacy and security risks, as these AI agents require extensive access to users' personal data, likely processing it unencrypted in the cloud. TechCrunch reports: 'So we can just put our brain in a jar because the thing is doing that and we don't have to touch it, right?,' Whittaker mused. Then she explained the type of access the AI agent would need to perform these tasks, including access to our web browser and a way to drive it as well as access to our credit card information to pay for tickets, our calendar, and messaging app to send the text to your friends. 'It would need to be able to drive that [process] across our entire system with something that looks like root permission, accessing every single one of those databases -- probably in the clear, because there's no model to do that encrypted,' Whittaker warned.

'And if we're talking about a sufficiently powerful... AI model that's powering that, there's no way that's happening on device,' she continued. 'That's almost certainly being sent to a cloud server where it's being processed and sent back. So there's a profound issue with security and privacy that is haunting this hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services [and] muddying their data,' Whittaker concluded.

If a messaging app like Signal were to integrate with AI agents, it would undermine the privacy of your messages, she said. The agent has to access the app to text your friends and also pull data back to summarize those texts. Her comments followed remarks she made earlier during the panel on how the AI industry had been built on a surveillance model with mass data collection. She said that the 'bigger is better AI paradigm' -- meaning the more data, the better -- had potential consequences that she didn't think were good. With agentic AI, Whittaker warned we'd further undermine privacy and security in the name of a 'magic genie bot that's going to take care of the exigencies of life,' she concluded. You can watch the full speech on YouTube.

Read more of this story at Slashdot.