Hackers Call Current AI Security Testing 'Bullshit'

Leading cybersecurity researchers at DEF CON, the world's largest hacker conference, have warned that current methods for securing AI systems are fundamentally flawed and require a complete rethink, according to the conference's inaugural 'Hackers' Almanack' report [PDF].

The report, produced with the University of Chicago's Cyber Policy Initiative, challenges the effectiveness of 'red teaming' -- where security experts probe AI systems for vulnerabilities -- saying this approach alone cannot adequately protect against emerging threats. 'Public red teaming an AI model is not possible because documentation for what these models are supposed to even do is fragmented and the evaluations we include in the documentation are inadequate,' said Sven Cattell, who leads DEF CON's AI Village.

Nearly 500 participants tested AI models at the conference, with even newcomers successfully finding vulnerabilities. The researchers called for adopting frameworks similar to the Common Vulnerabilities and Exposures (CVE) system used in traditional cybersecurity since 1999. This would create standardized ways to document and address AI vulnerabilities, rather than relying on occasional security audits.

Read more of this story at Slashdot.