Salesforce Study Finds LLM Agents Flunk CRM and Confidentiality Tests

A new Salesforce-led study found that LLM-based AI agents struggle with real-world CRM tasks, achieving only 58% success on simple tasks and dropping to 35% on multi-step ones. They also demonstrated poor confidentiality awareness. 'Agents demonstrate low confidentiality awareness, which, while improvable through targeted prompting, often negatively impacts task performance,' a paper published at the end of last month said. The Register reports: The Salesforce AI Research team argued that existing benchmarks failed to rigorously measure the capabilities or limitations of AI agents, and largely ignored an assessment of their ability to recognize sensitive information and adhere to appropriate data handling protocols.

The research unit's CRMArena-Pro tool is fed a data pipeline of realistic synthetic data to populate a Salesforce organization, which serves as the sandbox environment. The agent takes user queries and decides between an API call or a response to the users to get more clarification or provide answers.

'These findings suggest a significant gap between current LLM capabilities and the multifaceted demands of real-world enterprise scenarios,' the paper said. AI agents might well be useful, however, organizations should be wary of banking on any benefits before they are proven.

Read more of this story at Slashdot.