Google's AI Will Help Decide Whether Unemployed Workers Get Benefits

An anonymous reader quotes a report from Gizmodo: Within the next several months, Nevada plans to launch a generative AI system powered by Google that will analyze transcripts of unemployment appeals hearings and issue recommendations to human referees about whether or not claimants should receive benefits. The system will be the first of its kind in the country and represents a significant experiment by state officials and Google in allowing generative AI to influence a high-stakes government decision -- one that could put thousands of dollars in unemployed Nevadans' pockets or take it away. Nevada officials say the Google system will speed up the appeals process -- cutting the time it takes referees to write a determination from several hours to just five minutes, in some cases -- helping the state work through a stubborn backlog of cases that have been pending since the height of the COVID-19 pandemic.

The tool will generate recommendations based on hearing transcripts and evidentiary documents, supplying its own analysis of whether a person's unemployment claim should be approved, denied, or modified. At least one human referee will then review each recommendation, said Christopher Sewell, director of the Nevada Department of Employment, Training, and Rehabilitation (DETR). If the referee agrees with the recommendation, they will sign and issue the decision. If they don't agree, the referee will revise the document and DETR will investigate the discrepancy. 'There's no AI [written decisions] that are going out without having human interaction and that human review,' Sewell said. 'We can get decisions out quicker so that it actually helps the claimant.'

Judicial scholars, a former U.S. Department of Labor official, and lawyers who represent Nevadans in appeal hearings told Gizmodo they worry the emphasis on speed could undermine any human guardrails Nevada puts in place. 'The time savings they're looking for only happens if the review is very cursory,' said Morgan Shah, director of community engagement for Nevada Legal Services. 'If someone is reviewing something thoroughly and properly, they're really not saving that much time. At what point are you creating an environment where people are sort of being encouraged to take a shortcut?' Michele Evermore, a former deputy director for unemployment modernization policy at the Department of Labor, shared similar concerns. 'If a robot's just handed you a recommendation and you just have to check a box and there's pressure to clear out a backlog, that's a little bit concerning,' she said. In response to those fears about automation bias Google spokesperson Ashley Simms said 'we work with our customers to identify and address any potential bias, and help them comply with federal and state requirements.' 'There's a level of risk we have to be willing to accept with humans and with AI,' added Amy Perez, who oversaw unemployment modernization efforts in Colorado and at the U.S. Department of Labor. 'We should only be putting these tools out into production if we've established it's as good as or better than a human.'

Read more of this story at Slashdot.