Microsoft’s Copilot falsely accuses court reporter of crimes he covered

German journalist Martin Bernklau typed his name and location into Microsoft’s Copilot to see how his culture blog articles would be picked up by the chatbot, according to German public broadcaster SWR.

The answers shocked Bernklau. Copilot falsely claimed Bernklau had been charged with and convicted of child abuse and exploiting dependents. It also claimed that he had been involved in a dramatic escape from a psychiatric hospital and had exploited grieving women as an unethical mortician.

Copilot even went so far as to claim that it was “unfortunate” that someone with such a criminal past had a family and, according to SWR, provided Bernklau’s full address with phone number and route planner.
↫ Matthias Bastian

So why did Copilot (which is just OpenAI’s ChatGPT with sparkles) claim Bernklau did all sorts of horrible things? Well, his occupation – journalist – is a dead giveaway. He has written a lot of articles covering court proceedings in Tübingen on abuse, violence, and fraud cases, and since Copilot is just spicy autocorrect, it has no understanding of context and pinned the various crimes he covered on Bernklau. Adding in his address, phone number, and a damn planned route to his home is just the very disgusting icing on this already disgusting cake.

What makes matters even worse, if you can believe it, is that Bernklau has absolutely no recourse. He contacted the public prosecutor’s office in Tübingen, but they stated they can’t press charges because the accusations coming from Copilot aren’t being made by a real person. And to make it still even worse, Microsoft just threw its hands in the air and absolved itself of any and all responsibility by pointing to its terms of service, in which Microsoft discards liability for content generated by Copilot. Convenient.

This is nothing short of a nightmare scenario that can utterly destroy someone’s life, and the fact that Microsoft doesn’t care and the law isn’t even remotely prepared to take serious matters like these on is terrifying.