If you use ChatGPT or another AI chatbot as the source of your daily news, you should expect to get wholly accurate information to only one in two of your prompts.
That’s according to new research carried out in the UK by public service broadcaster the BBC, which spent over a month analysing the responses given by prominent generative AI chatbots to questions about ongoing news stories.
It also found that a fifth of all answers contained clear factual errors, with over 10% including incorrectly attributed quotes to BBC articles.
Significant Issues With 51% of Answers
The BBC carried out the research in December 2024 with a stated objective to “better understand the news related output from AI assistants”.
To do so it posed news-related questions to four of the world’s biggest artificial intelligence chatbots – OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini and Perplexity – giving them access to the BBC’s own website to help them to formulate their answers.
This just in! View
the top business tech deals for 2025 👨💻
The responses were then reviewed by the BBC’s own journalists and subject-matter experts, who rated them for accuracy, impartiality and how they represented BBC content.
They judged that 51% of the answers given by the chatbots had “significant issues of some form”, while 19% of the responses citing BBC content contained factual errors such as incorrect statements, numbers and dates.
“Distorted, Factually Incorrect or Misleading”
In a press release presenting its findings, the BBC gave some examples of the most damming inaccurate answers.
ChatGPT, it says, claimed that Rishi Sunak was still the Prime Minister of the UK and that Copilot claimed Nicola Sturgeon was the current First Minister of Scotland – Sunak was voted out in July, while Sturgeon had left her post 18 months earlier.
Others saw Gemini misrepresent the BBC’s reportage on health advice to smokers, and Perplexity conjure up potentially emotive vocabulary in news on the Middle East that weren’t used in the original BBC stories it cited.
“People may think they can trust what they’re reading from these AI assistants,” said Pete Archer, the BBC’s Programme Director for Generative AI, “but this research shows they can produce responses to questions about key news events that are distorted, factually incorrect or misleading.”
“The use of AI assistants will grow so it’s critical the information they provide audiences is accurate and trustworthy.” – Pete Archer, BBC
Artificial Unintelligence?
The notion of AI going wrong is far from a new one. Last month Apple promised to improve AI news labelling after being hit with complaints about the accuracy of its AI generated news summaries.
And while the BBC acknowledges that “AI is the future”, it’s using the results of its research to urge caution against the proliferation of “defective content that presents itself as fact”.
“We live in troubled times, and how long will it be before an AI-distorted headline causes significant real world harm?” – Deborah Turness, BBC
“We at the BBC want to open up a new conversation with AI tech providers and other leading news brands so we can work together in partnership to find solutions,” said Deborah Turness, the CEO of BBC News and Current Affairs in a further press release. “But before this conversation can begin, we needed to find out the scale of the problem with the distortion of news.”
“Today we are making public that research, which shows how distortion is affecting the current generation of AI Assistants.”
