Key Takeaways
- In worrying news to academics, new research from Deakin University finds that ChatGPT makes citation errors more than half of the time.
- Out of 176 citations, ChatGPT fabricated 35 (19.9%) and made errors on 141 (45.4%). It was found to be both real and accurate on just 77 occasions (43.8%).
- The findings should serve as a note of caution for academics everywhere, who are increasingly turning to AI tools to expedite the research process.
ChatGPT fabricates or erroneously cites references in more than 50% of cases, a new study has revealed.
According to new research from Deakin University, when tasked with writing six literature reviews on a variety of mental health topics, the AI chatbot only made 77 out of 176 accurate and real citations.
The results will alarm academics everywhere, many of whom are turning to different AI tools to expedite the lengthy research and citation process. In recent weeks, Anthropic unveiled its latest gambit, a play for the life sciences space known as Claude for Life Sciences. However, this evidence suggests that researchers should think twice before outsourcing their work to AI — at least for the time being.
ChatGPT Citations Incorrect More Than Half the Time, Says Study
New research from Deakin University posits that ChatGPT makes false or inaccurate citations more than half of the time. The study sheds light on the chatbot’s shortcomings in the field of academia.
To conduct the study, the researchers focused on three different psychiatric conditions: major depressive disorder, binge eating disorder, and body dysmorphic disorder.
This just in! View
the top business tech deals for 2025 👨💻
Deakin University scientists tasked the chatbot with writing six literature reviews on the chosen mental health topics, which vary in both public understanding and volume of research. Depression, for instance, boasts an extensive body of research, while body dysmorphic disorder is less well-understood.
Information Either Misleading or Totally Made Up
ChatGPT generated a total of 176 citations across the study. Nearly a fifth (19.9%) of these were found to be completely fabricated. Of the remaining 141 real citations, a significant portion (45.4%) contained inaccuracies, including incorrect publication dates, page numbers, or invalid digital object identifiers (DOI).
Shockingly, ChatGPT was found to be both real and accurate on just 77 occasions, approximately 43.8% of the time. To put it another way, 56.2% of overall citations were either made up or contained errors.
The errors in question weren’t always obvious. For instance, when ChatGPT provided the DOI for a fabricated citation (as it did on more than 94% of occasions), 64% of examples linked to research papers on totally unrelated topics. In other words, readers would only realize the error if they clicked through to the linked paper. The remaining 36% of fake DOIs, meanwhile, were completely invalid.
AI Not Yet Fit for Academic Research
The study should give academics everywhere pause for thought. AI tools, including ChatGPT, Gemini, and the new Claude for Life Sciences, have been heralded as invaluable tools that can save time and automate tedious parts of the research process. The Deakin University research, however, seems to pour cold water on much of this promise.
The researchers urge “careful prompt design, rigorous human verification…and stronger journal and institutional safeguards to protect research integrity.” Indeed, their findings should definitely serve as a note of caution in the field of academic research — and more widely.
In a short space of time, AI has turned the world on its head. Seemingly every week, another company lays off staff in favor of automation, while evidence is mounting that businesses are using AI to completely change their ways of working. However, businesses should be mindful that failing to think through their adoption strategies can lead to a costly buildup of AI debt.
