Irony alert: Hallucinated citations found in papers from NeurIPS, the prestigious AI conference

AI detection startup GPTZero scanned each 4,841 papers accepted by the prestigious Conference connected Neural Information Processing Systems (NeurIPS), which took spot past period successful San Diego. The institution recovered 100 hallucinated citations crossed 51 papers that it confirmed arsenic fake, the institution tells TechCrunch. 

Having a insubstantial accepted by NeurIPS is simply a resume-worthy accomplishment successful the satellite of AI. Given that these are the starring minds of AI research, 1 mightiness presume they would usage LLMs for the catastrophically boring task of penning citations.

So caveats abound with this finding: 100 confirmed hallucinated citations crossed 51 papers is not statistically significant. Each insubstantial has dozens of citations. So retired of tens of thousands of citations, this is, statistically, zero. 

It’s besides important to enactment that an inaccurate citation doesn’t negate the paper’s research. As NeurIPS told Fortune, which was archetypal to study connected this GPTZero’s research, “Even if 1.1% of the papers person 1 oregon much incorrect references owed to the usage of LLMs, the contented of the papers themselves [is] not needfully invalidated.” 

But having said each that, a faked citation is not a nothing, either. NeurIPS prides itself connected its “rigorous scholarly publishing successful instrumentality learning and artificial intelligence,” it says. And each insubstantial is peer-reviewed by aggregate radical who are instructed to emblem hallucinations.  

Citations are besides a benignant of currency for researchers. They are utilized arsenic a vocation metric to amusement however influential a researcher’s enactment is among their peers. When AI makes them up, it waters down their value.

No 1 tin responsibility the adjacent reviewers for not catching a fewer AI-fabricated citations fixed the sheer measurement involved. GPTZero is besides speedy to constituent this out. The extremity of the workout was to connection circumstantial information connected however AI slop sneaks successful via “a submission tsunami” that has “strained these conferences’ reappraisal pipelines to the breaking point,” the startup says successful its report. GPTZero adjacent points to a May 2025 insubstantial called “The AI Conference Peer Review Crisis” that discussed the occupation astatine premiere conferences including NeurIPS. 

Still, wherefore couldn’t the researchers themselves fact-check the LLMs enactment for accuracy? Surely, they indispensable cognize the existent database of papers they utilized for their work. 

What the full happening truly points to 1 big, ironic takeaway: If the world’s starring AI experts, with their reputations astatine stake, can’t guarantee their LLM usage is close successful the details, what does that mean for the remainder of us?