Tens of thousands of publications from 2025 might include invalid references generated by AI, a Nature analysis suggests.

https://www.nature.com/articles/d41586-026-00969-z

Anyone have thoughts on their own experience and what the options are to combat this?

Honestly, I would consider this fraud.

Shouldn't peer review be catching these fraudulent citations? If it isn't, what does that suggest about the feasibility of the current peer-review based scientific model?

I think anybody who relies on AI for any sort of facts, should be fired. It is hard to believe that anybody on the planet is not aware of how unreliable AI is. It regularly makes up facts. We all know this. Why would anybody use it to gather data, when you know the data is unreliable!?

WillyP said:

I think anybody who relies on AI for any sort of facts, should be fired. It is hard to believe that anybody on the planet is not aware of how unreliable AI is. It regularly makes up facts. We all know this. Why would anybody use it to gather data, when you know the data is unreliable!?

Either laziness or to increase output. An enterprising individual released at least 364 AI "history" videos in a month. They were entirely and deliberately fictitious.

I've had the experience. Grok (as I recall) produced a reference and a detailed quote that was exactly what I was looking for .... it was a couple of years ago, and I thought I had written about it but I just searched my computer but can't find it ... and I was so happy I rewrote parts of what I was working and used the reference. Then, maybe a week later I decided I'd check the entire paper, and lo and behold, it it didn't exist. So, yes, AI can produce convincing detailed references, but .... they're easy to check.

The reason I use AI to look for references is that it is incredibly good at its job, and It has found many references that I do use ...... after checking :).

All my preprints before chatGPT were read carefully by others, they weren't papers per se in journals.
I didn't think to myself that LLM can produce references that don't even exist. It's really upseting to hear that it does such a thing. But it also means we can't accept everything it writes without understanding the subject we asking about.
So it means that academics still have work to sort out the mess those LLMs do.

"I didn't think to myself that LLM can produce references that don't even exist." I didn't either ... but now I know .. and that's why they call it hallucinating. It's complete with all the details, and since it was saying what I wanted to hear, it was convincing. But, it's not that AI 'knows' that it's false. It doesn't .... but its discovery algorithm isn't perfect and it can discover things that don't exist. (IMO of course).