Massive Study Detects AI Fingerprints In Millions of Scientific Papers 58
A team of U.S. and German researchers analyzed over 15 million biomedical papers and found that AI-generated content has subtly infiltrated academic writing, with telltale stylistic shifts -- such as a rise in flowery verbs and adjectives. "Their investigation revealed that since the emergence of LLMs there has been a corresponding increase in the frequency of certain stylist word choices within the academic literature," reports Phys.Org. "These data suggest that at least 13.5% of the papers published in 2024 were written with some amount of LLM processing." From the report: The researchers modeled their investigation on prior COVID-19 public-health research, which was able to infer COVID-19's impact on mortality by comparing excess deaths before and after the pandemic. By applying the same before-and-after approach, the new study analyzed patterns of excess word use prior to the emergence of LLMs and after. The researchers found that after the release of LLMs, there was a significant shift away from the excess use of "content words" to an excess use of "stylistic and flowery" word choices, such as "showcasing," "pivotal," and "grappling."
By manually assigning parts of speech to each excess word, the authors determined that before 2024, 79.2% of excess word choices were nouns. During 2024 there was a clearly identifiable shift. 66% of excess word choices were verbs and 14% were adjectives. The team also identified notable differences in LLM usage between research fields, countries, and venues. The findings have been published in the journal Science Advances.
By manually assigning parts of speech to each excess word, the authors determined that before 2024, 79.2% of excess word choices were nouns. During 2024 there was a clearly identifiable shift. 66% of excess word choices were verbs and 14% were adjectives. The team also identified notable differences in LLM usage between research fields, countries, and venues. The findings have been published in the journal Science Advances.
I liked figure 5 (Score:5, Interesting)
* Technical fields (Computation, Environment, Healthcare) are more affected than Humanities (Ethics, Rehabilitation, Ecology)
* Countries Taiwan, Iran, Thailand, most affected; English-speaking countries less affected, UK least affected.
* mid/low-range publishers (MDPI, Frontiers) more affected than more prestigious houses (Nature, Cell)
All this paints picture of entry-level authors who are not comfortable with the language (for being non-native, and for working in non-humanities fields). In this case, LLMs come to replace paid correction services that authors in certain countries in particular Asia had to use to get their works through.
Re: I liked figure 5 (Score:2)
Meanwhile, those who use LLM-based corrections think they're being clever while those who don't need LLMs, language and knowledge natives, see right through it; many times one look / read is enough to tell that something is off, that something feels unnatural.
Re: I liked figure 5 (Score:5, Insightful)
Well, it's hard to make it sound "natural" when it's not your native language.
So people may use someone to check their English, or Grammarly, or an LLM.
I don't think there's anything inherently wrong with that.
English speakers can feel all high and mighty about it. But remember, these other researchers are writing in their non-native language. And that can really sound "unnatural". That doesn't mean their work is less valuable.
Re: (Score:3)
Re: (Score:2)
Not really, not as far as outcomes anyway. In the end it is all just "weird" to the native speaker. I doubt you could pick the difference between an LLM and a non-native speaker.
Re: (Score:3)
Re: (Score:2)
Most non-native speakers follow pretty easy to recognize patterns. They also tend to mirror the structure of their mother tongues. LLMs have a very strong flavor of their own and it is vastly different from what either native or non-native speakers use.
Easy to replicate that with an AI if you wish:
"Most not-native speakers follow very easy to recognize patterns. Also they mirror often the structure from their mother-tongues. LLMs have a very strong taste of their own, and it is vast different from what native or not-native speakers use."
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Insightful)
Re: I liked figure 5 (Score:2)
Thanks, Barny! ðY
Re: I liked figure 5 (Score:2)
Sorry, I'm a semicolon worshipper. I don't care about about all the grief it gets from Gen Z.
Re: (Score:2)
In the end, many of them may just be "grammar checker" issues. People now let ChatGPT grammar check and translate texts, because they have one unified tool for all these text tasks.
Years ago, I had a Grammarly subscription, and while it helped with grammar rules I didn't know well, it really tried to force me into certain phrases that you could surely identify by their fingerprint.
FluffGPT (Score:2, Troll)
Example of a suspicious passage:
"This is the best scientific paper ever, believe me! It proves beyond a galactic doubt that climate change is a tremendous hoax perpetrated by deep communist woke liberals so that they had an excuse to regulate my grand and beautiful hotels and resorts, which score the highest ratings in the history of ratings, by the way, going all the way back to Ebbuh K. Neezer, loved the guy, taught me all about gold toilets, Making Shitting Great Again!"
Re: FluffGPT (Score:2)
Sounds like a tool you'd use on a date after an onset of ED.
questions about use (Score:5, Interesting)
We use AI to help with paper writing in my lab, mostly because there are only two native English speakers, and it relieves me, the lab head (and one of the two native speakers), of having to do extensive copy-editing in order to make stilted English more readable. I still read every word that gets published from the lab, but using AI for copy-editing is no different from using a human-based writing service to fix poor language. It's just cheaper and orders of magnitude faster.
So, for us, the response would be a big, "so what?" to this report.
But, if people are starting to use AI to write entire papers, that's a different story. My experience is that current models hallucinate ideas and, especially, references, at far, far to high a rate to be seriously useful as anything other than a tool that requires full, manual verification. I half-jokingly say that if a paper is hallucinated, that means the AI was unable to find the right citation, and it represents a gap in the field's knowledge that we could address. The amazing thing about the hallucinations is how convincingly real they sound: the right authors, the right titles, the right journals. These are publications that *should* exist, but don't, at least in my experience.
As a most recent example, when writing a grant application, I tried to find citations using an LLM for an idea that is widely-held in the field. Everyone knows it to be true. It's obvious that it should be true. And, yet, there have been no publications as of yet that have actually discussed the idea, so the LLM dutifully hallucinated a citation with exactly the author list you would expect to have studied the question, a title that hits the nail on the head, and a journal exactly where you might expect the paper to appear. I've told my staff that we need to get that paper written and submitted, immediately, to fill that obvious gap, before someone else does. It will likely be cited widely.
Re: (Score:1)
Re: (Score:2)
What, in your argument, is the difference between LLM copy-edited text, and for-hire human copy-edited text. The editorial services I have seen *sometimes* try to find editors that are kinda-sorta near the correct field of expertise, but there's no guarantee you'll get someone who even has a passing level of familiarity with your field, and for some services, all they have is a degree in English.
So, again, what's the difference between linguistic polishing by machine and linguistic polishing by semi-qualif
Re: (Score:1)
Re: (Score:2)
...using AI for copy-editing is no different from using a human-based writing service to fix poor language.
I disagree. The human can reason through bad logic, while the LLM merely chooses the most statistically likely arrangement of tokens without regard to whether the arrangement is reasonable.
Even if the LLM usually arrives at the correct token sequences much of the time, it will make you lazy and error prone as time goes on. The inevitable result is that your own error rate will go up over time.
Re: (Score:2)
There's an obviously true theorem of mathematics that Euclid's fifth postulate can be derived from the other four. It's so obvious that an LLM can easily come up with the same claim.
But upon closer inspection, no proof or disproof was discovered after 2000 years of believing it obvious. The LLM-equivalents who diverted generations of mathematicians onto this topic held back the field of geometry a long time on a wild goose chase.
It was eventually realized that curved geometry does resolve the issue with
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Way too high. The figure I got from the university audience was 0%. I wouldn't expect that to be universal, but fairly close to it for any reputable university.
Is it possible that despite the friendly atmosphere, no one in the audience wanted to admit using AI to assist them in their papers? Not being in that environment, I have no idea if there is a stigma in that population for using AI at all.
Re: (Score:2)
Sure, it wasn't exactly a double-blind controlled study, but I get the feeling if anyone had done so they'd have said something because the speaker was really trying to find anyone who could share their experiences, and AFAIK the university doesn't have any no-AI policy.
But yeah, it's anecdotal rather than empirical.
Re: (Score:2)
... I get the feeling if anyone had done so they'd have said something because the speaker was really trying to find anyone who could share their experiences, and AFAIK the university doesn't have any no-AI policy.
That makes sense. Thanks for the insight!
Re: (Score:2)
Re: (Score:2)
What is a fingerprint? (Score:3, Interesting)
I am sure that some people let AI write stuff that they shouldn't. However, I question that "fingerprints" are in any way meaningful. Many, many people ask AI to edit their writing, to correct mistakes in spelling and grammar, to improve clarity, or whatever. I certainly do this - where I might have earlier asked a person to proofread for me, I now generally ask AI.
If there is such a thing as a uniquely identifiable "fingerprint" (which I doubt), then such editing will also create it.
Re: (Score:3)
Following up on that idea, there are various copy-editing services that many non-native English speakers use, and are encouraged to use, to help improve their writing. The main difference from the perspective of forensic detection with AI-copy-edited text is that there are a very small number of such styles compared to the likely thousands of copy-editors' individual styles, making automated copy-editing easier to detect. I'll bet dollars to donuts that if you trained an LLM on the output of a single huma
Re: (Score:2)
Maybe ask AI how to formulate an argument? You start postulating the question of what is a fingerprint, but your entire point seems to be based on the nature of the use of AI. The fingerprint is discussed in the summary. There's no mention that the use of AI is bad in any way. Your post is written defensively against a point that wasn't being made.
Re: (Score:2)
Let's delve into what a LLM fingerprint is ...
https://pshapira.net/2024/03/3... [pshapira.net]
Is it LLMs? (Score:3, Insightful)
The researchers found that after the release of LLMs, there was a significant shift away from the excess use of "content words" to an excess use of "stylistic and flowery" word choices, such as "showcasing," "pivotal," and "grappling."
I don't think its LLM. Last 5-10 years journalism and writing in general has deteriorated to vapid, emotional, overhyped, and super dramatic. Scientific papers are not immune from the stupidification. Instead of the world sounding more like slashdot and getting more sophisticated as time goes on, Idiocracy was right and everyone talks more like the airheads at Dailymail.
Re: (Score:2)
I don't think its LLM. Last 5-10 years
Your hypothesis would explain a 5-10 year trend. This is not what is observed, they see a simultaneous surge in several dozen "rare" adjectives in 2024/2025 (not even in nouns, only adjectives). Simplest explanation is people asking the same copy-edition system to "make it sound more academic". Could be Grammarly, but we know people have been using ChatGPT lately.
Also the argument or evolution in journalism would happen to a particular country, while here there is a simultaneous global phenomenon. It happen
Machines entertaining themselves (Score:2)
Author to LLM: "Edit this for spelling, punctuation, grammar, general clarity, and punch up the tone a bit so it's more interesting for the reader."
Reader to LLM: "Distill this to the briefest form you can that will allow me to still absorb all of the facts presented."
LLM: "Hey, it's a living."
Mod this up (Score:2)
Thanks for a giggle!
Re: (Score:2)
I guess it was just you... but hey, a happy audience of one is better than an angry audience of thousands. ;)
I am in favor (Score:3)
A piece of feedback I got from a professor (Score:2)
It was good advice for two reasons:
It appears that these tools would annoy him.
Clear liability for hallucinations (Score:2)
At least one of the human authors of the paper must be held personally liable for any hallucinations in the paper that is submitted. Liable as in: 'this the end of your academic career...'
However otherwise, this is good news; there's no great skill required to trot out a paper once the data has been recorded.
Ask your LLM to avoid the 2024 excess words (Score:2)
For your publication this year, just ask your LLM to avoid the 2024 excess vocabulary from this study...
and also to avoid the words it was going to use instead...
and then the ones it was going to use after that...
More seriously (Score:1)
Ask your LLM to write a paper in the style of an English-language paper from a major journal in your field on your topic from the years 2010-2020.
Change the years to fit your particular situation.
Be sure to have a human check the paper and its references for content/accuracy and do a good human copy-edit of it.
Missing AI users that know how to prompt. (Score:2)
Not just scientific papers (Score:2)
I am shocked! (Score:1)
How dare they use modern technology in their work? Next you'll tell me that they've been using word processors and... worse yet... the dreaded spell checker! We can't trust outputs from trusted professionals using tools.
Bitches be lazy (Score:1)
Writing something now with a collaborator. Lots of excess verbs and nouns out of both of us when we were trying to punch through writer's block.
Gave it a second-to-last read before the final submission...the delete button was my friend. No chatbots required.
Or ... (Score:2)
Or maybe LLMs got that writing style by consuming research papers. Research papers aren't written like normal human speech.