Are Researchers Using LLMs to Write Their Papers?
With LLM applications more abundant, have researchers been using them to assist their writing? We know they most likely have when writing peer reviews [1] (highlighted in previous post [2]), but how about doing so in writing their published papers?
Liang et al comes back to answer this question in [3]. They applied the same corpus-based methodology proposed in [1] on 950k papers published between January 2020 to February 2024, and the answer is a resounding YES, especially in Computer Science (up to 17.5%).
More interesting observations:
- Papers written by more prolific authors show more signs of LLM usage.
- Papers more similar to their peers show more signs of LLM usage.
- Shorter papers show more signs of LLM usage.
Originally posted on LinkedIn.
References
[1] Weixin Liang, Zachary Izzo, Yaohui Zhang, Haley Lepp, Hancheng Cao, Xuandong Zhao, Lingjiao Chen, Haotian Ye, Sheng Liu, Zhi Huang, Daniel McFarland, and James Zou. “Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews.” 2024. https://arxiv.org/abs/2403.07183
[2] Benjamin Han. “Generative AI Seeped into Research Peer Reviews.” synesis, 2024. Read on this blog.
[3] Weixin Liang, Yaohui Zhang, Zhengxuan Wu, Haley Lepp, Wenlong Ji, Xuandong Zhao, Hancheng Cao, Sheng Liu, Siyu He, Zhi Huang, Yang Diyi, Christopher Potts, Christopher Manning, and James Zou. “Mapping the Increasing Use of LLMs in Scientific Papers.” 2024. https://arxiv.org/abs/2404.01268