Accurate, Focused Research on Law, Technology and Knowledge Discovery Since 2002

AI-Generated Junk Science Is a Big Problem on Google Scholar

Gizmodo: “AI-generated scientific research is polluting the online academic information ecosystem, according to a worrying report published in the Harvard Kennedy School’s Misinformation Review. A team of researchers investigated the prevalence of research articles with evidence of artificially generated text on Google Scholar, an academic search engine that makes it easy to search for research published historically in a wealth of academic journals. The team specifically interrogated misuse of generative pre-trained transformers (or GPTs), a type of large language model (LLM) that includes now-familiar software such as OpenAI’s ChatGPT. These models are able to rapidly interpret text inputs and rapidly generate responses, in the form of figures, images, and long lines of text. In the research, the team analyzed a sample of scientific papers found on Google Scholar with signs of GPT-use. The selected papers contained one or two common phrases that conversational agents (commonly, chatbots) undergirded by LLMs use. The researchers then investigated the extent to which those questionable papers were distributed and hosted across the internet.” [h/t Pete Weiss]

Sorry, comments are closed for this post.