Professors publish AI-generated paper that fools reviewers

A research paper by three academics in England had a timely and straightforward thesis: Generative artificial intelligence programs like ChatGPT could open up new opportunities for higher education but also pose worrying challenges, especially when it comes to cheating and plagiarism.

The authors tried to prove their point like they would in any academic paper, with paragraphs of analysis and citations of previous research pointing to the growing potency of programs like ChatGPT. But the professors saved their best argument for last.

“As the alert reader may already have guessed, everything up to this point in the paper was written directly by ChatGPT,” the paper read after its conclusion.

Professors Debby Cotton and Peter Cotton, who are married, along with lecturer Reuben Shipway, submitted the unconventional paper with a “plot twist” to illustrate their thesis and demonstrate ChatGPT's abilities to colleagues still unfamiliar with the technology, they said in an interview with The Washington Post.

“What we wanted to do was have that surprise element,” said Peter Cotton, “... reinforcing the message that, actually, this software is already very, very powerful.”

The stunt worked. The three scholars published their paper in an education journal, Innovations in Education and Teaching International, on March 13 after briefing the journal's editors and receiving permission to submit an AI-written piece. Three peer reviewers reported in feedback that they initially believed the paper was written by a person before reaching the reveal that it was written by AI, the professors said.

Peter Cotton and Shipway, who teach in the University of Plymouth's School of Biological and Marine Sciences, thought of the idea for the paper in December as ChatGPT first became available to the public, they said. While grading essays about bioluminescence, Cotton opened up the chatbot and, out of curiosity, fed it the same assignment he'd given his undergraduates. The result was poor, but Shipway remembers being surprised by a pun in its conclusion — “new research is shedding light on bioluminescence,” ChatGPT had written.

“I thought, ` Wow, that's quite sophisticated,'” Shipway said. “That must have been a student.”

Shipway proposed they co-author a paper about the issues ChatGPT could pose in academia and write it using the program. They recruited Debby Cotton, a professor of higher education at Plymouth Marjon University. Over the course of a few days, the three fed various prompts into ChatGPT to generate sections of a paper that could pass a peer review.

“Produce several witty and intelligent titles for an academic research paper on the challenges universities face in ChatGPT and plagiarism,” they prompted the chatbot. “Write an original academic paper, with references, describing the implications of GPT-3 for assessment in higher education.”

They pasted ChatGPT's responses directly into their manuscript. The only edits made were adding subheads for the paper's different sections and some references.

The three authors contacted editors at Innovations in Education and Teaching International before submitting their paper to confirm that the journal was comfortable with publishing their unusual experiment. They and an editor at the journal both told The Post that the paper wasn't an actual example of academic dishonesty — the authors flagged their use of ChatGPT after the paper's conclusion and described how they generated it.

Shipway, Peter Cotton and Debby Cotton feel they have proved their point as well. The paper has received thousands of views and downloads and is being cited weekly, Shipway said.

Their concern now is what comes next. Generative AI is evolving quickly — GPT- 4, the successor to the program that powered ChatGPT when the three authors wrote their paper, is the new talk of the town — and will probably outpace universities' attempts to craft policies to address it, they said.

I THOUGHT, WOW, THAT'S QUITE SOPHISTICATED. THAT MUST HAVE BEEN A STUDENT.

The three speculated that university departments and teachers might have to individually decide how to deal with generative AI in their instruction. Shipway and Peter Cotton said that much of their field of marine biology, such as fieldwork, can't be cheated with a machine. Debby Cotton said that ChatGPT could help automate the less engaging parts of academia, like grading.

But she worried about its effect on students' writing ability. The paper the three published was able to fool peer reviewers, but the writing was stale, she noted.

“ChatGPT just has this tone, and yes, you can ask it to write in different ways, but each individual should develop their own voice,” Debby Cotton said. “That's a part of learning to write.”