ChatGPT: Tell-tale signs of essays written with AI tools revealed by researchers

The writing style of the AI tool is ‘bland’ and ‘journalistic’, according to Cambridge University Press and Assessment researchers
PA Wire
Eleanor Busby28 March 2024

Repetition of words, tautology and paragraphs starting with “however” are some tell-tale features of ChatGPT’s writing style, researchers have found.

The writing style of the artificial intelligence tool is “bland” and “journalistic”, according to a Cambridge University Press and Assessment study.

It comes after the rise of generative AI tools, like ChatGPT, has sparked concerns about cheating among pupils in the education sector.

Researchers compared essays written by three first-year undergraduate students, with the aid of ChatGPT, with 164 essays written by IGCSE students.

These essays were marked by examiners and the undergraduates were then interviewed and their essays were analysed.

The study found essays written with the help of ChatGPT performed poorly on analysis and comparison skills compared to non-ChatGPT-assisted essays.

But ChatGPT-assisted essays performed strongly on information and reflection skills.

Researchers identified a number of key features of the ChatGPT writing style, which included the use of Latinate vocabulary, repetition of words or phrases and ideas, and pleonasms.

Essays written with the help of ChatGPT were also more likely to use paragraphs starting with discourse markers like “however”, “moreover”, and “overall”, and numbered lists with items.

The researchers said ChatGPT’s default writing style “echoes the bland, clipped, and objective style that characterises much generic journalistic writing found on the internet”.

The report said: “The students found ChatGPT useful for gathering information quickly.

“However, they considered that complete reliance on this technology would produce essays of a low academic standard.”

Lead researcher Jude Brady, of Cambridge University Press and Assessment, said: “Our findings offer insights into the growing area of generative AI and assessment, which is still largely uncharted territory.

“Despite the small sample size, we are excited about these findings as they have the capacity to inform the work of teachers as well as students.”

She added: “We hope our research might help people to identify when a piece of text has been written by ChatGPT.

“For students and the wider population, learning to use and detect generative AI forms an increasingly important aspect of digital literacy.”

Create a FREE account to continue reading

eros

Registration is a free and easy way to support our journalism.

Join our community where you can: comment on stories; sign up to newsletters; enter competitions and access content on our app.

Your email address

Must be at least 6 characters, include an upper and lower case character and a number

You must be at least 18 years old to create an account

* Required fields

Already have an account? SIGN IN

By clicking Create Account you confirm that your data has been entered correctly and you have read and agree to our Terms of use , Cookie policy and Privacy policy .

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Thank you for registering

Please refresh the page or navigate to another page on the site to be automatically logged in