Challenging the Illusion: AI Text Detection and Human Perception

Once again, a word that appears with some frequency in my writing (delve) is maligned as an indicator of AI authorship. But, at least in this instance, it comes within the context of an essay that includes a warning that humans are only under the illusion that they can detect AI written material. The essay in which the aspersion occurs also accurately notes that studies have shown that a well-written AI prompt has often been adjudged by reviewers as more surely written by a human than a piece written, indeed, by a human. The study in circulation demonstrating the latter, written by individuals associated with the Department of Cognitive Science, UC San Diego, does not look like it was written by humans, no matter how many times I review it, but that is attributable to a different bias, not AI.

Ethan Mollick, a writer whose insights at www.oneusefulthing.org are always worth exploring, recently captured my attention with his thought-provoking essay Post-apocalyptic Education. In a section aptly titled, “The Illusions,” Mollick states:

People can’t detect AI writing well. Editors at top linguistics journals couldn’t. Teachers couldn’t (though they thought they could – the Illusion again). While simple AI writing might be detectable (“delve,” anyone?), there are plenty of ways to disguise “AI writing” styles through simple prompting. In fact, well-prompted AI writing is judged more human than human writing by readers

Ha! that parenthetical regarding the word delve, included to provide a small exception to the author’s point, is no exception, at least when it comes to my writing.