As AI-generated text continues to evolve, distinguishing it from human-authored content has become increasingly difficult. This study examined whether non-expert readers could reliably differentiate between AI-generated poems and those written by well-known human poets. We conducted two experiments with non-expert poetry readers and found that participants performed below chance levels in identifying AI-generated poems (46.6% accuracy, χ2(1, N = 16,340) = 75.13, p < 0.0001). Notably, participants were more likely to judge AI-generated poems as human-authored than actual human-authored poems (χ2(2, N = 16,340) = 247.04, p < 0.0001). We found that AI-generated poems were rated more favorably in qualities such as rhythm and beauty, and that this contributed to their mistaken identification as human-authored. Our findings suggest that participants employed shared yet flawed heuristics to differentiate AI from human poetry: the simplicity of AI-generated poems may be easier for non-experts to understand, leading them to prefer AI-generated poetry and misinterpret the complexity of human poems as incoherence generated by AI.
Poetry doesn’t need to rhyme. Rhyming is a mnemonic device, so a poem can be memorized and performed.
There are many other devices.
Also, nice poem. Did you write it or chatGPT?
I never thought I’d see the day
When someone writes a poem
The first thing that we say to them
Is “Did you use an LLM?” :(
If a poem neither rhymes nor scans,
Sorry for my spite
It’s no longer poetry
It’s someone talking shite
Rhyming has other purposes: creation of additional sonic rhythm and restricting possibilities for making matter more distinct and interesting (as rules do for any game).