Using AI to write about the dangers of AI
We shouldn't unquestioningly believe "AI detectors" but nor should we be so naive to think that anyone's immune form the temptation of using AI to write very long documents.
It’s possible that AI was used to write parts of Pope Leo XIV’s latest encyclical about AI’s impact on humanity. An analysis by Linch Zhang posted on the forum LessWrong found certain paragraphs of Magnifica Humanitas to be between 40 percent and 100 percent written by AI, according to the popular AI detector Pangram.
The document includes known traits that appear in AI-generated writing, such as a higher use of the word “genuinely” — which crops up in writing by Anthropic’s Claude — than previous encyclicals, Zhang says. Another person ran the text of the document section by section through Pangram, finding that 62 percent of its first chapter was flagged as AI generated. When The Verge ran roughly 2,000 words of the document through Pangram, it estimated that 46 percent was AI-written.
Still, other portions register as being written by humans. Zhang notes that Pangram flagged some sections as “essentially 0% AI.” The first 20 paragraphs of the last four encyclicals, when run through Pangram, had a 100 percent confidence of being human written. And a transcript of Pope Leo’s speech, run through Pangram, was also rated as 100 percent human.
AI detection isn’t foolproof. Different AI detectors can display different results, and even when there’s consensus there’s no guarantee they’re correct. But Pangram is generally respected among AI researchers. In March 2025, Pangram said it estimated its false positive rate of reporting human-written work as AI-generated “to be approximately 1 in 10,000.”
Recently, the author of The Future of Truth: How AI Reshapes Reality discovered that AI had put "synthetic quotes" in the book. Yes, we should all be doing better fact checking - there's a quotation attributed to Virginia Woolf I almost used recently until I noticed something slightly off about it. Sure enough, it was fabricated.
I think we would do well to remember that, if the total span of time the human race has been around was a clockface,the time period we're in right now - with cars and the internet and mass democracy - started at about a millisecond to midnight. We haven't really evolved for influencing people and events at great distance, but then nor is it new to encounter fake news or people using whatever means they have at their disposal to get things done.
Source: www.theverge.com
Are.na block: ↗
Collection: ai-critique-policy
Cristóbal Ascencio & Archival Images of AI + AIxDESIGN / Image to Audio Corruption 02 / Licenced by CC-BY 4.0