Why do we need a new test?

A common version of the Turing Test goes like this:

Place a human, let’s call them Alan, in a room with two computers with no visible difference between them. Yet unbeknownst to Alan each computer is connected with a different source in another room – one with a fellow human, and the other with an AI. Alan is then asked to engage in a conversation with each and note how well they respond. Here’s the key to the Turing Test though, it is not Alan we are testing, but the AI. When Alan, or any human for that matter, cannot tell the difference between the human and the AI on the other side, that AI is said to have passed the Turing Test.

Until the creation of publicly accessible “generative AI” such as ChatGPT, the average human had no access to an AI that could pass the Turing Test. This has changed. Yet, with few examples to the contrary, when asked to “write a poem,” these AIs tend to compose, to put it gently, less than inspiring poetry. Put less gently, when measured by a capacity to rock our souls with words, so far, AI falls miserably short.

This led us to propose a new test for AI and the digital humanities.

The Dickinson-Turing Test

“If I read a book and it makes my whole body so cold no fire can warm me, I know that is poetry. If I feel physically as if the top of my head were taken off, I know that is poetry. These are the only ways I know it. Is there any other way?”
- Emily Dickinson

Can an AI touch our souls with poetry, or any kind of art for that matter, so that our heads come away from our shoulders and our spines shimmer with technicolor fuzz? If so, then we can say that an AI has passed the Dickinson-Turing Test, a measure of AI outputs based on what they do for the observer, and for art, in a space that exists between observer and the creation. What happens in that space between observer and observed? The Dickinson-Turing Test allows us to peer into that liminal space, making it a necessary and often overlooked step for the humanities in the emerging age of AI. 

Can you pass the Dickinson-Turing Test? 

This is the ask, this is the gift, this is the open-handed, gently-placed gauntlet we offer to AI and its creators.