Poetic Language Between Humans and AI

Role: Conducted design research and developed speculative design experiments
Team: Solo project
Timeline: January 2018, 2 Months
Studio: Carnegie Mellon University, Environments Design Studio
Recognition: Exhibited at the Future Perfect Conference, Data & Society Research Institute, New York, 2018

Most of the ways we talk to AI are built on logic and precision. But human language isn't like that — it's full of metaphor, context, and feeling.

Poetic Language between Humans and AI is a speculative design exhibit exploring how metaphors, phrases, and poems can be used to communicate with AI, fostering deeper understanding of intangible emotions.

 

The Challenge

As AI increasingly integrates into our daily lives, designers must critically examine and provoke discussions about the complex interactions between human and nonhuman intelligences, addressing biases and hidden power structures to thoughtfully shape future environments. In this studio, the challenge was to interpret the topic: Where are the humans in AI?

The Process

I started by conducting exploratory research and brainstorming on social interaction topics regarding humans and artificial intelligence. I categorized insights into four themes:

  • Poetic/Lyricism: Does AI understand metaphors or symbols?

  • Social Status/Hierarchy: Often, AI is called an assistant. Does the AI world have social status or hierarchy?

  • Manners/Traditions: As younger generations spend more time with machines and AI, they might lose contact with human intelligence, culture, and roots. How might this affect the way we value daily manners and etiquette?

  • Emotional & Intelligent Quotient: While humans do not value EQ and IQ equally, how does AI perceive the importance of each?

1*0XEDvpU9m6kz-3XZZuKFgw.jpeg

Discussion

Meanwhile, I joined the Humans and AI discussions on the Carnegie Mellon University campus, focusing on ethics, machine learning, data, and interactions. Below are the speakers' insights that influenced my design:

  • Humans have qualities like multi-dimensional trust, emotional depth, and the ability to attribute mental states—beliefs, intents, desires, feelings, knowledge, etc.—to oneself and others, and to understand that others' mental states are different from one's own (Theory of Mind).

  • Technology should not replace us when it cannot be explained.

  • There is a difference between behavioral data on human actions and mind (e.g., using facial recognition to define mood) versus actual data.

Data and Society - Madeleine Clare Elish

Data and Society - Madeleine Clare Elish

Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department - Min Kyung Lee

Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department - Min Kyung Lee

Former Design Manager at Twitter, current Independent Design Consultant - Cennydd

Former Design Manager at Twitter, current Independent Design Consultant - Cennydd

Carnegie Mellon Universtiy — K & L Gates Conference on Ethics and AI

“Trina, Designing Text Technologies for Critical Interpretation” - Anne Burdick

“Trina, Designing Text Technologies for Critical Interpretation” - Anne Burdick

Speaker MiMi Onuoha — an artist and researcher using data and code to explore new forms of storytelling, social critique, and interaction

 

DESIGN PRINCIPLES

1*7EnlmRYBGlxk3Ix8nrIDjQ.png
1*fnUkX3pFW4evZ98wFstaPA.jpg

Venn diagram of Humans and AI

1*ID9FXVCf2rtj2S2ar8KomA.png

Pros and Cons of humans and AI

 

Scoping The Problem

I decided to dive into AI topics related to metaphors/lyricism and Emotional Intelligence. Communicating intangible emotions is challenging between people — even more so with AI. What if humans could communicate with AI in human terms, using metaphors, poems, and phrases to understand one another contextually?

 

Case Studies on Language and Expression

I researched and explored different expressive terms to communicate with AI. I examined existing expressions and probes, such as:

  • Rorschach Test: Psychologists use this test to examine a person’s personality characteristics and emotional functioning. “By asking the person to tell you what they see in the inkblot, they are telling you about themselves, and how they project meaning onto the real world,” says psychologist Mike Drayton for BBC.

  • Arrival: This sci-fi drama directed by Denis Villeneuve explores language, communication, intention, and time. In the movie, humans create an entire alphabet system by studying the communication patterns of aliens.

  • Human Languages: Human languages create associations for understanding context through the composition of compound characters.

Pages of poetry and art from a German doctor, Justinus Kerner’s Klecksographien (1890), inspired by accidental inkblot.

Rorschach test

A scene from the film, Arrival (2016)

A scene from the film, Arrival (2016)

1*PtLqSwgFLAdE98GtllKuHg.png

Aliens alphabet system and communicate patterns. - the film, Arrival (2016)

0.png

Sited from Teaching Culture through Language: Exploring Metaphor and Metonymy in Chinese Characters by Hu, Ying-Hsueh

3.png

German Compound words

Screen Shot 2021-02-13 at 7.43.33 PM.png

Chinese Compound words

Screen Shot 2021-02-13 at 7.43.11 PM.png
 

THE RESULT

I created two speculative design experiences that invite visitors to explore how human language is fundamentally built with metaphors that tell stories and provide context, as well as how we express and communicate intangible feelings. I encourage visitors to think about new ways for AI to communicate more humanely.

Poetic Language_Poem0-1.png

Introduction of my research and speculative designs

Poetic Language_Poem0.png

First Prompt

Poetic Language_Poem6.jpg

At the exhibition “Where are the Humans in AI?” 2018

Poetic Language_Inkblot1.png
Poetic Language_Inkblot2.png
Poetic Language_Inkblot7.png
Poetic Language_Inkblot4.png
Poetic Language_Inkblot6.png
Poetic Language_Inkblot5.png
Poetic+Language_Inkblot3.jpg

Inkblots I made

Poetic Language_Poem1.jpg

Google Image Reader’s interpretations of the inkblots

Poetic Language_Poem2.jpg
Poetic Language_Poem3.jpg

Visitors expression of the inkblots

Poetic Language_Form0.png

Examples of how human languages tell stories through compound words

Poetic Language_Form0-1.png

Second Prompt

Poetic-Language_Form28.jpg
Poetic-Language_Form21.jpg
Poetic-Language_Form22.jpg
Poetic-Language_Form23.jpg