Poetic Language Between Humans and AI

Timeline
8 Weeks

Year
2018

 

Today, code - numbers and logistics - is the communication medium between humans and artificial intelligence.

What if humans could introduce their way of communication to artificial intelligence?

Poetic Language Between Humans and AI explores how humans might use metaphors, phrases, or poems as the language with AI, allowing a more in-depth communication and understanding of intangible emotions.

Communicating with AI in humans terms

Poetic Language_Poem0-1.png
 

Language Through Poem

Poetic Language_Poem0.png
Poetic Language_Poem6.jpg

Inkblot Prints

The self-generated inkblot is inspired by the Rorschach test, which psychologists use to examine a person’s personality, characteristics, and emotions. “By asking the person to tell you what they see in the inkblot, they are telling you about themselves, and how they project meaning onto the real world,” — psychologist Mike Drayton for BBC.

Poetic Language_Inkblot1.png
Poetic Language_Inkblot2.png
Poetic Language_Inkblot7.png
Poetic Language_Inkblot4.png
Poetic Language_Inkblot6.png
Poetic Language_Inkblot5.png
Poetic+Language_Inkblot3.jpg

Humans and AI‘s Interpretation

I uploaded the inkblot prints on Google image reader. As a result, Google produced codes [numbers, words, logistic]. The audience then looks at the different ways of interpretation between how humans and AI.

Poetic Language_Poem1.jpg
Poetic Language_Poem2.jpg
Poetic Language_Poem3.jpg
 

Language that allows invention

Poetic Language_Form0.png
 

Language Through From

Poetic Language_Form0-1.png
Poetic-Language_Form28.jpg
 

Balloon Compound Words

Balloon Compound Words is inspired by how human languages create associations for understanding context through composing compound characters. Here, the audience makes their own words through a variety of size, wrinkles, fullness, stack, curves, colors, communicating with a rich context.

Poetic-Language_Form21.jpg
Poetic-Language_Form22.jpg
Poetic-Language_Form23.jpg

Exploratory Research

1*0XEDvpU9m6kz-3XZZuKFgw.jpeg

Where are the humans in AI? is the central theme of the research. I started to brainstorm on social interaction topics regarding humans and artificial intelligence. I categorized insights into four themes:

  • Poetic/ Lyricism — Does AI understands metaphors or symbols?

  • Social Status/ Hierarchy — Often, AI is called an assistant. Does the AI world have social status or hierarchy?

  • Manners/Traditions — While the younger generations spend more time with machines and AI, they might lose contact with human intelligence, culture, and roots. Moreover, to see the importance of daily manners and etiquette.

  • Emotional Quotient/Intelligent Quotient/Adversity Quotient — While humans do not value EQ and IQ equally, how does AI see each's importance?

 

Discussion

I joined the humans and AI discussion, regarding ethics, machine learning, data, and interactions, on the Carnegie Mellon University campus. Below are speakers' insights that influenced my design:

  • Human have qualities like multi-dimensional human trust, emotional, and the ability to attribute mental states — beliefs, intents, desires, feelings, knowledge, etc. — to oneself and others, and to understand others' are different from one's own (Theory of Mind)

  • Technology should not be us when it cannot be explained

  • There's a difference between behavioral data on human action & mind (technology like facial recognition to define mood ) vs. data

Data and Society - Madeleine Clare Elish

Data and Society - Madeleine Clare Elish

Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department - Min Kyung Lee

Carnegie Mellon University Human-Computer Interaction in the Machine Learning Department - Min Kyung Lee

Former Design Manager at Twitter, current Independent Design Consultant - Cennydd

Former Design Manager at Twitter, current Independent Design Consultant - Cennydd

An artist and researcher using data and code to explore new forms of storytelling, social critique, and interaction - MiMi Onuoha

An artist and researcher using data and code to explore new forms of storytelling, social critique, and interaction - MiMi Onuoha

“Trina, Designing Text Technologies for Critical Interpretation” - Anne Burdick

“Trina, Designing Text Technologies for Critical Interpretation” - Anne Burdick

K & L Gates Conference on Ethics and AI - Carnegie Mellon University

K & L Gates Conference on Ethics and AI - Carnegie Mellon University

 

Design Principles

With a critically informed stance in AI topics, I defined my design principles.  

1*7EnlmRYBGlxk3Ix8nrIDjQ.png
 

Scoping The Problem

I decided to dive into AI topics related to metaphors/lyricism and Emotional Intelligence. I asked myself questions:

  • What does an AI written poem look like?

  • If there's a group of AI assistants in the same space, what would the social hierarchy be like?

  • Why is the AI portray in our daily life mostly female? like Alexa

  • What would the world be like if AI helps humans develop different intelligence forms and test them in different ways?

  • When humans go to an AI bar, what are the humans feeling? How does stress, sadness, happiness plays in the scenario?

  • What might integrating social metaphor in environments look like?

1*fnUkX3pFW4evZ98wFstaPA.jpg
 

Human Behavior Analysis

Human's solution when facing difficult situation: conversations → decision → Reaction

Q: Why do we choose to have conversations with friends and family?

A: Because of the trust we built, the experiences we had, and the context we already know about each other

Q: How do we trust AI? How does AI understand what's going on with my life?

A: AI gets data about my life through facebook — how often I check others' social media. It gets my behavioral data based on how often I engage with a subject.

1*ID9FXVCf2rtj2S2ar8KomA.png
 

Defining The Problem

Communicating intangible emotions is challenging between people—even more so with AI. What if humans’ could communicate with AI in humans terms, using metaphors, poems, and phrases to understand one and another contextually?

1*r_w_jObRhiblX6PhE2MV-w.jpeg
 

Case Studies On Languages and Expressions

I researched and explored different expressive terms to communicate with AI. I looked at existed expressions and probes like:

  • Rorschach test— Psychologists used the test to examine a person’s personality characteristics and emotional functioning.“By asking the person to tell you what they see in the inkblot, they are telling you about themselves, and how they project meaning onto the real world,” — psychologist Mike Drayton for BBC.

  • Arrival, a sci-fi drama directed by Denis Villeneuve, is about language, communication, intention, and time. In the movie, humans create a whole alphabet system by studying how aliens communicate patterns.

  • Human languages create associations for understanding context through composing compound characters.

Pages of poetry and art from a German doctor, Justinus Kerner’s Klecksographien (1890), inspired by accidental inkblot.
A scene from the film, Arrival (2016)

A scene from the film, Arrival (2016)

1*PtLqSwgFLAdE98GtllKuHg.png
0.png
3.png
Screen Shot 2021-02-13 at 7.43.11 PM.png
Screen Shot 2021-02-13 at 7.43.33 PM.png

(L) Sited from Teaching Culture through Language: Exploring Metaphor and Metonymy in Chinese Characters by Hu, Ying-Hsueh (R) German Compound words (R) in Chinese characters, elements of symbols combine to create a new character and new meaning

 

Experience Design

I storyboarded an experience for audiences to use metaphors, phrases, and poems to communicate the intangible emotions with AI.

1*Mq0Y6VZS-LncH9BCOSMLSQ.jpeg
1*E5bYSe_k1I9YmgYr-ccn8A.jpeg
1*5hak2vm6p7bOuYom-zfyyw.jpeg