An exploration and vision strategy on how we can create interactions with machines that is natural and human-like.

Project Summary

The Project

Our goal was to showcase the power of AI through immersive and interactive experiences that would engage and inspire audiences, while also demonstrating the practical applications of Watson's capabilities. Overall, our artistic vision and business strategy for IBM Watson aimed to change the way people think about AI.

My Approach

With the IBM Design Leadership team, I was responsible for:

• An exploration of industry trends in the AI and Internet of Things tech areas.

• Creating a series of digital illustrations that depict Watson interacting with people and objects in various scenarios to showcase its versatility.

• Create a mural that visually represents the art direction for unique, human-like AI interfaces.

• Observing and capturing interactions between people and the Pepper Bot.

Check out Part 1 of this topic: Design strategy and vision for IBM Watson

Introduction

Science Fiction has always captured the imagination of both scientists and creative people across generations. At IBM Watson, it was easy to tap into this source of inspiration for how our products should look, feel, and behave. It is a competition in the AI world to be the person or company to push technology from Weak AI to Strong and Super AI and have it look, feel, and behave like the AI in Hollywood (easier said than done). But maybe it just needed a more creative approach.

Using my background as an artist, I remembered all the tricks I was taught to create the illusion of life and empathy. It’s actually easy to convince people they are interacting with something real, even when they are aware its not. But what is the underlying pattern that makes this happen?

Communication.

There are several theories on communication represented as models, the most well known being the Shannon-Weaver Model, which explains the interaction of radio and telephones. I chose the Emmert-Donaghy model because it is more complex and abstract than the other models, breaking down the steps into Perception, Motivation, and Reasoning. I thought this would be a good fit for AI.

But how do you translate a communication model into machine logic? How do you apply Perception, Motivation, and Reasoning to programming?

It started with a Pepper Bot

The Watson Lab gave the IBM Watson Platform team a Pepper robot. This adorable little robot is used for commercial, consumer, and academic purposes to assist people in their daily lives.

Over the course of a week, I observed the team interacting with the robot and started understanding their behavior.

I quickly sketched out a “journey” of communication in order to identify patterns and gaps in our understanding. It became clear that our focus should be on a very specific part of the communication model: the machine’s processing.

Creating the Model

Knowing now the focus, I used Emmert-Donaghy model of communication to see how it described processing. There are three phases that information will flow through before the communicator creates a response: Perception, Motivation, and Reasoning.

Using Philip Emmert and William Donaghy’s book Human Communication: Elements and Contexts, I looked deeply into each phase, looking for keywords and ideas that translate into machine behavior.

Perception means that the communicator is becoming aware of something through their inputs. In this phase, the mind is looking for patterns in its surroundings and labeling them.

Motivation is the intent of the communicator. Now that the communicator understands their surroundings, how does it react to it?

Reasoning gathers all of the information and prioritizes multiple outcomes, relying heavily on the communicator’s motivation.

I wanted to explain this concept to my fellow designers and get their feedback, so I created an illustrative diagram that walks through the definitions, gives examples, and shows how machine processing fits in with the entire cycle.

Meeting with the Watson Lab

After bouncing my initial ideas off the Design team, I then showed them to the IBM Watson Lab’s engineers. The engineers were impressed with my ideas and immediately started discussing how they could be implemented with IBM Watson's advanced AI technology. They started brainstorming ways to enhance the user experience by incorporating natural language processing and machine learning into the design.

As the meeting progressed, I could see the excitement building among the team members. It was clear that everyone was passionate about the project and eager to make it a success. The engineers shared their insights and technical expertise, further refining and enhancing my initial design concepts.

Taking feedback from this meeting, I went back and updated the model in digital form. This time, on the recommendation of the Lab, I added Memory which stores decisions and scoring. Each phase will not only pull from the one before it, but also from Memory.

Applying the idea

Using the knowledge gained from the research I conducted, I created a model that could be used in various healthcare settings. I explored user stories within the Healthcare industry while working through each phase of the cycle. I picked one and mapped the interaction to the machine’s “thinking”.

Outcomes

The Watson Lab team was pretty happy with the idea and started working on developing around this model. This meant that we needed to patent the idea quickly so that IBM could start working on it openly.

Several projects and products across the Watson portfolio were developed and inspired by this discovery work, including Enterprise Design Thinking for AI.

IBM Watson AI Services | Enterprise Design Thinking for AI Guidelines | Watson Intu | Everyday Ethics for Artificial Intelligence Guidelines | TJBot | Human-to-Machine Communication Model Medium Article | Communication Model for Cognitive Systems Patent | AI Essentials Framework