article by David Malouf on interaction design

Foundations of Interaction Design

by David Malouf on 2007/08/23 | 

Somehow, products, services, and systems need to respond to stimuli created by human beings. Those responses need to be meaningful, clearly communicated, and, in many ways, provoke a persuasive and semi-predictable response. They need to behave.

This basic definition of Interaction Design (IxD) illustrates the common threads between definitions crafted by esteemed designers Dan Saffer¹ and Robert Reimann² as well as the Interaction Design Association³.
It’s also important to note that Interaction Design is distinct from the other design disciplines. It’s not Information Architecture, Industrial Design, or even User Experience Design. It also isn’t user interface design. Interaction design is not about form or even structure, but is more ephemeral—about why and when rather than about what and how.

For any design discipline to advance, it needs to form what are known as foundations or elements. The creation of such semantics encourages:

• better communication amongst peers
• creation of a sense of aesthetic
• better education tools
• exploration

There are other reasons, but for now these seem sufficient for a discussion about foundations.

What Are Foundations?

“Foundations” first came to my attention while preparing for Masters of Industrial Design program at the Pratt Institute in Brooklyn, NY. The program was built by Roweena Reed Kostellow based on her educational philosophy of foundations (as detailed in the book Elements of Design by Gail Greet Hannah⁴).

To Kostellow there were six elements that made up the foundations of Industrial Design: line, luminance & color, space, volume, negative space, and texture. Mixing and experimenting with these was at the heart of designing in the 3D form discipline. Students at Pratt explored these foundations in a year’s worth of studio classes. They would press boundaries and discuss relationships while critiquing abstract and real projects.

I’m not the only person ever to think about this issue though I propose that we think about it differently. Dan Saffer, for example, in his book, “Designing for Interactions”[5] has a great chapter on what he calls the Elements of Interaction Design: Time, Motion, Space, Appearance, and Texture & Sound. Dan’s elements concentrate on what I would call the forms that carry interactions, but to me they are not the form of an interaction, except maybe time.

If there are indeed foundations of Interaction Design, they need to be abstracted from form completely and thus not have physical attributes at all.

Foundations of Interaction Design

Time

“Time” makes interaction design different from the other disciplines of user experience (UX). It is the wrapper of our experience of an interaction and must live over time.
But Time is not a single foundation in Interaction Design. There are too many interrelated facets of time to be manipulated. And as we all learned Time is relative; it is fungible; and exists on many axis all at the same moment. Let us consider three time related foundations of Interaction Design:

Pace

Interaction design is the creation of a narrative—one that changes with each individual experience with it, but still within constraints. For example, if I’m using an email client, I’m not going to turn on a stove burner during the process of writing an email.

Narratives have pacing. We experience that most clearly when we watch a movie. A great movie will have you coming out of a theater having never looked at your watch. Pace is also a part of interaction design, but in some cases a good experience may have you looking at your watch—hopefully not out of boredom, but because you need to know the current time so you can complete the goals of the interaction.

The way I think of pace in interaction design often correlates to how much can I do with any given moment. And not just how much can I do, but how much I have to do before moving to the next moment. For example, I can have a single really long form where all of my checkout information is presented in one presentation when I’m buying something, or I can separate different components of the checkout process into more discreet moments.

While it might take the same length of time to complete either experience because the number of form fields is the same, the experience of the pacing of these designs is quite different. Further, it has been argued that one long form is more efficient, and conversely that separating a form into chunks is more manageable. Maybe that means that the total positive experience needs to consider other things beyond efficiency for its own sake.

Reaction

A simpler way that we design for time in interaction design is “reaction time.” How long does it take for the system to produce a reaction to an event? We’ve all seen our cursor change to an hourglass or the proverbial progress bars as we wait for the system to do what we asked, but there are other reaction time considerations.
Actions done in real time (synchronous) have a level of relationship to the moment, while actions that seem to happen in a black box and come back later (asynchronous) lack that relationship. However, because some systems take time, we need to be cognizant of how we communicate these different types of reactions.

Context

Every major foundation element like time should probably have a “context” sub-element. What this means is that there is always something about the human being in the interaction that would change the course of the design itself. In the case of “time,” we cannot design any application without understanding and exploring the meaning of how much time a human will be spending in direct contact with the system.
How much time we spend with an application and how long we are in relationship to it inform our designs and also participate in the experience we create.

Alan Cooper & Robert Reimann in About Face 2.0⁶ speak about the context of time as the concept of “posture.” There are four postures:

• Sovereign – an application that takes our full attention.
• Transient – applications in the periphery of our attention that call us for short moments.
• Daemonic – alerting systems
• Parasitic – support interaction mode for both sovereign and transient applications.

Metaphor

Metaphor is a literary device which uses one well-understood object or concept to represent with qualification another concept which would be much more difficult to explain otherwise. The virtual nature of computers requires that we bring tangible metaphors to bear to help people understand the vagueness of it all. What type and how many metaphors we use directly impact the quality and emotional connection we have for a product.

A favorite metaphor is the trash can or recycle-bin (pick your OS). The idea your files are in waiting in some virtual “bin” or “can” so that if you were mistaken you can dig through the trash(Ick!) and recover them is ingenious. Of course, you can always “empty” it, making whatever was inside irrecoverable. The metaphor works well for most people mainly because of its preciseness and flexibility with the real. In thinking about the qualities of the metaphors for a bin/can between Mac OS and Windows, one might wonder if the nature of a trash can’s “dirtiness” makes it less likely that we will dig files out than recover files from the recycle-bin.

All metaphors break down at some point; where these metaphors break is how we get things into them. We still use the term “delete” to express how we add something to that bin or can. We don’t delete things into our real trash cans, do we? Despite the break down of the metaphor (and every computer metaphor does break down at some point), it still is tangible enough for us to grasp.

But sometimes metaphors go too far. They require a chasm wider than our ability to imagination. The literal desktop seems to make sense and has been tried in the past. If I have a blotter, a file cabinet, an inbox, a calendar, etc. laid out quite beautifully on my screen, I can call my objects files, use a notepad, keep my messages in an inbox, and keep appointments on a calendar, right?

But metaphors appear to succeed best when they are imprecise and the user has to fill in the gaps from their own understanding. Thus, we have an adaptation of that desktop metaphor on our computers today.

The interaction designer needs to strike this balance, cautiously using the metaphors of their predecessors and building on top of them, so long as the original (maybe convention-setting) metaphor can withstand the new direction.

Abstraction

Working in tandem with metaphor, Abstraction relates more to the physical and mental activity that is necessary for an interaction to take place. I first started thinking about abstraction after reading an article by Jonas Lowgren⁷ on what he has termed “Pliability.” After reading the article and using the term a few times in talks and discussions, it occurred to me that Jonas was really speaking about how abstracted an interface is from the response of the product.

By most accounts almost everything on a PC is pretty abstracted because you have two primary interface points for input—mouse and keyboard. Some people have placed their monitor inside of some sort of touch device lowering the level of abstraction for some types of interactions, mainly drawing. Still, most of us type, point, click, and move the mouse around on the screen.

Let us focus on “mousing”. We are looking at a monitor where there is a cursor (an icon) we were taught is related to the mouse. Without looking at that mouse (usually) we move it and in whatever direction we move the mouse, the icon on the screen (usually an arrow) moves. Well, sorta. Right and left seem to work, but moving a mouse away from us moves the cursor up and moving it towards us moves the cursor down playing on the metaphor of perspective possibly.

Then when we get the icon over a target, we click a button on the mouse. This is a strong level of abstraction. The mouse, monitor, and CPU work in unison to create a series of effects that communicate to the connection between the three devices. But the connection is very very abstract and must be learned.

Even in moused behaviors there are different levels of abstraction as well. My favorite comparison is between Google Maps and MapQuest. What makes Google Maps a success is that by mousing down and moving my arm I can change the focused area of the map. It has a very quick reaction time (see above), but the type of motion—moving my arm as if moving a piece of paper in my focused line of sight—is much less abstracted than in MapQuest, which is to simply click on the border or on the map (assuming the correct mode is set). Now one might say that the click is easier (a less complex set of behaviors), but this is more abstracted, arguably less engaging, and definitely less accurate. This makes Google Maps (and copycats) a much more pleasing and effective interaction.

Systems are both becoming highly complex and highly integrated into our lives. Many systems are losing abstraction completely, and not always for the better, while complexity is increasing abstraction of information. This is why everyone is so fascinated with touch-screens of late. They quickly reduce the level of abstraction for interacting with a computer device.

Other new and popular technologies will create challenges for the next wave of interact designers.. The expanding world of spatial gestures, RFID, and other near-field communications technologies create interaction experiences basically increase abstraction without any device to interface with directly. For these, we have not found similarly effective metaphors to guide the user’s understanding of the abstraction as we have for the mouse.

Negative space

All good design disciplines have a form of negative space. In Architecture and Industrial Design, it is the hollowness or the space between solids. In Graphic Design, it is “white space” what is left without color, line, or form—literally the white part of the paper to be printed on. Sound design uses silence, and lighting design looks at darkness.

So what is the negative of interaction?

There are many places where you can “lack” something, or, more accurately, there are many layers. Are we only talking about the product action? What about our action? What about the space in between either entity’s action?

Pause – So clearly a moment in time where no action is taking place by anything that is part of the interaction experience. Often in interaction design we try to fill these gaps, but maybe these gaps are useful.

Cessation of thought – What if doing nothing created a reaction from the system? Well, one student thought this up with BrainBall (http://w3.tii.se/en/index.asp?page=more&id=4) at Sweden’s Interaction Institute (http://w3.tii.se/en/). As you think less, the ball moves more.

Inactivity – Doing nothing, or the product doing nothing in reaction to an action may be a negative occurrence. This differs from pause, but in this case inactivity is the reaction to activity as opposed to just a cessation of activity.

Well, whatever the negative space of interaction design is, it isn’t.

Intersection in Interaction

Unlike form-creating design disciplines, interaction design is very intricate in that it requires other design disciplines in order to communicate its whole. For that reason, interaction design is more akin to choreography⁸ or film making than music or costume making. The foundational elements above only belong to interaction design, or are re-defined to be explicitly for interaction design.

For example, the use of color is an aesthetic tool and a functional tool that can enhance or detract from communication of core interaction styles. Language or semiotics as tools for communicating through another discipline called narrative or story telling also come together and make for a better interaction experience. Further, for many experiences, information architecture is required for the preparation and arrangement of information before the interaction can be created.

As Dan Saffer points out (see above), motion, sound, appearance, texture, and sound all make up the form and are used to create patterns of time, abstraction and metaphor.

It is the interaction designer’s attempt to manipulate these four foundations that separates the practice from industrial design, architecture, graphic design, fashion design, interior design, information architecture, and communication design.

In the end, interaction design is the choreography and orchestration of these form-based design disciplines to create that holistic narrative between human(s) and the products and systems around us.

References:

¹Reimann, Robert. “So you want to be an Interaction Designer”

²Saffer, Dan. “A Definition of Interaction Design”

³Interaction Design Association. “What is Interaction Design?”

⁴As captured in this recent book: Hannah, Gail Greet, Elements of Design: Rowena Reed Kostellow and the Structure of Visual Relationships, New York: Princeton Architectural Press, 2002.

⁵Saffer, Dan. Designing for Interaction: Creating Smart Applications and Clever Devices, New Riders, 2007.

⁶Cooper, Alan and Reimann, Robert, About Face 2.0, Indianapolis, IN, Wiley Publishing, Inc., 2003.

⁷Lowgren, Jonas. “Pliability as an experiential quality: Exploring the aesthetics of interaction design,” Artifact 12:1 (April 10, 2006): 55–66. (republished on the author’s website)

⁸Heller, David (NKA Malouf, David), “Aesthetics and Interaction Design: Some Preliminary Thoughts.” (ACM membership required), Interactions 12:5 (September-October 2005): 48-50.