Beta — We’re reformatting our articles to work with the new site, but this is one hasn’t been done yet; it may contain formatting issues. No need to report this. We’re working on it.

Designing Products That Work the Way People Work: An Interview with Kate Gomoll

by Christine Perfetti
on September 17, 2004

Designing Products the Way People Work

Kate Gomoll is a recognized expert in the area of Field Research and Usability
Testing. UIE’s Christine Perfetti recently had the opportunity to talk with
Kate about how she and her team at Gomoll Research & Design conduct field
studies. Here is what Kate had to say about her experiences.

UIE: Many of our clients say they stay away from conducting field research
projects because they are
too expensive and time-consuming
to implement. Why should designers invest the time and resources in field studies?

Kate Gomoll: Nothing replaces the power of direct observation. Designers
can gain so much information from watching people in their actual work environment
and capturing a user’s unarticulated needs. The work we conduct takes place
in the participant’s environment — such as home, work, stores, banks, and hospitals.
Our findings are based on realities, not preconceptions. As a result, design
teams can get much closer to their customers. Users will “show and tell” them
things they would incorrectly consider unimportant in a focus group, interview,
or usability test.

Also, in our experience, what people say rarely matches what they do. People
don’t remember the steps of a process, especially if it’s something they do
all the time. They may tell you something is easy, but when you actually observe
users doing that task in a field study, you can see all the problems and inefficiencies
they didn’t recall or couldn’t articulate.

In your full-day seminar at UI9, you and Ellen Story will demonstrate
how to plan and conduct a field study. Can you describe the major steps involved?

Before the field visit, we develop a participant screener to ensure that the
people we are visiting are representative users. We always make sure that key
stakeholders agree on the mix of participants we will visit so there is no debate
about the appropriateness of the participants after the data analysis. We also
create a detailed study plan with the session script and all of the materials
we need for the study.

A typical site visit lasts 1.5 – 2 hours. We spend the first
part of our visit asking users to prioritize their most important and frequent
tasks. We then observe them completing their most high-priority tasks. As we
watch the user, we capture the steps, constraints, and tools needed to accomplish
their tasks and any social interactions that take place. We also note any process
inefficiencies or opportunities for improvement.

Our sessions are a blend of interview and observation. We observe the user
as much as possible, but we also gather data from contextual interviews. We
have a basic script for the session, with interview questions and observation
opportunities. But, we move around in the script, asking questions when they
are relevant and weaving observation opportunities between questions. That’s
the best way to make sure we get as much data as possible for our efforts.

Do you have any tips for conducting the contextual interviews with

My colleague, Ellen Story, and I have worked out a really
good approach to “tag-team interviewing.” We have a study script that we work
from, but we are quite flexible with it. Ellen will ask a question, and while
the participant is answering her question, I will look through the script for
the next logical question, based on the participant’s response. This helps the
session feel more natural and informal. It also speeds things up, because we
can do the same thing with our observation opportunities.

Having our questions worked out ahead of time and reviewed by stakeholders
also helps us avoid asking leading questions. Of course, the ability to work
with people and draw out their honest responses takes some practice and finesse.
The more we do this kind of work, the better researchers we become.

How many customers do you recommend visiting in the field? 

We recommend 5-8 participants within a segment. So, the total number depends
on how the design team decides to create segments.

How do you determine how many user segments are necessary? 

Before we determine the user segments and the number of participants, we typically
ask the clients to assemble a group of key stakeholders. This group typically
includes people from the development team, marketing, engineering, usability/user
interface, and domain experts. We make sure that this group has input and buys
into the definition of a segment, as well the criteria for recruiting.

There are three main questions we ask stakeholders:

  1. How many different user groups are there in your target market?
  2. How do you distinguish the user groups from each other?
  3. How greatly do you think the usage patterns and preferences will
    differ among groups?

Before we conduct any of the field visits, we gain agreement from all of the
key stakeholders about the sites and people we visit, so that when the data
rolls in, nobody is telling us "you visited the wrong people."

 Many design teams have tried field research
but are unsuccessful when the time comes to interpret the piles of data they’ve
collected. How do you go about analyzing the data gathered from field observation?

At Gomoll Research & Design, we follow a 3-stage design process of Investigation,
Analysis and Design. During the Investigation Phase , we create user
profiles and personas and visit customer sites to understand the goals, attitudes,
and workplace of the intended users. This is the stage where the field research
process fits. When we’re developing our site visit materials, we think about
what deliverables we plan to develop after the study. For example, a typical
field study will yield scenarios, user profiles, storyboards / task maps, incremental
reports, and a summary report.

During the Analysis Phase, we’re looking for patterns in the data. We focus
our efforts on finding these key usage patterns and prioritizing design opportunities.
We create deliverables that will highlight the patterns and opportunities in
a compelling way. This phase includes Workflow Modeling, Task Analysis, and
Usability Benchmarks.

Finally, during the Design Phase , we use an iterative process to
develop user models and begin prototyping. We gather user feedback and conduct
usability tests to help us evaluate and revise our designs.

During each phase, we’re careful to gather and incorporate user feedback so
that we design products that work the way people work. That is the key.

(Editor’s Note: You can read more about Gomoll Research & Design’s Process
on their web site.

Can design teams gather insights from field visits they can’t uncover
with usability testing?

By making direct observations, design teams can discover product opportunities
they may never have imagined if they had only conducted usability tests. We
observe users doing real work in real time, which enables us to paint an accurate
picture of the how users interact with a product or process.

For example, we worked with Sonic Solutions to design MyDVD 4, the leading
DVD creation software, to better understand how people create DVDs today. To
accomplish this, we conducted observational research in Sonic’s customers’ homes.
By observing their customers firsthand, we uncovered common user goals, behaviors,
task flows, and the critical problems users encounter when creating DVDs.

What led Sonic Solutions to want to conduct field research? 

When we initially met with Sonic’s executives, they were already aware of the
power of field research. Their biggest challenge was deciding what kind of users
to study. The design team believed that the majority of their users were *early
adopters* of DVD creation technology, willing to do whatever it takes to become
technically proficient with relevant video and PC hardware and software. For
the next generation MyDVD, they wanted to target a larger mass market. But they
weren’t sure how well field research on early adopters would help with a design
for mass market users.

How hard was it to find customers who would let you in their homes?

To find customers, we sent email messages to registered customers. As is typical,
our acceptance rate was approximately 10%. Since the list of registered customers
in our target locale was quite large, the recruiting was fairly easy. We are
often surprised at how willing people are to invite us into their homes. In
this case, most of our participants were anxious to show us their DVDs because
they were so proud of them. They were also eager to positively impact the next
release of MyDVD and to get a few usage tips.

Were there any findings that surprised the team? 

One of the biggest surprises was that the design team’s assumptions about early
adopters were not always accurate. We visited users representing a variety of
profiles, from a professional videographer, to a teenager whose parents purchased
a more powerful PC and related video equipment just to get a VHS tape of his
grandfather archived for perpetuity. Most users had a keen interest in video
or film media, but little of the supposed relevant technical expertise of an
early adopter.

Whether the MyDVD application came with the PC, the camera, the DVD burner,
or the conversion device, the users were confused by the multiple shrink-wrapped
applications that came with their purchases. Initially, they didn’t understand
the total task flow or which parts of which applications to use to accomplish
each step in the task. The MyDVD application was just a piece in a very confusing
puzzle, not the focus of the user’s attention.

Can you share some of the design improvements that came from the research?

Our research findings led to immediate product design improvements, such as
the creation of a “MyDVD Tutorial” to help new users understand the technology,
the product, and the DVD-creation process. Our analysis made it clear that all
user profiles would benefit from this tutorial, and that the tutorial should
cover many aspects of the DVD creation process, even those not directly supported
by MyDVD.

Thanks, Kate.