Beta — We’re reformatting our articles to work with the new site, but this is one hasn’t been done yet; it may contain formatting issues. No need to report this. We’re working on it.

Honing Your Usability Testing Skills: An Interview with Ginny Redish

by Christine Perfetti
on September 9, 2004

Honing Your Usability Testing Skills

If you’ve ever done any usability testing, then you’ve been affected by
Ginny Redish’s pioneering work. Ginny is a world-renowned usability expert and
co-author of the books, “A Practical Guide to Usability Testing” and “User and
Task Analysis for Interface Design”.

While preparing for Ginny’s full-day seminar at the User Interface 9 Conference,
UIE’s Christine Perfetti had the opportunity to ask Ginny about her thoughts
on the best practices surrounding usability testing. Here is what Ginny had
to say about her experiences.

UIE: How has usability testing evolved as a technique over the past
10 years?

Ginny: More and more companies and government agencies are doing usability
testing. And, they are doing more of it, starting earlier in the process, and
testing iteratively. Over the past 10 years, usability testing has become more
integrated into the development process, especially in web design. As a result,
we are doing more informal, formative, diagnostic usability testing now than
ever before.

Has your philosophy changed at all since you first began usability

No – and yes. My philosophy of usability testing has always been that it is
the best way to find out how well a draft or prototype or product is doing for
its users. I’ve always believed that usability is about helping designers and
developers create products where users can quickly and easily find what they
need and understand what they find. I’ve always believed that usability specialists
should be part of the product team from the beginning and should do testing
with the team – not as the “usability police.”

But my philosophy of how to do usability testing has also expanded over the
last 20 years. When I started out, almost all usability testing was done in
a formal lab with a very “hands off” approach to interactions between participant
and facilitator. Today, the line between usability testing and field studies
has blurred quite a bit. Typically, today, I sit with the participant. Depending
on the stage the product is in, I may engage in much more dialogue than I did
when I started out. I’ve done usability testing in conference rooms and cubicles;
I even did one this summer in an airport hangar.

Your work focuses on helping usability practitioners to improve their
usability testing skills. What recommendations would you give first-timers?
Are there any common mistakes you see with design teams just starting out with
usability testing?

Some aspects that I find design teams often need help with are:

  • Thinking about the issues – what you want to learn from the usability test
  • Writing good scenarios – that test the web site or product without giving
    away too much

  • Facilitating comfortably – knowing when to talk and when not to, how to
    ask neutral questions, how to keep participants thinking aloud

  • Taking good notes without missing anything critical
  • How to report results so that the right people act on them

That’s why, in my full-day seminar at UI9, we’ll work on all the aspects of
planning, preparing, facilitating, note-taking, analyzing, and communicating

What recommendations do you have for usability professionals who want
to get their development team on board and excited about usability testing?

Involve them. Be a team together. Invite them to observe. If you have an observation
room for them, put out candy or other food as an incentive for them to come.

Have someone in the observation room with them to monitor their conversation
and, if necessary, join in to keep them from jumping to conclusions too quickly.
If you do not have an observation room, set up a schedule so that you only have
a few observers in the room at one time. Give them brief instructions on how
to behave.

In either case, give them note-taking forms and a bit of instruction on how
to use them. Involve them in a debriefing right after the testing; involve them
in helping to find good solutions to the problems. Include some positives in
the report.

In your experience, what are the typical costs associated with usability
testing? What are the budgeting considerations?

I’m always surprised when people think usability testing costs a lot of money.
Yes, it costs money – but it’s usually a tiny fraction of the money being spent
on the technical side of a project. Whether these are internal costs – or hard
dollars you give out to someone – depends on how much you can do inhouse. The
major costs for usability testing are

  • Time for internal people (and a usability consultant if you need one) to
    plan, prepare, conduct, analyze, and report

  • Recruiting and paying participants (You may find it is less expensive to
    use a recruiting firm than to have internal people spend lots of time on the
    phone trying to find the right people.)

  • Facilities (if you rent a lab, for example) – but you can also do usability
    testing in a conference room or even at people’s desks

Many usability practitioners believe that five to eight users is enough
to find the majority of usability problems on web sites. In your experience,
how many users are enough for testing?

“How many users” is one of the great controversies in usability testing today.
I think it’s a false controversy because there is no one answer. The number
you need depends on the type of testing you are doing, where in the development
process you are, how many different types of users the web site or other product
has, and a few other factors. I have slides in the seminar that explain all
the factors you need to consider to decide on how many users.

The type of usability testing that we’ll focus on during the seminar day is
the type that most companies and agencies are doing – testing prototypes of
web sites or documents or software in the design stages for the purpose of finding
and fixing major problems. My experience over many, many years of that type
of usability testing is that you’ll find the major problems with relatively
few users (I usually say six to 12 ).

Many design teams attempt to use heuristic evaluations as a less expensive
alternative to usability testing.  Where are your thoughts on this method?

Usability testing is only one of many techniques in the usability professional’s
toolbox. You can use a heuristic evaluation (having one or more experts review
the product) to catch major and obvious flaws in a product – if there are any.
If the developers are willing to accept the results, they can bring a better
product to usability testing. However, a heuristic evaluation is not an alternative
to usability testing. A heuristic evaluation is just a prediction of what users
will do. Until you see the real users, you don’t know whether those predictions
are right. Only usability testing shows you where the real problems are.

You recently conducted two usability studies with Mary Theofanos and
colleagues at the National Cancer Institute researching how vision-impaired
users interact with web sites. What were some of your key learnings from this

Our first study was with 16 blind users; our second study was with 10 low-vision
users. In both, we were watching and listening as our participants used web

From our first study, we came up with many specific guidelines to help web
designers and developers really make sites accessible – not just meet the letter
of the law. One seemingly simple but very important learning is that many blind
users do not know what Skip Navigation means. They want to skip over “all that
stuff” at the top of each page, but they don’t click on the link that would
help them do that. You can help them by changing the name of the link – and
I’ll talk about how well different names worked.

Our second study was even more interesting because we found that the needs
of low-vision users are so diverse that simple solutions are not going to help
everyone. Adding accessibility on after a web site has been developed is not
working. We need a new paradigm for thinking about “experience equity” – making
web sites work for everyone.

Thanks, Ginny.  

Besides being a usability expert – Ginny has a way with words.
In her virtual seminar Writing Web Content that works, Ginny shares how to get the optimum content with the right words, with the right design, in the right place to meet site visitors’ needs.