Menu

Beta — We’re reformatting our articles to work with the new site, but this is one hasn’t been done yet; it may contain formatting issues. No need to report this. We’re working on it.

Usability Labs: Our Take

by Tara Scanlon
on July 1, 1999

Many clients ask us whether they should build a usability lab—a special set
of rooms dedicated to usability testing, often with a one-way mirror and sophisticated
AV equipment.

We tell them, well…it depends.

Though we do hundreds of usability tests every year, we don’t have a lab.
We do most of our testing in conference rooms and offices ? or sometimes less
than that. Last summer, we rented a tent in the middle of a field and successfully
tested 91 users in it.

This doesn’t mean we disagree with people who place great value in usability
labs. We just have a different perspective.

On the One Hand…

There’s little question that a usability lab is a good place to test products.
There are clear advantages to having everything in one place and ready to go
whenever testing is scheduled. Having the lab on-site also makes it easier
for the whole team to attend testing sessions.

A usability lab can be a powerful political tool, visible evidence that the
company is willing to spend money and effort to make its products usable.

This need to be visible can encourage developers to make sure upper management
knows the lab is being used. At Lotus, for example, the lab is in a well-trafficked
part of the building and an LCD panel outside the door announces what’s being
tested today. We know one leader at another company who holds all his usability
team meetings in the lab, just so his boss remembers that the lab is there.

Furthermore, in an era where almost no one has a real office, labs represent
golden real estate. (They even have doors!) We visited one company where the
usability lab is literally in a tower that stands above a sea of cubicles.

…On the Other Hand

Some advocates see labs as the only place to garner scientifically valid data.
We’re not sure that’s true or necessary. Usability testing is not really a
statistical exercise, and experience has shown us that the first six or eight
users uncover the majority of serious usability problems—even without a fancy
lab.

Even with a lab, if it’s busy or if someone doesn’t know how to use it, there’s
often the perception that testing can’t be done. This can limit the amount
of usability testing.

Not only that, but usability labs are expensive. With a limited budget, money
set aside to build (or lease) a lab is money that can’t be used to do more
testing; it’s a matter of setting priorities.

Usability testing, with or without a lab, always puts users into an
unrealistic setting—by definition.

In a lab, we’ve found that users sometimes worry about who’s behind that one-way
mirror and what those folks are doing. This fear may have merit: we know about
some misbehavior behind the one-way mirror, such as playing with the camera
controls instead of watching the test, or using the invisibility to catch up
on sleep or to tell other observers what to think about the ongoing test.

By contrast, we’ve found that with our no-lab setup, once they begin the tests,
users quickly ignore observers who are sitting in the same room with them.
And these observers are forced to pay close attention to what’s happening (or
at least not distract the users).

Life Without a Lab

We’ve never felt we needed a permanent usability lab. Our testing setup is
simple and portable, and it adapts well to available space at our office or
at clients’ conference rooms.

We seat the user (or users) at the computer or prototype with the facilitator
sitting nearby. Observers from the development team sit where they can watch
the participant work, and they all have cards or note paper to write down their
observations and questions. The video camera, when we use it, records the screen,
what the user says, and part of the user’s profile. We sometimes mount the
camera tripod on a nearby desk to give us an unobstructed view.

People often ask us if we have trouble controlling observers who are in the
same room as the test participants. This isn’t as much of a problem as you
might think. Even the most rambunctious team members behave when they’re sharing
the room with people who might use their software. But to make sure the users
aren’t harassed, we also enforce strict rules for the observers. •