Conducting Inherent Value Testing
In last week’s article, I summarized the benefits of Inherent Value Testing,
a simple usability testing technique that can help you measure how your site
communicates your product’s value.
If you’ve conducted a usability test in the past, it’s simple to put together
an Inherent Value Test. As with all testing, you start with recruiting the
right participants. You’ll need two groups: people experienced with the site
and people who you will introduce to the service during the test.
Let’s say you want to understand the value of your HR department’s intranet
site. You would look for two groups of employees: those who are using the site
on a regular basis and those who have never (oralmost never) used it before.
You’d want to recruit a minimum of six from each group, to ensure you have
enough users to see important behavioral patterns.
We start with the experienced group in the first phase. We want them to interact
as "naturally" as possible with the site, so we may ask them to bring
some of their own work or data to use.
For example, when testing your HR site, you might look for people who currently
have questions about their benefits or need to make changes to their retirement
plan distributions. Resolving their own issues gives the participants a "focus" for
We’ve found having a focus works better than just a random tour without any
focus. We get to see how users work with the site as they discuss it. Their
commentary is easier to understand and more accurately represents true issues
with the site.
Don’t be afraid to guide the users to visit specific portions of the site.
Feel free to ask them what they think is important about it. In some types
of usability tests, "leading the user" is inappropriate. However,
in Inherent Value Testing, it’s a useful technique for discovering valuable
qualitative perspectives on a variety of portions of the site.
You also can ask the participants to visit a competitor’s site in addition
to the site you are studying. Having them take you on a tour of the competitor’s
site, describing what they like and what they don’t like, gives them a point
of comparison and highlights important benefits and deficiencies of your site.
Phase I: Extracting Benefits from Participants
As their working through the site, you want to ask them what they like and
don’t like about the service. Focus your discussion away from the design specifics,
such as fonts or colors, and more towards the utility of the service.
In our study of the HR site, we might ask: What parts of the site do they
like the best? What part is most helpful? What have they used before? What
did they do with that portion of the site?
Leverage any discussion on their comments. If the user says, "I like
how complete the information is on this page." You can respond with, "What
do you think makes it complete information?" It can be difficult for participants
to really analyze the site at first, so persistence will pay off.
At the end of the test session, just before you send them away, you can ask
two useful questions: "What are two things you really liked about the
site?" and "What are two things you really disliked about the site?" The
most important answers are the ones that just roll off their tongue. In our
experience, the more they have to think about it, the less important the like
or dislike turns out to be.
When you’re done with the first phase, you should easily have identified patterns
in what people like and dislike about your offerings. You’ll also have seen
which features people aren’t discovering and how they react to them the first
time. This feedback can help you brainstorm ideas on how you let users know
about these hidden features and benefits.
Phase II: How Do New Users View the Benefits
For the second phase, you’ll bring in your ‘new’ users. As with the experienced
participants, you’ll want to use test tasks that are as natural as possible.
If you can ask them to bring the same work as the experienced users, that will
However, if bringing in work isn’t feasible, you can ask the participants
to ‘role play’ in scenarios similar to those performed by the experienced users.
We’ve found that quality role playing can reveal many issues about a design.
For example, on the HR site, you could ask them to pretend that they are about
to have a new baby and ask them to investigate the company’s family leave policy
and procedures. Make up any appropriate but
necessary details, such as the baby’s due date, but let them use their own
situation as much as possible, such as their real manager and upcoming deadlines.
As their working with the site, you want to look at each page closely. Do
they visit the same pages as the experienced users? Do the pages reveal the
same benefits that you heard from the experienced users? If not, what benefits
are *they* seeing from the site?
Viewing the same site through these new users’ eyes is often enlightening.
For many sites we’ve studied, it’s a very disappointing experience the first
time, as we realize how poorly the site communicates the inherent values. However,
subsequent testing is encouraging as we see how simple design changes can completely
change the new users’ perspectives of the site.
Analyzing the Results
Once you’ve completed both phases, you want to look at the behavioral patterns
in each phase. You’ll also want to look at the differences between to the two
You’ll want to ask yourself, "If the phase 2 users saw the same inherent
value as the phase 1 users, what would be different?" You mayrealize that
you’d see more sales or reduced call center costs.
It’s very likely you can come up with a fairly accurate estimate of the increased
revenues or decreased costs, by estimating the number of people who fit into
the ‘new user’ category. Multiplying that estimate by the average increase
in revenues or savings in expenses will give you a powerful estimate.
You’ll also have new information to guide future development priorities. You’ll
know how much to invest in new benefits and features and how much to invest
in making the existing features more visible. Often, the latter can take a
smaller investment to yield good results.
Inherent Value Testing is a straight-forward variant of a basic usability
test. But don’t let its simplicity fool you. It’s an extremely powerful addition
to your usability toolbox. •