Producing Great Search Results: Harder than It Looks, Part 2

by Jared M. Spool

Creating an effective search results page takes hard work. In the first installment of this article, I introduced how designers need to understand the users’ tasks and ensure every result delivers great scent. In this installment, I’ll elaborate on the specific principles of the scent of information that play an important role when designing search results.

Prevent Pogosticking

The search results page is just a flavor of a gallery page—one produced dynamically from the query. One important property of gallery pages is they need to eliminate the need for the user to pogostick. This is just as true for search result pages.

Pogosticking is what happens when a user jumps from the gallery page to an individual content page and back, like a child on the spring-based toy of the same name—up and down, up and down. At first glance, letting a user bounce between the search results, to locate the specific content they seek, may seem like a good idea. After all, they can explore the offerings and find the one that best fits their needs.

However, our research has shown otherwise. When we’ve studied e-commerce sites, we found 66% of purchases on sites happened without any pogosticking at all. This meant that users were deciding what to purchase from the gallery page, and not clicking into the product detail page until after they’d decided what to buy.

In fact, the more they pogosticked, the less likely the session would result in a purchase. Pogosticking doesn’t promote exploration of the offers—it gets in the way of it. We found the same behaviors when we studied non-e-commerce sites: the more pogosticking we observed, the less likely the user found content they were happy with.

The best search results pages will prevent pogosticking by providing the relevant content before the user chooses a specific result.

Most Relevant Links Should Be First

As we watched users interact with search results, we quickly noticed a recurring pattern. Users lose momentum as the apparent relevancy of the results go down.

They scan the results from top to bottom. As they encounter each result, they make an assessment on whether it’s close to what they are looking for. As they encounter a result that seems less relevant, they start to slow down.

Then, an interesting thing happens. At some point, they start to look back up at the top of the results, only concentrating on that top area. This is when they’ve hit their relevancy threshold.

The best sites keep the most relevant results at the top. This is a challenge, as relevancy is very related to the task. If the relevancy depends on a sortable or filterable attribute, such as price or location, then providing tools to do that can be helpful. However, that means the tools themselves have to be easy to find and use.

For example, Kayak.com uses prominent sliders in their left margin to filter, and clearly displays sorting criteria at the top of the results listing. The efforts of skilled visual and interaction designers can make a big difference in the effectiveness of page.

Eliminate the Wacko Results

Nothing slows the user’s scanning momentum more than encountering results that are irrelevant, or as we like to call them, “Wacko.” (We started calling these wacko results when one of our test participants kept repeating, “I can’t believe these wacko results.” The name has just stuck with us.)

The problem with Wacko results is users can’t understand why the system is presenting them. They don’t realize that it’s algorithmic. That the engine is using some magical formula to retrieve the results and, to the algorithm, these results seem very likely related.

Instead, many users view it as a digital equivalent of Tourette’s Syndrome, where the system just spits out random items, unrelated to their search. Things are going well, then all of a sudden, something completely wacko. Why does that happen? The users don’t know.

As soon as they encounter a wacko result, it reduces their confidence in the search. For some users, this makes it unlikely they’ll continue using Search as a reliable method for completing their objective, since it works unpredictably.

For example, in one study, a shopper, looking for a low cost camcorder, searches for ‘Digital Camcorders on BestBuy.com. He clicks on a link that says “See all 827 results”, and sorts the list with the lowest priced items at the top. The shopper, expecting to see a list of the lowest price camcorders Best Buy offers, is surprised when they encounter a page with 15 results, none of which are cameras, but instead something called a whiteboard faceplate and digital camera pouch. Disappointed and confused, the shopper left the site without a purchase.

The best teams work hard to eliminate any wacko results. They regularly try the most popular queries to ensure the first page of results contain highly relevant listings.

Put More Results On Each Page

Another similarity that search results pages share with gallery pages is how users react to “chunking”. Chunking is when the results are displayed in groups, such as ten per page.

When we observe people searching, we see they rarely look beyond the first page of results. Getting past the second or third page never happens. We see users become flustered when there are eleven items, but only the first ten are displayed on the initial results page, requiring a new page to see the remaining item. Nobody seems pleased when this situation occurs.

Interestingly, people never complain because a search results page contains too many results. We’ve seen sites with 20, 40, and even 100 results on a single page and the users of those sites never were flustered or concerned.

So, limiting each page to ten results doesn’t seem to be for the benefit of the users. As I mentioned above, users will look down the list until they’ve found something really relevant or reached their relevancy threshold.

The best teams focus on delivering very relevant results and don’t force the user to continue looking on a second page. Orbitz, for example, put 25 hotels on their results page for a recent search on Aruba. They didn’t force the user to go to a second page to see the later listings.

Handle “No Results” Gracefully

What happens when the site doesn’t have the content the user seeks? Ideally, you want to clearly tell the user, “Sorry, but we don’t have that.” Oddly, few sites do. Instead, they often produce a list of random results, none of which complete the user’s quest.

For example, a shopper looking for a Wii game unit at Staples gets a search results page that doesn’t acknowledge the product exists and brings up a single result for a WiFi finder. It’s probably unreasonable to expect Staples’ design team to have a message for every possible product they don’t carry. However, the Wii is a popular and difficult to find product and Staples.com probably gets many shoppers hoping they might carry it.

Contrast this with the hotel reservation site for Best Western. A guest entering Newburgh, NY into their site gets a list of nearby hotels, even though Best Western doesn’t have any hotels in Newburgh. The team at BestWestern.com have gone out of their way to recognize cities they don’t have properties in.

On some sites, they initially look like they have the desired content, only to inform the user later that they really don’t. On Best Buy, the initial search results look like they have a Wii, but clicking on the item says sold out. From a developer’s perspective, this makes sense as Best Buy does have a page for the Wii. But the shopper doesn’t care if the site has a page for the device, they just want to buy the device. Best Buy’s competitor, CircuitCity, states that the Wii is sold out right on the initial search results page.

The best teams work diligently to gracefully handle a valid no results condition, ideally in the way a customer service representative might be helpful in person. If, there’s a reasonable substitute, the site should offer that. Otherwise, it should be up front about not having the content.

Ensuring an Effective Search Results Page

As I stated at the beginning, there’s no easy way to build an effective search results page. It takes solid research and regular testing.

The teams that achieve it do so because they’ve spent time scouring through their log data, watching their users, and looking for solutions that provide the best experience. They make frequent changes to the design and start the process again. It’s something that takes both time and resources, and needs properly scheduling and budgeting.

The alternative is leaving the results up to chance and, unfortunately, the odds are slim you’ll end up with an effective design that way.

Published here on July 14, 2008.