poorbuthappy > usability > finding blind spots in navigation

Finding blind spots in navigation.

How to find out what elements of your navigation people have actually noticed.

(Usability, navigation, interface design, July 16, 2000)

July 19: interesting methodology update (at the bottom of this page).

Introduction:

Banner blindness has been well documented, and recently people have been observing users ignore navigation elements as well (think amazon's tabs).

So basically you're putting in lots of navigation features, and some of them are being plain ignored. Or are they?

Since my boss gave me the funny look when I asked him about getting eyetracking devices, I came up with this method to find out what people actually see/look at on a webpage:

I had them draw a picture of the site on a piece of paper after doing a standard usability test.

Conclusions:

People often have a glance at navigation elements (anything that is NOT main content) and decide on it's usefulness by how it looks. They don't actually read it, or figure out what it does. Just a glance. If it looks useful somehow they may investigate. If they think it won't help them they won't bother. They'll make comments like: "I didn't look into that because I didn't think it was going to be useful for me.".
Different people will decide to ignore different elements, probably depending on their browsing style.
People will keep ignoring navigation elements once they've decided they're not useful, even if their decision was based on very superficial evidence and may be wrong, and even if they get lost and don't know where to turn.
Obviously, this was just a quick bit of research. But I'll continue doing it, because the results were interesting.

The study:

Here's the site as it was when the testing was done. Have a look at the navigation. And here are adapted screenshots showing how the users saw the site:

Chris:

Chris never noticed any of the navigation, but went straight for the search button. After finding some pages he looked for links within the main content. He was frustrated he didn't have enough links. The navigation he didn't see was explained as "It didn't seem relevant to me so I didn't really look at it".

Gen:

Gen didn't notice the dropdown or the search box. It's also interesting she didn't draw any of the colored bars at the top. She did see a few of the links on the top left, but only read two of them. She noticed the subscribe box but didn't read the text that goes with it and figured it would give her updates on the website.

Roger:

Roger noticed all the navigation elements (he's a graphic designer). He couldn't tell us what the links in the top left corner did, and he thought there were links in the whitespace below them. He mainly used the dropdown to navigate.

Footnotes:

How it was done:

I tested the site with several people. I had them do a classic usability test (I gave them 2 goals), for a few minutes. Afterwards I closed the browser and asked them to draw a picture of the site with all the navigation elements they could remember, as detailed as possible. Then I took screenshots of the real site and adapted them to the drawings.

Methodology update (July 19):

My methodology was a bit shoddy on this experimental test it seems. As was pointed out to me by Tim Drapeau from eyeTracking.com:

"The fact that users did not remember or draw particular elements on the screen does not mean that they did not look at them. Your assumptions that they did not look at something because they did not draw it on your paper are wrong."

Another good point was made by Howard Kiewe on the CHI mailing list:

"One type of IQ test includes a component that measures visual memory by having subjects draw what they remember of a geometric image after studying it for a certain time period. Many subjects cannot reproduce the details of the image even though they carefully studied it. It is possible that your test does not reveal what was ignored, but rather what could not be remembered of what was seen. This is especially important since in a real world context users only have to remember to select the appropriate link (assuming it is visible). If there were some way to factor out the influence of memory from the influence of attention in your test, its results would be easier to interpret."

However, the observations where interesting and not only deduced from the drawings but also from watching/questioning the users. Maybe the draw-what-you-saw methodology can be refined? Or maybe it just isn't reliable and we need to come up with something else. More questions than answers again, but that's what keeps it interesting. I certainly got some interesting ideas from this exercise.