Usability testing should be incremental

Long before unleashing your site on the world at large, you must test the site extensively with real users. Usability testing should be incremental, beginning as soon as there are several pages that are complete enough that testers can get some idea of at least the navigation. Even if the navigation is not yet wired up to go anywhere, you can show it to testers and ask, “Where do you think this will go? What would you expect to find if you click this link?” If the testers’ view of how the navigation works differs from the way it was actually intended to work, you have a problem. Better to discover that problem now, when only a few pages are constructed, rather than later, after hundreds of pages are just a few days shy of deployment. Of course, early testing does not in any way eliminate the need for later testing. Retest at least once each month throughout the project. If you are really, really good at listening to what your testers tell you, you won’t have any major problems left to fix on your fi nal test.

Just how involved, expensive, and time-consuming does usability testing have to be? Until just recently, many experts believed that bigger testing was better testing. They thought there needed to be bloated scenarios involving hundreds of testers, expensive usability labs with video taping of all tests, and convoluted and extensive reports complete with statistical analysis. The end result of such an overblown testing mentality is that testing was rarely done at all. Today, though, the belief is that more informal, less involved testing (termed “discount testing”) can produce huge benefi ts certainly more benefi ts than grandiose testing scenarios that aren’t done at all. Major usability issues tend to become quickly obvious even with informal testing. How do you do discount testing?

Choose three or four testers with widely different technical abilities for each round of testing. Avoid using team members as testers; only people who are “off the street” and have not actually helped to create your site can give you unbiased, laypersons’ opinions of what will and will not work with real visitors. Feel free to enlist friends and family in the test, as long as you are sure they will give you that unbiased opinion.

For instance, your mother’s “Yes, dear, this is just a wonderful site. I am soooooo proud of you!” might be a wonderful ego stroke, but it’s of little practical value as a critique. What you truly need here is blunt honesty. Only frank critiques can improve the site (as well as protect your reputation as a professional web designer). And what should be your reaction to a brutally honest review? A sincere, heartfelt, “Thank you so much for identifying these flaws, so that now we can fix them!”

When you are testing, you are actually looking at two key issues: do the testers “get it,” and can they accomplish the necessary tasks? The fi rst refers to whether or not they understand the purpose of a site, its value, how it’s organized, and how the navigation works. For this question, it’s best to ask the users to think aloud, and then turn them loose, observe what happens, and listen very carefully to their comments. A few well-considered questions can always spur more in-depth evaluation as well. Did areas of the site irritate or baffle them? Why? And what would they change? What drew the testers’ attention fi rst on a page? Second? Third? Probe for specifics in all of these areas.

To evaluate the second of the two key issues whether or not the testers can accomplish tasks you might want to use tasks of their own choosing as well as assign specific tasks that you feel will be critical to future visitors. Again, what confused them or lost them?

On the other hand, were there areas of the site that engaged them, or amused them, or intrigued them? Why? What would they suggest that would allow you to propagate those positive aspects to other areas of the site? Although you definitely need to take notes and perhaps even record each test session, a 30-page report detailing the results is overkill and very much a waste of your time. That time would be better spent in fixing the problems, not documenting them. A two-hour session with team members should allow you to focus on the major issues that simply must be fixed, as well as prioritize those issues that would be nice to repair but aren’t so critical.

Be particularly wary of requests for additional features, so-called “scope creep.” These features can end up cluttering the site to the point of un-usability, can have other unanticipated negative consequences to the site, and might be of value to only a very small subset of visitors not to mention the extra time, effort, and costs to construct the additional features. Often, you must fi nd a way to halt such scope creep in its tracks, because it can derail both the project’s budget and its timetable. Once you have incorporated what you learn from each set of testers, test the site again. And again. And yet again, until you and the latest set of testers are happy with the results, and the site can be launched.