Every usability plan should include some
actual usability testing. If you want a great site, you
need to do some testing. After you have worked on a site,
you can't see it freshly anymore. Testing will remind you
that not everyone thinks or uses the web in the same
way.
In usability testing, representative users work on typical tasks using the web site (or a prototype) and the evaluators use the results to see how the user interface supports the users in doing their tasks.
Testing does have limitations.
- Test results do not prove that a site works.
- Participants are rarely fully representative of target population.
- Testing is not always the best technique.
The Five Steps of Testing
- Develop the Test Plan
- Select Participants
- Prepare the test materials
- Conduct the test
- Make use of the test results
1. Develop the Test Plan
- It serves as the blueprint for the test.
-
It describes purpose, goals and objectives. For example
you may want to learn about the usability of:
- New or modified tasks.
- Tasks critical to operation.
- Tasks frequently done.
- Tasks done under pressure.
- Tasks that probe common usability problems.
- It describes or implies needed resources.
Sample Format of a Formal Plan
- Purpose.
- Problem statements / test objectives.
- User profile.
- Method (test design).
- Task list.
- Test environment.
- Evaluation measures.
- Report contents and presentation.
2. Select Participants
Number of Test Participants
- According to Jakob Nielsen, you need to test with only five users to find 85 percent of the problems. 1
- Alan Woolrych says that "the issue is not whether five users are enough. It's an issue of the costs, benefits and risks of running some number of users with known relevant characteristics. Three may be enough. Twenty may not be enough." 2
- Steve Krug asserts that in most cases the "ideal number of users for each round of testing is three, or at most four." Don't Make Me Think! A Common Sense Approach to Web Usability, Que, 2000. p. 146.
Selecting Test Participants
- Participants will vary depending on site.
-
Typical traits examined include age, gender, education,
computer experience, web experience, operating system
experience.
- Modify these to suit your particular audience (prospective students, current students, faculty/staff, alumni).
-
You want to have representation for all profiles in
your population.
- Do not use only highly seasoned users.
- Include a few "Least Competent Users". They are typically non-experienced, minimal computer experience, etc. The thought is that if they can make it through the test, the product is pretty usable.
- Beware of people who tend to volunteer for tests. They tend have a particular personality type - that becomes a subject variable that you cannot control for.
- Although selecting test participants is important it is more important to test early and often.
3. Prepare the test materials
Develop a task scenario questionnaire
- First develop a list of tasks you want users to perform.
- Then translate those tasks into scenarios for the questionnaire.
Task scenarios are representations of actual work that a user would likely perform using your web site. You use task scenarios to tell the participants what you want them to do. A good task scenario:
- Is short.
- Is in the user's vocabulary.
- Is clearly written.
- Doesn't use steps or instructions.
- Is not too much of a challenge for the allotted test time.
Try the scenarios tasks out yourself and refine them as needed.
Example: Task Scenario from ITSS web site evaluation 3
- Task: Find the Help Desk phone number.
- Task translated into a scenario: "You have a general question about ITSS. Can you find the Help Desk phone number?"
4. Conduct the test
- A self-reporting methodology is usually best for a beginning usability
specialist to use.
- Self-reporting involves gathering task information from the participant via your prepared task based questionnaire.
- (In formal usability experiments, a trained usability professional would watch closely to observe and record testers' behavior. In formal testing sometimes a special lab with an observation room and a two-way mirror is used. Sometimes the session is video taped. Sometimes eyetracking devices are used. But you don't really need any of that.)
-
Let your tester test your site.
- Unless you're planning on providing a personally-supervised guided tour of your site to all your readers, leave your tester to the business of testing your site. For more information visit: User Testing Techniques.
- Unless you're a trained usability professional, your presence will likely as not serve to inhibit your evaluators, and thus compromise their ability to test the site.
5. Make use of the test results
In formal usability testing, data and findings would be transformed into recommendations by way of a written report and/or presentation. It would include methodology, user profile, task list and results. But to make your site more usable a report isn't necessary. What you need to do is:
- Develop solutions to problems. Get expert advice if the solutions are not obvious.
- Fix the problems.
- Test the revised version to ensure you made the right design decisions.