Usability Testing Project (User-Centered System Design)
Cara Hanley
Southern New Hampshire University
IT-337-Q5010
Objective:
-
- Conduct a heuristic review of a website that sells computers and electronics
-
- Create a user test plan and document the results
-
- Create a User Satisfaction Questionnaire (QUIS) and collect feedback
-
- Provide a summary of the user testing experience
Heuristic Review
| Task | Subtask | Heuristic Effectively Implemented or Violated | Recommendation (to remedy violation or improve design of website) |
| Go to www.tigerdirect.com | Find laptops section | Aesthetic and minimalist design – Violated The cluttered page made it easy to become distracted.Despite the fact the tabs across the top of the page were yellow, the page was so cluttered it took a minute to realize where to begin.The website tried to fit so much on the home page that my screen cut off the edges. | I would improve the aesthetic and aim for a more minimalist design for this website by decluttering the home page. I remove any flashing or moving text and design the website with a consistent color scheme. If all the relevant information can’t fit on one page, that is a good indicator they are trying to fit too much information on that page. |
| Sort by laptops that have at least 16GB of RAM | Show only laptops with a 15” to 15.6” screen | Aesthetic and minimalist design – Violated The same side panels that extended outside of my screen on the home page also appeared on the laptop page. Consistency and standards – Effectively Implemented The sorting options were on the left-hand panel where I expected them to be. Match between system and the real world – Violated I felt limited by the number of options. For instance, I couldn’t view both the 15” screens and the 16” at the same time.The screen size options appeared to be in order by availability instead of by size, which is not what was expected | I recommend removing the distracting and unappealing side panels from the page. I would remove the moving text that takes up most of the page, which is another distraction. To maximize flexibility, increase user options, and appeal to the user’s logic, I recommend adding clickable boxes that allow the user to select multiple screen sizes and multiple RAM options at the same time. |
| Select Windows 10 Pro Operating System | Consistency and standards – Effectively Implemented The highlighting that appeared when I hovered the mouse over a selection was helpful in confirming my selection.Showing my category selections across the top of the page was what I expected to see. | Instead of highlighting when a user hovers their mouse over a selection, I think clickable boxes would be easier to use. Boxes are often used when more options are given. I would suggest expanding user options. | |
| Select an Acer, a Dell and a Lenovo laptop to compare. Ensure that each are at least $750 but not over $1300 | Recognition rather than recall – Violated There was nothing to indicate what I had already selected other than scrolling back to find the boxes I had checked. It was easy to forget what I had already selected and what I was looking for. Flexibility and efficiency of use – Violated I wanted to have the ability to select the three brands from the left-hand panel so that I don’t have to see the brands I don’t want to see but that was not an option.It took me a minute to figure out what to do next after I had selected the 3 laptops I wanted to compare. I just guessed and clicked on the ‘Compare’ button on my last selection in hopes it would bring up all three (it did). Help and documentation – Implemented (somewhat effectively) A chat box appeared only after I clicked the ‘Add to Cart’ button. | To ease the cognitive load of the user, I would have a clearly visible indicator, perhaps on a side panel, of what selections they had already made. I would make this section clickable and highly visible so they can start the comparison process at any time. I would once again add clickable boxes that can help the user narrow their search and spend less time scrolling through items they don’t want to see. I suggest making the chat box visible on every page, not just after the user has added items to the cart. | |
| Add the Acer model to the cart | Consistency and standards – Effectively Implemented The green ‘Add to Cart’ buttons made it clear how to add an item to the cart. Error prevention – Violated Once I hit the ‘Add to Cart’ button, a window opened that suggested other items to add to my cart. It was not immediately clear how to bypass this window or not make those selections because the bottom portion of the window did not show on my screen (this was until I realized there was a scroll bar)I went to move the scroll bar and instead accidentally clicked outside the box and the window disappeared. It did indicate only the laptop was successfully added to my cart. Aesthetic and minimalist design – Violated The page was cluttered and suggesting other items to add to my cart. The section of the page with the suggestions took up more space on the page than did the actual item I was purchasing. | I would remove any pop-up windows that suggest other items to add to the cart. If the website wants to upsell, they should have a smaller section at the bottom of the page suggesting additional items but have them clearly separated from what the user is actually intending to buy. Having a window that is not in proportion with user’s screen and has a scroll bar hanging off the page is not useful and makes it too easy for the user to make an error. I would make the window appear within the page with a clear indicator of how to reject suggested items. The website should also ensure that if the user accidentally (or intentionally) clicks outside of the box that they won’t lose their place. |
User Test Plan
Participant Information
Date: 6/13/2022
Participant #: Pilot
Name: Tony
Gender: Male
Age: 32
Occupation: Software Engineer
Participant’s technology expertise level: Advanced
Type of device being used: Desktop computer
Website being used: www.tigerdirect.com
Description of where evaluation is being conducted: Participant’s home office
Directions Read to Participant:
You have been invited here today to help us understand whether users have problems using this website to compare different computers to see what they might want to purchase in the future. Any problems that you encounter during the tasks are not your fault, but rather issues with the website that we are interested in fixing in the near future. If you need help with any task, you can use help provided on the site, or you are welcome to ask me and I can provide you with some hints as necessary. Any questions? The total time to complete this evaluation will be approximately 15 minutes.
List of Tasks (Provided to the Participant Prior to Testing)
| Task | Rating scales / Questions | Comments |
| Look for one computer you are interested in purchasing |
On a scale from 1 to 7, with 1 being very unsatisfactory and 7 being very satisfactory, how would you rate this task? On a scale from 1 to 7, with 1 being difficult to use and 7 being easy to use, how would you rate this task? |
Participant rated task a 3. He stated it was not intuitive how to filter. Once he figured out the filter, it almost showed everything he wanted. He stated the comparable feature was missing. Rated level of difficulty as a 3. |
| Click on where you would go to if you had questions about the computer you selected. |
Was this easy to find? If not, what would have made it easier to find? |
Participant stated No, it wasn’t easy to find because he had to scroll all the way to the bottom before he found the ‘Help – Email Us’ at bottom of the page. He also noticed the chat box disappeared while he was looking for a laptop. He stated what would have made it easier to find is “if the chatbox hadn’t disappeared on me”. And a ‘Contact Us’ was at the top of the screen instead. Since the chatbox wasn’t available, he scrolled to the bottom (since it wasn’t at the header either, so his next guess was at the footer). He said if the chat box had been available when he needed it, he would have used that instead. |
| Add the computer you selected to your cart and proceed to checkout |
On a scale from 1 to 7, with 1 being difficult to use and 7 being easy to use, how would you rate this task? Was there anything that made this task inefficient? |
Rated a 7 because when he went to the checkout page, he quickly saw the item and price. He did note this section was a bit small compared to the rest of the page. He said the pop-up was annoying but there were two big buttons to choose from that made it obvious. |
| Find a wireless router for under $230 | On a scale of 1 to 7, with 1 being very difficult and 7 being very easy, how would you rate the navigability of the site in completing this task? | He rated this a 4 for difficulty because from the checkout page to proceed back to shopping, the font was small and hard to see. He couldn’t click on the home page icon to go back to home page. He had remembered where to click and search from the previous task(s), but even so he said it wasn’t obvious. |
| Find the most expensive wireless headset made by Jabra |
How was your experience in completing this task? Are you confident in your selection for this task? |
Participant said it was “pretty fast”. He relied on memorability to search for this item as he had used the search and filter functions for previous tasks. Participant said yes he was confident. |
| View the item(s) in your cart |
Was the cart where you expected it to be? Is there anything you would do to improve the design or layout of this feature? |
Participant stated yes the cart was at the top of the page where he expected. No, he wouldn’t improve anything here. |
| Remove the item(s) in your cart |
On a scale from 1 to 7, with 1 being difficult to use and 7 being easy to use, how would you rate this task? Did the website do what you expected it to when completing this task? |
Participant removed one item from the cart at a time. He rated this task a 6 because there was no ‘Remove All’ option. No because once the shopping cart is empty, it should have returned to main menu he said. |
| Find out if the company carries smart power strips |
On a scale from 1 to 7, with 1 being difficult to use and 7 being easy to use, how would you rate this task? Was there anything you would improve for the process or design in completing this task? |
Rated it a 6 because search functionality is good but didn’t autocomplete when he began typing in ‘smart’ in the search box. Participant said he would add an autocomplete feature to search bar. |
| Compare two different brands of gaming computers |
What do you think about how the information and features are laid out? What, if anything, caused you frustration? |
Participant said “It’s alright. They’re missing a few things. Not sure why there’s two rows in graphics and GPUVPU”. (At this point I realized participant was focused on the comparison charts, which was not part of the task. I redirected him to thinking about the process of comparing instead). His response when asked about the way features were laid out for the comparison task: Frustration was caused by the appearance of being taken to a completely different site when he looked for gaming computers. |
| Find laptops for under $500 | On a scale of 1 to 7, with 1 being very frustrating to use and 7 being very pleasant to use, how would you rate your experience with the website for completing this task? | He rated this task a 5 because he stated price range selected is excluding everything under $200 but only one range can be selected at a time. |
| Where would you go to track your order? |
Was this feature where you expected it to be? What, if anything, would you change about this feature? |
Yes. Participant said he found it easily. He said he wouldn’t change anything. |
| If you were looking for a sound card, where would you expect to find it? |
Was this item where you expected it to be? On a scale from 1 to 7, with 1 being very difficult to find and 7 being very easy to find, how would you rate the findability of this task? |
Participant said “Yes”. He simply typed in ‘sound card’ under search. He rated this task a 7 because everything was where he expected it to be. |
User Satisfaction Questionnaire (QUIS)
The following questionnaire was provided to the participant to fill out upon completion of the usability test:
Tracking User Performance
| TASK | Time started | Time ended | Any help needed (describe) | Successfully completed? | Comments by participant | Comments by the observer |
| 1. Look for one computer you are interested in purchasing | 8:08 PM | 8:15 PM | N/A | Yes | He said it wasn’t obvious that he needed to click ‘Guided Search’ to filter selection. | It took a couple minutes before participant was aware there was a way to filter items. |
| 2. Click on where you would go to if you had questions about the computer you selected. | 8:15 | 8:20 | N/A | Yes | Laughing about the tweet button. | Participant eventually found the help section but initially was curious about the tweet button and even clicked on it to see where it would take him. |
| 3. Add the computer you selected to your cart and proceed to checkout | 8:20 | 8:23 | N/A | Yes | None | Participant selected ‘No thank you’ to a pop-up and then scrolled to look at the extensions and selected to add a supplemental item to the cart |
| 4. Find a wireless router for under $230 | 8:23 | 8:27 | N/A | Yes | Wondered aloud why home page icon doesn’t work. | From the page he was last on (the checkout page), the participant said the icon doesn’t work to go back to the home screen. He instead typed in “router” in the search tab. I had to remind the user of the task (he forgot the task list was printed out next to him). An item showed up (extender) that he didn’t select. (This could possibly have been due to a user error in miss-typing in the search box). |
| 5. Find the most expensive wireless headset made by Jabra | 8:28 | 8:30 | N/A | Yes | None | Participant had no problems with this task since he now understood how the filtering works |
| 6. View the item(s) in your cart | 8:30 | 8:31 | N/A | Yes | None | Participant quickly recognized the cart at the top of the screen where he expected it to be. |
| 7. Remove the item(s) in your cart | 8:31 | 8:32 | N/A | Yes | Mentioned he could only remove one item at a time. | Participant did not have any issues with this task. |
| 8. Find out if the company carries smart power strips | 8:32 | 8:34 | N/A | Yes | None | He had no difficulty with this task, for which he used the search bar (instead of filtering like I had). |
| 9. Compare two different brands of gaming computers | 8:34 | 8:41 | N/A | Yes | He expressed frustration with the process for comparing gaming computers. | Participant went back to home page and clicked on the gaming pc icon. Noticed page completely changed (dark screen now). Wasn’t sure how to search now? Went to ‘See All’. Went back to normal screen but no longer showed filters. Hit ‘Guided Search’ button again to show filters. Filtered first by brand he wanted (Lifestyle, then by RAM, processor speed, RTX 3080 (graphics card). Clicked on compare button (Asus vs CyberPower PC). Took a few seconds to realize he needed to click ‘compare’ again. |
| 10. Find laptops for under $500 | 8:41 | 8:44 | N/A | Yes | None | He used tabs this time (clicked on ‘Show all products’). Filtered by computers then laptops. Filtered by price range. Accidentally clicked on wrong price range then hit back button. Was able to complete task. |
| 11. Where would you go to track your order? | 8:44 | 8:45 | N/A | Yes | None | Participant knew exactly where to go for this task. |
| 12. If you were looking for a sound card, where would you expect to find it? | 8:45 | 8:47 | N/A | Yes | None | Participant used the search bar for this task and had no issues. |
Summary
Overall, I do believe the user test plan was effective in that it allowed me to gather the information I needed to determine the usability of the website in question. The testing environment was ideal for the participant and did not require much set-up. The task list was printed out and provided to the participant before the test again, however participant got so lost in one of the tasks that he forgot the task list was there and asked me to repeat one of the questions, which was fine. The test was of adequate length to capture all the information needed while being considerate of the participant’s time.
Timing the tasks as they were completed was somewhat difficult and interrupted the flow of the test. Initially I had pencil and paper ready to go for taking notes as the participant completed the tasks, however less than a quarter of the way into the session I switched to typing on my laptop, which I had open with the task list. I wanted to capture every comment and observation so I took detailed notes, but paradoxically the notetaking caused me to miss out on important observations. There were times where I had to ask participant what he had clicked on to get to a certain page because I had missed a keystroke while taking notes.
In addition, the participant did not understand one of follow-up questions I asked him upon completing a task and it had to be re-worded. Prior to the test, the question had made sense in my head but when I said it out loud to the participant, I realized it was poorly worded. I believe my choice of words in this case were too precise and hard to pronounce and a more straightforward question would have been better. Example: Instead of: How would you rate the navigability of the site for the purpose of this task? The question should have been: How would you rate the level of difficulty in finding this item?
Before presenting this user test plan to additional users, I would rethink the materials used in the testing environment and preparation. I think in this case audio would be very helpful in capturing the participant’s comments so that my notes would be reserved for observations only. I would also reword my follow-up questions to ensure they are clear and concise.
References
Usability.gov. (2020). Usability.gov: Improving the user experience. Retrieved from usability.gov.
Questionnaire for User Interface Satisfaction. https://garyperlman.com/quest/quest.cgi?form=QUIS
Based on: Chin, J.P., Diehl, V.A., Norman, K.L. (1988).
Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface.
ACM CHI’88 Proceedings, 213-218.