OVERVIEW
I worked in a team to conduct usability testing on various features of the Washington Trails Association website. Our intent was to derive design improvement recommendations based on feedback gathered from the usability test sessions. By implementing our recommendations, WTA can expect to minimize user frustration and improve the stickiness of the website--meaning visitors are more likely to return for future use.
|
Team
Maria Buan Lucia Choi Sol Choi Tuyen Truong |
Project Length & Setting
10 weeks | HCDE 417 |
Tools
Google Form, Sheets, Slides QuickTime recording |
Contributions
Heuristic evaluation Affinity analysis Define audience and issues Create test kit and procedures Administer test procedures Analyze results Draw recommendations Report findings |
CLIENT BACKGROUND
WTA aims to protect the places people love to hike to enable equal opportunity to access the outdoors. The overall mission of WTA is to preserve, enhance, and promote hiking opportunities in Washington state through the following four main areas:
- collaboration
- education
- advocacy
- volunteer trail maintenance
THE PRODUCTThe WTA website is a hub for users to gather information about hikes, volunteer trail maintenance opportunities, and WTA facts. For this study, my team focused on the web features pertaining to hikers. For hikers, the WTA website offers the following:
|
|
TARGET AUDIENCE
People of all demographics who are interested in exploring the great Pacific Northwest outdoors.
AUDIENCE ANALYSIS
After conducting user research, we gathered that hikers generally visit the website to seek information about hikes. Based on this finding, we decided to focus our research areas to the following pages:
|
Homepage
|
Hiking Guide
|
Driving Directions
|
Trip Report
|
HEURISTIC EVALUATION
To identify key test objectives, my team used Jacob Nielsen's Usability Heuristics to perform a heuristic evaluation. After each member conducted their individual heuristics evaluation, we gathered to rate each heuristic for each WTA website page that we deemed as most relevant to hikers. The rating was done on a pass or fail basis. We eventually narrowed down to three heuristics that we thought were most recurring and pertinent.
- consistency
- clutter and simplicity
- help documentation and error messages
USABILITY PROBLEM
Though abundant in information, the current WTA website is cluttered and disorganized-- resulting in a lack of navigability and intuitiveness.
RESEARCH QUESTIONS
Our research questions were based off of the issues identified from the heuristics evaluation.
|
1. Can users understand what services are offered to them from the homepage?
Vertical Divider
|
2. Can users effectively use the website's navigation or search tools to find what they are looking for?
Vertical Divider
|
3. Can users gather information about a hike using the WTA website?
|
These research questions inform future improvements of the WTA website because they target the main reason users visit the WTA website: to seek information.
THE 4 TASKS
We came up with 4 tasks that would help understand the research questions and examine the three heuristics that we determined as critical for improvement. These 4 tasks were presented to each participant in the study and informed us of their pain points and areas of confusion.
PARTICIPANTS
To ensure that the target audience was represented, we sought participants with varying hiking experience levels and genders who matched the following inclusion criteria:
- have limited experienced using www.wta.org
- unregistered on www.wta.org
- resides in Washington state
- able to come to the study site
- between 18-25 years old (this age group was most accessible to us as college students)
Participant |
P1 |
P2 |
P3 |
P4 |
P5 |
P6 |
Gender |
Male |
Female |
Female |
Male |
Male |
Female |
Hiking Experience |
Advanced |
Beginner |
Value |
Moderate |
Beginner |
Beginner |
WTA Familiarity |
Some |
None |
Value |
None |
None |
Some |
METHODS USED
Pre-screening filtered participants for the study. We created an online questionnaire using Google Forms to filter legible participants. The form was distributed across each team member's social media accounts including Facebook and Twitter.
The purpose of the pre-test questionnaire was to refresh the user about what they answered on the screener to ensure that they are qualified to participate in the study. The questionnaire included questions about how many hikes the user went on in the past year, the region they lived in, their hiking level, how they currently find hiking trips, and how familiar they are with the site. Each participant answered the questionnaire in 1-3 minutes.
There were a total of four tasks which focused on four main features surrounding a user’s preparation in hiking: the home page, hiking guide, driving directions, and trip report.The tasks took on a within-subjects study method which meant that each participant completed the sequential tasks in the same order. They were evaluated by success, learnability, and error rates. The tasks exposed usability deficiencies and gradually shaped the product in question.
The post-task questionnaires collected immediate thoughts right after each task was completed. Each task resulted in different past-task questionnaires. In general, we used Likert scales to understand how easy or useful the respective features were.
The post-test questionnaire was intended to gather information about each participant's overall experience with the website. During this time, team members asked any clarifying questions they had while observing the test session.
DATA COLLECTION
Adapting an assessment style usabilty test, my team captured data using the following techiques:
Data Recordings
Quantitative + Qualitative Data
Types of Metrics
Data Recordings
- Voice and screen recordings done through Camtasia
Quantitative + Qualitative Data
- Close-ended: multiple choice, likert scales
- Open-ended: think-aloud, short answer
Types of Metrics
- Success - can the participant complete the task?
- Learnability - given the task-at-hand, what is the ease-of-use of the website's tool?
- Error Rates - can the participant complete a task with little to no assistance from the moderator?
AFFINITY ANALYSIS
|
After the data was compiled and analyzed using affinity analysis, the study revealed a variety of good and bad findings pertaining to each of the four pages that we focused on in our study (homepage, hiking guide, driving directions, and trip report).
|
|
FINDINGS
THE GOOD
Vertical Divider
|
THE BAD
|
RECOMENDATIONS
My team generated recommendations relating to each of the three heuristics we identifed as most pertinent to the user's experience on the WTA website. The following recommendations are based on data and findings from usability test sessions.
ISSUE 1: CLUTTERED
- Reorganize content to emphasize visual and content hierarchy
- Remove some content such as the advertisment on the right side bar
ISSUE 2: MISSING FEATURES
- Offer filter to narrow down results based on experience level
- Designate a section for trail conditions
ISSUE 3: UNCLEAR FEATURES
- Reword "view larger map" to make it more clear that clicking on this link will lead users to a Google maps with step-by-step driving directions
FUTURE CONSIDERATIONS
If my group were to conduct repeat this study, we would make the following changes.
EXPERIENCE LEVELS
We noticed during the usability test sessions that many participants were unsure how to categorize their hiking experience level. In the future, I would be more clear by recommending that 0-3 hikes in the past year represent beginner level, 4-6 hikes represent intermediate, and 7+ represent expert level.
LOGISTICS
The next time I conduct a usability test involving screen, video, or voice recordings, I will make sure that there is enough storage available because during this study, we ran into the problem of not having enough media space.
PRE-TEST QUESTIONNAIRE
Many of our participants were unsure where cities belonged to in certain regions within Washington State. This made the task of "find a hike within your experience level and region" difficult because some participants were unfamiliar with geographical locations.
EXPERIENCE LEVELS
We noticed during the usability test sessions that many participants were unsure how to categorize their hiking experience level. In the future, I would be more clear by recommending that 0-3 hikes in the past year represent beginner level, 4-6 hikes represent intermediate, and 7+ represent expert level.
LOGISTICS
The next time I conduct a usability test involving screen, video, or voice recordings, I will make sure that there is enough storage available because during this study, we ran into the problem of not having enough media space.
PRE-TEST QUESTIONNAIRE
Many of our participants were unsure where cities belonged to in certain regions within Washington State. This made the task of "find a hike within your experience level and region" difficult because some participants were unfamiliar with geographical locations.
