Timeframe
January – April, 2019
My Role
- UX Researcher
- Test Moderator (Team of 4)
Client
Wegmans Food Markets, Inc.
Methods
- Secondary Research
- Heuristic Evaluation
- Usability Testing
- Questionnaire
OVERVIEW
Founded in Rochester, NY, Wegmans Food Markets, Inc. is a privately held American supermarket chain. The Wegmans Meals 2GO mobile app offers services to customers to order chef-prepared restaurant food at select Wegmans locations (delivery, carryout, or curbside pickup).
Wegmans needed to test the app with real customers to identify usability issues across the user journey and understand opportunities for optimization. We conducted usability testing sessions with 12 users and then provided structured feedback in a usability evaluation report.
Objectives
Wegmans wanted to test the end-to-end meal ordering process for the Wegmans Meals 2GO iOS app, particularly the usability of its user interface and user flow, aiming to:
- Assess the overall effectiveness of the app for users to create and customize orders.
- Gain an understanding of the app usability and identify usability issues.
- Understand users’ satisfaction and preferences for the app as a whole.
- Provide insights for improvement and optimization.
Deliverables
We conducted 12 in-person usability tests of the end-to-end process on the Wegmans Meals 2GO iOS app with representative users.
We created an evaluation report detailing the findings of the tests and providing actionable optimization recommendations. Wegmans adopted a number of our recommendations in following updates.
My Roles
I worked closely with my teammates and actively contributed to each part of this project. Particularly, I moderated all the usability testing sessions and made sure participants had a pleasant participation experience. I also took on the responsibility of reviewing and editing all the deliverables.
PROCESS
Prior to the tests, we conducted secondary research on the app’s target users and a heuristic evaluation on the app to inform the test design. We regularly met with the Wegmans Meals 2GO development team to communicate our project plan, progress, and insights.
Project Timeline
Part I. Secondary Research
Our project began with research on the app’s target users and context of use in order to understand mental models of users.
We had a project kick-off meeting with the Wegmans Meals 2GO development team to overview: product vision, primary features and target audience of the app, and the scope of this project.
Product Vision
Our client expected the app to provide the following user experience:
- Customers can order chef-prepared restaurant food as they are in the store.
- Customers can skip the line by placing the order with the app.
- Customers can customize their orders.
- The app provides the same great experience as other Wegmans applications do.
User Profiles
The Foodies
- Would like to try different types of ethnic cuisines.
- Want to track calories intake.
- Eat out and order delivery food weekly.
- Prefer to schedule orders a week ahead.
- Order food that they can not cook.
- Quite tech savvy.
The Family in a Rush
- Eat organic and non-GMO products.
- Avoid cooking to save time.
- Prefer to customize ingredients because of dietary restrictions.
- Have family members who have different food preferences
The Solo Diner
- Appreciate convenience and speed.
- Busy and spontaneous life type.
- Tech savvy.
- Compare different delivery applications to find best deals and fastest delivery time.
- Prefer choosing food based on appetizing pictures.
- Read user reviews and recommendations.
Expected Environments of Use
Part II. Heuristic Evaluation
We conducted heuristic evaluation on the app to inform the usability test plan.
The evaluation followed the meal ordering user journey on the app, which is from onboarding, browsing, customization, to checkout. Referring to the usability heuristics below, we first inspected the app independently. We then synthesized our findings to generate a heuristic evaluation report.
Heuristics
The usability heuristics we used were the 10 Usability Heuristics by Jakob Nielsen and several accessibility heuristics.
| External consistency | Minimize knowledge in the head |
| Internal consistency | User control and freedom |
| Widgets and labels near targets | Flexibility and efficiency of use |
| Group like widgets/functions | Error prevention |
| Frequently used functions optimized | Error recovery |
| Speak the user’s language | Novel interactions easily learned and recalled |
| Perceptibility of system state | Help & documentation |
| Appropriate selection of design patterns |
Insights
We identified 34 heuristic issues in the design of the app that resulted in user confusion and increased difficulty of use. Heuristic issues were primarily found in the customization and checkout steps, particularly related to perceptibility of state of system and feedback, flexibility and efficiency of use, and error prevention and recovery.
Therefore, we proposed to focus the usability test on the process of customization and checkout.
Part III. Usability Test
We gathered usability testing data from representative users to further evaluate the usability issues discovered from previous heuristic evaluation.
Test Design
Informed by the findings of our heuristic evaluation, we designed the tasks according to the order of user workflow: onboarding, customization, and checkout. In each task, we investigated the ease of use, intuitiveness, and user satisfaction with the app.
Methodology
- A Background questionnaire to collect demographic information of participants;
- Think-aloud protocol during the test to analyze their verbal expressions, and
- Post-task questions and a post-test questionnaire to assess their satisfaction with the app.
Measurements Taken
- Task Completion
- Task time
- Error count
- Verbal expressions
- Facial expressions
- Other specific task measurements
Test Environment
All the usability tests were completed in the Usability Lab at Rochester Institute of Technology. A participant and a moderator sat in the test room, a facial expression observer and a time/error tracking observer sat in the observation room.
Part IV. Analysis & Results
Consistent with the heuristic evaluation, usability issues were primarily found in the customization and checkout process, particularly regarding:
- Perceptibility of state of system and feedback
- Flexibility and efficiency of use
- Error prevention and recovery
1. Perceptibility of State of System and Feedback
For example, most participants (8 of 12) looked for pizza topping price but could not find it. Some of them did not realize that toppings charged extra until they added the pizza to the cart.
Participants expected to check the total price of a pizza before adding it to the cart. However, the app did not reflect the updated price of the pizza.
2. Flexibility and Efficiency of Use
Take editing an item in the cart as an example, the user was supposed to delete it from the cart, go back to the product page, and add a new item with intended customization.
The majority of participants (9 of 12) looked for a way to edit the item in the cart without having to delete it first.
3. Error Prevention and Recovery
For instance, an item in the cart may be deleted by mistake if a user pressed the minus segment “-” of the digit stepper too fast. The user may also accidentally deleted the item by tapping the close button “x” on the upper right corner.
In both cases, the app did not provide any alert dialog to prevent the user from deleting an item by mistake.
Post-Task & Post-Evaluation Questionnaire
After each task, participants completed a three-item questionnaire about the ease of use, intuitiveness and satisfaction of the app.
Once the participants had completed all the tasks, they filled out a detailed post-evaluation questionnaire designed to capture their overall impression and preferences for the app.
In general, participants’ reactions to the Wegmans Meals 2GO app were positive. The app has good usability with a System Usability Scale (SUS) score of 78.13.
OUTCOME
We created a detailed evaluation report and provided actionable optimization recommendations covering everything from user flows to user interface. A number of our recommendations were adopted in updates of the app.
Example:
TAKEAWAYS
From this project I learned:
- We should design flexible systems to allow users to complete tasks in a variety of ways instead of a prescriptive one.
- It is not feasible to test everything. Learning as much as possible about the product and the client early on and having heuristic evaluation on the product could help to prioritize usability issues.
- Have quick debriefs after each test session to identify what went well, what could be improve upon, and what was worth noting from our observation.
- Leave enough time for data analysis and report writing.