Wegmans 2go cover
Wegmans 2GO cover image

Wegmans Meals 2GO

Usability testing on a food catering and delivery app

Wegmans Meals 2GO

Usability testing on a food catering and delivery app

Timeframe

January – April, 2019

My Role

  • UX Researcher
  • Test Moderator (Team of 4)

Client

Wegmans Food Markets, Inc.

Methods

  • Secondary Research
  • Heuristic Evaluation
  • Usability Testing
  • Questionnaire

OVERVIEW

Founded in Rochester, NY, Wegmans Food Markets, Inc. is a privately held American supermarket chain. The Wegmans Meals 2GO mobile app offers services to customers to order chef-prepared restaurant food at select Wegmans locations (delivery, carryout, or curbside pickup).

Wegmans needed to test the app with real customers to identify usability issues across the user journey and understand opportunities for optimization. We conducted usability testing sessions with 12 users and then provided structured feedback in a usability evaluation report.

Objectives

Wegmans wanted to test the end-to-end meal ordering process for the Wegmans Meals 2GO iOS app, particularly the usability of its user interface and user flow, aiming to:

  • Assess the overall effectiveness of the app for users to create and customize orders.
  • Gain an understanding of the app usability and identify usability issues.
  • Understand users’ satisfaction and preferences for the app as a whole.
  • Provide insights for improvement and optimization.

Deliverables

We conducted 12 in-person usability tests of the end-to-end process on the Wegmans Meals 2GO iOS app with representative users.

We created an evaluation report detailing the findings of the tests and providing actionable optimization recommendations. Wegmans adopted a number of our recommendations in following updates. 

My Roles

I worked closely with my teammates and actively contributed to each part of this project. Particularly, I moderated all the usability testing sessions and made sure participants had a pleasant participation experience. I also took on the responsibility of reviewing and editing all the deliverables.

PROCESS

Prior to the tests, we conducted secondary research on the app’s target users and a heuristic evaluation on the app to inform the test design. We regularly met with the Wegmans Meals 2GO development team to communicate our project plan, progress, and insights.

Project Timeline

Part I. Secondary Research

Our project began with research on the app’s target users and context of use in order to understand mental models of users. 

We had a project kick-off meeting with the Wegmans Meals 2GO development team to overview: product vision, primary features and target audience of the app, and the scope of this project.

Product Vision

Our client expected the app to provide the following user experience:

  • Customers can order chef-prepared restaurant food as they are in the store.
  • Customers can skip the line by placing the order with the app.
  • Customers can customize their orders.
  • The app provides the same great experience as other Wegmans applications do.

User Profiles

Type 1 user the foodies

The Foodies

  • Would like to try different types of ethnic cuisines.
  • Want to track calories intake.
  • Eat out and order delivery food weekly.
  • Prefer to schedule orders a week ahead.
  • Order food that they can not cook.
  • Quite tech savvy.
Type 2 user the family in a rush

The Family in a Rush

  • Eat organic and non-GMO products.
  • Avoid cooking to save time.
  • Prefer to customize ingredients because of dietary restrictions.
  • Have family members who have different food preferences
Type 3 user the solo diner

The Solo Diner

  • Appreciate convenience and speed.
  • Busy and spontaneous life type.
  • Tech savvy.
  • Compare different delivery applications to find best deals and fastest delivery time.
  • Prefer choosing food based on appetizing pictures.
  • Read user reviews and recommendations.

Expected Environments of Use

Expected environment of use
Customers order food with the app from home, from workplace, during commute (on the bus, in the parking lot, etc), and when out and about.

Part II. Heuristic Evaluation

We conducted heuristic evaluation on the app to inform the usability test plan.

The evaluation followed the meal ordering user journey on the app, which is from onboarding, browsing, customization, to checkout. Referring to the usability heuristics below, we first inspected the app independently. We then synthesized our findings to generate a heuristic evaluation report.

Heuristics

The usability heuristics we used were the 10 Usability Heuristics by Jakob Nielsen and several accessibility heuristics.

 
External consistencyMinimize knowledge in the head
Internal consistencyUser control and freedom
Widgets and labels near targetsFlexibility and efficiency of use
Group like widgets/functionsError prevention
Frequently used functions optimizedError recovery
Speak the user’s languageNovel interactions easily learned and recalled
Perceptibility of system stateHelp & documentation
Appropriate selection of design patterns

Insights

We identified 34 heuristic issues in the design of the app that resulted in user confusion and increased difficulty of use. Heuristic issues were primarily found in the customization and checkout steps, particularly related to perceptibility of state of system and feedback, flexibility and efficiency of use, and error prevention and recovery.

Therefore, we proposed to focus the usability test on the process of customization and checkout.

Part III. Usability Test

We gathered usability testing data from representative users to further evaluate the usability issues discovered from previous heuristic evaluation.

Test Design

test design one product one group one set of tasks

1 Product

All participants used the same version of the app on a provided iPhone.

1 Group

We conducted the user test on a single participant group (n = 12). All participants had no previous experience using the app.

1 Set of Tasks

All participants went through the same set of scenarios and tasks.

Informed by the findings of our heuristic evaluation, we designed the tasks according to the order of user workflow: onboarding, customization, and checkout. In each task, we investigated the ease of use, intuitiveness, and user satisfaction with the app.

Methodology

  1. A Background questionnaire to collect demographic information of participants;
  2. Think-aloud protocol during the test to analyze their verbal expressions, and
  3. Post-task questions and a post-test questionnaire to assess their satisfaction with the app.

Measurements Taken

  • Task Completion
  • Task time
  • Error count
  • Verbal expressions
  • Facial expressions
  • Other specific task measurements
a test session in progress
I am moderating a test session while my teammates are monitoring from the observation room and taking notes.

Test Environment

All the usability tests were completed in the Usability Lab at Rochester Institute of Technology. A participant and a moderator sat in the test room, a facial expression observer and a time/error tracking observer sat in the observation room.

Test Environment: a participant and a moderator sat in the test room, a facial expression observer and a time/error tracking observer sat in the observation room.
Test Environment: a participant and a moderator sat in the test room, a facial expression observer and a time/error tracking observer sat in the observation room.

Part IV. Analysis & Results

Consistent with the heuristic evaluation, usability issues were primarily found in the customization and checkout process, particularly regarding:

  • Perceptibility of state of system and feedback
  • Flexibility and efficiency of use
  • Error prevention and recovery

1. Perceptibility of State of System and Feedback

For example, most participants (8 of 12) looked for pizza topping price but could not find it. Some of them did not realize that toppings charged extra until they added the pizza to the cart.

Participants expected to check the total price of a pizza before adding it to the cart. However, the app did not reflect the updated price of the pizza. 

I thought the price will show me at the bottom before I put into the cart.

pizza customiaztion
Most participants could not find the pizza topping price. The item page did not reflect the updated pizza price after adding toppings.

2. Flexibility and Efficiency of Use

Take editing an item in the cart as an example, the user was supposed to delete it from the cart, go back to the product page, and add a new item with intended customization.

The majority of participants (9 of 12) looked for a way to edit the item in the cart without having to delete it first.

items in the cart
Details of an item was not editable once it was added to the cart. The user was supposed to delete the item and start over.

It's really annoying. I was expecting to edit it in my cart. It's not flexible.

participant tried to edit an item in the cart
A participant tried to edit an item in the cart by tapping the item but accidentally deleted it.

3. Error Prevention and Recovery

For instance, an item in the cart may be deleted by mistake if a user pressed the minus segment “-” of the digit stepper too fast. The user may also accidentally deleted the item by tapping the close button “x” on the upper right corner. 

In both cases, the app did not provide any alert dialog to prevent the user from deleting an item by mistake.

result of the post-evaluation questionnaire
The app did not provide any alert dialog to prevent the user from deleting an item by mistake.

Post-Task & Post-Evaluation Questionnaire

After each task, participants completed a three-item questionnaire about the ease of use, intuitiveness and satisfaction of the app.

Once the participants had completed all the tasks, they filled out a detailed post-evaluation questionnaire designed to capture their overall impression and preferences for the app. 

In general, participants’ reactions to the Wegmans Meals 2GO app were positive. The app has good usability with a System Usability Scale (SUS) score of 78.13. 

answers to the post-evaluation questionnaire
The questionnaire was referred to the System Usability Scale (SUS) by John Brooke. The green and red cells represent the positive and negative answers respectively.

OUTCOME

We created a detailed evaluation report and provided actionable optimization recommendations covering everything from user flows to user interface. A number of our recommendations were adopted in updates of the app. 

Example:

pizza customization page before and after update
Extra cost of each topping is now displayed on the topping list. Users can now see the updated price of a customizable item before adding it to the cart.
item edit before and after update
Users can now edit a customizable item in the cart without having to delete it first.
cart page before and after update
The app now asks users to confirm their action to prevent them from removing an item by mistake.

(App store review)

This app is the easiest app I have ever used. Very simple and easy to navigate through the different choices of entrees. Customizing your order is a breeze!

Loribeth4 , 07/30/2020

TAKEAWAYS

From this project I learned:

  • We should design flexible systems to allow users to complete tasks in a variety of ways instead of a prescriptive one.
  • It is not feasible to test everything. Learning as much as possible about the product and the client early on and having heuristic evaluation on the product could help to prioritize usability issues. 
  • Have quick debriefs after each test session to identify what went well, what could be improve upon, and what was worth noting from our observation.
  • Leave enough time for data analysis and report writing.
If you would like to learn more, please contact me for a full evaluation report.