Timeframe
August – December, 2019
My Role
• Team Lead (team of 3)
• UX/UI Designer
• User Test Moderator
Platform
Desktop
Methods
• User Interview
• Persona
• Concept Testing
• Use Case & Storyboard
• Prototype (paper + Figma)
• Cognitive Walkthrough
OVERVIEW
Over 5% of the world’s population – or 430 million people – require rehabilitation to address their ‘disabling’ hearing loss (WHO, 2021).
Speech-to-text technology has been popularized by applications to capture speech and make the text available later. It also has potential to enhance real-time communication between deaf and hard of hearing (DHH) and hearing people.
How might we enhance real-time communication between deaf and hard of hearing (DHH) and hearing people in professional environments?
Solution
As part of a Rochester Institute of Technology design project, my team and I designed Clarity, a speech-to-text application that:
- Provides real-time transcription and speaker identification in a virtual space.
- Enables speech/text input editing during a discussion.
- Accelerates text input and highlights key information in transcripts/text inputs.
My Roles
I wore many hats in this project:
- As the team lead, I oversaw the progress of our project and organized work sessions.
- As a UX designer, I participated in each step of our design process, from conducting user research to sketching designs, creating prototypes, and collecting user feedback.
- I served as the moderator in all user testing sessions interacting with expert users.
DESIGN PROCESS
We reached our design solution by following the User-Centered Design (UCD) process in which users and their needs are primary focuses at each stage of the iterative design process. We met with target users at the beginning of the project and at the end of each phase to collect data and feedback.
Phase I. Needs Assessment
User Interviews
We interviewed 4 target users including one deaf, one hard-of-hearing, and two hearing individuals between the age of 18 and 35. The interview data helped us understand DHH people’s behaviors, frustrations, and preferences when communicating with hearing people in professional settings, and vice versa.
When adopting a new communication tool, our target users considered the following factors important:
Personas
We extracted behavioral variables of our interviewees from the interview data and created three personas: deaf, hard-of-hearing, and hearing users. The personas delineated behavioral patterns, frustrations, communication preferences, goals, and needs of our target users.
Phase II. Ideation
Sketches: 30 - 3 - 1
Based on the personas, each of us brainstormed and sketched 30 solutions to the design problem. We converged our 90 ideas to the 3 top ones and further consolidated them into a final sketch. Some of the functions we sketched:
- Captions with identification of speakers: DHH people often have difficulties identifying who is speaking in group conversations. We wanted our design to be able to indicate the speaker while transcribing the speech.
- Word prediction and writing suggestions: We wanted our design to predict words based on users’ writing style and offer writing suggestions. This function addresses the issue of inefficiency when typing to communicate.
- Dynamic presentation: To help users efficiently comprehend the conversation, we wanted to display information in different styles (e.g. size, color, and animation).
Concept Testing & Iteration
We collected feedback on our design idea from an expert user who self-identified as deaf. We then made the following changes:
- Users should be able to choose the layout of windows in the system.
- Our design should provide the flexibility to review the transcript during a conversation.
- We needed alternative ways to dynamically present information other than using color and animation to increase accessibility.
Use Cases & Storyboards
We created 3 sets of use cases and storyboards to visualize how the features of our design would be used by users.
Phase III. Lo-Fi Prototyping
After mapping out the information architecture and carefully considering several key user scenarios, we created a paper prototype for our design. The paper prototype helped us focus on the user flow and content of each screen.
Feedback & Iteration
We tested our paper prototype with 3 target users (2 deaf and 1 hearing). The conceptual model of our design was able to match our users’ mental models.
Our users indicated they were able to notice the key features while interacting with the prototype. We made a few modifications to our design based on their feedback.
- Position of minimized shared screen will be centered; user can remove the screen to focus on other content.
- The system will remember users’ microphone settings.
- Options for interacting with the transcript/text should be more salient.
Phase IV. Hi-Fi Prototyping
We designed a hi-fidelity prototype using Figma. This prototype reflected the latest version of our design which incorporated many features and changes inspired by user feedback in various evaluations.
Visual Design
We chose green as the primary color to represent balance and harmony.
We chose vivid orange (brown) as the secondary color to add the feeling of energy and compassion to our design.
We matched Lato, one of the most widely used Google Fonts, with Merriweather that is pleasant to read on screens.
Key Features
1. Real-Time Transcription with Speaker Identification
When users speak into the microphone of their devices, the system automatically transcribes and displays the speech along with their names.
2. Editable Transcript
Users can edit their own transcripts or text input during discussions. The system will indicate the text is being edited and mark the text has been edited afterwards.
3. Text Input Acceleration
Users can participate in the discussion by typing.
The system predicts words and offers writing suggestions based on users’ writing style. Users can confirm the suggested words to speed up the text input.
4. Dynamic Presentation of Text
The system automatically detects and highlights key information such as date and location in the transcript and text input to improve its scannability and readability.
Cognitive Walkthrough (User Testing)
We conducted cognitive walkthroughs with two expert users who are deaf to examine the usability of our hi-fidelity prototype. The results also helped us form our final design.
Our participants represented a user who used our design for the first time. Each participant successfully carried out three primary tasks with a few small tasks focusing on the main features of our design.
We received positive feedback on the prototype, especially the dynamic presentation and identification of speaker features. Our expert users also provided feedback on the UI components to make information more readable.
Overall, I like the design. Looks clean and professional. I like the split screen between Powerpoint, transcript, chat, and members. All look great!
Michelle
College Student
FINAL DESIGN
We updated the hi-fidelity prototype and released the final design at the course showcase in December 2019. We also created a design specification for communicating our design with developers.
REFLECTION
Technology has tremendous potential to make the workplace more inclusive. Understanding people’s needs and difficulties they face is crucial to achieve the full potential of technology. Driven by the User-Centered Design process, our design addressed most of the user needs we identified at the beginning of this project. I have learned that:
- Keeping the discussion open and working together as a team helps reduce confusion and improve collaboration and productivity.
- I further improved my skills on sketching and rapid prototyping, which are essential to quickly explore and test design solutions.
- Asking users to describe what they see on the sketch or prototype helps gauge the differences between the conceptual model of our design and users’ mental models.
- We should pay attention to the social accessibility of our design by drawing as little attention from people around our users as possible.
- Similar to any relationships, it is important to be patient and considerate when interacting with people with diverse abilities.