Clarity project cover
clarity cover


Enhance communication between deaf and hard-of-hearing and hearing individuals - UX/UI Design, Accessibility


Enhance communication between deaf and hard-of-hearing and hearing individuals - UX/UI Design, Accessibility

My Role

• Team Lead (team of 3)
• UX/UI Designer
• User Test Moderator


• User Interview
• Persona 
• Concept Testing
• Use Case & Storyboard
• Prototype (paper + Figma)
• User Testing


August – December, 2019


Speech-to-text technology has been popularized by applications to capture speech and make the text available later. It also has potential to enhance real-time communication between deaf and hard of hearing (DHH) and hearing people.

As part of a Rochester Institute of Technology design project, my team and I designed Clarity, a speech-to-text application that enhances the real-time communication experience in professional environments where DHH individuals work with hearing counterparts.


How to enhance real-time communication between deaf and hard of hearing (DHH) and hearing people in professional environments?


Clarity is an application that:

  • Provides real-time transcription and speaker identification in a virtual space.
  • Enables speech/text input editing during a discussion.
  • Accelerates text input and highlights key information in transcripts/text inputs.

My Roles

I wore many hats in this project:

  • As the team lead, I oversaw the progress of our project and organized work sessions.
  • As a UX designer, I participated in each step of our design process, from conducting user research to sketching designs, creating prototypes, and collecting user feedback.
  • I served as the moderator in all user testing sessions interacting with expert users.


We reached our design solution by following the User-Centered Design (UCD) process in which users and their needs are primary focuses at each stage of the iterative design process. We met with target users at the beginning of the project and at the end of each phase to collect data and feedback.

Phase I. Needs Assessment

User Interviews

We interviewed 4 target users including one deaf, one hard-of-hearing, and two hearing individuals between the age of 18 and 35. The interview data helped us understand DHH people’s behaviors, frustrations, and preferences when communicating with hearing people in professional settings, and vice versa.

When adopting a new communication tool, our target users considered the following factors important:


We extracted behavioral variables of our interviewees from the interview data and created three personas: deaf, hard-of-hearing, and hearing users. The personas delineated behavioral patterns, frustrations, communication preferences, goals, and needs of our target users.

Phase II. Ideation

Sketches: 30 - 3 - 1

Based on the personas, each of us brainstormed and sketched 30 solutions to the design problem. We converged our 90 ideas to the 3 top ones and further consolidated them into a final sketch. Some of the functions we sketched:

  • Captions with identification of speakers: DHH people often have difficulties identifying who is speaking in group conversations. We wanted our design to be able to indicate the speaker while transcribing the speech. 
  • Word prediction and writing suggestions: We wanted our design to predict words based on users’ writing style and offer writing suggestions. This function addresses the issue of inefficiency when typing to communicate.
  • Dynamic presentation: To help users efficiently comprehend the conversation, we wanted to display information in different styles (e.g. size, color, and animation). 

Concept Testing & Iteration

We collected feedback on our design idea from an expert user who self-identified as deaf. We then made the following changes: 

  • Users should be able to choose the layout of windows in the system.
  • Our design should provide the flexibility to review the transcript during a conversation.
  • We needed alternative ways to dynamically present information other than using color and animation to increase accessibility. 
We are collecting feedback on our design idea from an expert user.
We are collecting feedback on our design idea from an expert user.

Use Cases & Storyboards

We created 3 sets of use cases and storyboards to visualize how the features of our design would be used by users. 

Phase III. Lo-Fi Prototyping

After mapping out the information architecture and carefully considering several key user scenarios, we created a paper prototype for our design. The paper prototype helped us focus on the user flow and content of each screen. 

Feedback & Iteration

We tested our paper prototype with 3 target users (2 deaf and 1 hearing). The conceptual model of our design was able to match our users’ mental models. 

Our users indicated they were able to notice the key features while interacting with the prototype. We made a few modifications to our design based on their feedback.

  • Position of minimized shared screen will be centered; user can remove the screen to focus on other content.
  • The system will remember users’ microphone settings. 
  • Options for interacting with the transcript/text should be more salient.

paper prototype
The paper prototype we used in user feedback sessions.

Phase IV. Hi-Fi Prototyping

We designed a hi-fidelity prototype using Figma. This prototype reflected the latest version of our design which incorporated many features and changes inspired by user feedback in various evaluations.

Visual Design

We chose green as the primary color to represent balance and harmony. 

We chose vivid orange (brown) as the secondary color to add the feeling of energy and compassion to our design.

Key Features

1. Real-Time Transcription with Speaker Identification

When users speak into the microphone of their devices, the system automatically transcribes and displays the speech along with their names.

2. Editable Transcript

Users can edit their own transcripts or text input during discussions. The system will indicate the text is being edited and mark the text has been edited afterwards.

3. Text Input Acceleration

Users can participate in the discussion by typing. 

The system predicts words and offers writing suggestions based on users’ writing style. Users can confirm the suggested words to speed up the text input.

4. Dynamic Presentation of Text

The system automatically detects and highlights key information such as date and location in the transcript and text input to improve its scannability and readability.  

User Testing

We conducted user tests on the hi-fidelity prototype with two expert users who were deaf  to help us form our final design. 

Our participants represented a user who used our design for the first time. Each participant successfully completed three primary tasks with a few small tasks focusing on the main features of our design. 

We received positive feedback on the prototype, especially the dynamic presentation and identification of speaker features. Our expert users also provided feedback on the UI components to make information more readable. 

We are conducting a cognitive walkthrough on the hi-fidelity prototype with an expert user
We are conducting a cognitive walkthrough on the hi-fidelity prototype with an expert user.

Overall, I like the design. Looks clean and professional. I like the split screen between Powerpoint, transcript, chat, and members. All look great!


College Student


We updated the hi-fidelity prototype and released the final design at the course showcase in December 2019. We also created a design specification for communicating our design with developers.

A Glimpse of the Final Design
A Glimpse of the Final Design


Technology has tremendous potential to make the workplace more inclusive. Understanding people’s needs and difficulties they face is crucial to achieve the full potential of technology. Driven by the User-Centered Design process, our design addressed most of the user needs we identified at the beginning of this project. I have learned that:

  • Keeping the discussion open and working together as a team helps reduce confusion and improve collaboration and productivity.
  • I further improved my skills on sketching and rapid prototyping, which are essential to quickly explore and test design solutions.
  • Asking users to describe what they see on the sketch or prototype helps gauge the differences between the conceptual model of our design and users’ mental models.
  • We should pay attention to the social accessibility of our design by drawing as little attention from people around our users as possible.
  • Similar to any relationships, it is important to be patient and considerate when interacting with people with diverse abilities.
If you would like to learn more, please contact me for the full design specification.