Context-Aware Speech-Generating Devices for Autism
Role: UX Research, UX Design, UI Design
Skills: Qualitative Analysis, Sketching, Wireframing,
Prototyping, Usability Tests
Duration: September 2022 - Current
Project Type: Faculty Research Project
Overview
Design context-aware / emergency functions on the speech-generating devices for autistic children
What is Autism?
Autism spectrum disorder (ASD) is a developmental disability that can cause a wide range of challenges in social interactions, communication, and behaviors. Some of autistic children are non-verbal and use augmentative and alternative communication (AAC) aids in analog and digital forms of communication.
What is Speech-Generating Device?
AAC aids include analog picture boards and digital speech-generating devices (SGDs) that use text-to-speech technology to translate images or words selected by the user into speech output. Current SGDs are difficult to set up, use, and customize for the daily activities of the autistic children.
Goals
Create an empirically based, low-fidelity prototype from scratch that focuses on
-
Context-Aware Feature:
Add contextual information from the environment (such as the current activity based on time of day and location) to improve the usability
-
Emergency Mode:
Provide a way to call for help during an emergency and explain injuries of autistic children themselves and others through SGD devices
What is Context Awareness?
According to the Google Awareness API, there are five elements required to provide context-aware functionality.
Time
Location
Activity
Beacons
Headphones
Try Our Deliverables
Process
Responsible for a wide range of areas related to initial UX design
5 times of field observations and 3 times of interviews
1. Domain Analysis
-
Understand and analyze the function of AAC aids
-
Conduct digital ethnography
2. Field Observations
and Interviews
-
Visit a clinic that provides speech therapy for autistic children
-
Conduct interviews with speech-therapists
3. Prototyping
-
Sketch and build a low-fidelity prototype based on observations and interview result
4. Cognitive
Walkthrough
-
Receive initial feedback from stakeholders to improve the prototype
PACT Analysis
We considered our initial idea in the context of PACT (People, Activities, Contexts, and Technology) and summarize our intentions for the application.
People
-
Physical differences
-
Physiological / Psychological differences
-
Mental models
-
Social differences
Activities
-
Temporal aspects
-
Complexity
-
Safety critical
-
Nature of content
Context (analyzed with Activities)
-
Social
-
Organizational
-
Physical circumstances
Technology
-
Application (should have proper communication between user and system)
-
Input / Output
Personas
Through digital ethnography and based on the given interview data of our target users, I produced four personas: an autistic child, mother, father, and speech language therapist.
Affinity Diagram
We analyzed activity data / prompts based on the field observations.
Mind Map
To define specific contexts (time, location, activities) based on users, we listed out what kind of activities children are doing at home.
User Journey Map
Based on the previous user research, our team created two customer journey maps: one is based on "at home" scenario, and another one is "at the clinic" scenario.
Here is an example of the home scenario.
Interview Questions & Answers
I interviewed two speech-language therapists. Some of the highlights from my interviews are...
01.
Do autistic children usually use other applications besides the AAC app?
It is their voice, children don’t use other function or apps on their device.
02.
What features would be helpful to make the current AAC app easier to use?
Simplifying the steps for generating sentences.
“I” “want” “to” could be one button “I want to.”
03.
Do you have any concerns about using the AAC device on a daily basis?
Currently there is no emergency features.
It would be handy to have the information card on the device, too.
Sketch
My initial idea of the context-switching feature and the emergency mode.
Context-switching (detecting user's location)
Emergency mode
Lo-fi Prototype
Before
Feedback:
-
Descriptions are too long
-
Operating options are difficult to understand at a glance
-
There are some icons not related to the emergency mode (location button: dining room)
-
Emergency button may be better to not include in the bottom bar
After
Changes:
-
Descriptions are simple and easy to understand
-
Split screen transition for each flow
-
Eliminate unnecessary icons and add an exit button (x) in the upper right corner
-
Customizable Bottom Bar
Outcomes
Scenario1:
Child moves into the kitchen and wants to talk about lunch. The AAC user can manually switch to “kitchen” mode. The icons related to lunchtime appear.
Scenario2:
Communicate face-to-face with first responders during an emergency situation about themselves, whether they are hurt, how to contact their parent (and other information on the information card)
Next Steps
This project is a year-long research project that aims to make the product more practical for the future, and I believe the following process will help us achieve this goal. We hope that our research will continue to improve the product, and that we can successfully build the application to ultimately help improve people's lives.
1. Reverse-Engineering
Reverse-engineering the currently used AAC application
2. Diary Study
Conduct a diary study of daily family routine to enhance the context-aware feature
3. Beacon System
Support indoor beacon system to detect rooms
4. Prototyping and Testing
Medium-to-high fidelity iOS prototyping
Reflection
Gratitude and Flexibility
This was my first time participating in a project from its inception, and it was very meaningful to be able to take charge of the project ーstarting with the PACT analysis which involved learning about autism and incorporating its characteristics, creating a child-friendly consent form to accompany autistic children, and so on. Some of the interviews and usability tests were conducted remotely, which presented a challenge in contacting busy parents to get them to participate in the study. We learned to be more flexible and compassionate in our communication. We also came to understand the importance of building a close relationship with them. Thank you so much for everyone who participated in this study. We are grateful for their cooperation and kindness in sharing their experiences of autism with us.
Small Steps, Big Impact
In the early stages of the project, each member of the research team had various ideas and images to improve the product, but in order to move forward with the big picture of a single design, it was important to break the user flow down into smaller pieces for consideration. We have carefully observed how users behave, focused on the features they really need, and continued to think about how our designs can meet those needs. By stopping at times to receive feedback from multiple stakeholders, we were able to create a design that was ultimately more concise and user-friendly.
Expanding the Potential of the Product
SGDs should continue to be further promoted for non-verbal people in the future. Having worked closely with family members who have suffered from hearing loss and aphasia, I strongly believe that this application has the potential to greatly aid in future language training and communication not only for people with autism, but also for those who are unable to speak. The context aware and emergency modes that our project is focusing on will help make that more feasible. I will always keep innovative ideas in mind, such as supporting other languages and thinking about how we can specifically expand our target audience. To make it happen, I will continue to contribute to this project as a UX designer.
Special thanks to Dimensions ABA, Northwest Behavioral Associates, Dr. Varsha Koushik from Colorado College, and Dr. Annuska Zolyomi for supporting this project🤍
Let's Get
Social
