Speech-to-text communication
A dictation-based application that translates impaired speech into text and synthesized speech. It can support communication during a doctor's visit as well as in many everyday situations.
Back to GIDO LABS
THE PASS TO
SPEAKING
THE PASS TOSPEAKING
Selected R&D project · Human-centered AI
mPASS: Personalized Speech Recognition for Assistive Technologies
“Everyone should have the right to speak and to be understood.”
mPASS is a human-centered AI platform designed to help people create personalized speech recognition systems tailored to their needs, voice patterns, and communication contexts — with a strong focus on supporting people with speech impairments.
Disordered Speech RecognitionPersonalized ASRAssistive TechnologySpeech AIHuman-centered AIE-health
Challenge
Human-centered speech AI
People with speech or hearing difficulties can face communication barriers in everyday life, including social isolation, embarrassment, and reduced quality of life.
Standard speech recognition systems are often too generic to support highly individual or non-standard speech patterns. mPASS takes a different approach: it adapts speech recognition to the user, not the user to the technology.
Approach
Personalized recognition, built around context
mPASS combines AI, speech technologies, system development, and speech and language therapy perspectives to support people with different types of speech disorders.
At the same time, mPASS is not limited to disordered speech. It is a platform for creating personalized speech recognition systems in broader contexts, including collecting speech recordings and building user-adapted ASR datasets for further model training.
Workflow
How mPASS works
A user-driven workflow that turns everyday voice into a personalized speech recognizer.
mPASS is designed as an easy-to-use online platform enabling non-technical users, carers, therapists, or project teams to create personalized speech recognition resources and systems.
- 01
Define communication context
Users define contexts such as a doctor's visit, school, therapy, shopping, daily communication, or another domain-specific use case.
- 02
Record personal speech samples
Users record speech samples and associate vocal input, sounds, or utterances with intended words or phrases.
- 03
Build personalized speech data
The platform organizes recordings and mappings into structured, user-specific speech data for personalized ASR.
- 04
Create a personalized recognizer
The system supports the creation of custom speech recognition models or datasets adapted to the user's own voice and context.
- 05
Use it in assistive or domain-specific applications
The personalized recognizer can support mobile apps, games, therapy tools, communication aids, or other context-specific speech interfaces.
Use case scenarios
Mobile applications
To demonstrate how mPASS can be used in practice, we designed four mobile application scenarios powered by personalized speech recognition systems trained through the mPASS online platform.
Educational reading game
A mobile game helping children practice reading by showing syllables to pronounce. Correct pronunciation is recognized by the personalized speech recognition system and rewarded with points.
Voice-controlled communication book
A mobile communication book for people who use symbol-based communication and may also have motor impairments. The app can be controlled by voice, even with only a few distinguishable sounds, helping users build phrases and communicate more independently.
SMS and email dictation
A mobile application for dictating SMS messages and emails, including voice control of the sending process. Users can define frequently used phrases and commands, including urgent messages or calls for help.
Impact
Why it matters
mPASS shows how AI can be designed around human needs rather than forcing users to adapt to generic systems.
The project combines speech technology, machine learning, accessibility, assistive technology, and applied R&D — and remains an important example of Gido Labs' work in human-centered AI and personalized speech systems.
Scope
Our work
The mPASS project spanned research, system architecture, and applied design. Our contributions covered the full path from concept to working platform.
- system architecture for a personalized ASR platform
- user-centric workflow for creating custom speech recognizers
- speech recording workflows and personalized ASR data collection
- speech feature extraction and ASR pipeline components
- generation of phonetically rich and balanced texts for ASR training
- web platform and mobile application UX concepts
- mobile application scenarios for assistive speech interaction
- research evaluation of recognition performance and practical applicability
Research
Selected research output
Evaluation of an Automatic Speech Recognition Platform for Dysarthric Speech
Irene Calvo, Peppino Tropea, Mauro Viganò, Maria Scialla, Agnieszka Bętkowska Cavalcante, Monika Grajzer, Marco Gilardone, Massimo Corbo
Folia Phoniatrica et Logopaedica, 2021
Proof-of-concept Evaluation of the Mobile and Personal Speech Assistant for the Recognition of Disordered Speech
Agnieszka Bętkowska Cavalcante, Monika Grajzer
International Journal on Advances in Intelligent Systems, 2016
The project was funded by the Polish National Centre for Research and Development (NCBiR) within the framework of the Lider IV program.