On-Device Speech Recognition Now on Raspberry Pi

- The Whisper model comes in several sizes, from a 39 million parameter "tiny" model to a 1.55 billion parameter "large" model. On a Raspberry Pi 5, only the "tiny" model is fast enough for real-time transcription, processing a 10-second audio clip in about 5 seconds, whereas the "small" model takes roughly 30 seconds for the same clip. - Standard Automatic Speech Recognition (ASR) systems struggle with children's voices due to higher-pitched vocalizations, developing speech patterns, and natural disfluencies like hesitations or repetitions. Even Whisper, which was trained on 680,000 hours of audio, requires specific fine-tuning to achieve high accuracy with young learners. - A study focused on adapting Whisper for children on a Raspberry Pi created a lightweight, fine-tuned "tiny.en" model that achieved a 15.9% Word Error Rate (WER). Using low-rank compression, the model's encoder size was reduced, requiring approximately 2 GFLOPS fewer computations during inference on the device. - Deploying complex models on edge devices relies on optimization techniques like quantization, which reduces the precision of the model's parameters. Studies show that INT8 quantization can reduce a model's size by 75% and improve latency by 15-30% with minimal impact on transcription accuracy. - Whisper's underlying architecture is an encoder-decoder Transformer that was trained to support 99 languages. However, 65% of the supervised training data was in English, meaning performance can be significantly lower for non-English languages without additional fine-tuning. - Processing speech data on-device is critical for educational tools as it eliminates latency, enabling the real-time feedback necessary for effective learning, such as pronunciation correction. It also enhances privacy and safety by keeping sensitive voice data from young users off of cloud servers, which aids in complying with regulations like the Children's Online Privacy Protection Act (COPPA).