# esp32-pie-asr

> On-device live English speech recognition and voice-control demos for the ESP32-P4, running without cloud or a neural accelerator.

esp32-pie-asr is an open-source project by Boumedine Billal that runs live English speech recognition on the ESP32-P4 microcontroller. The board's microphone feeds a streaming model and the recognized words appear on the board's screen as you speak, with no cloud. The firmware can be flashed from a browser page using Chrome or Edge.

## What It Is

The project is embedded firmware plus an inference engine for the ESP32-P4X. It uses Kroko (by Banafo), a 65.7M-parameter streaming Zipformer2 transducer, with weights reduced from 71 MB to 26.65 MB using 2- and 4-bit quantization. Inference runs on the chip's two RISC-V cores using PIE SIMD instructions, with no neural accelerator. The firmware opens on live transcription and has a second page of six voice-control demos.

## Recognition and Voice Control

The README reports 4.34% WER on LibriSpeech test-clean and 10.26% on test-other, at 0.70x real time, with words final about 2 seconds after they are spoken. A text file on the microSD card lists phrases (hotwords) to listen for, which can be toggled on screen without retraining. The six voice-control demos (robot vacuum, smart home, car cabin, factory line, kitchen, presenter) map spoken orders to simulated device actions using word rules rather than a language model.

## Setup Path

The model file is copied to a FAT32 microSD card, since it does not fit the board's 16 MB flash. The browser installer then flashes the firmware over the USB Serial/JTAG port. It requires an ESP32-P4X-Function-EV-Board (chip revision v3). Alternatively, the firmware can be built with ESP-IDF v6.1. The code is Apache-2.0 and the weights are CC-BY-SA 4.0.

## Features
- Streaming on-device English speech recognition
- No cloud and no neural accelerator
- 2/4-bit quantized Kroko model (26.65 MB)
- PIE SIMD kernels in assembly across both RISC-V cores
- Custom phrase lists (hotwords) editable on microSD card
- Six voice-control demos driven by word rules
- Browser-based firmware flashing from Chrome or Edge
- Bit-exact results between PC build and chip
- Beam search decoding

## Integrations
ESP-IDF, Hugging Face, Kroko, sherpa-onnx

## Platforms
WINDOWS, MACOS, LINUX, WEB, API, CLI

## Pricing
Open Source

## Version
v1.0

## Links
- Website: https://boumedinebillal.github.io/esp32-pie-asr/
- Repository: https://github.com/BoumedineBillal/esp32-pie-asr
- EveryDev.ai: https://www.everydev.ai/tools/esp32-pie-asr