ssftalk

ssftalk is an open-source Rust crate from solidSF that covers voice synthesis, voice cloning, speech-to-speech conversion, audio isolation, multi-speaker dubbing, full-duplex rooms, and audio-LLM mechanistic interpretability. The README describes it as a pure-Rust engine with no C++ FFI (only serde and serde_json as dependencies), licensed under Apache-2.0. It can run in-process or through a cloud binary WebSocket API.

What It Is

ssftalk is a voice engine library that implements the ElevenLabs Voice API v1 surface. Developers can call it in-process via ElevenLabsEngine, or over HTTP/1.1 via a loopback server and client. The README lists coverage for text-to-speech (including streaming and timestamps), stream-input WebSocket, voice listing and cloning, text-to-voice design previews, speech-to-speech, audio isolation, sound generation, forced alignment, speech-to-text, pronunciation dictionaries, dubbing, and model listing.

Voice Features and Extensions

Beyond API parity, the README describes eight-axis emotion steering with inline tags such as laugh, sigh, and whisper, plus a custom j128 128 kbps voice codec. Output formats include WAV, PCM, ulaw_8000, mp3_44100_128, and j128_24000. Dubbing uses WSOLA pitch-invariant timeline alignment across multiple speakers.

Full-Duplex Rooms and Interpretability

The duplex module provides conversational rooms with binary audio framing, human barge-in truncation, backchannel versus turn speaking modes, and diarization slots. The mechinterp module targets audio-LLM workloads: bitplane-based token pruning of audio frames, 256-bit sparse autoencoder circuit matching, 64-dimensional speaker signatures, and counterfactual ablation of harmonics and prosody. The README reports an 88% KV token reduction, a vendor-published figure.

Cloud WebSocket API and Benchmarks

For batch workloads, a binary WebSocket endpoint at wss://api.solidsf.com/v1/ssftalk/ws dispatches vocoder, prosody, voice conversion, and duplex jobs to solidSF GPU clusters using prepaid credits. The README also includes a 14-stage benchmark example, runnable with cargo, whose figures are published by the author.