# ssftalk

> A pure-Rust, Apache-2.0 voice engine offering ElevenLabs v1 API parity for synthesis, cloning, dubbing, full-duplex rooms, and audio-LLM interpretability.

ssftalk is an open-source Rust crate from solidSF that covers voice synthesis, voice cloning, speech-to-speech conversion, audio isolation, multi-speaker dubbing, full-duplex rooms, and audio-LLM mechanistic interpretability. The README describes it as a pure-Rust engine with no C++ FFI (only serde and serde_json as dependencies), licensed under Apache-2.0. It can run in-process or through a cloud binary WebSocket API.

## What It Is

ssftalk is a voice engine library that implements the ElevenLabs Voice API v1 surface. Developers can call it in-process via `ElevenLabsEngine`, or over HTTP/1.1 via a loopback server and client. The README lists coverage for text-to-speech (including streaming and timestamps), stream-input WebSocket, voice listing and cloning, text-to-voice design previews, speech-to-speech, audio isolation, sound generation, forced alignment, speech-to-text, pronunciation dictionaries, dubbing, and model listing.

## Voice Features and Extensions

Beyond API parity, the README describes eight-axis emotion steering with inline tags such as laugh, sigh, and whisper, plus a custom `j128` 128 kbps voice codec. Output formats include WAV, PCM, ulaw_8000, mp3_44100_128, and j128_24000. Dubbing uses WSOLA pitch-invariant timeline alignment across multiple speakers.

## Full-Duplex Rooms and Interpretability

The `duplex` module provides conversational rooms with binary audio framing, human barge-in truncation, backchannel versus turn speaking modes, and diarization slots. The `mechinterp` module targets audio-LLM workloads: bitplane-based token pruning of audio frames, 256-bit sparse autoencoder circuit matching, 64-dimensional speaker signatures, and counterfactual ablation of harmonics and prosody. The README reports an 88% KV token reduction, a vendor-published figure.

## Cloud WebSocket API and Benchmarks

For batch workloads, a binary WebSocket endpoint at wss://api.solidsf.com/v1/ssftalk/ws dispatches vocoder, prosody, voice conversion, and duplex jobs to solidSF GPU clusters using prepaid credits. The README also includes a 14-stage benchmark example, runnable with cargo, whose figures are published by the author.

## Features
- ElevenLabs v1 API parity in-process and over HTTP
- Text-to-speech with streaming and character timestamps
- Zero-shot voice cloning
- Natural-language voice design previews
- Speech-to-speech conversion
- Audio isolation
- Sound effect generation
- Forced alignment and speech-to-text with diarization
- Pronunciation dictionaries
- WSOLA multi-speaker dubbing
- Eight-axis emotion steering and inline expression tags
- j128 128 kbps voice codec
- Full-duplex rooms with barge-in handling
- Audio-LLM token pruning
- 1-bit SAE voice circuit matching
- 64-D glass-box voice signatures
- Counterfactual ablation
- Cloud binary WebSocket API
- Pure Rust with no C++ FFI

## Integrations
ElevenLabs Voice API v1, GPT-4o Realtime, Gemini Live

## Platforms
API, DEVELOPER_SDK, WEB

## Pricing
Open Source

## Links
- Website: https://github.com/solidsf-inc/ssftalk
- Repository: https://github.com/solidsf-inc/ssftalk
- EveryDev.ai: https://www.everydev.ai/tools/ssftalk