Self-hosted speech server whose MCP endpoint exposes ASR transcription and file staging tools.
io.github.psyb0t/talkies MCP Server
Self-hosted MCP server for speech that groups speech services into one Docker image. It provides tools for ASR transcription, TTS synthesis, and file staging. The project topics include transcription, speech recognition, text-to-speech, and related models and protocols.
🛠️ Key Features
ASR transcription tools
TTS synthesis tools
File staging tools
Docker-based self-hosted deployment
🚀 Use Cases
Speech-to-text transcription
Text-to-speech generation
Staging audio/text files for processing
⚡ Developer Benefits
Includes MCP tooling for speech workflows
Published as a Docker image (with Docker pulls tracked)
CI, release version badges, and license metadata shown in the README excerpt
⚠️ Limitations
Tool details beyond ASR, TTS, and file staging are not present in the provided source data
Self-hosted speech services in one Docker image: OpenAI-compatible file
transcription and text-to-speech, Talkies live ASR over WebSocket, file
staging, model lifecycle controls, and an MCP endpoint for ASR workflows.
For the CUDA-only models (Parakeet-TDT, the larger Canary models, Qwen3 TTS
and Chatterbox Turbo), use psyb0t/talkies:latest-cuda with --gpus all. The
loopback port mapping keeps the service local; see
Getting started
for first boot and authentication.
The HTTP transcription and speech routes use the corresponding OpenAI wire
shapes where those contracts overlap. Streaming ASR, files, lifecycle controls,
and MCP are Talkies extensions.
Models at a glance
CPU: two Whisper models, Canary-180M-Flash, Nemotron ASR via parakeet.cpp,
four English Sherpa-ONNX Zipformer choices, Vosk small English, two phoneme
recognizers, and two Kokoro TTS backends.
CUDA: the CPU set plus Parakeet-TDT, Canary 1B/Qwen ASR, five Qwen3 TTS
variants, and Chatterbox Turbo.
Live ASR: bundled Nemotron, Sherpa-ONNX, and Vosk are native; bundled Whisper
is a bounded rolling decoder. Sherpa and Vosk also work through the
OpenAI-compatible file-transcription endpoint.
Phoneme recognition: wav2vec2-xlsr-53-espeak and zipa-ipa return the IPA
phones that were spoken, not words, with no language model correcting them
toward the nearest dictionary entry. Same transcription endpoint and
timestamp options as the other ASR models; see
Phoneme recognition.
Per-model concurrency limits cover WebSocket, HTTP, MCP, ASR, and TTS; the
bundled Nemotron CPU and CUDA entries admit two requests.
Streaming TTS: Qwen3 returns incremental raw PCM for
response_format="pcm"; other TTS formats and Kokoro are buffered.
Expressive TTS: Chatterbox Turbo (English) takes 19 inline tags such as
[sigh], [whispering] and [laugh] directly in the input text. Its output
carries a neural watermark by default; set TALKIES_CHATTERBOX_WATERMARK to
false to emit unmarked audio.
Voice cloning: drop a .wav into /data/custom-voices and it appears on
GET /v1/audio/voices. Qwen3 pairs it with an optional sibling .txt
transcript; Chatterbox needs only the clip, longer than five seconds.
wav2vec2-xlsr-53-espeak and zipa-ipa use the same transcription call as
every other ASR slug; only the model changes. text comes back as a
space-separated IPA phone stream rather than words, and no language model
corrects a mispronunciation toward a real word.
bash
curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
-F "file=@/path/to/clip.wav" \
-F "model=zipa-ipa"# {"text": "a ɪ m k ə n f j u z ...", ...}
Add -F "response_format=verbose_json" to get each phone as a words entry
with start and end in seconds.
Prompting Chatterbox with emotion
Tags go inline in input, in square brackets, lowercase. They are real tokens
in the model's tokenizer, so only these 19 do anything; any other bracketed
word is spoken as literal text:
Make targets, test suites, image builds, docs site
Agent integrations
The Talkies skill
teaches agents to use the HTTP, WebSocket, and MCP routes. It installs as a
Claude Code or Codex plugin from the shared psyb0t marketplace, and OpenClaw
installs the skill and a stdio MCP bridge from ClawHub. For Claude Code:
bash
claude plugin marketplace add psyb0t/agents
claude plugin install talkies@psyb0t
The Codex and OpenClaw commands and the bridge settings are in
Agent integrations.
Security in one minute
TALKIES_AUTH_TOKEN enables a shared bearer token for every HTTP and WebSocket
route except /healthz. It is unset by default. Keep the port loopback-only or
put Talkies behind TLS, authentication, and rate limiting. If untrusted callers
can supply remote file_path URLs, set TALKIES_BLOCK_PRIVATE_DOWNLOADS=true.
See Operations and security
for the complete posture.
Development
bash
make check # lint + unit tests in the dev image
make lint # flake8 + mypy only
make test-unit # fast offline unit tests
make run # run the CPU image locally
make test-streaming # real CPU native WebSocket ASR test
make test-streaming-custom # real CPU Sherpa/Vosk WebSocket + HTTP tests
make test-streaming-custom-cuda # real CUDA Sherpa WebSocket + HTTP test
make compile-heavy # regenerate the hash-locked ML requirements
make build-all # CPU and CUDA production images
make docs # build the documentation site into .testing/docs-site
make help lists every target, and
Development describes each one.
Talkies is released under the
WTFPL.
Model weights are downloaded at runtime and have their own terms; image
component notices are in
THIRD_PARTY.md.
Release notes are in
CHANGELOG.md.