CrispASR ships 62 registered open-weights TTS backends — roughly three
dozen distinct engine families once variants are folded; the authoritative
auto-generated list is docs/feature-matrix.md — behind
the same crispasr binary, each with a distinct voice / quality / footprint
trade-off:
--ref-text, base vs RL talker-tl / -sl, what each backend does--g2p-dict, number expansion, phoneme dialects
--tts-phonemes)CRISPASR_ZONOS_G2P) — built-in vs espeak, measured per language--tts-play)| Backend | Why pick it | Voice cloning | First-run download |
|---|---|---|---|
melotts |
Multilingual VITS2 (MeloTTS). 4 English speakers (US/BR/India/AU). 44.1 kHz output, ~102 MB GGUF. Neural G2P + CMU dict. BERT companion (Q4_K 52 MB) auto-downloads with -m auto; also via --codec-model or CRISPASR_MELOTTS_BERT env. |
No (per-speaker ID) | ~154 MB via -m auto |
piper |
Tiniest footprint (30 MB). rhasspy/piper VITS; 250+ community voices across 30+ languages. Built-in G2P (CMUdict + LTS rules) for English — no espeak-ng needed. Optional espeak-ng for other langs (loaded via dlopen). 22 kHz output. Use --g2p-dict to select dictionary source. |
No (per-voice GGUF) | ~30 MB default via -m auto; other voices are manual downloads |
kokoro |
Smallest + fastest. 82 M-param StyleTTS2-derived model. Multilingual via built-in G2P or espeak-ng (dlopen/popen fallback). | No (preset voice packs) | Manual wget (no -m auto) |
qwen3-tts |
Highest fidelity / strongest cloning. Speech-LLM (talker + code predictor + 12 Hz codec). Default voice auto-downloaded with -m auto; or supply your own WAV + ref-text. |
Optional (auto default voice; or WAV + ref-text or baked voice GGUF) | ~1.3 GB via -m auto |
miotts |
MioTTS-0.6B (Qwen3 LLM + MioCodec-v2). EN/JA. Single GGUF, 24 kHz output. Codec-aware mixed quantization (LLM Q4_K + codec F16). | Yes — --voice preset.emb.gguf (preset speaker embeddings) |
502 MB Q4_K via -m auto |
moss-tts |
MOSS-TTS-v1.5 (MossTTSDelay): Qwen3-8B backbone emitting 32 RVQ audio codebooks under a delay pattern, decoded by a 1.6B transformer codec. Needs the codec companion (--codec-model, or auto-downloaded/sibling). |
Yes — --voice ref.wav (the codec encoder clones the reference speaker) |
~5 GB Q4_K backbone + ~3.5 GB F16 codec via -m auto |
moss-tts-local |
MOSS-TTS-Local-Transformer-v1.5 (MossTTSLocal, 4B): Qwen3-4B backbone + a 1-layer local/depth transformer that autoregressively emits 12 RVQ codebooks per frame (RQ-Transformer; no delay pattern), decoded to 48 kHz stereo by MOSS-Audio-Tokenizer-v2 (downmixed to mono). Needs the codec companion (--codec-model, or auto-downloaded/sibling). |
--voice ref.wav --i-have-rights (needs the encoder-carrying codec, --codec-model moss-tts-local-v1.5-codec-enc.gguf; the plain -codec.gguf is decode-only and falls back to the default voice) |
~9.1 GB F16 backbone + ~2.1 GB codec via -m auto (F16 is the reliable target; Q4_K long-form runs away) |
omnivoice |
600+ languages, selected with -l / -tl (or "language" on /v1/audio/speech) — see the language note below. Qwen3-0.6B backbone with masked iterative 8-codebook TTS (SoundStorm-style). Zero-shot voice cloning from reference audio. Supports finetunes (omnivoice-singing). |
Yes (--voice <wav> --ref-text "...") |
~1.2 GB F16 + ~400 MB tokenizer |
vibevoice-tts |
Lowest-latency streaming TTS, designed for realtime. | Preset voice packs | ~636 MB via -m auto |
vibevoice-1.5b |
Base VibeVoice TTS model with WAV cloning. | Yes (CRISPASR_VIBEVOICE_VOICE_AUDIO=<wav> or --voice <wav>) |
~1.6 GB via -m auto |
orpheus |
Llama-3.2-3B talker + SNAC 24 kHz codec. 8 baked English speakers; expressive output. Greedy loops — pass --temperature 0.6. |
Preset names via --voice tara/leah/... |
~3.5 GB via -m auto (talker Q8 + 26 MB SNAC) |
chatterbox |
T3 AR + S3Gen flow-matching + HiFTGenerator. Built-in voice baked into the T3 GGUF; clones via a baked voice GGUF (see workflow below). EN/AR/DE variants share runtime. | Yes (--voice <voice.gguf>, baked from a WAV with models/bake-chatterbox-voice-from-wav.py) |
~880 MB via -m auto (T3 Q8 + S3Gen Q8) |
outetts |
OuteTTS-0.3-1B: OLMo-1B LLM + WavTokenizer single-codebook VQ-GAN. CC-BY-NC-SA-4.0 (non-commercial, ShareAlike). 24 kHz output. | Yes (--voice <speaker.json>, created with tools/reference_backends/outetts_create_speaker.py) |
~2.5 GB via -m auto (talker F16 + WavTokenizer decoder) |
f5-tts |
F5-TTS v1 Base: 22-layer DiT flow-matching TTS + Vocos iSTFT vocoder. MIT license. High-quality zero-shot voice cloning from 3-15s reference audio. 24 kHz output. English + Chinese (built-in pinyin g2p, #294). | Yes (--voice <ref.wav> --ref-text "transcript") |
~953 MB via -m auto (single F16 GGUF, DiT + Vocos) |
raon |
Raon-OpenTTS 0.3B (KRAFTON): F5-TTS DiT on the same runtime, paired with a 16 kHz HiFi-GAN vocoder (sbhifigan16k, slaney mel). English zero-shot voice cloning. CC-BY-NC-4.0 (non-commercial; auto-download prints the restriction). TTS→ASR roundtrip validated (0.90). Note: CPU vocoder ~40s/utterance. | Yes (--voice <ref.wav> --ref-text "transcript") |
~959 MB via -m auto (single GGUF: DiT + HiFi-GAN) |
raon-1b |
Raon-OpenTTS 1B (KRAFTON): the larger DiT (dim 1408, depth 28, 24x64 heads) on the same runtime and the same sbhifigan16k HiFi-GAN vocoder as raon. CC-BY-NC-4.0 (non-commercial; auto-download prints the restriction). TTS->ASR roundtrip validated on Kaggle GPU. Needs the default manual-SDPA attention path — CRISPASR_F5_FLASH=1 reintroduces the F16 KQ accumulation that made this size diverge to NaN on kernels ignoring the precision hint. |
Yes (--voice <ref.wav> --ref-text "transcript") |
~2.8 GB via -m auto (single GGUF: DiT + HiFi-GAN) |
supertonic |
Supertonic-3: non-autoregressive flow-matching TTS (Supertone). ConvNeXt+attention text encoder, 8-step Euler flow with baked-in CFG (4·cond − 3·uncond), ConvNeXt vocoder at 512 samples/frame. 44.1 kHz, 31 languages, ~99 M params. OpenRAIL-M weights (use restrictions + attribution). 10 fixed preset voices baked into the single GGUF — no cloning in the open release. --voice F1..F5/M1..M5, --tts-speed, --tts-steps. |
No (fixed presets) | ~200 MB F16 (single file) |
irodori-tts |
Irodori-TTS: RF-DiT flow-matching TTS with LowRankAdaLN + JointAttention + half-RoPE + SwiGLU. 48 kHz via Semantic-DACVAE-Japanese-32dim codec. MIT license. Japanese-focused (llm-jp-3 tokenizer). Zero-shot voice cloning from any reference WAV (DAC-VAE encoder + speaker CFG); emoji emotion control; duration predictor for output length. VoiceDesign (600M-v3): adds caption encoder for style/emotion control via text descriptions (--instruct "calm adult male, deep voice"); independent text/speaker/caption CFG. |
Yes (--voice <ref.wav> --i-have-rights) |
~526 MB Q4_K (VoiceDesign) / ~852 MB Q4_K (base) + DAC-VAE codec |
indextts |
IndexTTS-1.5: GPT-2 AR (24L/1280d) mel-code generator + BigVGAN vocoder. Designed for Chinese+English. Zero-shot voice cloning from any reference WAV. | Yes (--voice <ref.wav>) |
~2.4 GB via -m auto (GPT F16 + BigVGAN F16) |
cosyvoice3-tts |
Fun-CosyVoice3-0.5B-2512: Qwen2-0.5B AR speech-token LM + DiT-CFM (10-step Euler) + HiFT (NSF + iSTFT) @ 24 kHz. 9 languages + 18 Chinese dialects. Ships an 8-voice baked bank (zero_shot + fleurs-{en,de,zh,ja,fr,es,ko}). |
Yes — baked-bank name via --voice <name>, or native arbitrary-WAV cloning via --voice <ref.wav> --ref-text "..." (ports speech_tokenizer_v3 + CAMPPlus + matcha mel to ggml; speech tokens byte-exact vs ONNX). |
~1.2 GB via -m auto (Q4_K LLM + Q8_0 flow + HiFT + s3tok + campplus + voices) |
The default US Lessac voice needs no manual model download:
.\crispasr.exe --backend piper -m auto --tts "Hello from Piper." --tts-output piper.wav
For another voice, download its GGUF from
cstr/piper-voices-GGUF.
Either pass the full path, or put it in CRISPASR_MODELS_DIR and pass its bare
filename. CrispASR resolves that exact file; it does not replace an unknown
community voice with the registered US default (#397).
$env:CRISPASR_MODELS_DIR = 'D:\ai\crispasr'
.\crispasr.exe --backend piper -m piper-en_GB-cori-medium-f16.gguf `
--tts "Hello from the Cori voice." --tts-output cori.wav
For the HTTP server and PowerShell, keep -t 8 separate from -l en and call
the real curl executable (PowerShell aliases curl in some versions):
.\crispasr.exe --server --backend piper `
-m piper-en_GB-cori-medium-f16.gguf -l en -t 8 --port 8089 `
--no-spoken-disclaimer --accept-marking-responsibility
curl.exe -sS http://localhost:8089/v1/audio/speech `
-H 'Content-Type: application/json' `
--data-raw '{"model":"piper","input":"Hello, how are you today?","spoken_disclaimer":false,"response_format":"wav"}' `
--output piper-server.wav
Writing a WAV with --output avoids sending binary audio through PowerShell’s
object pipeline. To test raw streaming in cmd.exe, request
"stream":true,"response_format":"pcm" and pipe curl.exe to
ffplay.exe -f s16le -ar 22050 -ac 1 -nodisp -.
| Backend | Why pick it | Voice cloning | First-run download |
|---|---|---|---|
csm |
Sesame CSM-1B: Llama-3.2 1B backbone (first-codebook AR) + 100M depth decoder (codebooks 1–31) + Kyutai Mimi codec (32-codebook RVQ → SEANet) @ 24 kHz. Single GGUF. Apache-2.0. | No (single built-in voice) | ~1.4 GB via -m auto (single Q4_K GGUF) |
dia |
Nari Labs Dia 1.6B: byte-level text encoder (12L) + AR audio decoder (18L GQA) + 9-codebook DAC codec @ 44.1 kHz. CFG-guided, dialogue-style with [S1]/[S2] speaker tags. Apache-2.0. |
No (dialogue via speaker tags) | ~1.6 GB via -m auto |
zonos-tts |
Zyphra Zonos-v0.1-transformer: 26-layer GQA AR transformer → 9-codebook DAC @ 44.1 kHz. Rich conditioning: speaker embedding + text + emotion + FWHM pitch/tempo. CFG guided. Voice cloning from any reference WAV (pass via CRISPASR_ZONOS_SPEAKER_EMB_PATH or --voice <ref.wav>). Apache-2.0. |
Yes (--voice <ref.wav>) |
~1.6 GB Q8_0 (default) or ~931 MB selective-Q4_K (heads/embeddings kept F16, auto-retry guard) or ~3.0 GB F16, via -m auto + 104 MB DAC codec. |
bark |
Suno Bark: 3-stage GPT-2 (text→semantic→coarse→fine) + EnCodec 24 kHz decoder. All sub-models packed into one GGUF. Supports speaker conditioning via .npz prompts. MIT license. |
Yes (--voice <speaker.npz>) |
~423 MB via -m auto (selective Q4_K) |
speecht5 |
Microsoft SpeechT5 80M: char-level encoder (12L) + AR mel decoder (6L) + 5-conv postnet + HiFi-GAN @ 16 kHz. MIT. Speaker via 512-d x-vector. | Yes (--voice <xvector.bin>, raw float32) |
~300 MB via -m auto (F16 GGUF) |
fastpitch |
NVIDIA FastPitch 60M: non-autoregressive parallel TTS — 6L FFTransformer encoder + duration/pitch predictors + length regulator + 6L FFTransformer decoder + HiFi-GAN @ 22 kHz. Deterministic (no sampling). CC-BY-4.0. | No (single speaker) | ~230 MB via -m auto (Q8_0 GGUF) |
bananamind-tts |
BananaMind-TTS-V2.1 13M: Tacotron-lite (char tokenizer + Conv1d+BN+ReLU encoder + BiLSTM + AR GRU decoder with location-sensitive attention + postnet) + HiFi-GAN @ 22 kHz. English (LJ Speech) and German (ThorstenVoice). Apache-2.0. Runtime is designed as a template for standard Tacotron2 ports — add decoder_rnn_type=lstm to the GGUF to switch to the LSTM decoder path (architecture notes). |
No (fixed voice per locale) | ~40 MB Q8_0 / ~50 MB F32 per locale |
parler-tts |
Parler TTS Mini v1.1 (~900M): T5 encoder + MusicGen decoder + DAC 44.1 kHz. Apache-2.0. Prompt-conditioned: describe the voice in natural language via --instruct. |
No (prompt-conditioned) | ~900 MB via -m auto (Q8_0 GGUF) |
voxcpm2-tts |
VoxCPM2: 2B Qwen2 backbone + flow matching + VAE decoder @ 48 kHz. Zero-shot voice cloning via --voice <ref.wav>. |
Yes | ~2.4 GB via -m auto |
pocket-tts |
Kyutai Pocket TTS 100M: continuous-latent AR @ 12.5 Hz + one-step LSD flow head + Mimi VAE decoder → 24 kHz. English, German, Spanish, Italian, Portuguese, plus the upstream French 24L preview. CC-BY-4.0 plus gated-use conditions. Voice cloning via --voice ref.wav. |
Yes (--voice) |
~124 MB Q8_0 per non-English 6L model; English F16 ~220 MB; French 24L Q8_0 ~365 MB |
kugelaudio |
KugelAudio-0-Open: 7B Qwen2.5 backbone + 4-layer DiT diffusion head (20-step SDE-DPMSolver++) + acoustic VAE decoder → 24 kHz. 23 languages. MIT. | Pre-encoded voices (--voice voice.gguf) |
~17.3 GB F16 via -m auto — needs >16 GB VRAM, else --no-gpu. The ~5.7 GB Q4_K is not a usable substitute: it stutters and loops (WER 0.72 vs 0.056 for F16) |
tada-1b |
HumeAI TADA 1B: Llama-3.2-1B backbone + per-token flow-matching diffusion head + TADA codec → 24 kHz. English-only. -m auto downloads model + default tada-ref.gguf. |
Yes (--voice <tada-ref.gguf>, English voice refs only) |
~1.7 GB Q4_K + ~1 GB codec |
tada / tada-3b-ml |
HumeAI TADA 3B Multilingual: same architecture, 3B params. Supports ar, ch, de, es, fr, it, ja, pl, pt in addition to English. -l <lang> auto-downloads tada-ref-<lang>.gguf. |
Yes (--voice <tada-ref.gguf>) |
~4 GB Q4_K + ~1 GB codec |
lfm2-audio |
LiquidAI LFM2.5-Audio 1.5B: FastConformer encoder + LFM2 hybrid conv+attention backbone + 6L depthformer (8-codebook Mimi) + ISTFT detokenizer → 24 kHz. Interleaved text+audio generation. Also does ASR and speech-to-speech. LFM Open License v1.0 ($10M revenue cap). | No | ~1.5 GB Q4_K (JP) / ~1.6 GB Q5_K (EN) + ~157 MB detokenizer companion |
dots-tts |
rednote-hilab dots.tts-soar: Qwen2.5-1.5B LLM + 24L VAESemanticEncoder + 18L DiT flow-matching head (16-step Euler CFG) + BigVGAN vocoder → 48 kHz. Continuous-latent AR (patch-by-patch). Apache-2.0. The CFG flow-match needs an F16 DiT — a full-q8 core derails; use the mixed quant (crispasr-quantize keeps the DiT at F16, quantizes the LLM+PatchEncoder to Q8_0 or Q4_K). CAM++ reference-WAV voice cloning is supported when the speaker companion is present. |
Yes (--voice ref.wav --i-have-rights) |
~3.1 GB mixed-Q8 / ~2.2 GB mixed-Q4_K core + 330–345 MB vocoder companion |
fireredtts3 |
FireRedTeam/FireRedTTS3: Qwen3-1.7B LLM + 8L PatchEncoder + 11L DiT flow head over continuous 64-d 25 Hz RedAE latents; RedAE Qwen3 encoder/decoder + Vocos ISTFT head -> 24 kHz; CAM++ 512-d speaker encoder. Zero-shot ICL voice cloning (24 languages + 21 Chinese dialects): --voice ref.wav --ref-text "<transcript>" (auto-transcribed when –ref-text is absent); without --voice, a default English prompt baked into the core GGUF is used. Apache-2.0. Like dots-tts, the DiT flow head must stay F16 — crispasr-quantize quantizes only the LLM backbone. |
Yes (--voice ref.wav --i-have-rights) |
~1.6 GB mixed-Q4_K core + ~1.0 GB RedAE companion |
confucius4-tts |
NetEase Youdao Confucius4-TTS: GPT-2 T2S (24L/1280d, beam-sample num_beams=3, LlamaTokenizer vocab baked) + flow-matching DiT+WaveNet S2A (25-step Euler CFG 0.7, native w2v-BERT + CAMPPlus style/reference-mel conditioning) + BigVGAN vocoder → 22.05 kHz. Zero-shot voice cloning; 14 languages via Chinese LANGUAGE_TOKEN_MAP prompts (-l <lang>). Apache-2.0. Zero-shot only: without --voice conditioning the output is unintelligible by design. |
Yes (--voice ref.wav --i-have-rights) |
~376 MB Q4_K T2S + ~135 MB S2A + ~214 MB BigVGAN + w2v companion via -m auto |
mini-omni2 |
gpt-omni/mini-omni2: Whisper-small encoder + Qwen2-0.5B LLM with 8-stream architecture + SNAC 24 kHz decoder → 24 kHz. Also does ASR and speech-to-speech. MIT license. Requires --codec-model snac-24khz.gguf companion. |
No | ~1.0 GB Q4_K + ~80 MB SNAC companion |
voxtral-tts |
Mistral Voxtral-4B-TTS-2603: Ministral-3B AR backbone (26L GQA, NORMAL/adjacent-pair RoPE) + 3L bidirectional flow-matching acoustic transformer (8-step Euler ODE + CFG α=1.2, no positional encoding) + Voxtral codec decoder (ALiBi sliding-window attention + reflect-causal conv + ConvTranspose upsampling) → 24 kHz. 20 preset voices across 9 languages (en/fr/de/es/it/pt/nl/ar/hi); strong on French technical text. CC-BY-NC-4.0. | No (20 preset voices via --voice <name>, e.g. fr_female) |
~2.4 GB Q4_K / ~4.3 GB Q8_0 / ~8.2 GB F16 via -m auto |
All backends write mono WAV via --tts-output (22 kHz for piper/fastpitch/bananamind-tts/confucius4-tts, 16 kHz for speecht5, 44.1 kHz for melotts/dia/parler-tts/zonos-tts/supertonic, 48 kHz for voxcpm2-tts/dots-tts/irodori-tts/moss-tts-local/sidon, 24 kHz for most others). Programmatic callers don’t need this table: crispasr_session_output_sample_rate() returns the active backend’s output rate, with crispasr_session_input_channels() / output_channels() alongside (everything is mono today) — #332.
dots-tts supports zero-shot voice cloning from a reference WAV. Place the
CAM++ speaker companion (dots-tts-soar-spk-f16.gguf) beside the core model,
then pass the reference recording with the consent attestation:
crispasr --backend dots-tts \
-m dots-tts-soar-q4_k.gguf \
--codec-model dots-tts-soar-vocoder-q8_0.gguf \
--voice reference.wav --i-have-rights \
--tts "Hello from CrispASR." --tts-output cloned.wav
The speaker companion is loaded only when --voice is supplied. Without it,
the same backend produces text-conditioned speech using its default model
conditioning. If the speaker companion is missing, the CLI warns and ignores
the voice prompt; it does not silently claim that cloning succeeded.
The practical low-memory choice is the mixed Q4_K core: the LLM and PatchEncoder are quantized, while the DiT, projections, embeddings, and conditioning statistics remain at the precision required by the flow-matching loop. The Q8 core is the safer quality/size default; do not quantize the DiT.
The dominant cost is the DiT flow-match, not tokenization or the recurrent
LLM step. A local Apple M1 measurement using the Q4_K core, Q8 vocoder, Metal,
and an eight-patch cap took 24.2 s at 16 Euler steps: about 13.7 s in
flow-match, 10.4 s in the BigVGAN vocoder, and under 0.5 s in the LLM and
PatchEncoder. With the same seed and cap, CRISPASR_DOTS_ODE_STEPS=8 took
16.2 s (1.49x faster); this is a speed/quality trade-off, not a parity-preserving
optimization. Full utterances add patch-by-patch AR cost, so benchmark with a
known patch/audio cap and inspect the generated WAV.
For further experiments, CRISPASR_DOTS_FUSED_STEP=0 restores the legacy DiT
path for A/B testing; the persistent fused graph is the default on CPU/Metal.
CRISPASR_DOTS_CFG_INTERVAL=2 skips some unconditional CFG evaluations but is
approximate and should remain opt-in until the decoded output is checked.
CRISPASR_DOTS_TTS_BENCH=1 prints the stage timings used above.
TADA ships as two variants; only the 3B multilingual model can synthesise non-English text:
| Backend | Model | Languages |
|---|---|---|
tada-1b |
HumeAI/tada-1b | English only |
tada / tada-3b-ml |
HumeAI/tada-3b-ml | en + ar, ch, de, es, fr, it, ja, pl, pt |
TADA uses Llama 3.2 materials and is distributed under the Llama 3.2
Community License. Managed model and auxiliary downloads therefore require
--accept-license llama3.2; redistribution must retain the Llama notice and
follow its acceptable-use terms. The language reference packs are derived
from CC-BY-4.0 FLEURS clips and retain that attribution requirement.
Pass -l <lang> with the tada-3b-ml backend. CrispASR auto-downloads
tada-ref-<lang>.gguf from cstr/tada-tts-3b-ml-GGUF on first use (<200 KB
per language):
crispasr --backend tada-3b-ml -m auto -l fr \
--tts "La justice sans force est impuissante." \
--tts-output justice.wav
The tada-ref-fr.gguf encodes the acoustic fingerprint (speaker identity,
prosody) extracted offline from a 10 s FLEURS CC-BY-4.0 French clip. Once
cached it is reused for every subsequent French synthesis call. Available
language codes: ar, ch, de, es, fr, it, ja, pl, pt.
On the HTTP server the voice is per request — point voice at a different
tada-ref-*.gguf and the backend reloads it on the next call, no container
restart needed. Omitting voice (or "default"/"auto") keeps the
currently-loaded reference, so requests that don’t care pay nothing.
# request 1: French built-in voice
curl -s :8080/v1/audio/speech -d '{"input":"Bonjour.","voice":"tada-ref-fr.gguf"}' -o fr.wav
# request 2: German voice — switched live, same running server
curl -s :8080/v1/audio/speech -d '{"input":"Guten Tag.","voice":"tada-ref-de.gguf"}' -o de.wav
The name resolves like any other model path (absolute path, or a cache/registry
name that auto-downloads). Embedders going through the session C ABI get the
same capability via crispasr_session_set_voice(s, "tada-ref-de.gguf", NULL).
Generating a brand-new reference from raw audio+transcript at query time is not
yet wired into the server (it needs the encoder+aligner GGUFs loaded) — bake the
ref offline with the --make-ref pipeline below, then switch to it live.
To speak in a specific person’s voice, bake a ref GGUF from ~10 s of their speech. Two options — pure C++ (no Python) or the Python converter:
--make-ref (no Python needed)Place tada-encoder-f16.gguf and tada-aligner-en.gguf (they ship in the
same repo as the model — cstr/tada-tts-3b-ml-GGUF
for tada / tada-3b-ml, cstr/tada-tts-1b-GGUF
for tada-1b) next to your TADA model GGUF, then:
# Step 1 — bake the ref GGUF (one-time per speaker):
# --i-have-rights is required: baking IS the cloning step. The pack is
# stamped as clone-derived so step 2 still gets the consent gate and the
# spoken AI disclosure.
crispasr --backend tada-3b-ml -m auto \
--make-ref \
--voice speaker_10s.wav \
--ref-text "Exact words spoken in the audio." \
--make-ref-output tada-ref-custom.gguf \
--i-have-rights
# Step 2 — synthesise with that voice:
crispasr --backend tada-3b-ml -m auto \
--voice tada-ref-custom.gguf \
--tts "Bonjour, comment allez-vous ?" \
--tts-output result.wav \
--i-have-rights
The --make-ref pipeline runs entirely in C++: wav2vec2 aligner → BPE
tokenization → DP alignment → WavEncoder → LocalAttentionEncoder → GGUF
output. Auto-discovers the encoder + aligner GGUFs next to the model file;
override with --make-ref-encoder / --make-ref-aligner if needed.
pip install hume-tada
python models/convert-tada-ref-to-gguf.py \
--audio speaker_10s.wav \
--language fr \
--output tada-ref-custom.gguf
A 5–15 s clip of clean speech (no music/noise) produces the best fingerprint.
--language (Python) selects the language-specific TADA aligner; it must match
the language of the text you will synthesise.
The encoder pipeline (WAV+transcript → voice reference) is ported to C++/ggml.
Pre-converted GGUFs sit alongside the model in
cstr/tada-tts-3b-ml-GGUF
(and cstr/tada-tts-1b-GGUF for
tada-1b); --auto-download fetches them into the cache on demand:
| File | Size | Description |
|---|---|---|
tada-encoder-f16.gguf |
178 MB | Shared WavEncoder + 6-layer LocalAttentionEncoder + hidden linear |
tada-aligner-en.gguf |
1.1 GB | English aligner (wav2vec2-large + 128K-class Llama CTC head) |
The encoder GGUF is loaded by src/tada_encoder.{h,cpp}. The aligner GGUF
is loaded by the existing wav2vec2_load() runtime (same architecture).
To convert language-specific aligners:
python models/convert-tada-aligner-to-gguf.py \
--codec-repo HumeAI/tada-codec --language fr \
--output tada-aligner-fr.gguf
If --voice is omitted, the runtime uses tada-ref-<lang>.gguf when `-l
` / `--language ` to
select the language token for multilingual synthesis:
```bash
./build/bin/crispasr --backend chatterbox -m auto -l fr \
--tts "bonjour tout le monde" \
--tts-output out-fr.wav
```
The flag is wired into the T3 prompt, not concatenated into the spoken text.
With the rebuilt 2026-06-18 GGUFs, `-l fr` inserts the `[fr]` token after
`[START]` (token id 634 in the multilingual tokenizer) and changes the
generated speech-token stream. A local Q4_K smoke check with seed 123 showed
that no-language `bonjour tout le monde` roundtripped through Parakeet as
`Bonjour tout monde.`, while `-l fr` roundtripped as
`Bonjour tout le monde.`.
Quality is still model-dependent. The rebuilt artifacts fix the previous
tokenizer/model mismatch and make `-l` active, but some French Q4_K samples
remain heavily accented. Treat language-token checks as a wiring smoke test,
not a guarantee of native pronunciation.
For voice cloning, pass `--source-lang ` when the reference recording's
language differs from the requested output language (`-l` or `--target-lang`).
Chatterbox then follows the upstream V3 cross-lingual recommendation and uses
CFG weight 0 unless an explicit `--tts-cfg-scale` override is supplied. This keeps
the reference speaker's timbre while reducing transfer of the reference
language's accent. In server/session use, set the output language with the
target/source language setter and the recording language with
`set_tts_reference_language()`.
On the multilingual path the text is **NFKD-normalized** (then ASCII-lowercased)
before tokenization, matching upstream `MTLTokenizer.preprocess_text`. This
matters for scripts with precomposed diacritics: e.g. Arabic `أ`
(ALEF-WITH-HAMZA) decomposes to base alef + combining hamza, the form the model
was trained on. Without it, partial-diacritic Arabic produced spurious onset
letters (#170). Script-specific normalizers (zh cangjie / ja kakasi / he dicta /
ko jamo / ru stress) are not yet implemented.
> **Note on legacy GGUFs.** Some older multilingual T3 artifacts pair a
> 2352-token tokenizer with a 2454-vocab T3; the loader rejects that mismatch.
> Repair locally with `models/patch-chatterbox-gguf-add-merges.py` and the
> matching `grapheme_mtl_merged_expanded_v1.json` (2454 tokens).
### Performance
The compute-bound T3 AR decode is the slow stage. It runs on CPU by default on
Metal (GPU has higher per-step kernel-launch overhead for the many T=1 steps).
The CPU thread count defaults to `min(8, hardware_concurrency)`; override with
`CRISPASR_CHATTERBOX_THREADS=` (e.g. dial down on a heavily shared host).
Output is bit-identical regardless of thread count.
On **Vulkan** the T3 GPT-2 (turbo/nano) attention uses the explicit
softmax(QK^T)V path by default instead of `ggml_flash_attn_ext`: the Vulkan
flash-attention pipeline crashes on RADV (Radeon 780M, issue #402) while the
explicit path completes full syntheses there with both F16 and Q4_K weights.
Opt back into flash on Vulkan with `CRISPASR_CHATTERBOX_FLASH_ATTN=1`;
`CRISPASR_CHATTERBOX_NAIVE_ATTN=1` still forces the explicit path on every
backend (debug gate).
### Voice cloning
Two paths are supported. **The recommended path is the python baker
+ baked GGUF** — it's the workflow the upstream chatterbox project
ships, our parity is exact, and the C++ runtime treats the resulting
GGUF the same way it treats the built-in default voice. The native
24 kHz WAV path described below the baker is functional but
experimental — it ships its own caveats (see "Known issues" later).
**Step 1 — bake the voice GGUF (one-time per reference speaker):**
```bash
# Requires the upstream chatterbox-tts python package (pip install
# chatterbox-tts) or RESEMBLE_CHATTERBOX_SRC=/path/to/clone/src for a
# local source checkout. The model loads on CPU by default; pass
# --device mps / cuda for faster baking. Reference WAV can be any
# sample rate / channel count — the baker resamples to 16 kHz for
# the VoiceEncoder + S3Tokenizer paths and 24 kHz for the prompt mel.
python models/bake-chatterbox-voice-from-wav.py \
--input samples/jfk.wav \
--output my_voice.gguf \
--i-have-rights
```
`--i-have-rights` is required — baking is the cloning step, and everything
downstream just replays the pack. The baker stamps
`crispasr.voice.cloned_from_recording` into the output so the runtime's
consent gate and Art. 50(4) spoken disclosure recognise the `.gguf` as a
clone at synthesis time. Chatterbox has **no `.wav` cloning path**, so
before that stamp existed a baked voice could never trip either gate; see
[`eu-ai-act.md`](/CrispASR/docs/eu-ai-act.html#62-art-504--deepfake-disclosure). A pack baked
by an older CrispASR carries no stamp and reads as a preset — re-bake it.
The baker runs upstream `ChatterboxTTS.prepare_conditionals(wav)` and
writes five tensors plus two scalar metadata keys, using the same
names the runtime already accepts for the built-in default voice
(`conds.t3.{speaker_emb, speech_prompt_tokens}`,
`conds.gen.{prompt_token, prompt_feat, embedding}`,
`chatterbox.conds.{emotion_adv, gen_prompt_token_len}`). Output
size is ~150-200 KB regardless of reference WAV length.
**Step 2 — synthesise with the baked voice:**
```bash
./build/bin/crispasr --backend chatterbox -m auto \
--voice my_voice.gguf \
--tts "Cloned voice synthesising arbitrary text." \
--tts-output cloned.wav \
--i-have-rights
```
`--voice` is per-call cached, so server callers (`--server` mode) can
switch voices between requests without reloading on every synthesise.
**Direct `--voice .wav` — native cloning, no python required**
(experimental). The C++ runtime now runs the full VoiceEncoder +
S3Tokenizer V2 + CAMPPlus + 24 kHz Matcha mel pipeline in-process and
forks on the input sample rate:
- **24 kHz mono PCM16/F32 WAV** — atomic clone. Resamples 24 → 16 kHz
via a Kaiser-windowed sinc polyphase resampler, then computes all
five conds (`speaker_emb`, `speech_prompt_tokens`, `gen.prompt_token`,
`gen.prompt_feat`, `gen.embedding`) from the same source audio and
installs them together. ASR roundtrip on `samples/jfk.wav`
(resampled to 24 kHz) with prompt "Ask not what your country can
do for you." returns the prompt verbatim — the cloned voice path
works end-to-end.
- **16 kHz mono PCM16/F32 WAV** — **NOT a real clone**. Only the
T3-side conds (`speaker_emb`, `speech_prompt_tokens`) are
installed; S3Gen renders with the **default voice's** `gen.*`
bundle. The output sounds like the default voice, not the
reference speaker. The path exists as a stepping stone in the
module ladder; for actual voice cloning, use the 24 kHz WAV
branch above OR the python baker (recommended). Re-encode the
reference at 24 kHz mono (`ffmpeg -i in.* -ar 24000 -ac 1 ref.wav`)
to get a real clone.
**Known issues for the native path**:
- **Default backend split is hardware-dependent.** On Metal (Apple
Silicon) the default is full CPU because (a) T3's 30-layer × 86-step
AR loop is dominated by Metal kernel-launch overhead — measured 50 s
on CPU vs 75 s on T3-GPU + S3Gen-CPU on M1 — and (b) S3Gen UNet1D
on Metal has compound per-op precision drift across mul_mat / FA /
norm / add / gelu / tanh / softplus that the 10-step CFM Euler
solver amplifies ~1000× into the documented `s3gen_mel cos≈0.92`
collapse. On non-Metal GPU builds (CUDA / Vulkan), the default keeps
T3 on GPU and S3Gen on CPU — the S3Gen-on-GPU drift has only been
bisected on Metal but the same compound-precision class likely
applies to CUDA wmma / Vulkan cooperative-matrix paths with F16
intermediate state, so the safer S3Gen-CPU default ships on every
GPU backend.
- **Env knobs to override the default:**
- `CRISPASR_CHATTERBOX_FULL_CPU=1` — force everything to CPU.
- `CRISPASR_CHATTERBOX_T3_GPU=1` — opt T3 back into GPU on Metal
(useful for benchmarking on M3+ where tensor cores may flip the
balance, or to verify correctness on CUDA-without-S3Gen).
- `CRISPASR_CHATTERBOX_FORCE_GPU=1` — put T3 *and* S3Gen on the GPU
backend. Output is garbled on Metal (vocoder amplifies drifted /
NaN mel into saturated audio). Kept as a diagnostic.
- `CRISPASR_S3GEN_UNET_CPU=1` / `CRISPASR_S3GEN_ENCODER_CPU=1` /
`CRISPASR_S3GEN_VOCODER_CPU=1` — pin individual S3Gen sub-graphs
to CPU when the parent context uses GPU. Diagnostic: the
documented Metal scheduler "upgrade to higher-priority backend"
pass undoes user-set CPU assignments for host-mapped buffers, so
these don't fully isolate sub-graphs on unified-memory devices.
- `CRISPASR_S3GEN_UNET_PIN_CPU_OP=` /
`CRISPASR_S3GEN_UNET_KEEP_GPU_OP=` — op-type bisect (handover
round 7). Pins the named op to CPU (or keeps only that op on GPU)
inside `build_graph_unet1d`. Useful for localising the next
suspect op in further drift work; doesn't fix end-to-end TTS
audio on its own because the vocoder still amplifies whatever
mel the UNet produces.
- **F16 vs Q-mat weights:** F16 mul_mat is bit-identical CPU↔GPU on
every quant. Quantised weights (Q4_K/Q5_K/Q6_K/Q8_0) need the
CrispASR ggml-metal patches (`kernel_mul_mv_q4_K_q8_K`,
`kernel_quantize_q8_K_f32`, `kernel_mul_mm_*_hp`) plus
`GGML_PREC_F32` op tagging to reach the F32-precise path on Metal.
T3 carries those tags (see `chatterbox.cpp:build_graph_t3_kv`);
S3Gen tagging was tested and helped mul_mat alone but didn't break
the compound-drift chain, so it isn't currently applied. F16 T3 +
CPU S3Gen is the safest config if you need T3 on GPU and care
about exact-match output on Metal.
- T3 sampling can produce unrelated text on long technical prompts
(sampler drift). Short, common phrases work reliably; if a prompt
produces gibberish, try a different seed via `--seed ` (or the
legacy env `CRISPASR_CHATTERBOX_T3_SEED=`).
The parity-quality compute kernels are bit- or fp32-rounding-tight
against PyTorch — verified via `crispasr-diff chatterbox` on the
`ve_*`, `s3tok_*`, `campplus_fbank`, `campplus_xvector`, and
`prompt_feat_24k` stages. End-to-end output may drift from the
python baker due to the resampler differing slightly from librosa's
`kaiser_fast`; for perfect baker-equivalent cloning the
`models/bake-chatterbox-voice-from-wav.py` workflow remains
recommended.
If the WAV is not 16 kHz mono PCM16/F32, the runtime falls back to
the same hint-then-error path as before, pointing at the baker or
suggesting `ffmpeg -i in.* -ar 16000 -ac 1 ref.wav`.
The same `my_voice.gguf` works across all four chatterbox variants
(`chatterbox`, `chatterbox-turbo`, `kartoffelbox-turbo`,
`lahgtna-chatterbox`) since the cond tensor contract is shared.
**Optional: `--exaggeration`** is baked into the voice at conversion
time via `--exaggeration ` (default `0.5`); pass a different
value to the baker to produce a more / less expressive variant of
the same speaker. The C++ runtime reads
`chatterbox.conds.emotion_adv` from the loaded voice GGUF, so the
flag is honored without further wiring.
Companion sharing — the registry deliberately points multiple variants
at the same S3Gen file to avoid redundant downloads. Kartoffelbox-turbo
and chatterbox-turbo share the meanflow S3Gen verbatim; lahgtna and
chatterbox-base share the original S3Gen. Pulling any variant first
warms the cache for the rest.
| Variant | T3 default | S3Gen companion | Total |
|---|---|---|---:|
| `chatterbox` | T3 Q8_0 (610 MB) | base S3Gen Q8_0 (348 MB) | ~960 MB |
| `chatterbox-turbo` | T3 F16 (963 MB) | turbo S3Gen F16 (627 MB) | ~1.6 GB |
| `kartoffelbox-turbo` | T3 Q8_0 (623 MB) | turbo S3Gen F16 (shared) | ~1.25 GB |
| `lahgtna-chatterbox` | T3 F16 (1059 MB) | base S3Gen Q8_0 (shared) | ~1.4 GB |
Sampling controls:
| Flag | Default | Purpose |
|---|---|---|
| `--temperature` | runtime default 0.8 | AR sampling temperature (0 = greedy; runtime falls back to 0.8 when global default 0.0) |
| `--seed N` | 0 (non-deterministic) | RNG seed — same seed + same text = bit-identical audio |
| `--tts-steps N` | 10 (base/lahgtna) / 2 (turbo/kartoffelbox-turbo meanflow) | CFM Euler steps for the S3Gen mel-decoder |
| `--codec-model FNAME` | sibling autodetect | Explicit S3Gen GGUF path (overrides `-m auto` companion) |
Session-API knobs (runtime-settable via the session API, not CLI flags):
| Setter | Default | Purpose |
|---|---|---|
| `set_top_p(p)` | 1.0 | Top-p nucleus-sampling threshold for the AR T3 token loop |
| `set_min_p(p)` | 0.05 | Min-p sampling threshold |
| `set_repetition_penalty(r)` | 1.2 | Repetition penalty (1.0 = no penalty; > 1 discourages repeated tokens) |
| `set_cfg_weight(w)` | 0.5 | Classifier-free-guidance weight. 0 = unconditional; 0.5 = upstream default |
| `set_exaggeration(e)` | 0.5 | Emotion-exaggeration scalar. Raise for dramatic delivery, lower for monotone |
| `set_max_speech_tokens(n)` | 1000 | Upper bound on AR speech tokens per call (≈ 20 s at 50 Hz codes) |
| `set_tts_steps(n)` | (see CLI table) | Same as `--tts-steps`; settable without reloading the session |
**Quantized variants** (Q8_0, Q4_K) are supported — the
`crispasr-quantize` tool skips vocoder, F0-predictor, and embedding
tensors automatically (see [docs/quantize.md](/CrispASR/docs/quantize.html)). Turbo
size table for the alternate quants:
| Variant | T3 | S3Gen | Total |
|---|---:|---:|---:|
| Turbo F16 | 964 MB | 628 MB | 1,592 MB |
| Turbo Q8_0 | 629 MB | 350 MB | 979 MB |
| Turbo Q4_K | 457 MB | 245 MB | 702 MB |
The Conformer rel-pos parity gap that previously affected the C++
encoder closed in §80 (5 fixes: PE ordering, pos_bias_u/v transpose,
missing up_layer.conv, missing xscale-after-up_embed, attention
output head layout). encoder_out is now bit-exact to the Python
reference.
## Parler TTS — prompt-conditioned voice description
Parler TTS Mini v1.1 is a prompt-conditioned TTS model (~900M params):
T5 encoder processes a natural-language voice description, MusicGen-style
decoder generates audio codes, DAC codec decodes to 44.1 kHz PCM.
Apache-2.0 license. No reference audio or voice packs needed --- describe
the voice you want in text via `--instruct`.
```bash
# Auto-download (~900 MB Q8_0 GGUF on first run):
./build/bin/crispasr --backend parler-tts -m auto \
--instruct "A female speaker with a warm voice in a quiet room." \
--tts "Hello, this is a test of Parler TTS." \
--tts-output output.wav --seed 42
# Explicit model path:
./build/bin/crispasr --backend parler-tts \
-m parler-tts-mini-v1.1-q8_0.gguf \
--instruct "A young male speaker with an energetic tone." \
--tts "Welcome to CrispASR text-to-speech." \
--tts-output welcome.wav
```
The `--instruct` flag sets the voice description. If omitted, a default
description ("A female speaker with a warm, clear voice in a quiet room.")
is used. Output is 44.1 kHz mono PCM. Temperature (default 1.0) and seed
are supported for reproducible / diverse generation.
**Model file:**
[`cstr/parler-tts-mini-v1.1-GGUF`](https://huggingface.co/cstr/parler-tts-mini-v1.1-GGUF)
## TTS GGUF downloads
[`cstr/vibevoice-realtime-0.5b-GGUF`](https://huggingface.co/cstr/vibevoice-realtime-0.5b-GGUF) ·
[`cstr/vibevoice-1.5b-GGUF`](https://huggingface.co/cstr/vibevoice-1.5b-GGUF) ·
[`cstr/qwen3-tts-0.6b-base-GGUF`](https://huggingface.co/cstr/qwen3-tts-0.6b-base-GGUF) ·
[`cstr/qwen3-tts-1.7b-base-GGUF`](https://huggingface.co/cstr/qwen3-tts-1.7b-base-GGUF) ·
[`cstr/qwen3-tts-1.7b-voicedesign-GGUF`](https://huggingface.co/cstr/qwen3-tts-1.7b-voicedesign-GGUF) ·
[`cstr/qwen3-tts-tokenizer-12hz-GGUF`](https://huggingface.co/cstr/qwen3-tts-tokenizer-12hz-GGUF) ·
[`cstr/orpheus-3b-0.1-ft-GGUF`](https://huggingface.co/cstr/orpheus-3b-0.1-ft-GGUF) ·
[`cstr/kartoffel-orpheus-3b-german-natural-GGUF`](https://huggingface.co/cstr/kartoffel-orpheus-3b-german-natural-GGUF) ·
[`cstr/kartoffel-orpheus-3b-german-synthetic-GGUF`](https://huggingface.co/cstr/kartoffel-orpheus-3b-german-synthetic-GGUF) ·
[`cstr/snac-24khz-GGUF`](https://huggingface.co/cstr/snac-24khz-GGUF) ·
[`cstr/chatterbox-GGUF`](https://huggingface.co/cstr/chatterbox-GGUF) ·
[`cstr/chatterbox-turbo-GGUF`](https://huggingface.co/cstr/chatterbox-turbo-GGUF) ·
[`cstr/kartoffelbox-turbo-GGUF`](https://huggingface.co/cstr/kartoffelbox-turbo-GGUF) ·
[`cstr/lahgtna-chatterbox-v1-GGUF`](https://huggingface.co/cstr/lahgtna-chatterbox-v1-GGUF) ·
[`cstr/indextts-1.5-GGUF`](https://huggingface.co/cstr/indextts-1.5-GGUF) ·
[`cstr/parler-tts-mini-v1.1-GGUF`](https://huggingface.co/cstr/parler-tts-mini-v1.1-GGUF)
## F5-TTS — DiT flow-matching voice cloning
F5-TTS v1 Base is a DiT-based flow-matching TTS model with zero-shot
voice cloning from 3-15s of reference audio. MIT license. Architecture:
ConvNeXtV2 text encoder → 22-layer Diffusion Transformer with AdaLN-Zero
→ 32-step Euler ODE solver with CFG → Vocos iSTFT vocoder. Single GGUF
(~1.3 GB) containing both DiT and Vocos weights.
```bash
# Basic synthesis with voice cloning
./build/bin/crispasr --backend f5-tts -m auto \
--voice samples/jfk.wav \
--ref-text "Ask not what your country can do for you" \
--tts "Hello, how are you today?" \
--tts-output hello.wav --seed 42
# Without voice cloning (requires ref audio for now)
# F5-TTS always needs a reference audio + transcript pair.
```
The `--ref-text` flag provides the transcript of the reference audio.
This is required for F5-TTS (unlike indextts which conditions on audio
only). Output is 24 kHz mono PCM.
**Tokenization — English and Chinese (#294).** The model uses a 2545-entry
character/pinyin vocab. English (and other Latin text) tokenizes per character.
Chinese text is converted to pinyin the same way the upstream model expects
(`convert_char_to_pinyin`): a built-in g2p (`src/core/pinyin_g2p.*`) does
phrase-based segmentation → `pypinyin`-style TONE3 syllables (e.g. `zhong1`) with
tone-sandhi (不/一/third-tone), embedded from `pypinyin` via
`tools/gen_pinyin_data.py`. Token parity vs the reference is ~99.7% on the
`tests/test-pinyin-g2p` corpus; the residual is segmentation-boundary polyphones
that need full jieba (not embedded). Before #294 Chinese produced no audio (every
Han char hit the unknown token). Mixed English+Chinese in one sentence works.
**Output length (#294).** F5 estimates the number of mel frames from the
reference's speech rate (`ref_frames / ref_text_len`) × the generated text
length. A slow or expressive reference must not be truncated: the rate guard is
asymmetric (a generous upper bound, matching upstream which has none — an
over-estimate is just trailing silence that gets trimmed), so long sentences are
no longer cut off. `CRISPASR_F5_DURATION_CLAMP=0` restores the exact upstream
formula. Give an accurate `--ref-text`; a wrong/short reference transcript
distorts the rate.
**Model file:**
[`cstr/f5-tts-GGUF`](https://huggingface.co/cstr/f5-tts-GGUF)
### Performance (issue #294)
The whole cost is the ODE denoising loop — 32 steps × 2 (CFG) = **64 full 22-layer
DiT passes** over the ref+gen sequence. The vocoder and text encoder are <1%.
**Quantization is not an option** — flow matching accumulates per-op error 1408×
per synthesis, so anything below F16 is unintelligible (F16 is the only viable
format; see the model README). All speed comes from *fewer / smaller passes*:
| Lever | Speedup | How | Quality |
|-------|---------|-----|---------|
| **`--tts-steps 7`** | **~4.7×** | EPSS non-uniform step schedule (built in for n=5/6/7/10/12/16) | verified intact; drop to 10/12 (~3×) if a voice sounds rough |
| `--tts-steps 16` | ~2.0× | fewer uniform-ish steps | intact |
| `CRISPASR_F5_CFG_INTERVAL=2` | ~1.3× | reuse the unconditional CFG velocity between steps | intact at ≥16 steps |
| shorter reference clip (3–5 s) | scales with T | the DiT denoises ref+gen jointly, so a long ref inflates *every* pass (attention is O(T²)) | intact |
| `CRISPASR_F5_BATCH_CFG=1` | GPU-dependent | one 2×-batch CFG forward (matches upstream) instead of two; try on CUDA | intact |
| `CRISPASR_F5_EMBED_GPU=1` | ~1.3× (more on slow-CPU) | run the InputEmbedding (input_proj + conv-pos) on the GPU instead of the CPU; removes the per-step CPU stall | intact (byte-identical gate-off; roundtrip-identical gate-on). GPU builds only |
| `CRISPASR_F5_DIT_SKIP=2` | ~1.9× | DiTReducio temporal skip — reuse the cached step velocity every other step | intact at 32 steps; ≈ using `--tts-steps 16`, so use one or the other |
Stacking: `~4 s ref` + `--tts-steps 7` (+ `CRISPASR_F5_EMBED_GPU=1` on a GPU) compound.
**Do not** combine a low step count with `CFG_INTERVAL` or `DIT_SKIP` — too few
non-uniform steps + a stale/reused velocity degrades to noise (the runtime warns
below 16 steps). `EMBED_GPU` is orthogonal and stacks with any step setting.
`CRISPASR_F5_BENCH=1` prints the per-stage split (host_embed / dit_graph / vocos).
`-nfa` has no effect on F5 — the DiT always uses flash attention.
## IndexTTS — Chinese/English voice cloning
IndexTTS-1.5 is a zero-shot voice cloning TTS model. Given a short
reference WAV (~3-10 s), it reproduces the speaker's voice for arbitrary
text. Architecture: Conformer+Perceiver conditioning encoder → GPT-2
autoregressive mel-code generator (24 layers, 1280d, beam search) →
BigVGAN vocoder (24 kHz).
```bash
# Auto-download (~2.4 GB: GPT F16 + BigVGAN F16)
./build/bin/crispasr --backend indextts -m auto \
--tts "Hello world, this is a test." \
--voice reference_speaker.wav \
--tts-output cloned.wav
# Explicit paths
./build/bin/crispasr --backend indextts \
--model indextts-gpt.gguf \
--codec-model indextts-bigvgan.gguf \
--tts "Hello world." \
--voice reference_speaker.wav \
--tts-output hello.wav
```
The `--voice` flag points to any mono WAV file (16 kHz or 24 kHz) of the
target speaker. Longer clips (5-10 s) give better cloning fidelity.
The BigVGAN vocoder runs anti-aliased SnakeBeta on the CPU by default
(BigVGAN v2's upsample→activate→downsample sandwich; the raw activation
emits harmonics above Nyquist that fold back as audible click/buzz). On
M1 this adds ~5 % to the vocoder stage versus the aliased path. Two env
knobs for power users:
- `CRISPASR_INDEXTTS_VOCODER_RAW=1` — opt out of AA; fully GPU-graphable but
produces ~2 k impossible inter-sample jumps on speech. Use only for
reproducing the legacy / aliased benchmark.
- `--no-gpu` — keeps the whole vocoder graph on CPU. Recommended for
IndexTTS specifically: the GPT codes generate quickly either way, and
the AA custom op forces a GPU↔CPU sync per AMP block when mixed with
Metal, leaving GPU + AA the slowest of the four combinations.
Set `CRISPASR_INDEXTTS_DEBUG=1` for per-stage intermediate dumps (mel, conformer
blocks, perceiver output) useful for diff-testing against Python.
### IndexTTS Chinese text normalization
**Default in-process pipeline** handles ~95 % of real prompts:
- A port of upstream `tokenize_by_CJK_char` (every CJK codepoint is
whitespace-surrounded so SentencePiece emits one `▁` piece per
hanzi, matching the model's training distribution).
- The relevant subset of `TextNormalizer.char_rep_map` — full-width
CJK punctuation `，。：；！？、…“”‘’（）《》【】「」—～·` mapped to ASCII
before the CJK splitter runs. Notably `。` (U+3002) sits **inside**
the CJK Unicode range used by `tokenize_by_CJK_char`, so it must be
punct-mapped first or it gets split as a CJK character and the
model hallucinates an extra trailing syllable.
- ASCII upper-case for non-CJK runs (matches upstream's
`do_upper_case=True`).
Token IDs are bit-identical to Python `sentencepiece.SentencePieceProcessor.Encode`
on the preprocessed string. Mixed CJK+English
(`他用Python写了一个程序。`) works without special flags. Pass `-v` to
print the preprocessed string and token count (`indextts: text "..." ->
N tokens`) if output sounds wrong; the full BPE id dump
(`indextts: text_ids[...]`) needs verbosity ≥ 2, which only the library /
session API reaches — the CLI adapter pins verbosity to 1.
**ASR-validate Chinese output with a real Chinese ASR.** `whisper-base`
over-counts CER ~5× on Mandarin (we measured 21 % whisper-base vs
3.8 % `qwen3-asr-0.6b` on the same audio); use Qwen3-ASR, the upstream
Cohere transcribe API, or `whisper-large-v3` for CER numbers you can
trust.
**What the default does NOT do:** number → hanzi (`2025年` stays as
literal digits, which the model can't pronounce cleanly), pinyin tone
digits (`xuan4` is not restored to its real pronunciation),
English contractions inside Chinese text, dates / times / currency /
phone-number expansion. These need a full WFST-based normalizer
(upstream IndexTTS uses [wetext](https://github.com/pengzhendong/wetext)
on macOS, [WeTextProcessing](https://github.com/wenet-e2e/WeTextProcessing)
elsewhere). We deliberately don't vendor that engine — OpenFST + the
~1 MB of compiled `.fst` rule data is a heavy dependency for a feature
most TTS prompts don't need.
**Optional hook: `CRISPASR_INDEXTTS_TEXT_NORMALIZER`** delegates normalization
to any user-provided shell command that reads UTF-8 text on stdin and
writes the normalized text to stdout. The output is then run through
the default pipeline (CJK split + punct map + upper) before
SentencePiece.
Recommended setup with upstream wetext:
```bash
pip install wetext
# Then on every IndexTTS invocation:
CRISPASR_INDEXTTS_TEXT_NORMALIZER="python $REPO/tools/wetext-normalize.py" \
./build/bin/crispasr --backend indextts -m auto \
--tts "我有 3 个苹果，2025 年买的。" \
--voice ref.wav --tts-output out.wav
```
The wrapper at `tools/wetext-normalize.py` is a thin stdin → stdout
shim around `wetext.Normalizer`; it accepts wetext's normal flags
(`--lang zh`, `--traditional-to-simple`, `--remove-erhua`, etc.).
Without `wetext` installed, the wrapper passes input through
unchanged so the hook still degrades gracefully.
**Failure modes — all fall back silently to the raw text:**
- Hook command exits non-zero → warning to stderr, raw text used.
- Hook stdout is empty → raw text used.
- Hook isn't installed / `mkstemp` fails → warning, raw text used.
- Hook hangs → currently no timeout; the synthesis blocks until the
subprocess exits (no protection against a buggy normalizer). Use
with shell commands you trust.
**Security:** the env var IS passed to `system()` — the user is the one
setting it, so don't expose this hook to text inputs from untrusted
sources unless you also vet the env var.
When the hook fires you'll see no extra log line by default; pass
`-v` to confirm the post-hook tokenization (`indextts: text "..." ->
N tokens`).
---
## Irodori-TTS — Japanese voice cloning + emoji emotion control
Irodori-TTS v3 (500M) is an RF-DiT flow-matching model at 48 kHz via the
Semantic-DACVAE-Japanese-32dim codec. Needs the companion codec GGUF next to the
model (auto-discovered) or via `--codec-model`.
```bash
# Plain synthesis:
crispasr --backend irodori-tts -m irodori-tts-500m-v3-q8_0.gguf \
--codec-model dacvae-ja-32dim-f16.gguf --tts "こんにちは、世界。" --tts-output out.wav
# Zero-shot voice cloning from any reference WAV:
crispasr --backend irodori-tts -m irodori-tts-500m-v3-q8_0.gguf \
--codec-model dacvae-ja-32dim-f16.gguf \
--voice reference.wav --i-have-rights --tts "テスト。" --tts-output cloned.wav
# VoiceDesign (600M-v3): style/emotion control via --instruct:
crispasr --backend irodori-tts -m irodori-tts-600m-v3-voicedesign-q4_k.gguf \
--codec-model dacvae-ja-32dim-f16.gguf \
--instruct "落ち着いた大人の男性。深く響く声で丁寧に話している。" \
--tts "こんにちは、世界。" --tts-output voicedesign.wav
```
**Voice cloning** encodes the reference through the DAC-VAE encoder (resample →
−16 LUFS → latent) and conditions the DiT via speaker CFG. `--i-have-rights` is
the consent attestation; a spoken AI-disclosure is prepended unless
`--no-spoken-disclaimer`. A short, clean 5–15 s reference clones as well as a
long one and is faster.
**Emoji emotion control** — Irodori's emoji drive prosody (👂 whisper, 😮💨
breath, 😭 crying, …); include them in the text. See the model's
`EMOJI_ANNOTATIONS.md` for the supported set. Emoji outside the trained set are
harmlessly ignored.
**Output length** is set by the model's duration predictor (kanji unpack to a
variable number of mora, so a fixed chars/sec heuristic truncates). Pin the exact
frame count with `CRISPASR_IRODORI_T_LATENT=N`; the runtime's `duration_scale`
multiplier has no CLI flag today (the CLI adapter leaves it at 1.0), so the env
var is the only lever from the command line.
**Knobs (env):**
| Variable | Effect |
| --- | --- |
| `CRISPASR_IRODORI_CFG_SPEAKER` | Speaker-CFG strength for cloning (default 5.0). |
| `CRISPASR_IRODORI_T_LATENT` | Force the exact output latent-frame count. |
| `CRISPASR_IRODORI_DECODE_CHUNK` / `_CTX` | Overlap-save codec-decode window / context (auto for long outputs; `CHUNK=0` disables). Bounds peak decode memory; exact (byte-identical) output. |
| `CRISPASR_IRODORI_CODEC_GPU=1` | Run the DAC-VAE codec on the GPU under Vulkan (CPU by default there — validated clean on MoltenVK; confirm on your driver). `CRISPASR_IRODORI_CODEC_CPU=1` forces CPU. |
See also the shared [reference-conditioning cache](#reference-conditioning-cache)
and [streaming TTS output](/CrispASR/docs/streaming.html#streaming-synthesized-audio-out).
## Reference-conditioning cache
Voice cloning encodes the reference into a small conditioning blob — a DAC-VAE
latent (irodori) or Conformer/Perceiver + ECAPA conditioning (indextts) — which
is slow for long references. CrispASR caches it **content-addressed on the
reference audio**, in the runtime, so **every entry point** (CLI, server, C ABI,
language wrappers) skips the encode on a repeat reference automatically — output
is byte-identical to a fresh encode.
- Cache directory: `CRISPASR_TTS_REF_CACHE_DIR` (default: a `crispasr-tts-refcache`
folder under the system temp dir).
- Disable entirely with `CRISPASR_TTS_REF_CACHE=0`.
No flag is needed to enable it — it's on by default. (f5-tts caches its
auto-transcribed reference transcript separately, next to the voice file.)
---
## Local speaker output (`--tts-play`)
Pass `--tts-play` to play TTS output through the local speaker immediately
after synthesis, in addition to (or instead of) writing a file. The
spread-spectrum watermark is embedded before playback (unless disabled via
`--no-watermark` / `CRISPASR_NO_WATERMARK`), so the audio leaving the speaker
carries the provenance marker.
```bash
# Synthesize and play through default speaker
crispasr --tts "Hello world." --tts-play -m model.gguf
# Write file AND play
crispasr --tts "Hello world." --tts-play -m model.gguf --tts-output output.wav
# Select a non-default output device (miniaudio playback device index)
crispasr --tts "Hello world." --tts-play --tts-play-device 2 -m model.gguf
```
Playback is synchronous — the CLI blocks until audio drains, then exits.
Device -1 (the default) selects the system default output device.
**Implementation note:** the device is opened at the hardware-native sample
rate (`sampleRate=0 / channels=0`). The model's mono float32 PCM is
pre-resampled via linear interpolation before the device starts. This avoids
miniaudio's 4× upsampler artefacts on devices that run natively at 96 kHz
(MacBook Air Speakers and many Core Audio devices).
## AI-generated audio provenance & watermarking
All TTS output is marked as AI-generated by default through multiple
complementary layers. The waveform watermark is on by default and can be turned
off by an operator who takes on the marking responsibility themselves (see
[Disabling the watermark](#disabling-the-watermark-operator-opt-out) below); the
file-metadata tags are always written.
### Spread-spectrum watermark (built-in, on by default)
A frequency-domain watermark embedded in the PCM signal after synthesis.
Survives re-encoding, volume normalization, and moderate compression.
The embedder writes only a ramped watermark delta back into the signal and
leaves under-covered FFT boundary samples untouched, so quiet starts/ends do
not become click impulses.
```bash
# Detect watermark in any audio file (C API)
crispasr_watermark_detect(pcm, n_samples) # returns confidence 0..1
```
### AudioSeal neural watermark (optional upgrade)
Meta's AudioSeal (MIT) provides stronger robustness via a learned
SEANet encoder-decoder. 100% cosine parity with the PyTorch reference.
```bash
# Convert model (requires pip install audioseal gguf)
python3 models/convert-audioseal-to-gguf.py -o audioseal.gguf
# Use with TTS
crispasr --tts "hello" -m kokoro.gguf --watermark-model audioseal.gguf
# Or let the registry fetch it: "auto" / "default" resolves the AudioSeal
# GGUF (auto-downloading when --auto-download is on). An unloadable model
# falls back to the built-in spread-spectrum watermark rather than failing.
crispasr --tts "hello" -m kokoro.gguf --watermark-model auto --auto-download
# Debug: CRISPASR_AUDIOSEAL_DEBUG=1 for shape traces, CRISPASR_AUDIOSEAL_DUMP_STAGES=1 for binary dumps
```
### Disabling the watermark (operator opt-out)
The waveform watermark is on by default. It can be turned off two ways:
- **CLI flag**: `--no-watermark` — **requires `--accept-marking-responsibility`**,
else the run is refused (exit 12) with a `[MARKING]` audit line when it is
honoured. The same gate covers `--no-spoken-disclaimer` and `--no-c2pa`, on
the CLI and at server startup.
- **Env var**: `CRISPASR_NO_WATERMARK=1` — takes effect without the attestation
flag, so it is the escape hatch for embedders, not the documented CLI path.
Either one disables watermark embedding for the whole process and logs, once:
```
crispasr: warning: watermarking disabled. AI usage marking responsibility rests with the operator.
```
**The opt-out has a floor.** When the chosen output container cannot carry a
C2PA manifest (raw `.aac` / `.opus` with `CRISPASR_NO_C2PA_REMUX=1`, or a build
with C2PA compiled out) the watermark is the only machine-readable mark left, so
`--no-watermark` is overridden and the mark is kept, with a note naming the file.
**Rationale.** Disabling the mark does not remove any AI-disclosure obligation
that may apply to the generated audio — it transfers responsibility for meeting
it to whoever runs the binary. The warning text is deliberately
jurisdiction-neutral and names no statute at runtime, because the marking
obligation is jurisdiction-specific (e.g. EU AI Act Art. 50, and analogous US
state and other rules) and CrispASR is a tool/component, not the "provider" that
those rules bind. A downstream operator that ships synthetic media is that
provider. Keeping the mark on is the zero-config default so the compliant path
is the path of least resistance; the opt-out exists for legitimate cases
(content not represented as authentic, research/testing, or an operator that
substitutes its own compliant marking scheme). See
[`docs/issue-260/PLAN.md`](/CrispASR/docs/issue-260/PLAN.html) for the full background.
> **Scope.** The flag/env opt-out applies to the `crispasr` CLI and the server
> (process-wide — there is no per-request toggle). It does **not** affect the
> C-API/language bindings: `synthesize()` always watermarks. Binding consumers
> that need unwatermarked PCM call `synthesize_raw()` and skip
> `watermark_embed()`, thereby assuming the marking responsibility themselves
> (see [`bindings.md`](/CrispASR/docs/bindings.html)).
### File metadata (always active)
- **WAV**: `LIST`/`INFO` chunk with `ISFT="CrispASR (AI-generated audio)"` and `ICMT` notice
- **MP3**: ID3v2 `TXXX` frames: `AI_GENERATED=true`, `GENERATOR=CrispASR`
### C2PA Content Credentials
Cryptographically-signed provenance manifests with
`digitalSourceType=trainedAlgorithmicMedia` (a C2PA `c2pa.created` action),
embedded directly in the output file. This is the "signed metadata" layer that
complements the waveform watermark.
**Build.** WAV signing needs **no build flags and no external library** — it is
handled by the built-in native signer (`src/core/crispasr_c2pa_native.h`),
a pure-C++ implementation of C2PA (hand-built CBOR / JUMBF / COSE_Sign1, ES256
via the vendored BSD-2 micro-ecc + a header-only SHA-256). It compiles into
`crispasr-lib` on every platform with zero dependencies. Its output validates
in the c2pa-rs reference reader. To additionally sign **MP3/M4A/FLAC**, fetch the
optional c2pa-rs library:
```bash
./scripts/fetch-c2pa.sh # downloads the prebuilt c2pa-rs lib → third_party/c2pa
# then cmake reconfigure; look for "C2PA signing enabled" at configure time
```
(Set `-DCRISPASR_NO_C2PA_NATIVE=ON` to drop the built-in signer, e.g. a build
that wants only the c2pa-rs path.)
**On by default (self-signed), on every platform.** Output is signed
automatically — no flags needed. Signing uses a fixed self-signed certificate
**baked into the binary** (`crispasr_c2pa_default_cert.h`), so it works
identically on desktop, mobile, and in the **WASM browser sandbox** (no
filesystem or openssl needed at runtime). Self-signed manifests are valid and
machine-readable (EU AI Act Art. 50); C2PA verifiers show "unverified signer".
The bundled private key is intentionally public — it only marks content as
AI-generated, it is not a trust anchor. (Regenerate with
`scripts/gen-default-cert-header.sh`.)
```bash
crispasr --tts "hello" -m kokoro.gguf --tts-output out.wav # signed automatically
crispasr --detect-watermark out.wav # (or verify at contentcredentials.org)
```
**Bring your own cert** for a *trusted* signer identity (a CA-issued
code-signing cert; verifiers then show the named issuer):
```bash
./scripts/generate-c2pa-cert.sh # or use your CA-issued cert + key
crispasr --tts "hello" --c2pa-cert crispasr-c2pa.crt --c2pa-key crispasr-c2pa.key
```
**Format support.** **WAV** (RIFF), **MP3** (ID3v2 GEOB), **M4A/MP4** (ISO BMFF,
`c2pa.hash.bmff.v3`), and **FLAC** (ID3v2 GEOB prepend) are **all** signed by the
built-in native signer (the `c2pa-audio` submodule) — **c2pa-rs is no longer
needed for any audio container**; `fetch-c2pa.sh` is optional/legacy.
**AAC and Opus** have no C2PA embedding path in their raw streaming containers
(ADTS / Ogg) — and neither does c2pa-rs. So when C2PA is active, CrispASR
**muxes AAC/Opus into an MP4 container** (`.aac` → `.m4a`, `.opus` → `.mp4`, via
the in-tree glint encoder + `crispasr_mp4_writer.h`) and signs that natively.
Explicit `.m4a`/`.mp4` output is always AAC-in-MP4. Set
`CRISPASR_NO_C2PA_REMUX=1` to keep the raw `.aac`/`.opus` container (watermark +
file-metadata tag only). The muxed MP4 output validates in the c2pa-rs reference
reader and plays in standard players.
**Bindings / mobile.** C2PA lives in the core C API (`crispasr_c2pa_sign` /
`crispasr_c2pa_free`, and `c2paSign()` in the wasm/JS binding), so any consumer
of `crispasr-lib` can sign — pass a WAV/MP3 container and get signed bytes back.
Build the library for the target with `-DCRISPASR_C2PA_FETCH=ON`:
- **Android**: links `libc2pa_c.so`; bundle it in the APK `jniLibs//` next to
`libwhisper.so` (verified: the arm64 prebuilt links cleanly with the NDK).
- **iOS**: links `libc2pa_c.a` statically; `crispasr_enable_c2pa` auto-links the
required Apple frameworks (Security / CoreFoundation / SystemConfiguration)
(verified: the arm64 prebuilt links cleanly with the iOS SDK).
- **WASM** — WAV signing works out of the box (no `--c2pa`, no ~10 MB c2pa-rs).
Three ways, in order of preference:
1. **Native-JS signer (module-free).** `bindings/javascript/c2pa.mjs` +
`c2pa-verify.mjs` — **pure WebCrypto** C2PA sign + verify (ECDSA P-256/ES256
+ SHA-256, hand-built canonical CBOR / JUMBF / COSE_Sign1) for WAV/MP3/M4A/
FLAC — **no c2pa-rs, no wasm module at all**. These are **vendored from the
`c2pa-audio` submodule** (`third_party/c2pa-audio/js/`, the single source of
truth) — CMake copies them in at configure time; don't edit the copies. Runs
in any browser, Node ≥16, Deno, or a Worker. Also on npm/pub.dev as
`c2pa-audio` / `c2pa_audio`. Usage:
```js
import { c2paSignWav } from 'crispasr/c2pa';
const wav = Module.pcmToWav(float32Pcm, 24000); // interop WAV + AI tag
const signed = await c2paSignWav(wav, certPem, keyPem); // full C2PA manifest
```
Covered by `npm test` in `bindings/javascript/` — 12 hermetic unit tests
plus a live parity test through the c2pa-python reference reader.
2. **Built-in native C++ signer via `c2paSign()`.** The native signer
(`crispasr_c2pa_native`) compiles into every wasm build, so
`Module.c2paSign(wavBytes, "audio/wav")` works **without `--c2pa`** and adds
no c2pa-rs weight — use this if you already hold a wasm `Module`. Both 1 and
2 emit the same manifest and validate in the c2pa-rs reference reader (only
status `signingCredential.untrusted`, expected for the self-signed cert).
3. **c2pa-rs in wasm (opt-in, MP3/M4A only)** via `./build-wasm.sh --c2pa` —
adds ~10 MB (the full Rust stack), off by default. When enabled, the module
is built with **`-fwasm-exceptions`** + **`-sSUPPORT_LONGJMP=wasm`** because the
prebuilt c2pa-rs emscripten lib uses **native wasm exceptions** (it imports the
`__cpp_exception` tag — provided only by native wasm EH, not the default
no-exceptions runtime nor JS-based `-fexceptions`); needs a wasm-EH browser
(all modern browsers, 2023+). Only needed for `c2paSign(bytes, "audio/mpeg")`
and other non-WAV containers in the browser.
**VLC Playback Bug (Silence Padding).** Because the C2PA JUMBF metadata chunk is extremely large (often 2-3 KB), some audio players (notably **VLC**) stall their playback thread for ~1.5 seconds while parsing it from the end of the file before playing the stream. For short TTS clips, this causes the first 1-2 seconds of the generated speech to be silently dropped during playback.
To work around this while keeping the C2PA metadata intact, you can explicitly pad silence frames at the beginning of the audio. By the time the player unblocks, the silence has played out and the speech begins unharmed.
- **CLI:** `--tts-pad-silence-ms 1500`
- **C ABI:** `crispasr_session_set_tts_pad_silence_ms(session, 1500)`
- **HTTP Server:** Add `"pad_silence_ms": 1500` to the JSON request body.
### Voice cloning consent gate
Voice cloning (`.wav` reference files) requires explicit consent:
```bash
# CLI: --i-have-rights flag required
crispasr --tts "hello" --voice speaker.wav --i-have-rights
# Server: consent_attestation field required in JSON body
curl -X POST http://localhost:8080/v1/audio/speech \
-d '{"input":"hello","voice":"speaker.wav","consent_attestation":"I have consent"}'
```
All consent attestations are logged with ISO 8601 timestamps.
### Post-embed watermark verification (automatic)
After writing a watermarked WAV in TTS mode, CrispASR automatically
reads back the in-memory PCM and runs watermark detection on it. If the
score lands in the `No watermark detected` band (see the verdict table
below), a warning is emitted to stderr. The self-check only runs for the
built-in spread-spectrum detector (not AudioSeal) and only on clips long
enough to score — **≥ 2.5 s** for the default per-frame statistic, **≥ 5 s**
for the legacy sign test — because the detector averages across frames and a
short clip reads near chance even when the embed worked. A bare `< 0.6` bar
used to warn on most sub-two-second clips. No extra flags needed; the
self-check is a diagnostic and never the marking gate (embedding is
unconditional).
### `--detect-watermark PATH` — standalone watermark detection
Reads a WAV file, runs watermark detection (spread-spectrum by default,
or AudioSeal if `--watermark-model` is given), prints the file, the detector
used, the analysed duration, the confidence score and a human-readable
verdict, then exits.
The default spread-spectrum statistic is the **per-frame** one (t-statistic +
decoy specificity); `CRISPASR_WATERMARK_DETECT=sign` selects the legacy
averaged-spectrum bin-sign test, which additionally prints the exact
probability of the score arising without a watermark.
| Detector | Score | Verdict |
|---|---|---|
| spread-spectrum, per-frame (default) | > 0.65 | `AI-GENERATED WATERMARK DETECTED` |
| spread-spectrum, per-frame (default) | > 0.5 – 0.65 | `INCONCLUSIVE - consistent with a watermark, but not statistically significant` |
| spread-spectrum, per-frame (default) | ≤ 0.5 | `No watermark detected (this does NOT mean the audio is human-made)` |
| spread-spectrum, sign (legacy) | p < 0.01 | `AI-GENERATED WATERMARK DETECTED` |
| spread-spectrum, sign (legacy) | p < 0.20 | `INCONCLUSIVE …` |
| spread-spectrum, sign (legacy) | otherwise (or score ≤ 0.5) | `No watermark detected …` |
| AudioSeal (neural) | > 0.5 | `AI-GENERATED WATERMARK DETECTED` |
AudioSeal returns a probability rather than a bin-agreement fraction, so no
p-value is quoted for it. Note that an unwatermarked file scores ~0.5 on the
spread-spectrum detector, not 0.
```bash
# Detect watermark using the built-in spread-spectrum detector
crispasr --detect-watermark output.wav
# Detect with AudioSeal neural watermark model
crispasr --detect-watermark output.wav --watermark-model audioseal.gguf
```
### Spoken disclaimer (voice clones only)
Voice-cloned output is automatically prefixed with "This audio was
generated by artificial intelligence" using a neutral default voice
(not the cloned voice), with a 300ms silence gap before the cloned audio.
The spoken disclaimer can be disabled per-request while keeping all
machine-readable provenance (watermark + C2PA) intact:
- **CLI**: `--no-spoken-disclaimer` — **requires `--accept-marking-responsibility`**,
else the run is refused (exit 12).
- **Server**: `"spoken_disclaimer": false` in the request body — **requires a
`"marking_attestation": ""` field alongside it**, or a server launched
with `--accept-marking-responsibility` (the operator accepts the duty for every
response the process serves, which covers each request).
When the spoken disclaimer is suppressed, the caller assumes
responsibility for providing appropriate AI-disclosure to end users
(e.g. a visual label in the UI) — that is exactly what the attestation
records. The spread-spectrum watermark and C2PA metadata are always
embedded regardless of this setting.
**An unattested opt-out is denied, not refused** (#312, from v0.8.24). A server
request that sets `"spoken_disclaimer": false` without any attestation still gets
its audio — with the spoken disclaimer applied, i.e. the documented default. The
denial is announced, never silent:
```
X-Crispasr-Spoken-Disclaimer: applied
X-Crispasr-Marking-Warning: 'spoken_disclaimer': false ignored - it requires a
'marking_attestation' field (or a server launched with
--accept-marking-responsibility); served with the
spoken AI-disclaimer
```
plus a `[MARKING] … no_spoken_disclaimer=DENIED` line in the server log. Serving
the *stronger* default can't emit weaker-than-default output, and it doesn't hard-
break clients written before the field existed — v0.8.22/v0.8.23 returned
`400 marking_attestation_required` for that request instead, which took out voice
cloning for every Subtitle Edit build up to v5.1.0-rc16. The CLI keeps its hard
refusal: there the operator can just add the flag.
**Latency (by design).** The disclaimer is a *full synthesis pass* of the
neutral sentence on the loaded backend — not a pre-recorded clip — so it adds
latency proportional to that one fixed sentence. It is cached per process and
reused: a long-running **server** pays this cost once (on the first voice-clone
request) and not again, whereas the **CLI** re-synthesizes it on every
invocation because each run is a fresh process with a cold cache. For
continuous-latent AR backends such as dots.tts the disclaimer is a complete
autoregressive generation, so on a short clone it can roughly double the
wall-clock time of a single CLI call (a constant overhead that becomes
negligible for longer text and for repeated server requests). Pass
`--no-spoken-disclaimer` / `"spoken_disclaimer": false` to skip it when you
provide AI-disclosure another way — the watermark and C2PA provenance are
still embedded.
## Supertonic-3 — fast multilingual on-device TTS
[Supertone/supertonic-3](https://huggingface.co/Supertone/supertonic-3) — a
~99 M-param non-autoregressive flow-matching TTS built for on-device speed.
The upstream distribution is ONNX-only; CrispASR runs a native ggml port of
all four graphs (duration predictor, text encoder, vector estimator, vocoder),
converted into ONE GGUF that embeds the unicode indexer, the NFKD text tables
and all ten preset voice styles. 44.1 kHz output, 31 languages.
Weights are **OpenRAIL-M**: commercial use allowed, but the license carries
use restrictions and an attribution requirement (printed on first download).
The open release has FIXED preset voices only — `--voice` selects a preset
name (`F1`..`F5`, `M1`..`M5`), not a reference WAV; there is no voice cloning.
```bash
# English, default voice M1
crispasr --backend supertonic -m auto --tts "Hello from Supertonic." -o out.wav
# German, female preset, a little faster, fewer flow steps
crispasr --backend supertonic -m auto --tts "Guten Tag, wie geht es dir?" \
-l de --voice F2 --tts-speed 1.1 --tts-steps 8 -o out_de.wav
```
- `-l LANG` — one of: en ko ja ar bg cs da de el es et fi fr hi hr hu id it
lt lv nl pl pt ro ru sk sl sv tr uk vi (language is conditioned via
`` tags in the character stream; there is no phonemizer).
- `--tts-steps N` — flow-matching Euler steps (default 8). Each step runs two
vector-field passes (classifier-free guidance is part of the model's
update rule: `v = 4·v_cond − 3·v_uncond`).
- `--tts-speed X` — multiplies the upstream default rate 1.05.
- Long text is chunked at sentence boundaries (300 chars; 120 for ko/ja) and
joined with 0.3 s silences, matching the upstream SDK.
- Env: `SUPERTONIC_BENCH=1` prints per-stage timings.