RedNote Open-Sources dots tts, Challenging the Token-Based TTS Route

RedNote has released dots tts, a 2 billion-parameter speech model that challenges the discrete-token architecture behind many leading text-to-speech systems. The company says its fully continuous pipeline achieved the best average content accuracy and speaker similarity on three Seed-TTS-Eval subsets. That combination matters because open TTS developers have usually had to trade fidelity, stable generation, and practical inference speed against one another.

The June release includes training and inference code, pretrained and post-trained models, and a distilled checkpoint under Apache 2.0. RedNote also published a technical report on June 5 and revised it on August 10. The resulting package gives developers more than a polished demonstration. It provides a reproducible base for voice cloning, multilingual synthesis, fine-tuning, and low-latency serving.

The central contest is architectural. Models such as Qwen3-TTS use learned speech tokens, while dots tts keeps speech inside a continuous latent space throughout its core generation path. VoxCPM2 also follows a continuous-latent route, making RedNote's release part of a broader test. The industry is now measuring whether continuous representations can scale without sacrificing control, language coverage, or deployment efficiency.

What RedNote Actually Released

The important change is not another voice demo, but the release of a trainable and deployable TTS foundation under a permissive license.

RedNote's dots team released three initial versions of the same 2 billion-parameter architecture. The base checkpoint represents the pretrained model and supports further fine-tuning. The SOAR checkpoint adds self-corrective post-training and targets higher speaker similarity. The MeanFlow-distilled version reduces the acoustic sampling workload for latency-sensitive applications.

The complete release includes model weights, inference code, fine-tuning code, and the process needed to distill a new MeanFlow checkpoint. Code and released checkpoints use the Apache 2.0 license. That scope distinguishes the project from releases that provide weights but leave key training or adaptation stages private.

The model generates 48 kHz audio and supports zero-shot voice cloning. Zero-shot cloning means reproducing a speaker from reference audio without retraining the model for that person. Its recommended continuation mode uses both a short audio sample and the sample's exact transcript. An x-vector mode can work from audio alone, although RedNote says the paired transcript method produces better similarity.

The accompanying technical report was first submitted on June 5, 2026. Its August revision added experiments, efficiency results, and updated technical details. RedNote describes the system as multilingual but does not disclose the training corpus size in the report. That omission limits direct scale comparisons with competitors that publish training-hour totals.

Its headline Seed-TTS-Eval results cover English, Chinese, and a difficult Chinese subset. The SOAR checkpoint recorded error rates of 1.30%, 0.94%, and 6.60% across those respective sets. Its speaker-similarity scores were 77.1, 81.0, and 79.5.

Averaging those published results gives dots tts a 2.95% error rate and a 79.2 similarity score. RedNote says both are the best averages among the systems in its comparison. The claim refers to a benchmark table assembled from original publications or default open-source releases, not a new independent evaluation of every competitor.

Those qualifications do not erase the result. They define it. RedNote has released an open system whose reported benchmark balance is unusually strong, while also exposing the code needed for others to test that balance.

Why dots tts Avoids Speech Tokens

dots tts treats continuous speech representation as the foundation, not as a final reconstruction step after token generation.

Many modern TTS systems convert audio into discrete codes. Those codes resemble a vocabulary that a language model can predict. This design fits existing language-model infrastructure, but quantization can discard acoustic detail and divide speech across several codebooks.

RedNote instead trains an AudioVAE, a variational autoencoder that compresses waveforms into continuous acoustic representations. Its design uses multiple objectives so the latent space retains both linguistic structure and fine audio information. The model predicts those representations directly, without inserting discrete speech tokens into the generation chain.

The pipeline starts with a semantic encoder and a Qwen2.5-based language-model backbone. An autoregressive acoustic head then predicts the next continuous speech patch. Flow matching, a generative method that learns a path from noise to data, turns each prediction into an acoustic latent for the 48 kHz decoder.

Autoregression still gives the model a sequential dependency. Each generated patch depends on prior context. However, the predicted object is a continuous acoustic representation rather than a codebook entry. This structure lets dots tts retain an LLM's sequence-modeling strengths while avoiding a separate discrete codec vocabulary.

The acoustic head receives full generation history instead of a narrow recent window. RedNote argues that this conditioning helps preserve long-range speaker and acoustic consistency. It also targets drift, where a cloned voice or speaking style gradually changes during a longer passage.

A continuous design creates its own computational problem. Flow-based acoustic generation can require several evaluations for every output patch. A system may sound good in an offline test yet remain too slow for an assistant, live translation tool, or interactive character.

RedNote addresses that problem through classifier-free guidance-aware MeanFlow distillation. Distillation trains a smaller or faster generation process to imitate a more expensive one. In this case, the MF checkpoint reduces acoustic generation to four recommended evaluations while retaining much of the original model's benchmark performance.

The report claims first-packet latency of 85 milliseconds in output-streaming mode and 54 milliseconds in dual-streaming mode. These figures describe the authors' experimental configuration and should not be read as universal production latency. Hardware, concurrency, prompt length, compilation, and audio buffering can all change the result.

RedNote later added an optimized inference path. Its repository reports median real-time factors between 0.13 and 0.20 for selected text-only and cloning configurations. A real-time factor below one means generation completes faster than the resulting audio duration.

The architecture therefore makes a specific bet. Continuous representations can preserve more acoustic information, while distillation and optimized serving can make their higher decoding cost manageable. If that bet holds across independent deployments, discrete speech tokens stop looking like the only practical route to scalable neural TTS.

The Benchmark Win Is About Balance

RedNote's strongest argument is not that dots tts wins every column, but that it avoids a severe average tradeoff between words and voices.

Seed-TTS-Eval measures two different properties. Word error rate estimates whether an automatic speech recognizer recovers the intended content. Speaker similarity compares generated audio with the reference speaker using a speaker-verification model. Lower error and higher similarity are desirable.

Dots tts does not lead every individual subset. Its SOAR checkpoint posts a 0.94% Chinese error rate, while MiniMax-Speech records 0.83% in RedNote's table. Qwen3-TTS reaches 6.76% on difficult Chinese, close to dots tts at 6.60%. Seed-TTS records 79.6 similarity on Chinese, compared with 81.0 for dots tts.

The average tells a different story. RedNote reports 2.95% average error and 79.2 average similarity for SOAR. Its table places the model ahead of CosyVoice 3, F5-TTS, FireRedTTS-2, IndexTTS 2, Qwen3-TTS, Seed-TTS, and VoxCPM2 on the combined average.

The pretrained checkpoint actually produces a slightly lower 2.92% average error, while SOAR improves average speaker similarity. That difference illustrates the model's post-training tradeoff. Alignment can strengthen identity preservation without improving every content metric.

RedNote says SOAR uses reward-free self-corrective post-training. The system generates candidate speech, identifies difficult or defective outputs, and learns corrections without depending on a separately trained reward model. This approach concentrates training on failure cases while avoiding reward-model bias.

The benchmark comparison also pressures better-known open systems. The Qwen3-TTS report describes a family trained on more than 5 million hours across 10 languages. Its architecture uses discrete speech tokenizers and offers voice design, cloning, and streaming generation. RedNote reports strong average results without revealing a comparable training-hour figure.

CosyVoice 3 provides another scale reference. Its authors expanded training from tens of thousands of hours to 1 million hours, increased model size to 1.5 billion parameters, and covered nine languages plus 18 Chinese dialects. Its post-training study emphasizes tokenizer design and a differentiable reward model.

This comparison makes the dots tts result more interesting, but not decisive. Seed-TTS-Eval emphasizes zero-shot cloning from roughly three-second prompts. It does not cover the complete product experience around instruction following, voice design, long conversations, operational cost, or safety controls.

Automated similarity is also an imperfect proxy for human judgment. A model can preserve a speaker embedding while missing cadence, emotional restraint, pronunciation, or recording style. Likewise, an ASR-based error score depends partly on the recognizer used by the benchmark.

RedNote supplies audio demonstrations and reproducible evaluation commands, which gives researchers a way to inspect its claims. Independent reruns on fixed hardware and shared decoding settings remain more valuable than copying a leaderboard rank.

The immediate result is still meaningful. dots tts places a continuous model near the front on content fidelity and speaker preservation at the same time. Competitors now need to answer that balance, not merely produce an attractive sample.

Continuous TTS Has Become a Real Opposing Route

The competitive pressure falls on token-based TTS because continuous models now combine credible quality, open weights, and increasingly practical serving.

Qwen3-TTS illustrates the token route clearly. Its 12 Hz tokenizer compresses speech into a low-rate stream, while a dual-track language model supports streaming synthesis. The report claims a 97-millisecond first packet and offers models at 0.6 billion and 1.7 billion parameters.

That structure benefits from mature autoregressive language-model tooling. Discrete tokens can be batched, cached, sampled, and served through infrastructure already designed for text generation. They also create a shared representation that may connect speech generation with broader audio-language tasks.

dots tts accepts more work inside the acoustic path. Its language-model backbone predicts conditions for a flow-matching head, which generates continuous patches before waveform decoding. This can preserve acoustic detail, but it complicates scheduling and adds multiple calculations per patch.

VoxCPM2 shows that RedNote is not alone. Its continuous architecture also avoids an external discrete speech tokenizer. The 2 billion-parameter model combines multilingual synthesis, voice design, style-controlled cloning, and continuation cloning. Its authors report training on more than 2 million hours covering 30 languages.

VoxCPM2 therefore offers broader declared language coverage and more integrated controls. dots tts answers with stronger reported Seed-TTS-Eval averages, 48 kHz continuous latents, full-history conditioning, and a focused distillation path. Neither model settles the architectural contest alone.

CosyVoice 3 occupies a hybrid position. It uses a learned speech tokenizer for semantic generation and a flow-matching model for acoustic reconstruction. Its design shows why the debate is not simply autoregression versus diffusion. The important distinction concerns where information becomes discrete, what the model predicts sequentially, and how the final acoustics are recovered.

Deployment infrastructure is starting to reduce the continuous route's disadvantage. The dots tts release provides an OpenAI-compatible speech endpoint through SGLang Omni, with continuous batching for its MeanFlow and fixed-step variants. Streaming PCM output also lets applications begin playback before the complete waveform is ready.

SGLang's published integration reports peak throughput of 4.76 requests per second and 19.86 generated audio seconds per second at concurrency 16. That test used one H100, the four-step MF checkpoint, and Seed-TTS-Eval English. Its reported error rate was 1.35%.

Those serving numbers come from a specific stack and workload. They cannot establish cost parity with every token model. They do show that continuous autoregressive TTS has moved beyond isolated research inference.

This shift matters for developers selecting a foundation. Discrete tokens still offer simpler sequence handling and a clearer path to multimodal token unification. Continuous systems offer a chance to avoid quantization loss and preserve subtle acoustic traits. Distillation determines whether that quality survives under production constraints.

The competition will likely turn on repeatable deployment outcomes. Teams will compare concurrent throughput, memory demand, first-audio latency, long-form stability, and quality after quantization. A leaderboard average opens the contest, but those operational measurements will decide adoption.

What the Numbers Do Not Establish

The release supports a credible technical claim, but it does not yet establish universal language quality, production safety, or independent benchmark leadership.

The first limitation appears in RedNote's multilingual results. On a 24-language benchmark, SOAR records the highest average speaker similarity at 83.9. Its average word error rate is 6.8%, compared with 2.8% for MiniMax and 3.7% for Fish Audio S2 in the same table.

Performance also varies sharply by language. SOAR reports word error rates of 36.19% for Arabic, 14.24% for Hindi, 4.96% for Turkish, and 3.89% for Vietnamese. Several European languages produce much lower errors, while speaker similarity often remains strong across the weaker subsets.

RedNote attributes this gap partly to the BPE text backbone. Byte-pair encoding divides text into learned subword units, and language coverage depends heavily on available training data. Scripts or languages underrepresented during training require more data before the model maps text reliably into speech.

That creates a practical warning. A multilingual label does not mean uniform readiness. A developer could clone a speaker's vocal identity convincingly while generating incorrect words in a lower-resource language. Similarity and intelligibility must be evaluated separately for each target market.

The released model is also speech-heavy. Its AudioVAE may be broadly capable in principle, but the backbone was trained mainly for speech. RedNote says the release does not cover singing or unified speech-and-sound generation. Product teams should not infer those capabilities from the continuous audio representation.

Long-form performance requires further scrutiny. Full-history conditioning targets drift, but longer history also raises memory and inference questions. Public benchmarks built around short utterances cannot establish stable narration across chapters, meetings, or continuous interactive sessions.

The benchmark evidence mostly comes from the model's authors. RedNote says baselines use original publications or default open-source configurations, but model versions and decoding settings can influence the comparison. Proprietary systems also prevent full replication of their training and inference conditions.

Expressiveness evaluation adds another uncertainty. RedNote reports results from EmergentTTS-Eval, where Gemini judges samples against an OpenAI TTS reference across six scenarios. Automated model judging helps scale evaluation, but it does not replace blinded human listening across accents, devices, and recording conditions.

Voice cloning creates the clearest nontechnical risk. Three seconds of speech can be enough to imitate a recognizable voice. That enables personalized accessibility tools, localized media, and authorized digital characters. It also lowers the effort required for impersonation, fraud, harassment, and fabricated evidence.

The project's risk guidance calls for consent-aware reference policies, synthetic-speech detection, watermarking, and clear labeling. Those recommendations are not built-in safeguards. Apache licensing gives downstream developers broad freedom, leaving implementation and enforcement to each deployment.

A serious production system therefore needs more than a model endpoint. It needs proof of consent for reference audio, restrictions on sensitive identities, auditable generation records, abuse monitoring, and a response process. Watermarking must also survive common edits and compression before it offers meaningful protection.

These constraints do not negate the release. They prevent a benchmark result from becoming an unsupported claim about every language or deployment. dots tts is a credible foundation, but its adopters still carry the burden of validation and governance.

Three Signals That Will Decide the dots tts Bet

The next stage is about ecosystem evidence, not another headline score.

The first signal is independent reproduction of Seed-TTS-Eval and multilingual results. Researchers need to rerun dots tts, Qwen3-TTS, CosyVoice 3, VoxCPM2, and other open models under matched hardware and decoding policies. Human listening tests should accompany automated content and speaker metrics.

A successful reproduction would strengthen RedNote's central claim that continuous speech modeling improves the quality balance. A large ranking change would weaken the benchmark narrative, although the release would remain useful as an open research base.

The second signal is deployment performance outside RedNote's own scripts. SGLang Omni already supports dots tts with streaming audio and continuous batching. Community projects have also produced Python and Swift MLX ports for Apple Silicon, plus a ComfyUI integration.

The official model collection now includes base, SOAR, MeanFlow, one-step, two-step, and double-streaming variants. That growing checkpoint range can lower latency, but each compression step must preserve pronunciation and speaker identity in real workloads.

Developers should watch sustained concurrent throughput, first-audio latency, memory use, and failure rates on long input. Broad support from serving frameworks would strengthen the case that continuous generation can match token systems operationally. Narrow dependence on expensive accelerators would weaken it.

The third signal is whether RedNote closes its language and safety gaps. Better Arabic, Hindi, Turkish, and Vietnamese content accuracy would show that the architecture can scale beyond its strongest data regions. Publishing corpus composition or targeted evaluation details would also make those improvements easier to assess.

Safety progress should be equally concrete. Useful evidence includes a durable watermark, a public cloning-consent interface, identity restrictions, and tested detection performance after compression. Documentation alone will not answer whether open voice cloning can be deployed responsibly.

RedNote has already extended the family with instruction-controlled speech editing and lower-step checkpoints. Those additions suggest that dots tts is being developed as a reusable speech foundation, rather than a one-time leaderboard release. They also expand the surface for quality and misuse testing.

For developers, the practical action is straightforward. Test the exact checkpoint, language, voice, hardware, and passage length required by the product. Compare cloned identity and content accuracy separately, then measure latency under realistic concurrency.

For researchers, dots tts provides a valuable experiment: can continuous acoustic modeling become the default base for scalable TTS? The release supplies enough code and weights to test that question directly. Its benchmark averages make the test worth running, while its multilingual and safety limits prevent an early verdict.

The dots tts story will not be decided by whether one demonstration sounds natural. It will be decided when independent teams can reproduce its balance, serve it economically, and control how cloned voices are used.