Tencent Open-Sources AngelSpec, Challenging One-Size-Fits-All Speculative Decoding

Tencent Hunyuan has open-sourced AngelSpec after reporting up to 2.40 times faster Hy3-A21B inference than standard autoregressive decoding. The framework combines training, evaluation, and deployment support for two different speculative decoding routes. Its central claim challenges a common assumption: one drafting method cannot efficiently handle every language model workload.

AngelSpec includes multi-token prediction, or MTP, which proposes several future tokens through a lightweight autoregressive drafter. It also introduces DFly, a block-diffusion system that drafts a longer group of tokens in parallel. Tencent says DFly delivered 10.5% to 11.8% higher throughput than DFlash across its tested serving conditions.

That comparison matters because DFlash had already pushed speculative decoding away from strictly sequential drafting. AngelSpec does not simply propose another faster drafter. It organizes two contrasting approaches around the workload each handles best, then adjusts verification according to runtime conditions.

The result is an open framework with a more operational view of inference acceleration. Developers can inspect the training path, test the released Hy3 draft models, and examine how performance changes with traffic. However, Tencent’s reported gains come from its own models, configurations, and evaluation design. Independent production results remain the decisive missing evidence.

AngelSpec Turns a Research Choice Into a Deployment Choice

AngelSpec treats speculative decoding as a workload-selection problem, not a contest for one universal drafting architecture.

Tencent’s researchers submitted the first version of the AngelSpec paper on July 28, 2026. A revised version followed on July 29. The company then announced the open-source release with training code and Hy3-A21B draft model weights.

Speculative decoding uses a smaller or cheaper drafter to propose several future tokens. The full target model checks those proposals together and accepts the matching prefix. When proposals are accurate, the system produces several valid tokens while avoiding an equal number of expensive target-model passes.

The method preserves the target model’s output distribution when implemented with the required verification and rejection process. It therefore changes how efficiently tokens are generated, rather than substituting the drafter’s answers for the target model’s answers.

Draft quality still determines whether that theoretical advantage becomes a practical speedup. A weak drafter produces rejected tokens, adding work without advancing generation. A large drafter may predict accurately but consume too much time and memory. Verification can also become expensive when systems test more candidates than a request justifies.

AngelSpec places those variables inside one training and evaluation framework. It supports MTP and block-parallel speculative decoding, including hidden-state generation, long-context training, rollout workflows, and online acceptance evaluation. The release is intended to cover more of the path between a research checkpoint and an operating inference service.

The framework’s two drafting routes divide the problem by output behavior. Tencent trains its MTP drafter on varied conversational data, where language is open-ended and the next tokens remain relatively uncertain. It trains the block-diffusion route on code and mathematics, where longer continuations often follow more predictable structures.

This specialization is the event’s most important change. Many speculative decoding comparisons ask which method wins on average. AngelSpec instead asks which drafting structure fits a particular distribution, then exposes both structures through a common development path.

The accompanying weights also make the release more testable than a paper describing an unavailable system. Tencent says it has released MTP and DFly drafters for Hy3-A21B through its model channels. Developers still need the corresponding target model and suitable serving hardware, but they do not have to recreate every training stage before beginning evaluation.

Hy3 gives the release a demanding target. The official Hy3 model repository describes a mixture-of-experts model with 295 billion total parameters and 21 billion active parameters. It also lists a 3.8-billion-parameter MTP layer and a 256,000-token context window.

Those specifications make inference efficiency economically important. Only a fraction of the full model activates for each token, yet serving still requires substantial memory, communication, and target-model computation. A drafter that increases accepted output per verification step can improve latency or throughput without retraining the primary model.

AngelSpec therefore arrives as more than another Hy3 accessory. It offers an explicit framework for choosing, training, and evaluating draft systems as serving conditions change. That broader scope creates the pressure on one-size-fits-all speculative decoding.

The Reported Gains Put Static Drafting Strategies Under Pressure

Tencent’s strongest claim is not the peak speedup alone, but DFly’s lead across every tested concurrency level from 4 to 64.

According to the paper, DFly delivered a 1.98 to 2.40 times end-to-end speedup over autoregressive decoding on Hy3-A21B. Tencent also reports 10.5% to 11.8% more throughput than DFlash and roughly 30% longer average accepted output.

Concurrency measures how many requests the serving system processes at once. It changes the balance between computation, batching efficiency, memory pressure, and verification overhead. A method that looks fast for one request can lose its advantage once many requests compete for the same accelerators.

Tencent says DFly achieved the highest average throughput at each tested concurrency from 4 through 64. That range matters because it includes relatively light traffic and more heavily batched service. The result suggests DFly’s scheduler and verification strategy contributed beyond the draft model’s raw acceptance rate.

The numbers still require careful interpretation. A 2.40 times speedup does not mean every user sees responses arrive 2.40 times sooner. End-to-end speedup depends on the benchmark workload, output length, request mix, batching policy, hardware profile, and baseline configuration.

Throughput and latency also answer different questions. Throughput measures how much work a system completes over time. Interactive users often care more about time to first token and the delay between later tokens. A service can increase total throughput while producing smaller or uneven improvements for an individual request.

That distinction places pressure on inference teams using fixed drafting policies. A static policy may choose one draft length, one verification depth, or one drafter for all traffic. AngelSpec argues that these choices should respond to the domain, request characteristics, online load, and deployment hardware.

The target under pressure is therefore not one company. It is the route that treats speculative decoding as a fixed model attachment. If Tencent’s findings hold elsewhere, operators will need routing and scheduling logic that understands when drafting work remains valuable.

DFlash provides the most direct comparison. Its block-diffusion design uses hidden states from the target model to predict multiple draft tokens in parallel. The approach avoids generating every proposal through a separate sequential drafter step.

DFlash’s authors reported more than sixfold acceleration in selected experiments and gains over EAGLE-3. Those results used different targets and test conditions, so they should not be compared directly with AngelSpec’s 2.40 times maximum. Tencent’s relevant claim is the controlled DFly comparison reported within its own Hy3 evaluation.

EAGLE-style systems remain another important reference. They use target-model features to guide a smaller autoregressive drafter, often arranging proposals for efficient verification. These systems can deliver stable results across varied text, but sequential dependencies inside drafting can limit how quickly long candidate sequences are proposed.

MTP occupies a lighter version of that autoregressive route. It is easier to integrate when the target already exposes compatible prediction layers. Tencent’s own Hy3 release includes an MTP layer, making the model a natural test bed for comparing lightweight drafting with a specialized block-parallel system.

AngelSpec’s pressure on existing deployments is practical. Teams must decide whether the additional model, scheduler logic, profiling, and memory use produce enough accepted tokens to justify their complexity. The framework gives them code and weights to explore that question, but it does not make the answer universal.

How AngelSpec Makes DFly More Selective

DFly combines parallel drafting with autoregressive information, then spends verification work where the expected return is highest.

A block-diffusion drafter predicts several positions together instead of completing them one at a time. Parallel prediction reduces drafting latency, especially when the candidate block is long. However, tokens inside a sentence or code sequence depend strongly on earlier tokens in that same block.

That dependency creates a weakness. If every position is predicted from masked or incomplete neighbors, later proposals can miss information that an autoregressive drafter naturally receives. One early error can shorten the prefix accepted by the target model, wasting much of the proposed block.

DFly addresses this tension with two connected components. Its backbone conditions on features from the target model while retaining block-parallel generation. A predecessor-conditioned autoregressive head also gives predictions information about earlier tokens, improving dependencies within the proposed block.

The architecture does not turn the entire draft into a conventional sequential process. Tencent’s design tries to preserve parallel work in the backbone while adding enough predecessor information to improve candidate consistency. That balance is central to the claimed increase in average accepted length.

Accepted length is the number of proposed tokens the target model approves before encountering a mismatch. Longer accepted prefixes spread each expensive verification step across more useful output. Yet maximizing accepted length alone can be misleading if producing and checking those candidates consumes too much time.

AngelSpec therefore adds an adaptive verification method called D-Cut. Verification depth refers to how much of each proposed continuation the target model checks. A fixed depth can overinvest in uncertain candidates or stop too early on highly predictable ones.

D-Cut treats verification capacity as a shared batch-level resource. It estimates the expected value of retaining additional candidate positions and compares that value with a profiled runtime cost. The scheduler can then direct more verification work toward high-confidence prefixes across multiple requests.

This is a serving decision, not merely a model decision. Two requests running through the same model can justify different verification depths. A structured code completion might sustain a long, confident prefix, while an open-ended chat response may diverge after only a few tokens.

Hardware changes the calculation as well. A larger verification batch can be efficient on one accelerator configuration and expensive on another. Communication costs, memory bandwidth, kernels, and target-model parallelism all affect whether another candidate position saves time.

That is why AngelSpec profiles runtime cost instead of relying only on probability estimates. A candidate can look likely to be accepted but still offer poor utility if checking it expands a batch into an inefficient shape. The scheduler needs both confidence and measured system cost.

Nvidia’s speculative decoding documentation illustrates how deployment frameworks already expose method-specific configuration. Its DFlash support requires a draft model, draft length, mask token, and selected target layers. AngelSpec pushes the problem further toward adaptive allocation across live requests.

The domain split complements that runtime adaptation. MTP remains the lightweight route for conversational outputs with higher uncertainty. DFly targets code and mathematics, where parallel blocks can capture longer predictable sequences. Neither method receives an automatic claim to every request.

Consider an AI coding service generating a repetitive test suite. Imports, function signatures, and assertion patterns can make the next block relatively predictable. DFly can propose a longer continuation, and D-Cut can preserve deeper verification when confidence remains high.

Now consider the same service answering an ambiguous architecture question. Several valid explanations can begin from the same prompt. The accepted prefix may shorten because the drafter and target choose different wording. A smaller MTP proposal can avoid spending resources on a long candidate block with low survival odds.

This mechanism makes the release more consequential than a single benchmark improvement. It reframes speculative decoding as a policy spanning training data, draft architecture, request classification, and serving cost. The speedup comes from coordinating those layers rather than maximizing one isolated score.

What the AngelSpec Numbers Do Not Establish

AngelSpec provides credible first-party evidence, but it does not yet establish that DFly wins across models, hardware, or production traffic.

The paper’s results come from the team that designed the framework and trained the draft models. The reported comparisons have not been independently reproduced at release time. Readers should treat the speedups as measured company claims, not universal performance guarantees.

Model dependence is the first limitation. DFly uses internal target features, so a drafter is tied closely to the architecture and training characteristics of its target. A Hy3-A21B drafter cannot simply become a drop-in accelerator for an unrelated model family.

That connection increases training and maintenance costs. Each supported target may need its own hidden-state extraction process, training data mixture, checkpoint, and validation cycle. Updates to the target model can also require renewed compatibility testing or fresh drafter training.

Hardware dependence creates another uncertainty. Tencent says D-Cut uses profiled runtime cost, acknowledging that the best verification policy changes across systems. A policy tuned for one cluster may need new profiles before it performs well on different accelerators or network topologies.

The public headline numbers also compress several workloads into ranges. Code, mathematics, and conversation have different predictability. Average throughput can conceal weak categories, unfavorable prompt lengths, or traffic patterns where drafting overhead approaches the saved target computation.

Long-context behavior deserves particular scrutiny. AngelSpec supports long-context training, and Hy3 lists a 256,000-token context window. However, large key-value caches increase memory pressure and can change the relative cost of drafting and verification. Results at shorter contexts cannot settle performance near the model’s maximum window.

Quality preservation requires implementation discipline too. Speculative decoding can preserve the target distribution through the proper acceptance and rejection algorithm. Deployment shortcuts, approximate verification, altered sampling, or incompatible quantization can change outputs. Operators must validate both speed and behavioral equivalence.

Memory is another real cost. The target model, draft model, hidden-state interfaces, and additional runtime buffers must coexist. Even a lightweight drafter can reduce room for key-value cache or larger batches. The resulting capacity tradeoff may erase a throughput gain in memory-constrained deployments.

Operational complexity also matters. A production service must monitor acceptance length, draft time, verification time, queue behavior, and fallback performance. Routing between MTP and DFly introduces another decision layer whose errors can send the wrong workloads to the wrong drafter.

The open-source release makes these questions testable, which is valuable. It does not answer them automatically. Teams should reproduce an autoregressive baseline on their own hardware before comparing either AngelSpec route.

They should then separate measurements by workload and traffic level. Useful categories include interactive chat, code completion, mathematical reasoning, tool-use traces, and long-form generation. Each category should include latency percentiles, throughput, memory consumption, acceptance length, and output-equivalence checks.

A fair DFlash comparison also requires matched conditions. The same target model, precision, serving framework, prompt distribution, output length, and concurrency should be used. Otherwise, architectural claims can become entangled with kernel quality or configuration differences.

Community feedback offers early signals but not a substitute for controlled reproduction. Local deployment reports often use quantized models, consumer hardware, or modified serving engines. These results can reveal compatibility problems, though they rarely match the paper’s setup closely enough to validate its headline range.

The benchmark gap does not make AngelSpec unimportant. It defines the next stage of the story. Tencent has supplied an architecture, code path, and model weights that external teams can challenge under conditions the original authors did not control.

Three Signals Will Determine Whether AngelSpec Travels Beyond Hy3

AngelSpec’s importance will depend on independent replication, broader model support, and evidence that adaptive routing survives real production traffic.

The first signal is a reproducible Hy3-A21B benchmark from an independent inference team. The strongest test would use the released MTP and DFly weights while reporting hardware, precision, framework versions, prompt mix, and output lengths. It should compare autoregressive decoding, MTP, DFlash, and DFly under matched conditions.

Replication near Tencent’s 1.98 to 2.40 times range would strengthen the core claim. Consistent gains over DFlash would also suggest that predecessor conditioning and D-Cut add value beyond general block-diffusion drafting. Smaller or unstable gains would narrow AngelSpec’s practical appeal.

The most useful independent report would publish more than average throughput. It should include time to first token, inter-token latency, tail latency, memory use, accepted length, and performance across concurrency levels. Those measurements would show whether aggregate efficiency improves the user experience.

The second signal is support for another major target-model family. AngelSpec currently has its clearest evidence and released draft weights around Hy3. A successful port to Qwen, Llama, or another widely deployed open model would test whether its framework generalizes beyond Tencent’s architecture.

Porting would also expose the real cost of adoption. Researchers would need to generate target hidden states, train specialized drafters, integrate verification, and profile runtime behavior. A documented port with reasonable engineering effort would strengthen the framework’s claim to end-to-end usability.

Failure to attract ports would suggest that AngelSpec is primarily a Hy3 optimization package. That outcome could still benefit Tencent’s model users, but it would weaken the broader case for a unified speculative decoding framework.

The third signal is production evidence from mixed workloads. Tencent’s argument relies on heterogeneity, so a static benchmark cannot fully validate it. The decisive test is whether a live service can choose between MTP and DFly while traffic, domains, and hardware utilization change.

Operators should watch how often routing decisions improve accepted output per unit of verification cost. They should also measure fallback rates and routing mistakes. A complex adaptive system must beat a simpler baseline after its monitoring and scheduling overhead are included.

This test is especially relevant for services combining chat, coding, mathematical work, and agent actions. Such products generate outputs with very different entropy and length. They offer the conditions where specialized drafting should outperform a universal policy.

If mixed-workload deployments show stable gains, competitors will face pressure to expose similar routing controls. Serving frameworks could evolve from selecting one speculative algorithm at launch toward assigning algorithms and verification budgets per request.

If the gains collapse outside curated workloads, simpler methods will remain attractive. MTP can be easier to operate, especially when a model already ships with compatible layers. Static draft strategies also reduce the number of models and policies that teams must maintain.

Developers evaluating the release should preserve their test results and configuration decisions in a searchable engineering knowledge base. Speculative decoding experiments involve enough interacting variables that undocumented runs quickly become impossible to compare.

AngelSpec has already changed the question facing inference engineers. The choice is no longer only whether to enable speculative decoding. It is whether the drafter, training distribution, verification policy, and hardware profile match each request well enough to save real work.

The next one to three months should reveal whether outside teams reproduce Tencent’s numbers, port DFly beyond Hy3, and validate adaptive routing under live demand. Until then, AngelSpec is a serious open experiment with promising first-party results, not a settled winner.

For teams serving Hy3 today, the useful next step is a controlled benchmark against their existing autoregressive and MTP configurations. For everyone else, the key question is narrower: does workload-aware drafting deliver enough sustained efficiency to justify another model and scheduling layer? The answer will determine whether AngelSpec becomes a broadly adopted inference framework or remains a well-engineered advantage tied mainly to Tencent’s own model family.