DiDi Contact Center QA Replaces a Black Box With Amazon Bedrock

DiDi replaced an opaque quality assurance tool after its intent checks reached only 38% accuracy. Its new DiDi contact center QA system reportedly lifted that figure to 86%.

The company built the replacement with AWS for its International Business Group. It processes Spanish and Portuguese support conversations across ride-hailing, food delivery, and financial services. The system also scores compliance and finds emerging customer complaints.

This is more than another company adding a large language model to customer support. DiDi moved a consequential internal control from an outside vendor into an architecture its own team can inspect and change. The main contest is therefore between opaque outsourced automation and transparent, self-owned AI.

AWS and DiDi disclosed the system in a September 8, 2026 contact center QA case study. Most performance figures come from DiDi’s production validation and have not been independently audited.

The results still deserve attention. They show how narrow context, deterministic checks, and structured outputs can matter more than repeatedly rewriting a prompt.

What Changed Inside DiDi Contact Center QA

DiDi did not replace one model with another. It divided quality assurance into three controlled pipelines with different jobs and evidence requirements.

The first pipeline verifies intent. Customer service representatives assign each conversation a contact reason, which places the case within DiDi’s hierarchical classification tree. The system checks whether that assigned reason matches what the customer discussed.

If the label appears wrong, the pipeline recommends an alternative. It also examines conversations labeled “Other,” where no existing category seemed appropriate. Those cases can expose gaps in the taxonomy itself.

The second pipeline evaluates compliance. It scores several quality criteria while extracting operational insights from the same conversation. DiDi says its average compliance scoring accuracy exceeded 90% during production validation.

The third pipeline analyzes Voice of Customer data. Voice of Customer, or VOC, means aggregated evidence about customer problems, sentiment, outcomes, and recurring causes. DiDi uses this pipeline on demand when operations teams need to understand a developing pattern.

Live-chat records and phone transcripts enter a preprocessing layer first. Speech-to-text output from calls is normalized into the same conversation format used for chat. The resulting records then move through the appropriate QA pipelines.

That common schema is important. Differences between voice and chat should not force every downstream analysis component to implement its own ingestion logic. The system separates channel processing from the reasoning tasks that follow.

DiDi’s international business operates across 14 countries and regions, according to the companies. Its support organization handles Spanish and Portuguese conversations for tens of millions of users across three business lines.

At that scale, manual review can inspect only a limited portion of interactions. However, the outsourced alternative had its own problem. DiDi says its previous system did not expose enough reasoning to explain historical judgments or support rapid changes.

A failed compliance score can affect coaching, audits, and operational priorities. A wrong intent label can distort reports about why customers need help. An unexplained judgment is therefore not just inconvenient for a developer.

The replacement attaches a reasoning chain to each model judgment. Reviewers can see the stated basis for a score, examine the original conversation, and compare the decision with the applicable rule.

That traceability creates the article’s central tension. Bringing the system inside DiDi improves visibility and control, but it also makes DiDi responsible for validation, security, configuration, and ongoing accuracy.

DiDi now owns the definitions that shape the result. It also owns the consequences when those definitions are incomplete or wrong.

Why Context Isolation Beat More Prompt Tuning

The largest reported accuracy gain came from showing the model less information at the right moment, not from adding more instructions.

DiDi’s initial intent-verification design followed an intuitive approach. It placed the complete contact-reason tree beside the conversation and asked the model to judge the representative’s selected label.

The model then compared the chosen label with every available alternative. When it found a category that seemed slightly more precise, it tended to reject an otherwise reasonable selection.

This behavior created over-correction. A model tasked with finding the best label can behave differently from one asked whether an existing label is acceptable. Combining those decisions inside one prompt blurred the distinction.

DiDi says several rounds of prompt tuning did not resolve the problem. The team concluded that the model had been given the wrong decision environment.

The redesigned intent pipeline separates verification from classification. During verification, the model receives the conversation and only the representative’s current contact reason. It decides whether that label reasonably fits the interaction.

The complete classification tree stays hidden at this stage. That information isolation prevents the model from searching for marginally better alternatives before it answers the narrower question.

Only a failed verification opens the classification stage. The model then receives the full taxonomy, along with reasoning from the first stage. It recommends another category and supplies a confidence score and rationale.

DiDi handles “Other” labels through a separate three-level process. The system first searches sibling categories for a suitable match. It then searches the entire tree if the local branch offers no answer.

If neither search produces an appropriate category, the pipeline identifies a possible taxonomy gap. It can then suggest that operators add a label instead of forcing the conversation into an unsuitable existing bucket.

According to DiDi, this two-level design raised intent-verification accuracy from 38% to 86%. That is a 48 percentage-point increase, based on the company’s stated production validation.

The mechanism offers a practical lesson for enterprise AI teams. More context is not automatically better context. Extra options can change the task the model appears to solve.

This matters because many enterprise prompts combine several decisions for efficiency. A single request might ask a model to classify a conversation, justify the result, check policy compliance, summarize the case, and recommend an action.

Each additional objective creates another opportunity for instructions and evidence to interfere. It also makes failure analysis harder because developers cannot easily identify which part of the context shifted the answer.

DiDi’s approach instead treats context as part of application architecture. The team decides which information becomes visible at each decision point, just as conventional software controls access to variables and state.

That design also produces clearer failure boundaries. A questionable verification result belongs to the first stage. A poor replacement label belongs to the second. A missing category belongs to taxonomy governance.

The result is not a universal claim that multi-stage systems always outperform single prompts. Each added stage creates latency, operational complexity, and another component to monitor.

DiDi has not published sample size, class distribution, confidence intervals, or performance by language and business line. Those omissions limit comparisons with other contact center QA systems.

Still, the reported change challenges a common enterprise habit. Teams often respond to disappointing model behavior by expanding instructions or switching foundation models. DiDi changed the information flow instead.

That is a deeper form of prompt engineering. It moves responsibility from clever wording into system design, data definitions, and explicit decision boundaries.

A Self-Owned System Puts Outsourced AI Under Pressure

DiDi’s strongest challenge to third-party QA vendors is not the model accuracy claim. It is the argument that judgment logic has become strategic infrastructure.

An outsourced platform can reduce implementation work. It can package transcription, evaluation, dashboards, and workflows into one managed product. Buyers avoid maintaining every component themselves.

However, that convenience becomes a constraint when the vendor does not expose how it reached a judgment. DiDi says its prior solution made QA decisions without a sufficient audit trail.

The limitation became more serious as standards changed. Spanish support for ride-hailing does not necessarily use the same criteria as Portuguese support for financial services. New policies create further combinations.

A vendor-controlled implementation can require custom changes, retraining, or product updates before those standards appear in production. During that interval, old and new rules can coexist.

DiDi moved policy definitions into external configuration. The evaluation pipeline uses one prompt template, then injects the language, business line, criterion definition, passing rule, and failing rule for each ticket.

Adding another criterion does not require rewriting every language-specific prompt. Operators update configuration, and the application assembles the necessary context when it receives a conversation.

The pipeline evaluates several compliance items in one model call. It also returns structured business insights, including measures related to issue resolution and customer satisfaction.

Amazon Bedrock Tool Use constrains the response to a defined JSON structure. Each scoring item contains a judgment and its accompanying rationale. The relevant tool-use capability lets applications describe the expected tool input and process the resulting structured data.

Structured output solves only the response-format problem. Valid JSON does not guarantee that the underlying score is correct. DiDi therefore adds deterministic checks after generation.

For example, the model can identify possible spelling errors. Application code then counts only errors in the representative’s messages and applies the defined threshold to the verified number.

The system also calculates response-wait times in code. It injects those values into the prompt instead of asking the model to infer them from timestamps.

This hybrid pattern assigns semantic judgments to the language model and computable facts to conventional software. It avoids using probabilistic generation where a direct calculation can provide a reproducible answer.

That distinction is central to the self-owned approach. DiDi can inspect which decisions belong to the model, which belong to code, and which depend on business configuration.

The architecture also uses Amazon Bedrock because the service exposes multiple foundation models through a common interface. DiDi says that model choice can change without rebuilding each pipeline around another provider-specific integration.

Model portability still has limits. Different models interpret prompts and schemas differently, so changing models requires fresh evaluation. A common API removes some integration work but does not make behavior interchangeable.

Security creates another pressure point. Customer conversations can contain names, contact information, financial details, location data, and sensitive complaints. Moving those records into an AI workflow expands the system that must be governed.

DiDi says the deployment uses VPC endpoints through AWS PrivateLink, encryption, and fine-grained Identity and Access Management controls. AWS documents that a private connection can reach Bedrock without an internet gateway or public IP address.

The system also applies Amazon Bedrock Guardrails before model inference. It masks personally identifiable information and uses contextual grounding checks to flag answers that lack sufficient support.

AWS describes Guardrails as policies that evaluate prompts and responses for areas including sensitive information, denied topics, and unwanted content. Its guardrail controls can block or mask content according to the configured policy.

These measures do not eliminate governance work. DiDi must still define access, retention, escalation, review, and regional handling rules for customer data.

The same burden applies to business logic. Owning a transparent system means maintaining its taxonomies, evaluation criteria, validation sets, and monitoring process. The company has traded vendor opacity for internal accountability.

Third-party vendors are therefore pressured on two sides. They must match the convenience of a managed product while exposing enough evidence, configurability, and control for enterprise customers to trust consequential judgments.

The response does not have to be full code disclosure. Vendors can offer decision traces, versioned rules, evaluation tools, exportable results, and clearer separation between model output and deterministic checks.

DiDi’s case suggests that a simple accuracy dashboard is no longer enough. Buyers increasingly need to know which model, prompt version, policy definition, and post-processing rule produced each score.

That requirement turns observability into a product feature. It also makes internal ownership more attractive for companies with enough engineering capacity and sufficiently specialized operations.

What the Accuracy Numbers Do Not Show

The reported gains are meaningful, but the public evidence cannot establish how the system performs across every market, language, category, or changing policy.

The 38% and 86% intent figures come from DiDi’s production validation. The AWS account does not disclose how many conversations were tested or how evaluators defined a correct result.

It also does not provide accuracy by Spanish versus Portuguese. Performance may differ across accents, regional vocabulary, support channels, business lines, and classification depth.

Class balance matters too. A dataset dominated by common ride-hailing questions can produce a strong overall score while masking weak results for less frequent financial-service cases.

The same caution applies to compliance scoring above 90%. The published material does not state how many criteria were included, whether each criterion received equal weight, or how humans resolved disagreements.

Accuracy can also hide different error costs. A false spelling violation is inconvenient, while an incorrect assessment involving regulated financial-service conduct can carry more serious consequences.

A production evaluation should therefore track precision and recall for individual criteria, not only a single average. It should also monitor disagreement between human reviewers and the automated system.

Reasoning traces help reviewers investigate decisions, but they are not proof. A language model can produce a plausible explanation for a wrong answer.

The deterministic validation layer reduces this risk for measurable facts. It cannot convert every policy judgment into a calculation. Tone, resolution quality, empathy, and contextual appropriateness still require interpretation.

Guardrails introduce another limitation. They can filter sensitive information and test grounding, but AWS itself recommends continued validation as underlying safeguards change. A configured control should not be treated as a permanent guarantee.

Human oversight remains necessary, particularly for disputed scores and higher-risk criteria. Review teams need an appeal path that can correct both the individual decision and the underlying rule.

The public account also lacks operational measurements. It does not report model latency, processing cost, override rate, reviewer workload, or the percentage of conversations requiring escalation.

Those figures would reveal whether improved model accuracy translates into better operations. An accurate system can still struggle if it responds too slowly, requires frequent manual correction, or becomes expensive at full volume.

Comparison with other implementations adds useful perspective. Financial services provider Empower previously described another Bedrock-based QA deployment that processed thousands of transcripts daily.

Its automated QA system combined Amazon Connect Contact Lens with Bedrock. Empower said it expanded QA coverage twentyfold and reduced review time from days to minutes.

The implementations are not directly comparable. Empower used pre-redacted transcriptions from an AWS contact center stack, while DiDi described its own preprocessing and three-pipeline architecture.

Yet both cases point toward the same competitive direction. Enterprises want to review more conversations, explain evaluations, and shorten the gap between customer problems and operational action.

DiDi’s distinct contribution is its account of a failed design. Feeding the whole taxonomy into one call produced poor intent verification, despite repeated prompt adjustments.

That failure makes the case more useful than a simple vendor success story. It shows that model access alone does not deliver reliable quality assurance.

The remaining uncertainty concerns maintenance. Taxonomies change, customer behavior shifts, and policy language evolves. Model providers also update available versions and supporting features.

DiDi will need versioned evaluation sets that preserve representative examples from each language, channel, business line, and high-risk criterion. Otherwise, a configuration improvement in one area can quietly reduce performance elsewhere.

Teams building similar systems should preserve the evidence behind each release. A searchable engineering knowledge base can connect requirements, test results, prompt versions, and incident reviews without treating generated explanations as ground truth.

That practice supports the real promise of transparency. Visibility is useful only when teams can reconstruct what changed, why it changed, and how the new version performed.

The Next Test Is Whether DiDi Can Scale Control

Three signals will determine whether DiDi’s architecture becomes a durable operating system for QA or remains a successful deployment with limited public validation.

The first signal is performance across additional languages and business lines. DiDi says it plans to expand the system beyond its current coverage.

That expansion will test whether dynamic configuration genuinely limits maintenance work. A new language changes more than translated instructions. It can introduce regional phrasing, cultural expectations, transcription errors, and different policy requirements.

If accuracy remains stable across new deployments, the result will strengthen DiDi’s context-management thesis. Large declines would suggest that the current gains depend heavily on the existing Spanish and Portuguese validation environment.

The second signal is cross-pipeline integration. DiDi plans to connect intent verification, compliance scoring, and VOC analysis more deeply.

Today, each pipeline has a distinct purpose. Integration can create a feedback loop where complaint trends reveal missing classifications, repeated classification failures update the taxonomy, and compliance findings shape coaching.

It can also spread errors. A faulty issue cluster could influence taxonomy changes, which then affect intent checks and management reports.

Successful integration therefore requires provenance. Every downstream recommendation should retain links to the conversations, extracted fields, configuration version, and model output that produced it.

The third signal is operational evidence beyond accuracy. Future disclosures should include human override rates, false-positive patterns, processing latency, review time, and performance by category.

Those measurements would show whether the system remains useful after the initial validation period. They would also help buyers compare self-owned architectures with managed contact center products.

VOC analysis offers the clearest immediate test. DiDi described a surge in cancellation-fee complaints across Latin American markets. Operations staff triggered an analysis that clustered multilingual conversations and produced a structured report within minutes.

The pipeline begins by extracting fields from each conversation in parallel. These fields include the issue type, sentiment, outcome, and possible root cause.

An embedding model then measures semantic similarity between issue labels. Embeddings are numerical representations that place related meanings near each other, allowing the system to merge differently worded complaints.

That stage uses distance calculations and frequency rankings instead of generative output. DiDi says the design makes clustering deterministic and reproducible.

The language model receives only the high-frequency clusters during report generation. It creates an executive summary, identifies pain points, and suggests actions from a smaller, structured evidence set.

This pipeline again applies information isolation. The model does not receive thousands of raw conversations in one oversized prompt. Each stage narrows the evidence required for the next decision.

DiDi says the process reduced work that previously took hours to minutes. The claim will become more convincing if future reporting shows whether operations teams acted faster or prevented repeated customer harm.

Speed alone is not the goal. A rapid report with an incorrect root cause can direct resources toward the wrong intervention.

The strongest deployment will pair faster detection with measurable outcomes. Those might include fewer repeat contacts, better issue resolution, lower complaint recurrence, or quicker correction of a policy problem.

For developers, the immediate lesson is to design model boundaries before polishing prompts. Decide which evidence each call needs, which outputs require a schema, and which facts should be calculated in code.

Enterprise buyers should ask vendors for the same clarity. Request versioned rules, auditable decisions, per-category validation, escalation workflows, and access controls that reflect the sensitivity of conversation data.

Knowledge workers should care because the pattern extends beyond support. Any system that classifies documents, evaluates work, or summarizes recurring issues can suffer when it sees irrelevant options or combines incompatible tasks.

DiDi contact center QA is therefore a test of organizational capability as much as model performance. The company has taken ownership of context, definitions, and validation instead of outsourcing the entire judgment layer.

Will that ownership produce stable results as languages, policies, and models change? Watch the expansion metrics, cross-pipeline evidence trails, and human override rates. Those signals will show whether transparent enterprise AI can outperform the black box over time.