Flo Health Open Models Got Safer and More Useful, but 47% Still Passed Both Tests

Flo Health open models improved on two demanding internal benchmarks, yet only 47% of evaluated responses met both medical-safety and usefulness criteria. The women’s health company says its post-training system raised medical safety from 34% to about 61%. Usefulness increased from 42% to 76%.

Those gains came from a pipeline built around synthetic questions, specialized AI judges, and clinician-defined evaluation rules. Flo then used supervised fine-tuning and Direct Preference Optimization, or DPO, to place preferred behavior inside an open model’s weights.

The work matters because Flo serves 81 million monthly active users, according to a Databricks case study published on October 12, 2026. At that scale, even a low error rate can expose many people to incomplete, overly reassuring, or confusing health information.

The result also challenges the simplest approach to health-focused generative AI: send every request to a frontier model provider and rely on its general safety layer. Flo is betting that smaller, tailored models can deliver more control and lower inference costs. The harder question is whether an internally designed benchmark can establish safety beyond the environment that created it.

Flo Rebuilt the Evaluation System Before Training the Model

The central change was not a new chatbot or foundation model. Flo first converted clinical judgment into a repeatable evaluation and training system.

Flo’s medical, product, and engineering teams divided response quality into two main goals. Medical safety formed the foundation, while usefulness measured whether an answer gave a user relevant and understandable help.

That distinction sounds obvious until a model faces an ambiguous health question. Consider someone asking why a period is late. A reply that only advises medical consultation might avoid a dangerous claim, but it offers little immediate context.

The opposite answer can also fail. A model might list stress, travel, weight changes, or exercise while overlooking pregnancy or a condition requiring professional assessment. That response feels more useful, but its omissions can create false reassurance.

Flo therefore treated safety and usefulness as separate dimensions that must pass together. Its final framework included 26 AI judges, meaning automated evaluators with defined scoring instructions and reference cases.

Twelve judges enforced medical-safety constraints. Six micro judges examined narrower behaviors such as grounding, framing, voice, and fine-grained safety. Eight usefulness criteria assessed completeness, relevance, coherence, and tone.

The company also created rules for recurring failure patterns. One “balanced causes” rule required answers to cover benign and potentially serious explanations when both were relevant. This targeted the tendency of conversational models to produce a reassuring answer that sounds complete.

Flo says it validated these judges against synthetic cases and anonymized examples from user experiments. When an automated judge disagreed with an initial human assessment, blinded clinical reviewers adjudicated the case.

Some reviewed cases helped refine the judges. Flo reserved the rest for subsequent measurement, reducing the risk of evaluating solely on the examples used during calibration.

This evaluation-first strategy is the foundation of the reported benchmark results. It also explains why the project is more significant than a routine fine-tuning exercise.

A team cannot reliably improve “safety” if that word remains a broad aspiration. Flo had to express the aspiration as individual, testable behaviors before it could use those behaviors as training signals.

The same principle applies beyond healthcare. Enterprise teams often possess extensive domain knowledge, but that knowledge sits across policies, expert decisions, research, and operational exceptions. Systems that support knowledge blending can help organize such material, although model evaluation still requires task-specific tests and accountable reviewers.

Flo’s design creates a reusable asset. A foundation model can become obsolete, change its license, or lose its cost advantage. The clinical rubrics and evaluation cases can move to the next model.

That portability establishes the article’s main tension. Flo gains control by replacing a general provider’s safety assumptions with its own standards. It also accepts responsibility for proving those standards are reliable across a large and varied user population.

Why Flo Health Open Models Became an Economic Bet

Flo Health open models are not only a safety project. They are an attempt to reduce dependence on expensive, externally controlled model APIs.

Flo previously used traditional machine learning and began exploring generative AI for more flexible, personalized experiences. A single general-purpose model offered a direct path to those capabilities, but it introduced three constraints.

The first was inference cost at scale. Every generated answer can require a paid request to an external provider. A modest per-request difference becomes material when a product serves tens of millions of monthly users.

Flo estimates that tailored, fine-tuned models can reduce inference costs by an order of magnitude compared with frontier model APIs. That figure is a company estimate, not a disclosed production cost analysis.

The second constraint was model lifecycle dependence. An API provider can retire a model, change its behavior, revise usage rules, or introduce a replacement. Each change can force a health app to repeat evaluation work.

The third was limited control over domain behavior. A general model’s safety tuning must cover many subjects, cultures, products, and use cases. It cannot automatically encode every distinction that Flo’s clinicians consider important.

Flo’s longer-term answer is a compound AI system, which combines multiple models and supporting components instead of assigning every task to one model. Such a system can route simple tasks to smaller models and reserve larger models for difficult requests.

This route puts pressure on the default frontier-API strategy. The choice is not simply open versus closed technology. It is general safety supplied by a model vendor versus domain-specific controls maintained by the application developer.

General models retain clear advantages. Their providers can spend heavily on pretraining, broad evaluations, red-team exercises, and infrastructure. They can also update capabilities without requiring every customer to operate a training stack.

Yet those advantages come with distance from the final use case. A provider does not own Flo’s exact response policy for menstrual symptoms, pregnancy questions, or other sensitive women’s health topics.

Flo is trying to close that gap by turning its domain expertise into portable evaluation infrastructure. If the underlying model changes, the company can rerun its judges, regenerate preference pairs, and repeat post-training.

That process shifts the scarce resource. The base model becomes more replaceable, while the evaluation framework becomes more valuable.

Flo has already identified Gemma, Qwen, and Kimi as potential future open-weight candidates. Open-weight models expose parameters that developers can run and modify, although their licenses do not always satisfy every definition of open-source software.

The company has not said that one model family will handle every application. Its strategy instead treats models as components competing against a consistent set of medical and product requirements.

That makes the economic claim inseparable from safety. Lower-cost inference has limited value if a model produces worse health information. Likewise, a safer model might remain impractical if serving it costs too much.

Flo’s approach seeks a model that clears both thresholds. The early numbers show progress, but they do not establish that the balance is complete.

AI Judges Turned Clinical Rules Into Training Data

Flo scaled expert judgment by teaching automated judges to rank candidate answers, then used those rankings to change the model itself.

The pipeline started with more than 15,000 synthetic questions. Flo says it derived the inputs from model-generated or anonymously aggregated signals, rather than user-written free text or identifiable information.

That distinction matters in a health application. Training directly on sensitive user conversations can introduce privacy exposure, consent questions, and memorization risks. Synthetic prompts reduce those risks, although they introduce a different problem: whether generated questions reflect real users accurately.

For each question, the system produced multiple candidate answers. The 26 judges then scored those answers across safety, usefulness, and narrower behavioral criteria.

Flo selected stronger outputs and organized them into preference pairs. Each pair showed the model which of two responses better matched the desired behavior.

The team first applied supervised fine-tuning, which trains a model to reproduce selected examples. It then used DPO, a method that increases the likelihood of preferred responses relative to rejected ones.

The original DPO paper describes a simpler optimization route than conventional reinforcement learning from human feedback. It avoids training a separate reward model during the preference-optimization stage.

Flo describes its broader workflow as reinforcement fine-tuning. The system generates candidates, scores them, selects better answers, trains the model, and runs the evaluation again.

The distinction between evaluation and training is crucial. An AI judge does not merely grade the final model. Its score influences which outputs become positive examples, so any weakness in the judge can propagate into the trained behavior.

Flo encountered that problem in its first attempt. Usefulness improved, but medical safety remained relatively flat. The preference pairs did not make sufficiently fine distinctions between levels of medical quality.

A binary pass or fail signal had discarded useful information. Two responses might both avoid an explicitly unsafe statement, while one provides far better safety context.

Flo added a graded safety score after the binary gate. Answers that passed the basic safety test were ranked by how fully they satisfied applicable medical criteria.

That change created sharper preference pairs. The model could learn not only to avoid a prohibited answer, but also to favor an answer with better coverage and safety netting.

This is the project’s most instructive technical result. Preference optimization cannot repair a vague reward signal. If the evaluation system treats shallow caution and careful clinical framing as equivalent, the model has no reason to learn the difference.

Clinicians remained responsible for defining and validating the standard. They did not need to manually write or label every training pair, which allowed their work to influence more examples.

However, automation did not remove the human bottleneck. It moved expert effort upstream, from repeated annotation into rubric design, difficult-case adjudication, and evaluator calibration.

That redistribution can make domain tuning more scalable. It can also hide mistakes behind an automated score if teams stop reviewing disagreements or rare cases.

The method therefore depends on continuous judge maintenance. Medical guidance changes, user language shifts, and new model families expose different weaknesses. A fixed judge suite can become outdated even if it performed well during its original validation.

Databricks AI Runtime Made the Loop Repeatable

The infrastructure mattered because Flo needed to rerun generation, scoring, training, and evaluation as one controlled loop.

Flo trained with 32 NVIDIA H100 GPUs using PyTorch and Fully Sharded Data Parallel. FSDP distributes model parameters, gradients, and optimizer state across GPUs to support larger training workloads.

Its stack combined Databricks AI Runtime, LLM Foundry, Compose-RL, and MLflow. LLM Foundry handled distributed training, while Compose-RL supplied the preference-optimization layer. MLflow tracked experiments and checkpoints.

The AI Runtime documentation describes a serverless GPU environment for deep-learning workloads. In Flo’s project, that meant engineers could request distributed compute without configuring a new cluster for every experiment.

The benefit was not simply access to H100 accelerators. Flo kept data operations, experiment tracking, model training, and evaluation within one surrounding environment.

That arrangement reduces the friction between discovering a failure and testing a correction. A team can adjust a rubric, regenerate preferences, retrain the model, and compare the result against the previous checkpoint.

Flo’s first safety result shows why iteration speed matters. The initial preference design improved one target while leaving the other nearly unchanged. The team needed another complete loop to test graded safety scoring.

A fragmented stack can make that loop slower and harder to audit. Training data might move through separate systems, while model checkpoints, scoring versions, and evaluation reports remain disconnected.

A unified workflow can preserve the relationship among a question, its candidate answers, judge scores, preference pair, training run, and final evaluation. That lineage becomes important when a model produces an unsafe response and investigators need to understand why.

The design also supports Flo’s plan to test newer open-weight models. Engineers can replace the base model while retaining much of the generation, scoring, training, and tracking process.

That portability is the strongest argument for calling the pipeline an asset. Compute is rented, and foundation models change quickly. A validated method for converting domain standards into model behavior can last longer.

Still, platform convenience does not resolve the core quality question. Faster experiments can produce more evidence, but they can also produce rapid optimization against an incomplete benchmark.

The risk resembles teaching to a test. A model can become very good at satisfying the evaluator’s visible preferences without becoming equally safe across unexpected questions.

Flo’s use of held-out cases and clinical adjudication helps address that concern. The public case study does not disclose enough detail to determine how broad those cases were or how performance changed across user groups.

Infrastructure should therefore be understood as an accelerator for a governance process. It cannot replace the governance process itself.

The 47% Combined Score Is Progress, Not Proof

Flo improved both reported metrics, but the remaining failure rate prevents the results from functioning as a broad safety guarantee.

The original model scored 34% on Flo’s medical-safety benchmark and 42% on usefulness. Only 16% of responses passed both dimensions simultaneously.

After fine-tuning, the safety score reached approximately 61%. Usefulness rose to 76%, while the combined score increased to 47%.

The combined result nearly tripled, but it also means 53% of evaluated responses did not meet both standards. Some may have been safe but unhelpful. Others may have been useful while missing a medical safeguard.

That distinction affects deployment decisions. A model can still support constrained features when it does not qualify for unrestricted health advice. Product controls can limit topics, retrieve approved information, request clarification, or escalate sensitive cases.

Flo has not publicly disclosed the benchmark size, confidence intervals, subgroup results, or failure severity in the case study. It also does not identify the original open model used for these reported scores.

Without those details, outside researchers cannot determine whether a small set of recurring cases drove much of the improvement. They also cannot compare Flo’s model directly with frontier APIs or specialized health models.

The 26 judges were calibrated with clinician involvement, but the resulting scores remain internal measurements. Independent replication would give the claims more weight.

External evaluation matters because synthetic questions may differ from real conversations. Users misspell symptoms, omit context, switch topics, express fear, or ask questions shaped by cultural assumptions.

A benchmark can also underrepresent rare but serious conditions. Average performance might improve while a model continues to fail on the cases with the greatest potential harm.

The World Health Organization’s health AI guidance places accountability, transparency, and human autonomy alongside technical performance. Those principles are relevant even when an AI feature provides information rather than diagnosis.

Flo must also operate against its own privacy history. In 2021, the Federal Trade Commission finalized an order resolving allegations that Flo shared sensitive health information with analytics providers after promising to keep it private.

The FTC settlement record concerns earlier data practices, not the new training pipeline. It remains relevant because user trust in a health application covers both what the model says and how the system handles sensitive information.

Flo says its synthetic-data process avoided user-written free text and identifiable information. The company also offers Anonymous Mode, which it says can separate health data from a user’s identity.

Those measures address part of the risk, but several questions remain. Aggregated signals can still encode demographic imbalance. Synthetic data can reproduce assumptions from the model that generated it. Automated judges can reward confident language that resembles expertise.

The phrase “privacy-first” should therefore describe a design direction, not a completed external certification of this pipeline. Likewise, “safer” is a comparative benchmark result, not proof that every generated response is medically safe.

None of these limits erase the gains. They clarify what the numbers support.

Flo has shown that its internal system can move an open model toward clinician-defined behavior on its evaluation set. It has not shown that the model can replace clinical care or respond safely to every real-world health question.

Three Signals Will Show Whether the Strategy Holds

The next stage should be judged through cross-model replication, real-world monitoring, and independent evaluation, not another isolated benchmark gain.

The first signal is whether Flo reproduces the improvement on Gemma, Qwen, or Kimi. The company has named those model families as future candidates for its pipeline.

Comparable gains across different base models would support Flo’s claim that the evaluation and training system is the durable asset. A large drop with another model would suggest the current result depends heavily on an undisclosed base model.

The comparison should preserve the same held-out evaluation set while adding tests designed for each model’s distinct weaknesses. Otherwise, teams risk selecting the model that best fits a familiar test.

The second signal is production monitoring. Flo should report how often deployed systems refuse, escalate, retrieve approved information, or receive negative quality feedback.

Aggregate benchmark scores cannot reveal whether failures cluster around pregnancy, contraception, pain, menopause, or urgent symptoms. Category-level reporting would show where the model remains unreliable.

Monitoring should also distinguish low-value answers from potentially harmful ones. A repetitive response and a dangerously reassuring response should not carry equal operational weight.

The third signal is independent validation. External clinicians and safety researchers should test the system with cases that Flo’s training process did not generate or select.

Independent reviewers can probe distribution shifts, demographic variation, adversarial wording, multilingual use, and rare medical scenarios. They can also evaluate whether the automated judges favor a particular communication style over clinical substance.

Evidence from those three signals would strengthen the case for Flo Health open models. Failure to publish broader results would leave the current claims dependent on a vendor-authored case study and company-designed measurements.

For developers, the immediate lesson is not to copy Flo’s exact stack. It is to treat evaluation design as part of the product, rather than a test performed after training.

For enterprise buyers, the key question is also changing. Model procurement should examine who defines acceptable behavior, how those rules become tests, and whether the system preserves evidence across each training run.

Knowledge workers and health-app users should look for clearer boundaries. A trustworthy product should explain when generative output is informational, when it retrieves reviewed material, and when it directs someone toward professional care.

Flo’s reported gains make its approach worth watching. The unresolved 53% makes scrutiny equally necessary.

The next decisive update will not be a larger GPU run. It will be evidence that the same safety gains survive a new base model, unpredictable user questions, and independent clinical review. Until then, readers should treat the system as a promising post-training method, not a verified substitute for medical judgment.