Xiaohongshu HELMSMAN Moves Vector Search to Flash and Cuts Hardware Costs Over 90%

Xiaohongshu HELMSMAN has moved production vector search onto about 40 all-flash servers, replacing workloads that consumed roughly 35,000 CPU cores and 350 TB of DRAM. The company says the change reduced associated hardware costs by more than 90%.

That result challenges a familiar infrastructure assumption. Low-latency vector retrieval has usually depended on graph indexes held almost entirely in expensive DRAM. Flash-based systems reduced memory requirements, but their slower and less predictable access patterns often excluded them from demanding online services.

HELMSMAN takes a different route. It combines a clustering-based index with direct access to NVMe storage, learned search pruning, and a distributed index-building pipeline. The result reportedly approaches in-memory performance without preserving the same memory-heavy architecture.

The system is more than a laboratory benchmark. According to the OSDI paper, Xiaohongshu has operated HELMSMAN in production for several months. It supports workloads connected to search, recommendations, advertising, content moderation, and retrieval-augmented generation.

The important contest is therefore not Xiaohongshu against another social platform. It is all-flash clustering against the in-memory graph architecture that has dominated low-latency vector search.

Xiaohongshu HELMSMAN Turns a Research Result Into Production Infrastructure

The central change is operational: flash storage now handles online vector search that Xiaohongshu previously considered an in-memory workload.

Approximate nearest neighbor search, or ANNS, finds vectors that are likely to be closest to a query without scanning every stored vector. It supports semantic search, content recommendations, advertising retrieval, moderation, and RAG systems.

Xiaohongshu operates these services at an unusually demanding scale. Its researchers report managing hundreds of billions of embedding vectors and millions of queries per second across the platform.

Search alone can cover as many as 20 billion vectors. Its typical peak traffic reaches about 300,000 queries per second, with an average latency target near 10 milliseconds.

Recommendation services create a different pressure profile. Individual indexes range from one million to 100 million vectors, while aggregate traffic can reach about 2.5 million queries per second.

Advertising involves around one billion vectors and similarly tight millisecond-level latency requirements. These services can request up to 3,000 candidates before later filtering and ranking stages make final decisions.

The older infrastructure used distributed HNSW indexes stored in DRAM. HNSW is a graph-based index that navigates connections between nearby vectors to find promising candidates quickly.

Keeping the graph and vectors in memory minimizes storage latency. However, it also ties capacity directly to DRAM, even when compute resources remain underused.

Xiaohongshu reports that its broader in-memory vector fleet had grown beyond 100,000 CPU cores across about 4,000 nodes and roughly 50 clusters. By 2025, its HNSW indexes were consuming petabyte-scale DRAM.

HELMSMAN does not replace that entire fleet yet. The production comparison covers workloads formerly using about 35,000 cores and 350 TB of DRAM.

About 40 HELMSMAN machines now carry those workloads. Each production server contains 12 NVMe SSDs and between 700 GB and 1.1 TB of DRAM, according to the paper.

That remaining memory matters. An all-flash server is not a memory-free server. HELMSMAN keeps cluster centroids, search-routing data, and pruning models in DRAM while storing large vector lists on SSDs.

The shift still changes the hardware balance. Instead of making DRAM hold every graph edge and vector, the system reserves memory for compact routing structures and active computation.

The researchers say this deployment has operated stably for several months. They are now rolling it out as a unified vector-search layer across more services.

Xiaohongshu also published a proof-of-concept implementation. That code offers a starting point for examination, although it does not reproduce every component of the internal production environment.

Why In-Memory HNSW Became the Cost Target

HELMSMAN attacks a capacity problem disguised as a performance requirement.

In-memory HNSW delivered the latency Xiaohongshu needed. Yet the infrastructure carried far more throughput capacity than normal traffic required.

The researchers found that online workloads used only about 32% to 43% of the throughput available from their in-memory deployments. The remaining capacity was not simply wasted provisioning.

Those resources stored complete indexes and absorbed traffic bursts without creating latency spikes. Removing them was difficult because graph-based search depends on many fine-grained memory accesses.

Meanwhile, Xiaohongshu says its stored vector corpus has been doubling annually. Billions of new vectors arrive as users publish content and models generate new representations.

Larger embedding models compound the pressure. A higher-dimensional vector consumes more storage, while retrained models can require new vectors for previously indexed content.

That growth affects more than capacity. It also increases the time and resources needed to build, distribute, and replace indexes.

The obvious response was to move vectors onto SSDs. Xiaohongshu already used hybrid DRAM and SSD systems for workloads with relaxed latency requirements, including some moderation and RAG applications.

The online path was harder. Systems such as DiskANN keep compressed vectors in memory while storing full vectors and graph data on SSDs.

Graph traversal creates a dependency chain. The system reads one section of the graph, evaluates its neighbors, and then determines which storage locations to read next.

Those serialized decisions prevent the storage layer from issuing enough independent requests to saturate a high-bandwidth SSD array. Fast drives remain underused while each query waits for its next graph step.

Xiaohongshu tested DiskANN, Starling, and PipeANN on servers with 12 PCIe Gen5 SSDs. DiskANN and Starling reportedly missed average and tail-latency targets in the tested online workloads.

PipeANN reduced latency through parallel graph exploration. However, its throughput remained between 10 and 25 times lower than in-memory HNSW under the required latency limits.

This is the key reason HELMSMAN did not simply place an existing graph index on newer storage. Faster flash could not remove the access dependencies built into graph traversal.

Clustering changes that access pattern. A query first identifies relevant clusters, then reads several cluster lists in independent batches.

That approach can use multiple drives concurrently. It exchanges a sequence of small dependent reads for a smaller number of wider, parallel operations.

Microsoft’s SPANN research established an important version of this architecture. It keeps compact centroid data in memory and places larger posting lists on storage.

Xiaohongshu found that SPANN scaled more predictably as SSDs were added. Twelve drives produced close to a twelvefold throughput increase in one experiment.

Still, unmodified SPANN reached only about 12% to 14% of HNSW throughput in Xiaohongshu’s tests. It also used only 26% to 59% of available SSD bandwidth.

That gap defined the real engineering problem. Flash capacity was affordable and its aggregate bandwidth was substantial, but the software stack could not convert that bandwidth into production query throughput.

How HELMSMAN Makes All-Flash Vector Search Competitive

HELMSMAN works because storage, search decisions, and index construction were redesigned as one system.

The first component is an ANNS-oriented userspace storage stack. Userspace storage lets an application communicate with NVMe devices without sending every operation through the normal kernel file-system path.

A conventional read can pass through system calls, the file system, the block layer, device mapping, and the NVMe driver. Each stage adds software work and coordination.

That overhead becomes visible when many CPU cores issue small, frequent reads. Lock contention and context switching can limit throughput before the SSD hardware reaches its own ceiling.

HELMSMAN uses SPDK, a userspace storage framework designed for direct, asynchronous device access. It stripes cluster lists across raw NVMe devices and manages queues closer to the application.

Search threads submit asynchronous read commands for selected clusters. They then poll hardware completion queues and calculate vector distances after the data arrives.

This design aligns storage operations with the clustered index. Cluster reads are independent, so HELMSMAN can batch them across multiple drives without waiting for a graph traversal.

The paper reports that graph-based competitors used less than 20% of SSD bandwidth in one production-derived workload. SPANN reached about 55% on PCIe Gen4 drives.

HELMSMAN reached about 85% utilization on the same generation. It reached roughly 70% on Gen5 SSDs, where the larger available bandwidth exposed new bottlenecks elsewhere in the server.

The second component is leveling-learned search pruning. Pruning decides which candidate clusters can be skipped without reducing recall below a target.

A fixed pruning rule behaves poorly across varied queries. Easy queries may scan unnecessary clusters, while difficult queries may stop too early and miss relevant results.

Workload differences make this problem worse. Search requests can ask for between 100 and 3,000 candidates, while RAG requests may ask for only 10 to 100.

HELMSMAN first uses a router model to select an initial search range. A second model then evaluates clusters and removes candidates unlikely to improve the answer.

The models consider the query, requested result count, centroid distances, and local distribution patterns. Bucketing limits model overhead by routing similar cases through appropriate pruning models.

This learned approach remains compatible with batched storage access. It chooses the batch before reading instead of adding a fine-grained decision after every storage operation.

Under the same average recall, Xiaohongshu reports that a fixed policy missed the 90% recall target for more than 40% of individual queries. HELMSMAN’s method brought more than 80% of queries above that target.

The distinction between average and per-query recall matters. A system can report an acceptable average while producing many weak results for individual users.

The third component addresses index construction. Clustering-based systems must partition vectors, balance clusters, duplicate some boundary vectors, and build routing structures.

CPU-only SPANN construction could take several hours for smaller production indexes and multiple days at billion-vector scale. Such delays conflict with frequently retrained embedding models.

HELMSMAN assigns coarse clustering to GPUs. It then distributes finer balancing work across an elastic pool of CPU workers and merges the output into the final index.

A 100-million-vector index required about nine to 12 hours with CPU-only construction on a 192-core machine. Four Nvidia L20 GPUs reduced that process to less than one hour.

For a 10-billion-vector dataset, elastic CPU scaling reduced end-to-end construction from more than 16 hours to about four to seven hours.

Xiaohongshu can temporarily borrow up to 10,000 CPU cores from online clusters during low-traffic periods. Online workloads retain priority, and construction tasks can be reassigned when workers become unavailable.

That operational integration is important. Fast query serving alone would not help if a new embedding model required days of unavailable or stale indexes.

The Real Contest Is Flash Clustering Against Memory Graphs

HELMSMAN does not make HNSW obsolete; it narrows the situations where an all-memory index is economically justified.

In-memory HNSW remains difficult to beat on raw latency and throughput. Its critical data stays close to the processor, and graph traversal avoids scanning large vector groups.

HELMSMAN instead aims for enough performance under a defined service-level agreement. That is a different optimization target from winning every benchmark.

Across evaluated workloads, Xiaohongshu says HELMSMAN delivered two to 16 times the throughput of existing DRAM and SSD systems. It reached between 47% and 85% of production HNSW throughput on two 10-billion-vector datasets.

The comparison used a single HELMSMAN machine with 96 CPU cores and between 160 GB and 330 GB of DRAM. The HNSW deployment used ten shards, 320 cores, and 2.5 TB of DRAM.

HELMSMAN reportedly reduced CPU use by about three to four times and DRAM consumption by nearly one order of magnitude in those tests. It still met the target latency.

This trade becomes attractive when an HNSW fleet is provisioned mainly to hold indexes. If normal traffic consumes less than half of its throughput, paying for maximum memory capacity produces poor utilization.

HELMSMAN’s clustering route also benefits more directly from faster storage. Upgrading from Gen4 to Gen5 SSDs improved its throughput by about 55% on lower-dimensional workloads.

The gain reached 87% on a 1,024-dimensional RAG workload. Graph-based SSD systems improved by only 10% to 30% in the same generation change.

Those results suggest graph systems remained limited by serialized I/O or software overhead. HELMSMAN converted more additional drive bandwidth into useful search work.

The comparison is not universal, however. Vector workloads differ in dimensions, recall targets, result counts, filters, update rates, and latency requirements.

Xiaohongshu’s online systems often retrieve hundreds or thousands of candidates. Downstream ranking models then filter and rescore those candidates using richer content and user signals.

Clustering works well in that setting because broad batched reads produce large candidate pools. A workload seeking very small result sets at extreme recall may favor a different index.

Other research is also exploring alternatives. Huawei and university researchers presented DistVS, which separates compute, memory, and SSD storage across three tiers.

DistVS keeps low-precision vectors near compute, higher-precision data in remote memory, and exact vectors on SSDs. Its PRESS algorithm progressively removes candidates across those tiers.

That architecture treats memory as a shared, independently scalable service. HELMSMAN instead concentrates high-bandwidth flash within each serving node and redesigns access around local devices.

Neither path establishes a single answer for every operator. Together, they show that vector infrastructure is moving beyond a binary choice between full DRAM and slow disk.

For engineering teams, the broader lesson is architectural. Hardware economics matter only when the access algorithm and software stack can exploit the cheaper medium.

Teams evaluating retrieval infrastructure should therefore measure complete workload behavior. A searchable collection of technical documents also depends on indexing, updates, and retrieval quality, not storage capacity alone.

What the 90% Cost Claim Does Not Settle

The headline savings are credible as a reported deployment result, but they are not a universal purchasing formula.

The first limitation is evidence scope. Most detailed performance and cost figures come from Xiaohongshu’s own paper and production measurements.

OSDI publication adds peer review and makes the methodology available for inspection. It does not create an independent reproduction of Xiaohongshu’s internal deployment.

The released proof of concept can help outside researchers examine design choices. It cannot reproduce proprietary datasets, traffic distributions, cluster management, or hardware procurement conditions.

The second limitation is the comparison boundary. The 90% figure applies to hardware costs for workloads migrated from a particular in-memory deployment.

It does not necessarily include engineering labor, migration risk, spare capacity, operational tooling, or every networking and storage expense. The paper discusses device costs more directly than total ownership costs.

HELMSMAN servers also retain substantial DRAM. Forty machines with 700 GB to 1.1 TB each imply a combined memory footprint between 28 TB and 44 TB.

That is far below the former 350 TB allocation, but it remains a meaningful infrastructure requirement. The system shifts the ratio toward flash instead of eliminating memory.

The third limitation appears inside the paper’s operational lessons. Flash bandwidth is not evenly available under every query pattern.

During early recommendation trials, bursts sometimes targeted the same clusters and logical blocks. Those hotspots caused conflicts inside SSD chips, even when overall bandwidth remained below 20%.

Xiaohongshu addressed the problem by storing redundant copies of selected cluster lists. That raised throughput by 1.5 to two times with a small storage increase, according to the researchers.

Replication is a practical solution, but it shows why aggregate bandwidth can mislead. A dozen drives cannot help when many requests collide on the same internal resources.

Server memory bandwidth creates another ceiling. Twelve Gen5 drives provide about 140 GB per second of aggregate external bandwidth in the tested configuration.

The system used only about 70% of that capacity in practice. Effective DDR5 bandwidth, usually between 300 GB and 350 GB per second, became the bottleneck first.

Memory must absorb transfers from SSDs, feed CPU distance calculations, and support centroid search. Adding more drives produces limited gains after these paths saturate.

The fourth limitation concerns updates. HELMSMAN accelerates complete index rebuilding, but it does not solve high-rate in-place changes.

A typical 100-million-vector recommendation workload may combine hourly rebuilding with 25,000 to 30,000 search queries per second. Replacing rebuilds would require a similar rate of concurrent insertions and deletions.

The researchers say current dynamic ANNS systems cannot sustain both rates for that workload. HELMSMAN therefore uses a hybrid freshness design.

The main index resides on SSD and is rebuilt periodically. Recent insertions live in an auxiliary in-memory index, while a bitmap records deleted vectors.

Queries search both indexes and merge the candidates. This preserves freshness, but it adds memory consumption, merge work, and ongoing rebuild costs.

The final uncertainty is end-to-end product quality. The paper evaluates recall, latency, throughput, bandwidth, construction time, and infrastructure efficiency.

It does not disclose business metrics such as search satisfaction, advertisement conversion, or recommendation engagement. Those outcomes depend on later ranking stages and broader product behavior.

Readers should treat the result as strong systems evidence under Xiaohongshu’s workload. They should not assume that moving an unrelated vector database to flash automatically saves the same percentage.

Three Signals Will Show Whether HELMSMAN Changes the Market

HELMSMAN becomes an industry reference only if broader migration, outside reproduction, and sustained product quality follow the paper.

The first signal is Xiaohongshu’s own rollout. The company says all-flash servers are gradually replacing in-memory deployments across search, recommendation, advertising, and other vector services.

The current 40-server deployment covers only part of a much larger vector fleet. A majority migration would show that the architecture survives more datasets, traffic patterns, and failure modes.

Watch for updated server counts, DRAM reductions, and workload coverage. Continued expansion would strengthen the claim that flash clustering can become the default serving layer.

A stalled rollout would suggest that hidden workload differences limit the system. Particular services might still require in-memory HNSW for latency, recall, or update behavior.

The second signal is independent reproduction. The preprint record and open implementation give researchers a stable basis for comparison.

Useful replications should test modern Gen5 arrays, varied vector dimensions, large result counts, skewed traffic, and strict tail-latency targets. Small public benchmarks alone would miss the deployment’s hardest conditions.

Comparisons should also include current versions of DiskANN, SPANN, DistVS, and dynamic indexes. The central question is not whether HELMSMAN beats an outdated baseline.

It is whether its combination of clustering and userspace I/O remains efficient across different hardware, data distributions, and service objectives.

Independent confirmation of bandwidth use and per-query recall would strengthen Xiaohongshu’s case. Large gaps would reveal how much the reported result depends on internal tuning.

The third signal is whether production quality holds as the workload grows. Infrastructure savings have little value if users receive weaker search or recommendations.

Xiaohongshu emphasizes large candidate sets with about 90% recall because downstream models perform final filtering and ranking. That choice reflects its multi-stage product architecture.

Operators with different pipelines may require 98% or higher recall for smaller result sets. Those targets can increase scans and weaken the cost advantage.

Watch for evidence covering tail latency, low-recall outliers, hotspot behavior, rebuild freshness, and downstream ranking quality. These measures will show whether the system’s efficiency survives operational stress.

HELMSMAN’s most important contribution is not the claim that flash has replaced memory. It shows that modern SSD bandwidth becomes useful only after the search path is reorganized around parallel access.

For infrastructure leaders, that creates a concrete decision. Measure whether DRAM is serving traffic or merely holding indexes, then test clustered flash under the actual latency and recall targets.

For researchers, the next task is equally clear. Reproduce the result outside Xiaohongshu, pressure-test dynamic updates, and determine which workloads still justify in-memory graphs.

Xiaohongshu HELMSMAN has already moved the debate beyond a synthetic benchmark. The next question is whether other large operators can obtain similar savings without sacrificing the quality their users notice.