LlamaIndex Releases Legal-KB Agentic Retrieval Over Index v2

LlamaIndex has released legal-kb, a reference application that applies agentic retrieval over its Index v2 platform for legal documents.

The project supplies a TanStack Start web app that turns uploaded PDFs into a versioned, searchable knowledge base. Users can bring their own API keys for OpenAI or Anthropic models. The agent operates through a ToolLoopAgent built on Vercel AI SDK 6.

The release addresses a concrete workflow gap. Legal teams need agents that locate the right file first, then extract precise passages, rather than guessing across an entire corpus. Legal-kb demonstrates this sequence with four purpose-built tools.

What Legal-KB Actually Ships

Legal-kb equips the agent with retrieve, findFiles, readFile, and grepFile. Retrieve runs hybrid semantic search with optional reranking and citation return. FindFiles supports exact or fuzzy filename lookup. ReadFile returns raw text with byte offsets. GrepFile performs regex matches and reports character positions.

The agent is required to call findFiles before invoking the other three tools. This ordering prevents the model from scanning irrelevant documents and keeps token usage bounded. Every new upload of the same filename creates a new version record that can be filtered by metadata at query time.

All processing runs on LlamaParse within the Index v2 backend. Parsed chunks carry layout, table, and heading metadata that the retrieval harness preserves.

Why Legal Teams Need File-First Agents

Legal document sets frequently exceed several thousand pages across dozens of matters. Standard vector search returns passages without confirming the source document still exists or that the correct version is being read. Legal-kb forces the agent to declare the file list first, then operate inside that list.

The pattern mirrors how junior associates work: identify the binder, open the exhibit, then locate the clause. By codifying this sequence in tool definitions, the application reduces hallucinated citations and improves auditability.

ToolLoopAgent and Model Choice

The application uses Vercel AI SDK 6 ToolLoopAgent rather than a custom ReAct loop. This choice lets developers swap between OpenAI and Anthropic models by changing a single environment variable. Users supply their own keys at runtime, so no data leaves the local instance except through the chosen provider.

The web interface is a TanStack Start application. Drag-and-drop upload triggers parsing and indexing automatically. Version metadata appears in the file list so teams can restrict retrieval to a specific matter date or revision.

Comparison With Standard RAG Pipelines

Standard retrieval-augmented generation pipelines treat the entire index as a flat collection. An agent may receive passages from multiple matters in one response. Legal-kb instead surfaces the filename constraint before any passage is read.

Tool capabilities

• Retrieve supports rerank and citation metadata

• FindFiles returns only matching filenames with version counts

• ReadFile accepts offset and length parameters for large exhibits

• GrepFile returns start and end character positions inside the file

This explicit sequencing replaces the single-step retrieve-then-generate pattern common in most open-source demos.

Limits of the Current Reference

The repository supplies no evaluation harness for citation accuracy or version filtering correctness. The example agent follows the prescribed tool order, yet nothing prevents a different developer from bypassing findFiles. Production use would require additional guardrails around version selection and access control.

The project also depends on the stability of LlamaParse output. Any change in chunk boundaries affects downstream grep and read operations that rely on character positions.

What Developers Should Watch Next

Teams testing legal-kb should track three signals over the next three months. First, whether LlamaIndex publishes an official evaluation set that measures citation precision after findFiles filtering. Second, whether the community extends the same four-tool pattern to non-legal domains such as financial filings or regulatory dockets. Third, whether Vercel AI SDK releases additional loop controls that make mandatory ordering easier to enforce without custom scaffolding.

These signals will show whether the file-first retrieval pattern remains a legal-specific reference or becomes a reusable template for other document-heavy industries.