Ipsoot.Data

Home / Ethical AI / Legal RAG

Retrieval comes first: how we build legal RAG

A cited answer is only as good as the passage it was found in. The research shows that finding the right passage matters as much as the model that writes the answer, if not more.

Structuredtexts split by article, section and subsection
Embeddedwith models chosen for legal language
Matchedthe local model best suited to each job

What RAG is

Our cited answers use retrieval-augmented generation, or RAG. Instead of answering from memory, the system first retrieves the passages that bear on a question (contract articles, board decisions, statutes), then the model writes its answer from those passages and cites them. Legal RAG applies this to legal text, where an answer is only as good as the source it can be checked against.

What the research shows

LegalBench-RAG (Pipitone and Houir Alami, 2024) built the first benchmark for the retrieval step of legal RAG. It found that how documents are split into passages changes the results, and that general-purpose tools did poorly on specialized legal text.

Legal RAG Bench (Butler and Butler, 2026) tested complete legal RAG systems. It found that the quality of retrieval, more than the choice of language model, drove correct answers, and that many errors blamed on a model making things up began with a retrieval failure.

How we build ours

We build retrieval with the same care as everything else. Texts are split meticulously along their own structure, by article, section and subsection, so each passage is a complete provision. Passages are indexed with embedding models chosen for how well they find legal language. And each job uses the local model best suited to it, running on our own servers or yours, as described on our Ethical AI page.