[EDRM Editor’s Note: EDRM is grateful to Trusted Partner Merlin Search Technologies for permission to publish. The opinions and positions are those of the authors. All images in the article are from Merlin Search Technologies.] 

In June, researchers at Redgrave LLP threw down the gauntlet in a debate that has been building across discovery for years. Their study compared Relativity’s generative AI review tool with a conventional review team using Continuous Active Learning (CAL). The headline results were arresting: GenAI found 88% of the responsive documents. The CAL-assisted review team found 64%.

One attorney working with AI appeared to outperform a 24-person team. Legaltech News declared: “Better Than TAR. Nearly Expert.”

Not so fast.

Earlier this month, Gordon Cormack, Maura Grossman and Jeremy Pickens published a response. After reanalyzing Redgrave’s data, they argued that the comparison was not apples to apples. Properly used, CAL could deliver comparable results and do so efficiently. We know the researchers on both sides. Robert Keeling and his colleagues deserve credit for putting a difficult question to an empirical test and sharing the data. Maura and Gordon helped put technology-assisted review on a scientific foundation. Jeremy and John developed commercial CAL together at Catalyst, and William has spent much of his career researching effective retrieval and active learning.

We respect the work. But after examining both studies, we think the researchers have framed the debate around the wrong question. And the answer has important implications for the future of document review.

Round One: The Case for GenAI

Redgrave tested 45,004 documents against a demanding legal standard involving pharmaceutical marketing and controlled-substance regulations. One attorney developed and refined the Relativity aiR prompt over approximately 18 hours and 43 iterations. An AI model then classified the collection. In parallel, a 24-person managed review team used CAL to prioritize documents for human coding, spending approximately 1,123 attorney hours.

Against the judgments of an independent expert reviewing a random sample, GenAI found 88% of the responsive documents; the managed CAL-assisted process found 64%. GenAI flagged more nonresponsive documents too, but missed fewer documents the expert regarded as important.

For a general counsel confronting a large discovery bill, the implications are unmistakable. AI can make complex review decisions without staffing a large human team. But the result does not prove that the GenAI model is a better search or prioritization technology than CAL.

Round Two: The CAL Researchers Respond

The response identified the crucial mismatch: in Redgrave’s CAL workflow, the algorithm found documents for human reviewers to judge. In the GenAI workflow, the AI made the judgments itself. If a human reviewer mislabeled an important document, the CAL process lost it even if the algorithm had surfaced it perfectly.

Cormack, Grossman and Pickens showed that the apparent gap largely disappeared when the methods were compared on equivalent terms. They also showed that strong CAL methods could achieve comparable outcomes using relatively little training. Their point was not that GenAI lacks promise. It was that Redgrave had compared two complete workflows, not two algorithms doing identical jobs.

So both studies teach us something valuable: GenAI can make sophisticated review decisions, and CAL can locate evidence efficiently. Why insist on choosing one?

The Hidden Cost of Prompt Engineering

There was another number in Redgrave’s study that caught our attention: roughly 18 hours spent by an attorney refining the AI instructions. At the rates charged by experienced law firm partners, that could be a substantial bill before the document review even begins.

And the prompt work is not incidental. An AI reviewer can be brilliantly consistent at applying instructions that turn out to be ambiguous or incomplete. The real challenge is to discover what the instructions are missing before applying them across hundreds of thousands of documents. That is why we built Document-Driven Review into Merlin’s Review Partner. The system starts with the legal team’s protocol, improves its structure, and then examines actual matter documents to uncover ambiguities and edge cases. AI identifies where the instructions fail, presents focused questions with supporting documents, and proposes changes. The review manager decides. The system revises and retests the protocol until it achieves the required clarity, then independently validates review accuracy.

With Review Partner, much of that refinement is automated. The system examines the documents, identifies ambiguities, proposes improvements and tests the revised protocol. Instead of an experienced lawyer spending 18 hours manually refining a prompt, our goal is to complete the automated, document-driven refinement process in under two hours, leaving counsel free to focus on the legal decisions that matter.

The Answer: One Integrated Review Process

Once the protocol has been refined and validated, we do not send GenAI blindly through every document and hope for the best. Nor do we ask teams of contract lawyers to review documents selected by CAL.

We put the technologies together. Hybrid search identifies promising evidence. GenAI applies the tested review protocol to documents. Its high-confidence decisions teach CAL which remaining documents are most likely to matter. CAL continually reprioritizes the collection, so GenAI spends its effort where it can produce the greatest value.

That pairing answers the central concerns raised by both papers. Redgrave demonstrated the cost and inconsistency of a large human review operation: we substitute AI reviewers. Cormack, Grossman and Pickens highlighted the efficiency and adjustable stopping decisions available through CAL: we preserve those advantages rather than insist on full linear AI review.

The result is not merely two technologies running side by side. It is one process in which each makes the other more useful.

Document-Driven Review improves the instructions. GenAI makes the substantive decisions. CAL directs the work. Independent validation tells counsel whether the combined process is meeting the matter’s objectives.

The Real Winner Should Be the Client

John helped bring CAL into commercial document review because it could move important documents to the front instead of making lawyers read in arbitrary order. Generative AI gives us something CAL originally lacked: a scalable reviewer capable of applying detailed legal instructions to those documents. Document-Driven Review supplies the missing first step: ensuring the instructions have been tested against the evidence.

Redgrave deserves credit for demonstrating what GenAI can do on a hard legal task. Cormack, Grossman and Pickens deserve credit for reminding us that CAL remains one of the best tools for finding what matters without reading everything. They are not mutually exclusive insights. They are the ingredients for the next generation of review.

For trial lawyers and general counsel, the goal is not to pick a winning algorithm. It is to find the documents that matter, control costs, support defensible decisions and spend more time on the case than on managing a review factory.

GenAI makes the decisions. CAL finds the next evidence. Document-Driven Review makes both more effective. That is where smarter discovery is headed.

Assisted by GAI and LLM Technologies per EDRM’s GAI and LLM Policy.