[EDRM Editor’s Note: EDRM is grateful to Trusted Partner Merlin Search Technologies for permission to publish. The opinions and positions are those of the authors. All images in the article are from Merlin Search Technologies.] 

For the first 20 years of my career, I asked questions for a living.

As a trial lawyer, I wanted to know what really happened. Who knew about the problem? When did they know? What did they tell the client? Which document changed the story? Which witness had it wrong?

My question was simple: What happened?

The answers were somewhere in the evidence. Getting to them consumed an extraordinary amount of our time.

Electronic search gave us a remarkable shortcut. We could find words across enormous collections in seconds. But we still had to imagine the language people might have used, read what came back and work out what it meant. To get closer to an answer, we first had to answer the search box’s question: “Which keywords?”

That gap has occupied much of my professional life, first in the courtroom and then building discovery technology at Catalyst and Merlin.

Dr. William Webber approached it from another direction: information retrieval, statistical evaluation and machine learning. His perspective brings an equally important question to the investigation:

How do we know it happened?

Together, those questions came to define the work. We need to understand the events, and we need evidence that supports our account. Our recent research gave us a new way to think about how technology could help with both.

Moving Beyond Search

Over the past few weeks, William and I put our research and experience at Merlin into three articles exploring how keyword search, semantic search and Continuous Active Learning could work together.

We began with a simple proposition: keyword search wasn’t dead, but searching with keywords alone should be. Our first article explored why relying on the words we think to search can leave important evidence undiscovered.

The second examined what semantic search adds: the ability to find relevant information expressed in different words. But it also exposed that method’s limitations. Our experiments showed why combining keywords and semantic search could be more effective than relying on either alone.

The third brought CAL into the search process. AI assessed the initial results, and CAL learned from the clearest assessments to improve what the system found next. As we evaluated more documents and retrained the classifier, the evidence kept teaching the system what to look for.

That was meant to complete our study of hybrid search.

We thought Article Three was the end. It turned out to be the doorway.

Once AI could assess what search found and help the system learn from it, why stop at retrieving documents? It could also read them, compare accounts, explain what the evidence appeared to show and help us pursue the next question.

That became the subject of our fourth, unexpected article.

Four Eras, One Continuing Evolution

Working through that fourth article helped us see the larger pattern: search had been gaining capabilities that could now work together throughout an investigation.

Each era gave the machine another capability. Keyword search found the words. Semantic search added the ability to find meaning. Continuous Active Learning allowed the system to learn from the evidence. Now generative AI can bring those capabilities together, read what they find and work with us to understand what happened.

These eras overlap. Their capabilities accumulate. Keywords remain essential when exact language matters. Semantic search reaches beyond the words we thought to use. CAL learns what relevance looks like in the matter itself. The fourth era draws on all three to support something larger: an AI teammate helping investigate the evidence.

Much of discovery still operates primarily in the first era. To see what becomes possible beyond it, we need to begin with the capability that changed how we worked with evidence: finding words across a collection in seconds.

Era One: Find the Words

For most of the history of electronic discovery, every search began with the same question:

Which keywords should we use?

The human supplied the keywords, woven into a complex web of ANDs, ORs, wildcards, proximity operators and parentheses. The machine found the matches.

I watched that revolution unfold as a trial lawyer. Suddenly, we could search electronic documents in seconds instead of working through boxes of paper. Over time, the tools grew more sophisticated and the results more useful.

But the essential task remained ours: imagine how the people in the documents might have described what we wanted to find.

If we were investigating a failed project, did they call it a delay, a slip, a reset or a disaster? Did they use an internal acronym or a project name nobody on the legal team knew?

Good searchers learned as they went. A document revealed a new name. Searching that name uncovered an acronym. The acronym led to a project code. Each discovery helped us improve the next query.

The machine supplied the speed. We supplied the vocabulary.

That dependence on vocabulary created a persistent problem. Relevant evidence could use language our searches never included. And the missing documents were invisible: a convincing set of results could tell us what we found, but not what we missed.

This does not diminish the importance of keyword search. Exact language can be the strongest possible signal. A contract clause, account number, unusual name, email address or internal code may be precisely what matters. Keywords remain a foundation of search in our industry and at Merlin.

The next advance built on that foundation by addressing its central limitation: finding relevant evidence even when we did not know the words its authors had used.

Era Two: Find the Meaning

Semantic search opened a new possibility. We could describe the evidence we wanted to find without first guessing all the words its authors might have used.

The question began to change from “Which keywords?” to “What evidence am I looking for?”

Suppose a lawyer wanted to know whether management realized a project would miss its deadline before telling the client. Relevant documents might say “we are not going to make the date,” “this is slipping badly,” “we need another month” or “do not tell them yet.”

Keyword search depends on anticipating those expressions or discovering them along the way. Semantic search can look for the underlying idea even when the documents use different language.

For lawyers and investigators, that is an exciting advance. We can begin closer to the way we think about a matter: Who knew about the delay? Was anyone trying to conceal it? What changed after the regulator got involved?

The machine had gained another capability: searching for meaning as well as words.

What Project Checkmate Taught Us

It is tempting to assume that searching for meaning must be better than searching for words. Our research gave us a vivid reason to question that assumption.

In the second article in our series, we tested a search for “Project Checkmate,” an internal name for a collaboration involving pandemic-virus research. We supplied only the name, without explaining what the project concerned.

Keyword search found 34 responsive passages among its first 50 results. Semantic search found none.

The reason was straightforward. “Checkmate” suggests chess, winning or defeat. “Project” tells us little. Neither word conveys the research behind the name. The identifier mattered because people used it to refer to a particular project.

Keyword search could find that identifier. Semantic search lacked the context needed to connect it to the subject we wanted.

That became a central lesson of our work at Merlin. Combining the methods gives the system more to work with: exact names, codes and phrases, alongside relevant material expressed in language we did not anticipate.

Together, keywords and semantic search gave us a stronger starting point. But the evidence they found could teach us more than we knew when we framed the query. That suggested the next opportunity: let the system use those discoveries to improve the search still underway.

Era Three: Learn From the Evidence

A useful document might reveal a project name, a new participant or an unexpected relationship. More importantly, it shows what relevant evidence actually looks like in this matter.

A good lawyer uses those discoveries to decide where to look next. We wanted the search system to do the same.

The question now becomes: What have we learned, and what comes next?

Bringing Learning Into Search

Continuous Active Learning, or CAL, learns from examples. Give it documents that are relevant and documents that are not, and it begins identifying the patterns that distinguish them. As more relevance judgments arrive, it updates its model and prioritizes additional material.

This was familiar territory for both of us. William had spent years studying information retrieval and machine learning. At Catalyst, I had seen CAL transform document review by helping teams find relevant material earlier and work through collections more efficiently.

Our research explored how to bring that capability into search itself. Could the system learn from its initial results and improve what it retrieved next, without requiring the lawyer to stop and manually train it?

Generative AI gave us a way to do that.

An AI model could read the initial results and assess their relevance to the question. The clearest assessments could then become training examples for CAL. Keyword and semantic search would find the starting evidence; AI would evaluate it; CAL would learn from those evaluations and help find more.

The evidence we found could improve the search still underway.

What Our Experiment Showed

In the third article in our series, we compared two paths beginning with the same first 50 hybrid-search results.

One continued down the original ranking. The other used AI’s clearest relevance assessments to train CAL, then combined that learning with the search results to select what came next.

The second path found a higher proportion of responsive material. Among the next 50 results, precision rose from roughly 66 percent to 77 percent. In practical terms, more of what the system returned was relevant.

Project Checkmate made the idea especially clear. Keywords found the internal project name. AI could read the retrieved passages and assess whether they concerned the project. Those assessments gave CAL examples of the subject behind the name, helping it identify related material beyond the literal phrase.

The experiment showed the benefit of learning from the first results. The larger opportunity was to keep that learning going.

The Deeper We Go, the More Learning Matters

CAL’s real strength emerges as we work through hundreds or thousands of documents.

A conventional search ranks documents against the query. The strongest matches generally appear near the top. As we move down that ranking, we encounter weaker matches and spend more time finding useful evidence. Unless we revise the query, the search has no way to benefit from what the documents we have already read taught us.

CAL changes that pattern. After each batch of 50 documents, we use the clearest AI relevance assessments to train the classifier again. It then ranks the remaining collection using what it has learned.

The next batch adds more examples. Those examples inform the next round of training. The process continues, giving the classifier an increasingly informed picture of relevance in this particular matter.

Standard search works through a ranking. CAL keeps learning how to improve it.

That difference becomes especially important deeper in the collection. Relevant documents may use unexpected language or look quite different from the strongest initial matches. CAL can use patterns learned from earlier batches to bring that material forward, while a fixed search ranking continues toward its weaker results.

Keywords and semantic search provide the starting evidence and contribute signals throughout the process. CAL adds the ability to keep adapting as the evidence teaches us more.

By then, AI was reading the evidence to help decide what we should retrieve next. We began asking what else it could help us understand.

Era Four: AI Joins the Team

In our fourth article, we followed that question beyond retrieval. AI could compare accounts, connect events, identify contradictions and explain what the evidence appeared to show.

That opened a much bigger possibility: give the system an investigative assignment and work through it together.

Finally, the starting question becomes: What do you want to know?

Let’s walk through how AI and hybrid search can help a team pursue the answer.

1. Define the Assignment

Consider a question from the UK Post Office investigation:

What did Fujitsu know about problems with the Horizon system, when did it know, and what did it do in response?

That is an investigative assignment. It involves technical problems, internal communications, decisions, dates and potentially conflicting accounts. At the outset, the team may not know the terminology or even all the issues it should investigate.

The first task is to sharpen the question using what the collection can tell us. An AI prompt helper can examine an initial selection of documents, surface unfamiliar names and terminology, and ask clarifying questions. Does the assignment concern a particular period? Which kinds of system problems matter? Should the investigation examine what technical staff knew, what management knew or what was disclosed in litigation?

The professional supplies the objective and context. AI helps turn that objective into a clearer assignment grounded in the collection.

2. Find the Relevant Evidence

That clearer assignment gives the search system its direction. The three methods then work together:

• Keyword search finds exact names, error codes, ticket numbers and distinctive phrases.
• Semantic search finds the underlying issues even when people describe them differently.
• CAL learns from AI’s clearest relevance assessments, retraining after each batch of 50 documents to improve what the system finds next.

Together, these methods give AI a stronger body of evidence to examine. The system can search beyond the language of the initial question and use what it learns to reach deeper into the collection.

3. Read the Evidence and Develop an Account

As relevant material emerges, AI can read it, compare documents and prepare an initial account of what the evidence shows.

For the Horizon assignment, that might include a timeline of reported problems, who received those reports, what action followed and where internal discussions differed from statements made outside the company. The account can point back to supporting documents and identify questions the available evidence has not resolved.

The team can begin developing and testing its understanding while the investigation is still underway.

Alongside the documents, we have an organized account that helps us identify what to investigate next.

4. Challenge the Answer and Follow the Evidence

That account gives the lawyer a basis for more focused questions:

What evidence points the other way? Was the supposed fix actually tested? Did the people handling litigation know what the technical team knew?

Those questions may require closer examination of documents already found or a further search into the collection. New evidence can change the account, reveal another participant or suggest a different explanation.

Each exchange helps direct the next inquiry. The process continues as the team learns more.

This is the rhythm of an investigation, with AI taking on more of the work required to pursue each lead. The professional directs the inquiry, examines critical evidence and decides what the record supports.

That is what we mean by AI-Integrated Discovery: AI and hybrid search working with the professional from the first question through the continuing investigation, helping the team build and test its understanding of what happened.

As the account develops, the team needs to test both the conclusions and the evidence behind them. That brings us back to William’s question.

How Do We Know It Happened?

A clear narrative is a starting point. Its value depends on the evidence supporting it and how well it survives examination.

Three disciplines help us make that assessment.

Check the supporting documents. Factual conclusions should point back to the evidence behind them. The professional needs to examine the source, read it in context and determine whether it supports the account. A citation makes that examination possible; it does not settle the question.

Test what the search may have missed. Combining keywords, semantic search and CAL gives us more ways to find relevant material. We still need to assess coverage. Sampling documents outside the retrieved set can reveal missed evidence and help estimate how much relevant material remains.

Pursue competing explanations. Ask what evidence contradicts the account. Does the timeline hold up? Did a different participant describe the events differently? Is a conclusion resting on one ambiguous email? Those questions direct the next round of investigation.

This is where our two perspectives meet. My courtroom question asks what happened. William’s question asks how the evidence establishes it. AI can help us develop an account and investigate its weaknesses. The professional decides what the record supports.

We have more help answering both questions, and more time to examine the answers.

A Renaissance for Lawyers and the Practice of Law

That additional time is, for me, the most exciting consequence of this evolution. It gives us room to do more of the work that drew us to the profession.

Few lawyers went to law school because they dreamed of constructing Boolean searches, sorting email chains or spending nights reading thousands of documents to assemble a timeline. We did that work because it stood between us and the work we wanted to do.

We wanted to understand a difficult problem. Counsel a client who needed an answer. Find the weakness nobody else saw. Prepare a witness, test a theory, build a strategy and make an argument. Help someone whose business, reputation, freedom or future was at stake.

Those are the reasons I became a trial lawyer. They are also the reasons I have spent so much of my career trying to improve discovery.

An AI teammate can take on more of the searching, reading, comparing and organizing required to reach that work. Teams can begin understanding a matter earlier, test more explanations and pursue leads that might previously have demanded too much time or money.

The difficult decisions remain. Evidence will conflict. Witnesses will disagree. Clients will face choices with uncertain consequences. Our judgment, creativity, skepticism and experience belong at the center of those decisions. The opportunity is to spend more of our time applying them.

The promise of an AI teammate reaches beyond faster discovery. It gives lawyers more time for the responsibilities that make law a noble calling: protecting rights, challenging abuses of power, resolving disputes and helping achieve justice. Those responsibilities have always been there. As AI takes on more of the mechanical burden, we have an opportunity to return them to the center of our practice.

That is why the fourth era could become a renaissance for both lawyers and the practice of law.

We began this journey looking for better ways to search. What we now see is an opportunity to reclaim more of the work that brought us to the profession. AI can do more of the heavy lifting. We can spend more time doing what we went to law school to do.

The best future for legal technology is not one where lawyers do less law. It is one where they finally get to do more of it.

The AI Search Series

• Keyword Search Isn’t Dead. Searching With Keywords Alone Should Be
• AI Search Can Find the Meaning. But Still Miss the Evidence
• Search Finds the Initial Evidence. CAL Learns What Comes Next
• Beyond Search: From Keywords to AI-Integrated Discovery

Assisted by GAI and LLM Technologies per EDRM’s GAI and LLM Policy.