Understanding GenAI Response Limits: What Every Legal Professional Should Know

[EDRM Editor’s Note: The opinions and positions are those of John Tredennick and Dr. William Webber.] 

As some readers will know, we have spent many hours working with Generative AI-based Large Language Models (“LLMs”) like GPT (aka ChatGPT) and Claude. We use both in our DiscoveryPartner^® software to summarize and synthesize across documents, providing answers to user questions about large document sets. We also use both daily to assist with writing projects, reports for clients and the like.

Maybe you have noticed that these LLMs seem to have a limit on the length of their answers (or even document summaries). Plead, coddle or demand as you will, the LLM will still limit its answers to something in the range of a few thousand words. That can be frustrating to the user who wants the LLM to provide a longer analysis, perhaps for an in-depth report. Like writing a closing argument based on a review of a trial or arbitration transcript and the related exhibits?

“Why Can’t My LLM Write My Closing Argument,” you might ask?  And what can I do to fix that? We wrote this article to help legal professionals understand the answer to that question. We also wanted to tell you how you can get any LLM to write much longer pieces. In fact, one author had Claude prepare a 17-page, single-spaced closing argument for a team of lawyers we were helping, and did it in about an hour one Sunday.

The approach we will share isn’t just about circumventing length limitations—it’s about maintaining the rigorous analytical standards our profession demands while leveraging AI’s capabilities effectively. Much like how we structure complex litigation or transaction documents, this method provides a systematic framework that enhances both efficiency and quality. It allows us to harness AI’s analytical power while maintaining precise control over the final work product.

So here’s why LLM answers are limited and here’s a practical way to overcome that limitation.

The Technology Behind the Limits

To understand why these limits exist, consider how LLMs operate. These sophisticated systems process and generate text through complex neural networks that require immense computational resources. Each word generated demands significant processing power, with longer responses exponentially increasing these demands. It’s analogous to how a complex legal database search might slow down as we expand its parameters–there are practical limits to what can be processed efficiently and effectively at once.

Memory management presents another crucial constraint. Just as human memory becomes less reliable during lengthy depositions, LLMs must maintain context throughout their responses. As responses grow longer, the risk of losing coherence or introducing inconsistencies increases. This challenge mirrors how legal professionals must maintain consistent argumentation throughout lengthy briefs – the longer the document, the more challenging it becomes to maintain logical flow and avoid contradictions.

Quality control further necessitates these limits. Experience shows that extended AI-generated responses are more prone to inconsistencies and logical gaps. Much like how court-imposed brief length limits often improve the quality of legal argumentation, controlled response lengths help ensure AI outputs maintain their utility and reliability.

Response Limits for the Major LLMs

Understanding the specific limits of major AI platforms helps set realistic expectations for their use in legal practice. OpenAI’s GPT-4, currently among the most sophisticated models available, can generate responses of approximately 4,000 to 8,000 words. Anthropic’s Claude maintains a cap of about 5,000 words per response, while Google’s Gemini offers similar constraints. These limits aren’t arbitrary–they’re carefully calibrated to balance comprehensive analysis with reliable performance. And, in our experience LLMs tend to provide much shorter answers than these numbers, often more like 1,000 words, even when longer answers are appropriate and requested.

Impact on Legal Practice

These technical constraints directly affect how we can leverage AI in legal work. When analyzing complex agreements, for instance, we can’t expect an AI to process and comment on an entire merger agreement in a single response. Similarly, comprehensive legal research queries might require strategic structuring to work within these boundaries effectively.

Building on our understanding of AI’s technical constraints, let’s explore practical strategies for maximizing these tools’ potential in legal practice. The key lies not in viewing these limitations as obstacles, but rather as parameters that guide effective implementation–much like how we’ve long adapted our legal writing to meet various court-imposed requirements.

How to Coax an LLM into Creating a 17 Page Closing Argument

So how did we get an LLM like Claude to create a closing argument? Actually, there is a simple, three-step workaround that you might find handy. Here is how you can do it:

1. Ask the LLM to create an outline of the argument

First, ask the LLM to create an outline of your argument, or submit your own outline as a starting point. Either way, you have created a roadmap for the argument, report, article, etc. Work with the LLM to refine the outline so that it reflects your thinking and the points you want to make.

The LLM can easily craft a thoughtful outline in less than 5,000 words, one which covers all the points you or the LLM wants to make. This crucial first step helps ensure no critical elements are overlooked and provides a structured framework for subsequent analysis. Think of it as creating a detailed brief structure or case management plan before diving into the substantive work.

To illustrate the point, we asked Claude Sonnet 3.5 to prepare a comprehensive outline relating to the trial testimony and the exhibits admitted in the Phase One causation and liability trial regarding the Deepwater Horizon Oil Spill involving British Petroleum and several other defendants including TransOcean and Halliburton.

As a starting point, we gave Claude this prompt:

Here was Claude’s response:

Answer: Based on the document summaries provided, I can create a comprehensive closing argument outline for TransOcean: 

I. Introduction     

     A. Acknowledge the Tragedy         

          1. Express deep sorrow for loss of 11 lives and environmental impact (BP_00001474)         

          2. Recognize the gravity of the April 20, 2010 incident (BP_00000001 §3)     

     B. Frame Key Issue         

          1. Multiple parties involved in complex operation         

          2. Evidence shows BP bears primary responsibility 

II. TransOcean's Demonstrated Commitment to Safety     

     A. Crew Qualifications and Training         

          1. Highly experienced crew with proven safety record (BP_00000001 §24)         

          2. Regular safety drills and training conducted (BP_00000001 §24)     

     B. Equipment Maintenance and Compliance         

          1. BOP maintained to industry standards (BP_00001161 §2)         

          2. Regular testing and maintenance documented (BP_00000035 §29) 

III. BP's Primary Responsibility as Well Operator     

     A. Operational Control         

          1. BP was majority owner and operator of Macondo well (BP_00001391 §2)         

          2. BP had ultimate authority for well design and operations (BP_00001391 §2)     

     B. Decision-Making Authority         

          1. BP made critical operational decisions from onshore (BP_00000001 §22)         

          2. BP controlled temporary abandonment procedures (BP_00001391 §2)

IV. BP's Critical Failures

     A. Corporate Culture of Cost-Cutting         

          1. Prioritized financial savings over safety (BP_00000001 §16)         

          2. Ignored internal safety warnings (BP_00001402 §14)     

     B. Technical and Operational Failures         

          1. Inadequate well design with insufficient drilling margins (BP_00000042 §18)         

          2. Failed to conduct proper cement bond testing (BP_00000001 §12)         

          3. Misinterpreted negative pressure test results (BP_00000001 §23) 

V. BP's Knowledge of Risks   

     A. Prior Warnings Ignored         

          1. Professor Bea's warnings about safety risks (BP_00001402 §14)         

          2. History of accidents at other BP facilities (BP_00001402 §14)     

     B. Deliberate Risk-Taking         

          1. Used inadequate number of centralizers (BP_00000001 §12)         

          2. Rushed temporary abandonment procedures (BP_00001391 §20) 

VI. TransOcean's Appropriate Actions     

     A. Emergency Response         

          1. Crew followed established procedures (BP_00000001 §23)         

          2. Attempted appropriate emergency measures (BP_00000001 §23)     

     B. Post-Incident Cooperation         

          1. Immediate response to support investigation (BP_00001474)         

          2. Collaboration with authorities and other parties (BP_00001474) 

VII. Conclusion     

     A. Restate BP's Primary Responsibility         

          1. BP's decisions and actions led to the disaster         

          2. BP's corporate culture prioritized profit over safety     

     B. TransOcean's Limited Role         

          1. Acted within industry standards         

          2. Followed BP's direction as the well operator     

     C. Final Appeal         

          1. Evidence clearly shows BP should bear responsibility         

          2. TransOcean should not be held liable for BP's failures

2. Ask the LLM to prepare a comprehensive argument based on each section of the argument.

The next step is to work methodically through the argument. With a comprehensive outline in hand, address each component individually, allowing the AI to dedicate its full response capacity to detailed analysis. This approach mirrors how many professionals typically handle a complex writing project. Work on each section independently with the LLM until you are satisfied with the results.

There are a number of techniques we can use to get this information, with some requiring the LLM to make an extensive review of the most relevant sections of the trial testimony and exhibits. While approaches may differ, we can state confidently that the LLM cost for such an exercise will be a very small fraction of the legal costs for a human team to do the work.

Here, for example, is a draft argument based around the second argument presented in the closing: II. TransOcean’s Demonstrated Commitment to Safety:

Our point with this example isn’t to suggest that an LLM can craft a better closing argument than a senior trial lawyer who has lived with the case for years, including through what had to be an exhausting and complicated trial. Rather, think of this as an initial draft. If you could create something like this for a few hundred dollars in LLM costs, wouldn’t you?

3. Combine each of the sections into your final paper.

The last step is simple. Take each of the answer sections and paste them into a Word or Google document. You might need to format the headings or adjust bullets or numbered lists, but this is quick and easy. Stitching together even a long argument or report can be done in minutes.

To be sure, you will want to review the transitions between sections, but in our experience the LLM will do a good job at this work. Refine the language to your liking and voila, you have a finished argument/report/paper that provides detailed analysis across far more than the 5,000 word limit of a single LLM answer.

Is the work ready for prime time? I think you will be amazed at the quality of an LLM’s writing output, but you aren’t bound to use it as received. Rather, consider it a good (actually very good) first draft. In our experience, the output is every bit as good as one might receive from an excellent associate. Take the writing and make it your own. It is a lot better than starting from scratch.

The Future of Legal Practice: Strategic AI Implementation

Understanding AI’s response limits isn’t just about managing technical constraints—it’s about adapting our legal workflows to harness these powerful tools most effectively. As we’ve explored, these limitations stem from fundamental technical considerations: computational demands, memory management, and quality control mechanisms that ensure reliable outputs. While current models like GPT-4, Claude, and Gemini maintain specific word limits for their responses, these constraints need not limit the scope or quality of AI-assisted legal work.

Through methodical approaches like the three-step strategy outlined above—comprehensive outlining, systematic section development, and strategic integration—legal professionals can effectively generate extensive, high-quality legal content that far exceeds standard AI response limits. This structured methodology aligns naturally with how lawyers traditionally approach complex legal analysis, making it both practical and intuitive to implement.

When we created that 17-page closing argument with Claude in about an hour, it wasn’t because we found a way to override the system’s constraints. Rather, we embraced them, using a methodical approach that mirrors how we naturally handle complex legal analysis. By breaking down our work into discrete components—outline, section-by-section development, and strategic integration—we can harness AI’s capabilities while maintaining the rigorous standards our profession demands.

As AI technology continues to evolve, response limits will likely expand, but the fundamental principles of strategic implementation will remain valuable. The key to successful AI integration in legal practice isn’t just understanding technical limitations—it’s developing systematic approaches that leverage these tools’ capabilities while working within their constraints. By viewing these limitations not as obstacles but as parameters for effective implementation, legal professionals can maximize AI’s potential while maintaining the rigorous standards our profession demands.

Just as we’ve adapted our practice to court-imposed brief limits, electronic filing requirements, and other technological advances, mastering AI response limits represents another step in the evolution of legal practice. The future belongs to professionals who can skillfully navigate these constraints while leveraging AI’s capabilities to enhance their legal analysis and advocacy.

About Merlin Search Technologies

Founded in 2019, Merlin Search Technologies is a leader in AI and secure cloud technology, harnessing the transformative power of Generative AI to reengineer search, investigations, and discovery workflows for the digital age. As industry trailblazers, Merlin offers the first-ever On/Off Cloud Utility Pricing, allowing clients to reduce hosting costs, support green computing, and make discovery more secure, flexible, and cost-effective.

With a team bringing two decades of experience designing and building advanced discovery platforms for the largest companies and law firms worldwide, Merlin is poised to redefine the future of search, investigations, and discovery. Driven by innovation and backed by cutting-edge AI and cloud technologies, Merlin’s mission is to revolutionize how legal professionals handle complex challenges by enabling clients to find critical information faster, more effectively, and at lower costs than ever before.

Assisted by GAI and LLM Technologies per EDRM GAI and LLM Policy.