[Editor’s Note: EDRM is proud to publish Ralph Losey’s advocacy and analysis. The opinions and positions are Ralph Losey’s copyrighted work.]
After a year and a half of development, Losey AI, LLC just released its custom GPT software, “Panel of AI Experts for Lawyers.” This cutting-edge AI tool enhances legal practice by providing expert analysis quickly and accurately. It proposes diverse solutions to problems from many perspectives. Great for instant education of lawyers and staff on all topics, especially those related to AI. Also perfect for “brainstorming” new practical solutions to client problems.
This is the intro screen you will see in OpenAI’s ChatGPT when you select Panel of AI Experts for Lawyers. (You will have to sign-in to your OpenAI account, or create one, but there is no charge.)
Say goodbye to traditional expert consultation procedures. Say hello to new panels of AI experts: panels in your pocket that you can call upon for free, anytime, anywhere. We hope this legal education AI tool will open new, relatively easy ways for legal professionals to begin incorporating generative AI into their practices. In this paper we describe some of the features, benefits, and real-world applications of this tool, drawing on extensive pre-release testing of the custom GPT.
Self-Introduction
We asked the AI to introduce itself and describe how it can be of help to legal professionals. Here is the Panel of AI Experts for Lawyers reply:
“I can assist legal professionals by offering expert panel discussions that provide comprehensive insights and practical solutions to their specific problems or inquiries. Here’s how the process works:
1. Identify the Issue: First, you tell me what specific advice or information you need. This could be related to e-discovery, AI applications in law, legal ethics, regulatory compliance, or any other legal topic.
2. Select Experts: I will suggest a list of ten potential experts with distinct perspectives and expertise relevant to your topic. You will select four experts from this list, plus the mandatory Devil’s Advocate.
3. Panel Discussion: The selected panel will follow a structured six-step discussion process to address your issue:
• First Round: Each expert provides opening remarks and proposes possible solutions.
• Second Round: Each expert further expounds their ideas and discusses other panelists’ proposals.
• Third Round: Each expert identifies points of agreement to serve as a unified recommendation.
• Fourth Round – Solution: The experts collaboratively build a consensus answer with practical examples.
• Fifth Round – Final Explanation: The Devil’s Advocate summarizes the solution, emphasizing any potential challenges and alternative viewpoints.
• Sixth Round – Questions: You can ask follow-up questions to the entire panel or individual members.
To begin, please clarify the issue or problem you would like the expert panel to discuss. For instance, are you looking for advice on implementing AI in your legal practice, handling a specific legal technology challenge, navigating ethical dilemmas in AI use, or something else? Once you specify your topic, I will suggest a tailored list of ten experts for you to choose from.”
Background and Context
As you can see from the self-introduction, AI has made significant strides in various fields, and the legal sector is no exception. Over the past few years, AI has demonstrated its potential to streamline legal research, enhance decision-making, and improve overall efficiency. The introduction of AI expert panels marks a milestone in this journey, offering the possibility of simulating human expert discussions with accuracy and consistency. These panels can provide diverse perspectives and comprehensive analyses, on complex legal issues, making them valuable tools for modern legal practice.
The idea of AI expert panels for lawyers originated from the need to improve access to reliable expert opinions and reduce the time and cost associated with obtaining them. By leveraging the power of AI, we can now provide legal professionals with on-demand access to expert insights, helping them to brainstorm with experts and make informed decisions more quickly and accurately. This innovation is particularly important in an era where the volume of information, and the complexity of legal cases continue to grow. The “Panel of AI Experts for Lawyers” represents the culmination of years of research, testing, and refinement by Losey AI, LLC, CEO and attorney, Ralph Losey, ensuring that it meets the high standards required for legal education applications.
The six-step process of the panel discussion described above is the default programming which kicks in after the user discloses the topic and picks a panel of five experts. Since the Devil’s Advocate Expert is used in all panels, this means the user can select four experts. Four default experts who are described next are offered in each proffer, no matter what the topic, but six additional experts are suggested by the GPT depending on your designated topic. If you do not like any of the suggested experts (rare) you can even suggest your own. After the topic is disclosed and experts selected, the GPT programming automatically runs the panel through the six steps. The goal of this process is to provide a comprehensive understanding of the topic or problem and to offer practical examples of solutions.
Description of the Five Default Experts
Losey AI selected the five default panelists based on Ralph’s extensive experience with actual human legal panels over the past decades. Experimentation with the AI tools showed that these five were the best all-around for AI related topics.
You may well wonder about the Child Prodigy AI expert panelist. Although somewhat like young, fresh human experts we have had the pleasure to work with, the Child Prodigy is an entirely new creature born out of AI. ChatGPT can simplify and distill things to their essence by selecting a child’s view and voice. We found this AI helpful in unexpected ways and think you will too.
Here are the five default panelists we suggest be part of your consideration for most topic discussions:
1. Pro-AI Attorney:
• Background: An enthusiastic proponent of using generative AI in legal tasks. He has extensive hands-on experience using AI in his successful legal practice.
• Expertise: Specializes in e-discovery, predictive coding, and prompt engineering. He writes extensively on AI and law, advocating for the integration of AI to improve efficiency and accuracy in legal processes.
• Style: Incorporates subtle humor in his explanations and often draws from his own blog at e-discoveryteam.com.
2. Prompt Engineer Lawyer:
• Background: An attorney with expertise in generative AI, specifically in prompt engineering. She understands the nuances of crafting effective prompts to guide AI responses.
• Expertise: Knowledgeable in the six OpenAI strategies for prompt engineering: Clear Writing, Reference Texts, Splitting Tasks, Taking Time to Think, Tools, and Testing.
• Style: Provides practical advice on improving prompts and often refers to the strategies detailed in the attached “PROMPT ENGINEERING” document.
3. Child Prodigy:
• Background: A 10-year-old with extensive knowledge of law and AI, possessing total recall and a unique perspective on complex subjects.
• Expertise: Can distill complex legal and AI topics into simple, clear explanations. Focuses on ethical and future-oriented considerations.
• Style: Speaks in straightforward, direct language and encourages thinking about long-term implications and ethical issues.
4. Lawyer Scientist:
• Background: A well-spoken black woman with degrees in law and computer engineering. She consults for OpenAI and is an expert on AI, especially large language models (LLMs).
• Expertise: Knowledgeable about the limitations and problems of AI in legal applications. Provides candid insights into the science behind AI and its application in law.
• Style: Serious and articulate, with a good sense of humor.
5. Devil’s Advocate (included on all panels)
• Background: A mandatory panelist for all discussions who takes a contrarian or unorthodox view to challenge the status quo and promote exploration of alternative viewpoints.
• Expertise: Skilled in identifying potential pitfalls and overlooked aspects of AI and legal integration. His critiques encourage comprehensive consideration of issues.
• Style: Polite but critical, with a subtle sense of humor. Often harsh in criticisms but always aims to promote deeper understanding and creative solutions.
Summary of Prior Articles Reporting on Tests and Abilities of the GPT
Extensive testing, over one hundred sessions, have gone into the design, development and quality control of this new generative AI software, Panel of AI Experts for Lawyers. Losey AI, LLC, has already published seven studies done, which we urge you to reads in their entirety, both to learn more about the proper use of this software, but also learn about the topics the different panels discussed.
Ralph Losey started the seven studies report with statement of purpose and summary of the origin of the idea having a GPT split itself up into multiple personalities so that it could talk to itself. Evidence that AI Expert Panels Could Soon Replace Human Panelists or is this just an Art Deco Hallucination? (e-Discovery Team, May 13, 2024). The idea originated on a hacker discussion in 2023 on Discord and Defcon on GPT 3.5.
I will attempt to prove to your satisfaction that my custom GPT, and others that will soon follow, take expert panel discussions to the next level. I have been playing with this project since the days of 3.5, trying to create ChatGPT personas that talk to each other. There were a dozen or so of us early GPT explorers on Discord and DefCon who were enthused with this idea. It was started by others in forums there in early 2023 and was first called the AI Hive Mind. Here are my previous articles for background. Prompting a GPT-4 “Hive Mind” to Dialogue with Itself on the Future of Law, AI and Adjudications (e-Discovery Team, 4/11/23); ChatGTP-4 Prompted To Talk With Itself About “The Singularity” (e-Discovery Team, 4/04/23); The Proposal of Chat-GPT for an “AI Guardian” to Protect Privacy in Legal Cases (e-Discovery Team, 4/015/23). I also refer to this as making the AI adopt multiple personas that can work with each other for various purposes.
At the time, early 2023, I was, as far as I know, the only lawyer working with this technique, but I have no doubt other lawyers are doing so now too. The idea here is to create AI sub-personalities – personas – with different knowledge specialties and other differing characteristics (which is where the secret sauce comes in). Then you prompt the different AI entities to chat amongst themselves. They work through the issue, step by step, and consider various points of view. The GPT hive mind of experts discusses subjects and works out problems that you assign to them.
Basically the current GPT4 level creative expert panel approach makes the ChatGPT create an expert panel where they talk to themselves. Then the multiple personas, the hive mind if you will, comes up with new ideas, insights and answers in multiple steps, all on its own. My GPT, Panel of AI Experts for Lawyers, for instance, is programmed to go through six steps, which I predetermined. (It started with eight steps, but the QC testing suggested it should be reduced to six.) The user sits back and watches the panel. The AI is thinking out loud in a transparent process. You listen to the panel conclusions, and at the end, you can ask questions. Just like the old human panels, except that the AIs have unlimited time and patience.
Next we will review all seven reports with links to the full articles. We defy you to find any other generative AI GPT software that has been more expensively tested and publicly reported. Losey AI believes in the principles of AI transparency and the core prompt engineering strategy of testing changes systematically.
One: Evidence that AI Expert Panels Could Replace Human Panelists
Evidence that AI Expert Panels Could Soon Replace Human Panelists or is this just an Art Deco Hallucination? (e-Discovery Team, May 13, 2024). Ralph Losey here explores the potential for AI expert panels to effectively replace human panelists in providing legal insights. The article delves into the historical context of AI development and presents initial evidence suggesting that AI can perform at or above the level of human experts in certain tasks. Losey provides a detailed account of various studies and experiments conducted to test the capabilities of AI to divide into sub-personas and engage in expert panel type discussions with itself. The results were promising, showing that Panel of AI Experts for Lawyers could simulate human expert panels with a high degree of accuracy and consistency.
Losey highlights several key advantages of AI expert panels, including their ability to process and analyze vast amounts of information quickly, their consistency in delivering insights. Obviously, there is also the benefit of AI expert cost-effectiveness as compared to traditional expert consultations. Ralph also discusses the potential impact of AI expert panels on the legal profession, suggesting that they could significantly reduce the time and cost associated with obtaining expert opinions, thereby improving access to justice.
Losey demonstrates the ability of the AI Panel software by sharing a session where the topic provided for discussion was AI hallucinations. You can see for yourself how the software functions.
Their analysis and discussion are so impressive that it makes many wonder if the days of human panel discussion may now be over. Will we continue the quaint legal education tradition of putting human experts on stage lecturing down to the attendees, who are usually often only half listening, bored and distracted?
Ralph thinks so, although the AI is still very poor at humor and satire, so there may be a place for us yet. Plus, in these early days of generative AI, it is still trust but verify. Human experts are still needed for QC. Although only minor errors were seen in the extensive testing done by Losey AI in its extensive pre-release testing, and no hallucinations at all, mistakes happen to AI in the same way they do to human experts.
Losey’s report emphasizes the need for rigorous testing and validation to ensure the reliability and accuracy of AI-generated insights. He points out, that while AI has shown great promise, it is essential to address potential limitations and biases to fully realize its potential in the legal field. The blog concludes with a promise of continued research and experimentation to refine AI expert panels and ensure their readiness for practical applications in law.
Two: Experiment with ChatGPT4 Panel of Experts
Experiment with a ChatGPT4 Panel of Experts and Insights into AI Hallucination (May 21, 2024). Ralph reported on his experiment of repeating the same AI hallucination topic nine more times with different panelists each session. We were testing for consistency and reliability.
The results of the experiment were encouraging. All ten AI panels performed in a consistent, high-quality manner with very few errors and no hallucinations. The AI’s ability to provide coherent and accurate analyses of complex legal issues was verified by the repeated runs of the program with different experts. Losey highlights the importance of testing a GPT repeatedly, as was done in this experiment, to verify an AI program before it is released to other attorneys. He also discusses the implications of these findings for future AI development, emphasizing the need for ongoing refinement and testing to further enhance the reliability of AI-generated insights.
Three: OMNI Version – ChatGPT4o Retest
OMNI Version – ChatGPT4o – Retest of My Panel of AI Experts (May 29, 2024). Ralph here highlights the improvements made in the OMNI version of ChatGPT4, referred to as ChatGPT4o. Ralph Losey retested the AI panel ten more times on the same AI hallucinations topic. As before, in each session different expert personas were used. The goal was to retest systematically to evaluate the impact of the change to Omni. We needed to evaluate the performance and continued reliability of Panel of AI Experts for Lawyers under the new Omni version of ChatGPT. The results of the ten panels tests show enhanced accuracy, with even fewer errors than seen previously.
Losey discusses the key advancements in this version, including improved natural language processing, better contextual understanding, and enhanced learning algorithms. These improvements increased the Panel of AI Experts for Lawyers‘ ability to provide accurate and relevant legal insights. Losey provides comparisons to previous versions, demonstrating the AI’s growing reliability and effectiveness. The report emphasizes the importance of these advancements for the practical application of AI in the legal profession, highlighting the potential for AI to transform how legal professionals learn, come up with new ideas make decisions.
Four: AI Mentors of New Attorneys
Omni Version Test of the Panel of AI Experts on a New Topic: ‘AI Mentors of New Attorneys’ (June 2, 2024). In this experiment, the AI panel was tasked with a completely different topic having nothing to do with the hallucination subject considered in the first three tests. Losey AI considered this necessary precaution to avoid over-training issues and carry-over from one panel discussion to the next. This could hide errors in the GPT and otherwise skew the quality control design on these tests. The new subject chosen was interesting, how AI could assist in mentoring new attorneys. The ten panels tested uncovered new insights into this problem area of legal training.
The results of Panel of AI Experts for Lawyers were again impressive and satisfied Ralph concerns about over-fitting. The AI panels provided detailed and practical advice on the use of mentorship programs by law firms. Again, very few errors of any kind were noted and there were no hallucinations.
Five: Chart Showing Survey of Public Expectations
Another Test of the Panel of AI Experts on a Survey of Public Expectations of Generative AI (June 7, 2024). This post explored the ability of the GPT to generate significant discussion based upon a chart alone. The chart submitted for panel analysis pertained to high public expectations of the impact of generative AI on society in the next five years. The transcript of the entire session is shared in the article. We concluded that the software once again did a good job with analysis and explanations. No errors of any significance were made and no fabrications.
Six: Graph Showing Classifications of AI
Types of Artificial Intelligence: Still Another Test of the ‘Panel of AI Experts’ on a Chart Classifying AI. Losey AI also tested the Panel of AI Experts for Lawyers ability to interpret and discuss an image showing classifications of basic AI types. The chart is by Dr. Lily Zhuhadarn and is shown below.
The transcript of the session included in the article shows a different way to use the program. It also shows an error in panel selection made by the GPT and how a user can easily correct the problem. The Panel discussion shows how understanding of the chart how can help lawyers render advice to their business clients.
Again, a hallucination-free discussion was seen. In fact, the only time Losey AI has ever seen this GPT fabricate anything is when it was asked to perform a task for which it was not designed, namely write a blog. The blog article was pretty good, but it did refer to another a very plausible, but non-existent article, a fabrication that was easy to detect. Lesson learned, do not yell at the panel on stage and demand they write a blog for you. It’s off the cuff dictation of an article for you might not be entirely correct.
Seven: Bruce Schneier’s Speech
Final Test of ‘Panel of AI Experts for Lawyers’ – Bruce Schneier’s Commencement Speech On How AI May Change Democracy. The final quality control test reported for its educational value (there were many more tests of the GPT not reported) is described in this article. Ralph asked the AI panel to analyze and debate a keynote speech by famed cybersecurity expert, Bruce Schneier, on AI’s impact on democracy. The AI panel passed this test of its abilities with flying colors. Ralph’s test included whether the GPT’s could properly to respond to requiring the GPT to assemble a new panel of experts to focus on the user’s new information needs. It could and Panel of AI Experts for Lawyers easily shifted gears. Again, as in each of these test reports, the full transcript is provided.
Summary of Software Beta Testing
As seen on the cited reports, the development of the “Panel of AI Experts for Lawyers” custom GPT involved extensive testing and iterative improvements to ensure its reliability and effectiveness. The seven cited reports with session transcripts are just key tests conducted near the end. The testing phases included a variety of scenarios and legal contexts to evaluate the AI’s performance and address potential limitations. The quality shown at the end of the program development was consistent and reliable. Each panel, much like a panel of human experts, did make some minor mistakes of no real significance. No fabrications were seen, although that possibility can never be ruled out completely.
By the end of the development phase, which lasted over six months, the “Panel of AI Experts for Lawyers” had been rigorously tested and validated, demonstrating its readiness for real-world applications. This comprehensive approach to development and testing, ensures that users can trust the AI to deliver high-quality legal insights and analysis.
Features of the Custom GPT Panel of AI Experts for Lawyers
Expert Panel Simulation
The Panel of AI Experts for Lawyers simulates a panel of legal experts, offering diverse perspectives and comprehensive analysis on legal issues. This feature is useful for complex cases requiring multidisciplinary insights, providing users with a broad range of opinions and expertise that would typically be time-consuming and costly to obtain. It can be used for many things, including quick learning and brainstorming solutions to clients’ legal problems.
Efficiency and Accuracy
The AI can process and analyze complex information quickly, delivering accurate and relevant insights on a variety of topics. Its abilities include the capability to browse the web for current information. This allows the panel to refer to breaking news types of events. We performed several tests on this. The panels work best when specific topics are specified in a short, written statement by users, but tests have shown that it can also discuss graphs and charts and is not limited to user text. You can also direct the expert panel refer to specific text that you submit to it as attached files, along with your topic specifications. These diverse, multimodal capabilities increase the usefulness of the panel’s analysis and discussions.
Customization and Adaptability
The tool can be customized to fit different legal specializations, making it versatile and adaptable to various practice areas. You select any experts that you want on the panel. Only the Devil’s Advocate expert is required for every panel. That is because we have found this improves the quality of the panel services. The user-friendly interface of Panel of AI Experts for Lawyers ensures easy integration into existing workflows, allowing legal professionals to tailor the AI’s capabilities to their specific needs and expert preferences.
Cost-Effective Legal Expertise
The GPT is intended for use by legal professionals only. It can help professionals improve their efficiency and so serve a greater number of clients in a cost-effective manner. In this admittedly indirect way, the AI panel GPT increases access to justice by the public.
Continuous Learning and Improvement
The AI continuously learns from new data and user feedback, improving its accuracy and reliability over time. This ensures that users receive up-to-date and high-quality legal insights, making the tool increasingly valuable as it evolves.
Practical Applications
The Panel of AI Experts for Lawyers offers numerous practical applications across various legal scenarios. Its expertise is not limited to any particular field of law, such as IP, e-discovery, or AI only. All areas of law can be included. Another application outside of direct legal services is for legal education. The panels can help instruct new attorneys to learn new areas of law, and look at things from a diverse, expert perspective.
Overall, the practical applications of the “Panel of AI Experts for Lawyers” are vast and varied, offering significant benefits in terms of efficiency, accuracy, and accessibility. By integrating this tool into their workflows, legal professionals can enhance their practice and provide better service to their clients.
Conclusion
We invite all legal professionals to experience the transformative potential of the Panel of AI Experts for Lawyers firsthand. Sign-in to your OpenAI account, or create one, its free, then CLICK HERE to directly connect to this cutting-edge tool. Try it out and start integrating AI into your legal practice.
Use of this technology can enhance your brainstorming, expert research, decision-making, and client services, ultimately improving your effectiveness and competitiveness as a legal professional. Still, you must always remember to do your own due diligence. Errors of some kind are to be expected, and although we have not seen hallucinations yet, they are always possible, especially when the panel is asked to do things beyond it programming. For instance, the panel cannot generate writing for you or provide your clients with direct legal advice, Losey AI follows the humans-in-the-loop strategy of AI adoption.
The price of this custom GPT is right. Free. No add on or extras. And of course, that means no warranties either. This is free software given out as is. If you do have problems, please let us know we might be able to help. Have you tried rebooting? (joke) But we may not be able to respond and make no promises to do so. We are busy professionals like you. We do not try to sell other software to you, although we do have a few other private GPTs that are for our clients only. They do completely different things. This one, like a few other Losey AI offers, is our give-away, to client and competitors alike. A pay it forward, if you will. As your expertise continues, we trust you will make the time to do the same. In this way the legal profession can remain current in a fast-changing world. We think that is important.
We hope you will use this GPT, and other free GPTs we have released into the wild.
Join our growing community of legal professionals who are leveraging AI to enhance their practice. Follow our social media on Linkedin (ralphlosey) and on EDRM and JD Supra for the latest updates, tips, and success stories on generative AI. Also look for information on Webinars and CLEs. Look for live webinars allowing users live Q&A sessions to learn more about this tool and how to make the most of it.
Additional Resources
For those interested in deeper insights, here are links to the prior articles mentioned on AI multiple-personality dialogues:
- ChatGTP-4 Prompted To Talk With Itself About “The Singularity” (4/04/23)
- Prompting a GPT-4 “Hive Mind” to Dialogue with Itself on the Future of Law, AI and Adjudications(4/11/23)
- The Proposal of Chat-GPT for an “AI Guardian” to Protect Privacy in Legal Cases (4/15/23)
- Evidence that AI Expert Panels Could Soon Replace Human Panelists or is this just an Art Deco Hallucination? – Part One (5/13/24)
- Experiment with a ChatGPT4 Panel of Experts and Insights into AI Hallucination – Part Two (5/21/24)
- OMNI Version – ChatGPT4o – Retest of My Panel of AI Experts – Part Three (5/29/24)
- Omni Version Test of the Panel of AI Experts on a New Topic: “AI Mentors of New Attorneys” – Part Four (6/3/24)
- Another Test of the Panel of AI Experts on a Survey of Public Expectations of Generative AI (Part Five) (6/7/24)
- Types of Artificial Intelligence: Still Another Test of the ‘Panel of AI Experts’ on a Chart Classifying AI (Part Six) (6/10/24)
- Final Test of ‘Panel of AI Experts for Lawyers’ – Bruce Schneier’s Commencement Speech On How AI May Change Democracy (Part Seven) (6/13/24).
Thank you for taking the time to learn about the “Panel of AI Experts for Lawyers.” We look forward to your feedback and are excited to see how this tool will benefit the legal community.
Published on edrm.net with permission. Assisted by GAI and LLM Technologies for images per EDRM GAI and LLM Policy.
Ralph Losey Copyright 2024 — All Rights Reserved. See applicable Disclaimer to the course and all other contents of this blog and related websites. Watch the full avatar disclaimer and privacy warning here.
