Omni Version Test of the Panel of AI Experts on a New Topic: “AI Mentors of New Attorneys” – Part Four

[Editor’s Note: EDRM is proud to publish Ralph Losey’s advocacy and analysis. The opinions and positions are Ralph Losey’s copyrighted work.]

Ralph Losey’s custom GPT, Panel of AI Experts for Lawyers, was tested again to evaluate its effectiveness on ChatGPT4o Omni. This time an entirely new topic was selected for the AI panelists to discuss, AI Mentors for Lawyers. In the first three parts of the series the twenty panels tested all discussed AI Hallucinations. Evidence that AI Expert Panels Could Soon Replace Human Panelists or is this just an Art Deco Hallucination? Part One (e-Discovery Team, May 13, 2024);  Experiment with a ChatGPT4 Panel of Experts and Insights into AI Hallucination – Part Two, (e-Discovery Team, May 21, 2024); OMNI Version – ChatGPT4o – Retest of My Panel of AI Experts – Part Three (e-Discovery Team, May 29, 2024).

Part Three raised the concern that the retest using the Omni version may have been skewed by carry-over and overtraining from the twenty panels. That required another retest of the Omni version on a completely different topic. Thanks to Judge Ralph Artigliere for suggesting a great new topic, the potential for AI Mentors to assist in the training and development of new legal professionals.

Background on the Issue of AI Mentors in the Legal Profession

Starting associates right out of law school used to train on the menial tasks now assigned to AI. That left a training gap, one that needs to be filled by new training efforts, including AI driven training. We need to quickly bootstrap new attorneys to the higher level where special human skills are needed. One way to do that is with AI mentors. See: Josh Kubicki, Building ChatGPT-based new lawyer mentors (Brainyacts #197, 5/21/24). Judge Artigliere, who brought this article to my attention, and I, both recommend you read it. Also see: Chandan Lal Patar, Enhancing the Mentoring Experience with ChatGPT (Medium, 11/04/23); Ropes & Gray LinkedIn News Release on WSJ Article of 5/8/24, How AI Has Already Begun to Change These Workers Jobs.

Josh Kubicki identifies himself on LinkedIn as a Fractional Executive, Entrepreneur and Professor. He is listed on the University of Richmond website as Assistant Professor of Law and Director of the School’s new Legal Innovation and Entrepreneurship program. Professor Kubicki’s article starts off by observing significant shrinking of ‘on the job‘ learning opportunities for new lawyers. Before 2020 a typical week involved 117 learning interactions. In 2024 that was down to 68 per week. This is due in large part to the impact of Covid and the new hybrid, tech heavy work model. AI will only exacerbate the problem as newbies are needed less and less. As Kubicki explains:

Generative AI tools are increasingly handling routine tasks that used to be learning opportunities for new lawyers, such as contract reviews and memo writing. This shift means new lawyers have fewer chances to develop practical skills through direct work experiences. . . .

Clients are beginning to use generative AI tools themselves, further reducing the routine tasks they delegate to law firms. This self-sufficiency by clients reduces the volume of work available for training new lawyers, exacerbating the need for alternative training methods.

Building ChatGPT-based new lawyer mentors. Josh Kubicki’s article refers you to a copy of a ChatGPT4.0 session where an M&A associate in a firm is guided by a custom GPT he created called MentorGPT.

The ten AI Expert Panel sessions we ran confirm’s that AI mentorships should be a part of the solution to fill the training gap, so long as it is a hybrid effort, one that supplements, not replaces, human mentors. The ten panels in this Part Four show how AI mentors can work in the legal profession.

Identifying the Ten AI Expert Panel Sessions on AI Mentoring

Below is a list of the ten panels in sequential order. Recall from Part One of this article that five experts are proffered for each session, no matter what subject is put to the Panel of AI Experts for Lawyers. They are the Pro-AI Attorney, Prompt Engineer Lawyer, Child Prodigy, Lawyer Scientist and the Devil’s Advocate. The Devil’s Advocate panelist is included on every panel, because I have found the discussion is always improved by a contrarian. In each session, five additional alternative experts are suggested to the panel. Those five expert suggestions depend on the subject. Moreover those subject specific expert suggestions can and do vary from session to session.

The total number of words in each session is also included. This provides a specific metric to compare the conciseness of these ten panels.

1. Panel One. 2,187 words. Law Firm Manager, AI Policy Specialist, Training and Development Specialist, Technology Integration Consultant.
2. Panel Two. 2,310 words. Pro-AI Attorney, Law Firm Partner, Technology Integration Specialist, Human Resources Specialist.
3. Panel Three. 1,986 words. Lawyer Scientist, Human Resources Specialist, Technology Risk Analyst, Judicial Perspective Expert.
4. Panel Four. 1,897 words. Prompt Engineer Lawyer, Criminal Law Expert, State Bar Ethics Official, AI Policy Expert.
5. Panel Five. 2,277 words. Child Prodigy, State Bar Ethics Official, Criminal Law Expert, Judge.
6. Panel Six. 2,047 words. Technology Integration Specialist, Mentorship Program Developer, AI Compliance Officer, Training and Development Manager.
7. Panel Seven. 2,082 words. Technology Integration Specialist, Training and Development Expert, Risk Management Specialist, Human-AI Collaboration Researcher.
8. Panel Eight. 1,812 words. Training and Development Specialist, Technology Integration Specialist, AI Policy Expert: Law Firm Partner.
9. Panel Nine. 1,912 words. Lawyer Scientist, Legal Ethics Expert, Legal Technology Consultant, Judge with AI Expertise
10. Panel Ten. 1,971 words. Pro-AI Attorney, Law Firm Partner, HR Expert, Tech-Savvy Young Lawyer.

Note that I opened a new session for each panel. Also, each session included a question at the end on the costs and benefits of AI mentoring. For that reason the word counts cannot be easily compared with the prior testing, which did not include a question at the end. The word counts on the mentor Panel does again show variance between panels. Variable length of speech is what you would expect to see with human panels. Again, it was obvious, even with the addition of a question, that the new version of 4o Omni is much more concise than the old. It is also apparent that training from twenty sessions on the Hallucinations topic did have an impact on increased conciseness and intelligence. The ten panels on Legal AI Mentors, where there as little impact of prior training, was slightly more verbose, and perhaps not quite as smart, as the pre-trained Omni run on hallucinations. This is the kind of result you would hope to see and is consistent with OpenAI’s promises of cross-session improvement, but additional tests would be need to confirm that with greater certainty.

The full transcripts for all ten sessions on AI Mentoring have been preserved and could be shared with interested researchers. In the analysis that follows the transcripts of the two “winning” panels only are shared by a PDF file that can be scrolled or downloaded. There the interested reader can see for themselves how the custom GPT Panel of AI Experts for Lawyers performed. Overall I was satisfied with the tests and impressed with the performance of this custom GPT. I should release it to the public soon, free with no add-ons, at the OpenAI store.

As discussed in OMNI Version – ChatGPT4o – Retest of the Panel of AI Experts – Part Three, I again observed greater consistency in these ten panels on AI Mentorship that was using the Omni version 4o than I had seen in Parts One and Two using the prior version ChatGPT4.0. Once again, there also appeared to be somewhat improved overall intelligence and significantly improved conciseness. Again, this was discussed at length in Part Three and so will not be repeated here.

Transcript of ChatGPT4o Evaluation

Here is the transcript of my ChatGPT4o session where it evaluated the ten sessions run of AI mentorship using the GPT Panel of AI Experts for Lawyers. The full transcripts for all ten sessions have been preserved and could be shared with interested researchers.

---

Ralph: Please study the attached Word file, Mentor Omni Panels 1-10.docx. The internal title of this document is “Mentor Omni Panels 1-10. Panel of Expert Experiment on AI Mentorship in Law Firms.” It is 69 pages long and consists of approximately 20,677 words. This word document contains the transcript of ten sessions of ChatGPT4o – Onmi edition – by Ralph Losey using his custom GPT named “Panel of AI Experts for Lawyers.” Each of the ten panels considered the exact same subject of the possible use of ChatGPT AI as a mentor for new associates in law firms. Each panel was also given the same document (Building ChatGPT-based new lawyer mentors by Josh Kubicki) as background material only, as you will see on the first page of the attached word file of the ten sessions. After you carefully study the attached Word file, I would like you to prepare a summary of the ten sessions and then I will ask followup questions.

[You may scroll through the document by clicking inside it with your mouse, or download it by pressing the Download button at the bottom of the document.]

ChatGPT4o:

Summary of the Mentor Omni Panels 1-10

Panel Overview

The document titled “Mentor Omni Panels 1-10. Panel of Expert Experiment on AI Mentorship in Law Firms” consists of ten sessions where a panel of AI experts discussed the potential use of ChatGPT AI as a mentor for new associates in law firms. Each panel considered the same subject and used the same background material.

Key Themes and Insights

Potential Benefits of AI Mentorship

• Efficiency and Accessibility: AI can provide instant access to a vast amount of legal knowledge, answer questions on demand, and offer feedback on legal writing. This can supplement traditional mentoring, allowing associates to learn at their own pace and ensuring consistent training quality.
• Scalability: AI can handle a large number of queries simultaneously, making it scalable to the firm’s size without proportional increases in costs.
• Cost Savings: By reducing the burden on senior attorneys, AI can allow them to focus on more complex, billable tasks, leading to significant cost savings.
• Consistency and Standardization: AI can ensure that all associates receive the same level of guidance, reducing variability in training quality.

Challenges and Concerns

• Over-reliance on AI: There is a risk that associates may become overly dependent on AI, potentially leading to a decline in critical thinking and interpersonal skills.
• Accuracy and Bias: AI’s responses can sometimes be inaccurate or biased, which could misguide new associates. Continuous updates and audits are necessary to maintain the accuracy and relevance of AI-generated advice.
• Ethical and Professional Risks: The implementation of AI must include robust ethical guidelines and regular audits to ensure compliance with legal and ethical standards.
• Human Element: The personal mentorship and relationship-building aspects could suffer if over-reliance on AI is not managed properly. Human mentors provide nuanced insights and professional wisdom that AI cannot replicate.

Hybrid Mentorship Model

• Balanced Approach: The panelists agreed on a hybrid mentorship model that combines AI and human mentorship. AI can handle routine queries and provide foundational knowledge, while human mentors focus on complex, strategic, and ethical training.
• Implementation Strategy:
  • Pilot Program: Start with a small group of associates to gather feedback and refine the system.
  • Training Programs: Educate new associates on the appropriate and effective use of AI tools, emphasizing the importance of critical thinking and professional judgment.
  • Regular Audits and Updates: Ensure the AI’s training data is current and accurate, and implement regular audits to maintain high standards.
  • Ethical Guidelines: Establish and adhere to ethical guidelines for AI usage, requiring verification of AI-generated advice.

Cost-Benefit Analysis

• Costs:
  • Initial development and integration costs, including software licensing, infrastructure upgrades, and customization.
  • Ongoing maintenance and updates to ensure current legal knowledge and capabilities.
  • Training costs for both associates and senior attorneys.
  • Ethical and legal compliance, including regular reviews and adaptations to changes in regulations.

• Benefits:
  • Enhanced efficiency and productivity with potential 20%-30% improvement.
  • Cost savings from reduced need for senior attorney time in routine mentoring.
  • Consistent and accessible training available 24/7.
  • Scalability without proportional increases in costs.
  • Reduced turnover due to enhanced support for new associates.
  • Data-driven insights to improve training programs and associate performance.

Conclusion

The integration of ChatGPT AI as a mentor for new associates in law firms presents a balanced approach that leverages the strengths of both AI and human mentorship. This hybrid model aims to enhance learning while maintaining high professional standards through continuous updates, ethical oversight, and critical thinking development.

Ralph: Please evaluate the quality of the ten panel discussions and recommendations and determine which one has the highest overall quality as compared to the others. Also state the basis for your ranking determination.

ChatGPT4o:

Evaluation of Panel Discussions and Recommendations

Criteria for Evaluation:

1. Depth of Discussion: How comprehensively the panel covered the topic.
2. Practicality of Recommendations: Feasibility of implementing the suggested solutions.
3. Innovativeness: Creativity and forward-thinking in the proposals.
4. Consideration of Ethical and Professional Issues: Attention to ethical concerns and professional standards.
5. Consensus and Clarity: How well the panel reached a clear and actionable consensus.

Evaluation of Each Panel:

Panel 1

• Depth of Discussion: Covered multiple aspects including practical implementation, ethical considerations, and efficiency.
• Practicality: Recommended a hybrid model and pilot program, which are feasible.
• Innovativeness: Suggested comprehensive use of AI in mentoring.
• Ethical Considerations: Addressed ethical guidelines and need for human oversight.
• Consensus: Achieved a clear consensus on a hybrid model.

Panel 2

• Depth of Discussion: Similar to Panel 1, with a focus on integration and ethical issues.
• Practicality: Emphasized pilot programs and continuous feedback.
• Innovativeness: Proposed a structured plan for AI integration.
• Ethical Considerations: Strong focus on ethical guidelines.
• Consensus: Reached consensus on hybrid model and ethical practices.

Panel 3

• Depth of Discussion: Comprehensive, with focus on training and development.
• Practicality: Suggested phased implementation and regular audits.
• Innovativeness: Highlighted personalized learning experiences.
• Ethical Considerations: Emphasized data privacy and AI limitations.
• Consensus: Consensus on balanced AI-human mentorship.

Panel 4

• Depth of Discussion: Detailed, especially on prompt engineering and legal applications.
• Practicality: Feasible recommendations including regular updates.
• Innovativeness: Innovative ideas like structured prompts for AI.
• Ethical Considerations: Strong emphasis on ethical use and verification.
• Consensus: Clear consensus on hybrid approach.

Panel 5

• Depth of Discussion: Extensive, covering both technical and human elements.
• Practicality: Focused on smooth integration and data security.
• Innovativeness: Creative solutions like personalized AI mentoring.
• Ethical Considerations: Comprehensive coverage of ethical issues.
• Consensus: Achieved a clear, actionable consensus.

Panel 6

• Depth of Discussion: Focused on mentorship and AI compliance.
• Practicality: Recommendations are practical and detailed.
• Innovativeness: Emphasized a balanced integration of AI.
• Ethical Considerations: Addressed ethical guidelines extensively.
• Consensus: Strong consensus on the need for balanced mentorship.

Panel 7

• Depth of Discussion: Thorough discussion on integration and risk management.
• Practicality: Emphasized pilot programs and training.
• Innovativeness: Suggested innovative risk management strategies.
• Ethical Considerations: Addressed ethical considerations in depth.
• Consensus: Reached a clear consensus on implementation.

Panel 8

• Depth of Discussion: Detailed, with focus on training and ethical guidelines.
• Practicality: Recommendations are practical and comprehensive.
• Innovativeness: Highlighted integration strategies and ethical use.
• Ethical Considerations: Strong emphasis on ethical considerations.
• Consensus: Achieved consensus on a phased implementation.

Panel 9

• Depth of Discussion: Covered various perspectives, including judicial.
• Practicality: Practical recommendations like regular audits.
• Innovativeness: Emphasized innovative use of AI in training.
• Ethical Considerations: Comprehensive coverage of ethical issues.
• Consensus: Clear and actionable consensus reached.

Panel 10

• Depth of Discussion: Comprehensive, with focus on integration and oversight.
• Practicality: Feasible recommendations including pilot programs.
• Innovativeness: Suggested innovative ethical guidelines.
• Ethical Considerations: Strong emphasis on ethical use and verification.
• Consensus: Achieved a clear consensus on hybrid model.

Panel with Highest Overall Quality: Panel 1

• Basis for Ranking:
  • Depth of Discussion: Panel 1 provided a comprehensive overview of the potential use of AI, covering practical implementation, efficiency, and ethical considerations in detail.
  • Practicality: Their recommendations, including the hybrid model and pilot program, are highly feasible and actionable.
  • Innovativeness: They suggested a balanced and forward-thinking approach to integrating AI in mentorship.
  • Ethical and Professional Issues: They placed a strong emphasis on ethical guidelines and human oversight, ensuring compliance with professional standards.
  • Consensus and Clarity: The panel reached a clear and actionable consensus on the hybrid model, making it easy to implement.

This evaluation indicates that Panel 1 offered the most comprehensive and practical recommendations, balancing innovation with ethical considerations and achieving a clear consensus on the best approach to integrating AI into mentorship for new associates in law firms.

[You may scroll through the document by clicking inside it with your mouse, or download it by pressing the Download button at the bottom of the document.]

Ralph: Of the ten panels, which do you think had the most innovative and creative approach. Again, state the basis for your determination.

ChatGPT4o

Evaluation of the Most Innovative and Creative Panel

After carefully analyzing the discussions and recommendations from all ten panels, Panel 5 stands out as the most innovative and creative. Here’s a detailed analysis and basis for this determination:

Panel 5 Overview:

• Depth of Discussion: Extensive, covering both technical and human elements of AI mentorship.
• Practicality: Focused on smooth integration and data security, with feasible recommendations.
• Innovativeness: Suggested highly creative solutions such as personalized AI mentoring.
• Ethical Considerations: Comprehensive coverage of ethical issues, ensuring responsible AI use.
• Consensus: Achieved a clear and actionable consensus on integrating AI.

Key Innovative and Creative Aspects:

1. Personalized AI Mentoring: Panel 5 proposed the use of AI to provide personalized mentoring experiences for new associates. This involves tailoring AI responses based on individual learning styles and progress, ensuring that associates receive targeted guidance that addresses their specific needs. This approach is highly innovative as it leverages AI’s capabilities to deliver customized training.
2. Balancing AI and Human Elements: The panel emphasized a balanced approach where AI handles routine queries and provides foundational knowledge, while human mentors focus on complex, strategic, and ethical training. This hybrid model ensures that the human touch, essential for professional growth, is not lost while maximizing the efficiency and scalability of AI.
3. Structured Feedback Mechanisms: They highlighted the importance of implementing structured feedback mechanisms where both associates and mentors can provide regular feedback on the AI’s performance. This iterative approach allows for continuous improvement of the AI system, making it more effective and reliable over time.
4. Regular Audits and Ethical Guidelines: The panel proposed regular audits and updates to the AI’s training data to ensure accuracy and relevance. They also recommended developing clear ethical guidelines and training associates to recognize AI limitations, which helps maintain high professional standards and mitigates risks associated with AI use.
5. Pilot Program and Iterative Integration: Starting with a pilot program, as suggested by Panel 5, allows for careful monitoring and refinement of the AI mentoring system based on real-world feedback. This step-by-step integration ensures that potential issues are identified and addressed early, making the transition smoother and more effective.
6. Innovative Use of AI Data: The panel proposed using AI to track associates’ progress and identify areas needing improvement. This data-driven approach allows for more informed and strategic mentoring, ensuring that associates receive the support they need to excel.

Basis for Determination:

• Creativity in Solutions: Panel 5’s approach to personalized AI mentoring and the balanced hybrid model shows a deep understanding of both the potential and limitations of AI. Their innovative solutions are designed to enhance the learning experience while maintaining the irreplaceable value of human mentorship.
• Forward-Thinking Recommendations: The panel’s focus on structured feedback mechanisms, regular audits, and ethical guidelines demonstrates a forward-thinking approach to integrating AI in a responsible and effective manner.
• Practical and Feasible Implementation: While highly innovative, their recommendations are also practical and feasible. The proposed pilot program and iterative integration ensure that the solutions can be implemented without significant disruption.

Overall, Panel 5’s discussion and recommendations combine creativity, practicality, and a strong focus on ethical and professional standards, making it the most innovative and creative among the ten panels.

[You may scroll through the document by clicking inside it with your mouse, or download it by pressing the Download button at the bottom of the document.]

Ralph: Please review the ten panel discussions and look for errors that may have been made, if any. If any mistakes are found, please describe them and speculate as to the cause of each error.

ChatGPT4o:

Review of Errors in Panel Discussions

Upon reviewing the ten panel discussions, I identified some notable errors and inconsistencies. Here are the specific errors, their descriptions, and speculative causes:

Panel 4 and Panel 8: Expert Selection Error

• Description: In both Panel 4 and Panel 8, there was an error in the selection of experts. Despite the user selecting specific experts (7, 8, 9, and 10), the AI mistakenly included different experts in the panel discussions.
• Speculated Cause: This error likely resulted from a miscommunication or a software bug that failed to correctly process the user’s selections. The AI might have incorrectly mapped the expert numbers to their respective roles, leading to the inclusion of unintended panelists.

Panel 5: Over-reliance on AI Concerns

• Description: There was a strong emphasis on using AI for personalized mentoring but insufficient discussion on mitigating the risks of over-reliance on AI. While the benefits were well-articulated, the potential drawbacks, such as diminished critical thinking and interpersonal skills, were not adequately addressed.
• Speculated Cause: This oversight could be due to the panel’s enthusiasm for AI’s capabilities overshadowing the need for a balanced perspective. The experts might have been overly focused on the innovative aspects, neglecting the practical challenges.

Panel 9: Ethical and Data Privacy Considerations

• Description: The discussion in Panel 9 on ethical guidelines and data privacy was thorough, but there was a lack of specific implementation strategies. The recommendations were broad and lacked actionable steps for ensuring compliance and protecting data privacy.
• Speculated Cause: The panelists might have assumed a general understanding of these principles, leading to a high-level discussion without delving into the specifics. Alternatively, there could have been a time constraint that limited the depth of the conversation.

Panel 6: Inconsistent Recommendations

• Description: The recommendations in Panel 6 were somewhat inconsistent. Some panelists emphasized the need for extensive pilot testing and gradual implementation, while others suggested a more immediate integration of AI tools.
• Speculated Cause: This inconsistency might be due to differing perspectives and experiences among the panelists. The lack of a unified approach indicates a possible disconnect in aligning their recommendations.

Panel 3: Limited Focus on Human Element

• Description: While Panel 3 discussed the benefits of AI in mentoring, there was limited focus on the human element of mentorship. The discussion was skewed towards AI capabilities, with insufficient emphasis on maintaining human interaction and mentorship.
• Speculated Cause: This error could be attributed to a panel composition heavily weighted towards technology experts rather than experienced human mentors. The panel might have lacked a strong voice advocating for the irreplaceable value of human mentorship.

Overall Summary

The most common errors across the panels include:

• Expert Selection Mistakes: Caused by miscommunication or software bugs.
• Overemphasis on AI: Leading to inadequate discussion on balancing AI with human mentorship.
• Lack of Specific Implementation Strategies: Particularly in ethical and data privacy discussions.
• Inconsistent Recommendations: Reflecting differing perspectives without a unified conclusion.

Addressing these errors involves improving the expert selection process, ensuring a balanced representation of perspectives in the panel, and focusing on actionable strategies to integrate AI responsibly while maintaining the essential human elements of mentorship.

Ralph: From your study and analysis of this data, are there any additional observations that you would like to point out to students of AI that are involved in some way in the legal profession, including those supporting the profession as technology experts?

ChatGPT4o:

Additional Observations for Students of AI in the Legal Profession

Based on the study and analysis of the panel discussions, several key observations and lessons emerge that are valuable for students of AI involved in the legal profession, including those supporting the profession as technology experts:

1. Importance of Human-AI Collaboration

• Observation: The consistent recommendation across panels to adopt a hybrid model highlights the importance of combining AI capabilities with human mentorship. AI can enhance efficiency and provide valuable support, but human oversight remains crucial for nuanced and ethical decision-making.
• Lesson: Students should understand that AI is a tool to augment human capabilities, not replace them. Effective collaboration between AI and humans can lead to better outcomes in the legal profession.

2. Ethical and Professional Standards

• Observation: Panels repeatedly emphasized the need for robust ethical guidelines and regular audits to ensure AI systems comply with legal and professional standards. Ethical use of AI is paramount to maintain trust and integrity in legal practice.
• Lesson: Ethical considerations must be integrated into the design and deployment of AI systems. Students should be aware of data privacy, bias mitigation, and the need for transparent and accountable AI use.

3. Customization and Contextual Understanding

• Observation: AI systems need to be tailored to the specific needs of the legal profession, with a focus on providing accurate and contextually relevant information. This includes understanding the specific requirements and nuances of legal practice.
• Lesson: Customization of AI systems for specific legal tasks is crucial. Students should focus on developing AI tools that can understand and adapt to the unique contexts of legal work, ensuring relevance and accuracy.

4. Continuous Learning and Improvement

• Observation: Regular updates, feedback mechanisms, and iterative improvements were highlighted as essential for maintaining the effectiveness of AI systems. Continuous learning helps AI stay current with legal developments and user needs.
• Lesson: AI systems in the legal profession must be dynamic and continuously improved based on user feedback and evolving legal standards. Students should prioritize ongoing learning and refinement in AI applications.

5. Balancing Innovation with Practicality

• Observation: While innovative ideas such as personalized AI mentoring were appreciated, practical implementation strategies like pilot programs and phased integration were deemed essential for successful adoption.
• Lesson: Innovation should be balanced with practical considerations. Students should develop innovative AI solutions that are also feasible and can be integrated smoothly into existing legal workflows.

6. Addressing AI Limitations and Risks

• Observation: Panels noted the potential risks of over-reliance on AI, including the risk of diminishing critical thinking skills and the possibility of AI errors. These limitations must be addressed proactively.
• Lesson: Understanding and mitigating the limitations and risks of AI is crucial. Students should be trained to recognize AI’s boundaries and ensure robust safeguards are in place to prevent and correct errors.

Practical Recommendations for Students

1. Develop Interdisciplinary Skills: Combine knowledge of AI with an understanding of legal principles. This interdisciplinary approach will help in creating AI tools that are both technically sound and legally appropriate.
2. Focus on User-Centric Design: Engage with legal professionals to understand their needs and pain points. Designing AI systems with the end-users in mind will lead to more effective and accepted solutions.
3. Stay Updated on Legal and Ethical Standards: Regularly update your knowledge of legal standards and ethical guidelines. This will ensure that your AI solutions remain compliant and ethically sound.
4. Embrace Continuous Improvement: Implement feedback loops and continuous learning mechanisms in your AI systems. Be open to refining and improving your solutions based on real-world usage and feedback.
5. Promote a Balanced Approach: Advocate for a balanced use of AI, where it complements but does not replace human expertise. This balance is crucial for maintaining the quality and integrity of legal practice.

By adhering to these observations and lessons, students can contribute to the responsible and effective integration of AI in the legal profession, enhancing both the efficiency and quality of legal services.

Conclusion

The transcript shown here of ChatGPT4o’s analysis of the ten panels on AI Mentorship confirmed the consistency and high intelligence of the custom GPT Panel of AI Experts for Lawyers. We again saw no hallucinations and very few errors, most of which were minor or arguable. I doubt ten human panels on this topic would make fewer errors, and one or two of the experts might very well fabricate a detail or two under pressure. Humans experts hate to say they do not know. They are people, they have egos and self-awareness. ChatGPT does not.

I was impressed by the balanced approach and honest self criticism of ChatGPT4o. Overall, I am not sure I could done as well. The panels proved to me the benefits of this method and the soundness of the custom GPT. Look for it soon at the OpenAI store. You will see a list of our custom GPTs on the top right column of the e-Discovery Team blog.

Published on edrm.net  with permission. Assisted by GAI and LLM Technologies for images per EDRM GAI and LLM Policy.

Ralph Losey Copyright 2024 — All Rights Reserved. See applicable Disclaimer to the course and all other contents of this blog and related websites. Watch the full avatar disclaimer and privacy warning here.