Generative AI

About this Project

Chat GPT and other Generative Artificial Intelligence Products and advances are taking the world by storm, and the legal world is no exception.  This team would build on the work of prior EDRM Teams, including the Team that drafted and published the Use of Artificial Intelligence in Discovery white paper in 2021, and the recently released Professional Responsibility Considerations in AI for eDiscovery.  An initial project of this Team would be to develop guidelines for the use of generative AI by legal professionals, and the Team could then determine whether to work on other related materials—or just let the AI take it from there.

Contact info@edrm.net for more information on how to get involved in our global project community.

Mission Statement for the EDRM Working Group on Generative AI1

A brief overview of the objectives, activities, and outcomes  proposed for the new EDRM GenAI Project

The Project on Generative AI established by the EDRM is a collaborative initiative aimed at exploring both the potential uses and challenges of applying GenAI techniques to various stages within the Electronic Discovery Reference Model (“EDRM”).  The Project seeks the participation of legal professionals, academics, researchers, eDiscovery experts, data scientists, and others who share a common interest in the emerging field of generative AI (“GenAI”) and its implications for the legal industry, but through a distinctly critical and empirical lens.  The Project leadership has identified three goals for the Project’s initial tasks:

  1. Developing a checklist for EDRM members (e.g., individuals, law firms, and other organizations) to use when considering the use of GenAI in their eDiscovery processes.  The checklist will cover the key elements and questions that EDRM members should consider  when developing, evaluating, or adopting GenAI solutions, such as the specific use case or purpose, validity, reliability, bias, technical requirements, data privacy and security, cost and efficiency, and best practices and standards.
  2. The development of a standard framework for a benchmarking test akin to the MMLU or TREC (Meaningful and Measurable Legal Understanding or Text Retrieval Conference, respectively) for assessing the performance of large language models (“LLMs”) for use as part of eDiscovery systems.  The test will aim to measure the ability of LLMs to generate relevant, accurate, and coherent responses in connection with various eDiscovery tasks, such as document summarization, query answering, issue spotting, analysis, and document generation.  The test will also aim to provide a standard benchmark or method of comparison for different GenAI approaches and systems in eDiscovery.
  3. A series of webinars addressing the topics indicated above as well as others.  We intend to invite notable eDiscovery experts that are testing and using GenAI systems to talk about their evaluation and implementation efforts, as well as  prominent data scientists and researchers from industry and academia.  The webinars will provide an opportunity for members and others from the EDRM community to learn about the state-of-the-art and the future directions of GenAI in eDiscovery, as well as to exchange ideas and feedback.

The Project is seeking members of the EDRM community interested in working on the projects outlined above in the hopes of contributing to the advancement and adoption of GenAI in the legal domain, as well as fostering an open dialogue and a collaboration among the diverse stakeholders involved in eDiscovery.  The Project welcomes new members who are interested in joining or supporting the initiative.

 1The original draft of this synopsis was prepared by Matthew Golab in Word online using copilot. It was subsequently manually edited by Maura R. Grossman.  The prompt used was as follows.  It took circa 60 seconds for Matthew to draft the prompt, which was longer than it took copilot to generate the text:   “I want to write a short synopsis of a working group on Generative AI for the EDRM.  I would like an introduction, then the 3 components in bullet points and some descriptive text for each. The tone should be professional as the audience are legal professors and academics. There are 3 components – 1) a series of webinars where we get notable eDiscovery experts from the main eDiscovery systems to talk about their plans for implementing Generative AI, as well as other notable eDiscovery experts in the industry and also some notable data scientists.  2) the working group compiles a checklist for EDRM members (law firms) for consideration of Generative AI – what things you need to look out for and a series of questions you should ask.  3) drafting an eDiscovery test such as MMLU for evaluating the performance of LLMs in eDiscovery systems.  Of course, it took Maura longer than both to edit Copilot’s response.

Generative AI Project Feedback

Please let us know if you would like to volunteer to contribute or if you have suggestions by sending the form below:

Contributors

Co-Project Trustees

  • Dr. Maura R. Grossman, Research Professor at the University of Waterloo (Ontario, Canada) and Principal at Maura Grossman Law (Buffalo, NY)
  • Matthew Golab, Director Legal Informatics and R+D at Gilbert + Tobin (Sydney, Australia)