EDRM Evergreen/Review/Developing the Review Plan
From Working EDRM
| Comments: Please submit comments to the EDRM Evergreen Review forum |
Categories
add introduction
Contents |
Meet and Confer
Early in the litigation the case team must make a determination about the scope of the review: what is to be reviewed, how it is to be reviewed, and what is the intended outcome. To a great extent, the discovery order will direct what data is to be collected and reviewed. A meet-and-confer conference between the parties should be conducted early on in the planning stages to make these decisions. The Federal Rules of Civil Procedure 26(f) states that these initial meeting are to include discussions and decisions on matters pertaining to electronically stored information (ESI). Topics to be determined include preservation of ESI, the form of production for the ESI, and approaches for asserting privilege and work-product protection claims. Each of these decisions impact the planning of the document review.
Scope & Review Objectives
Historically, lawyers were obligated to review every document presented to them by the client or opposing counsel in the litigation for relevance, responsiveness to discovery requests, and privilege. With the explosion of email and other electronic documents, reviewing each and every document is often unrealistic. While the scope of the review is usually dictated by the discovery request, a key strategy is to control the scope of the review through the use of technology or other aids. These techniques should be discussed and agreed upon during the initial planning meetings.
Techniques that can be useful to limit the scope include:
- Strategic collection of data – it may be possible to do a more selective collection of key custodian data rather than a broad collection. There may be agreed upon file types or directory locations or key personnel.
- Strategic filtering of collected data – this may take the form of applying date limitations and/or keyword filters against the data that has been collected.
- Keeping ESI in its native format rather than a rendering to TIF or other image format.
In addition to limiting the scope, you also need to determine if the scope of your review is going to be "all ESI" or a hybrid of ESI and paper. Knowing this ahead of time will help you to determine what additional techniques to use to limit the scope and plan the review.
Once the scope of what is to be reviewed is known, you will next need to make the decision and selection of a knowledgeable review manager to manage the review process is an essential first step in the planning phase. The review manager must assure that those selected to conduct the review have a clear understanding of the case objectives. Additionally, it can be helpful to determine who else will comprise the review team and create an organization chart to ensure that all parties to the review know their role: corporate counsel, outside counsel, and, if needed, external reviewers.
The review manager should create clear documentation of the key issues and a distinction between issues of fact and issues of law. This documentation is generally captured in the form of Review Guidelines. These guidelines should not only identify the scope of the review and what the team will be looking for, but also include exemplar documents of the kind of evidence the litigation team will need to effectively prosecute or defend the matter will make the review team aware of what to keep their eyes open for. An understanding of who the key players are and what roles each plays in the case will enhance the review teams' ability to hone in on the critical pieces of evidence to the litigation. Many times it helps to create an organization chart of the key players to further identify their relationships to one another and to the matter.
The review objectives may evolve as a case progresses through the different phases of litigation. As it does, it is the responsibility of the review manager to update the review guidelines so that they remain current with the ongoing review. The following represents a few types of review that may be undertaken:
- Determine the relevancy of the information or documents collected or produced
- Use database fields, marks or tags within the application to categorize documents and facilitate improved search and retrieval of the documents collected or produced
- Determine whether or not any privilege applies to the documents subject to production
- Determine to which request(s) for production the documents are responsive
- Identify documents that should be marked as “confidential” or have portions redacted
- Relate key documents to alleged facts or legal issues previously outlined in the case
- Relate key documents to key players who may testify about the documents
- Identify other subjective information
In a few circumstances, a single review may encompass all these objectives. In reality, they are generally done in phases and by a variety of different reviewers, each with a clearly identified role in the review.
A key step in planning for the electronic review is to make sure that those selected to manage and conduct the review have a clear understanding of what the case is about. All those involved in selecting the review platform and spearheading the review process should review copies or summaries of the petition/complaint and answer(s).
All members of the review team should be provided copies or summaries of any discovery requests and answers thereto. Any special discovery orders or stipulations should also be summarized and available to the review team to make sure that the review is conducted in compliance with those orders or agreements. As a matter of course, the review team should receive training on the matter itself, the workflow designed, and to the review application selected.
Defining Review Protocols
Review Guidelines are typically prepared to define not only the objectives of the review but the rules of engagement for the review team. Historically, a coding manual defined the bibliographic fields that were to be captured by the reviewers into the review database. Today, with many reviews being done in digitized databases that already capture much of this information, the review guidelines are really designed to address how to categorize documents for their use in the course of the matter.
Examples of information that may be captured in the review guidelines include:
- Rules for adding additional information to the database such as: type of document (memorandum, letter, email), entities named in the document, marginalia found on paper docs that have been scanned, etc.
- Publication of look-up table to aid in identification of key players and known entities
- Rules for identification and subjective categorization of the responsive and producible documents with an outline of the issues to be identified and possibly sample documents that are representative of the issues at hand
- Rules for the handling of documents requiring redaction
- Rules for identifying and categorizing privilege documents
- Rules for identifying and handling confidentiality issues
- Rules for adding annotations to documents in the form of attorney notes and workproduct
- Rules for categorizing emails and their attachments and email threads
- Rules for the identification and tagging of non-responsive documents, spam and junk email
- Rules for handling of unreadable, password protected or other faulty documents
Determining Review Methodologies
There are many different methodologies and workflow used for document review. Sound choices usually are impacted by multiple factors:
- Agreements made by counsel during the meet and confer phase or as ordered by the court
- Time and cost constraints for the project
- Collection and restoration methods
- Format of the documents (ESI or paper or combination)
- Functionality available within the review platform
- Available resources to support different formats
Physical Format
Determining the physical format for the review should be established during the initial planning stages. A review may take many forms and utilize several different formats:
- Manual paper review
- ESI or paper review using an in-house review system
- ESI or paper review using a hosted online system
- ESI or paper review using temp attorneys
- ESI or paper review using an external service provider (domestic, offshore, or hybrid)
Review Assignments
In a typical scenario, the documents to be reviewed are divided up such that each reviewer is reviewing a specific numbered range of records that do not overlap with any other reviewers.
In more complex scenarios, the review sets may be divided based on the source of the records or a subset of records that meet search specifications for a particular key player, date range or case issue. The review assignments are then based on the importance of the source or key player, for example, with higher level attorneys reviewing the most critical key players’ records, and more junior level or paralegals reviewing less important ones.
To aid in an expedited privilege review, searches may be conducted for names of attorneys or law firms that would indicate a high probability of the result set containing attorney/client privileged communications. Documents meeting the privileged search criteria could be assigned to an attorney whose task is to review for privilege and mark the documents as such.
Establishing Project Timetable & Workflow
The design of a project timetable or workflow is critical to the review process and should be established at the outset of the project. The workflow should be designed by the review manager with the assistance of any outsourced service provider retained and the organization’s internal litigation support personnel. Often, those most familiar with the technology will be able to provide invaluable guidance on how to leverage that technology to maximum advantage.
Agreements made during the meet-and-confer process will influence the review objectives, techniques used, platform chosen and the overall project workflow and milestones. Discovery deadlines will also influence the review strategy. For instance, a DOJ second request with a 15-day turnaround might be handled differently than a standard litigation matter with a more long-term discovery schedule. Accordingly, there are different strategies which are determined by the nature of the request and the resources of your organization.
The review team should work with the electronic discovery outsourced service provider to develop the most efficient workflow for the project. Often, the outsourced service provider will be able to counsel the review team on the most efficient approach based on the strengths or limitations of the review tool. Do not assume that the workflow that worked in your last case using a different tool will work the same way on the next case and a different review platform. Work with the entire team to develop the methodology and workflow that makes the most sense given the needs of each specific case.
Other factors to consider regarding the creation of a project timetable include:
- Overall volume of data to be reviewed and complexity of the issues being reviewed
- Discovery deadlines
- Internal resources devoted to the review project
- Volume of redactions and the workflow provided to perform them
- Resource availability of all stakeholders: corporation, outside counsel, and service provider
- Ability of the service provider to produce to requirements; a sample run should be performed well in advance of any deadlines
- Production of key custodians
The timetable should include:
- Deliverable deadlines for each phase of the project:
- Collection and restoration of the data
- Processing, searching, and hosting the data
- Review of the data (first pass, second pass, privilege reviews)
- Production of responsive documents (as well as privileged documents if there is to be separate production)
- Daily, weekly, monthly document totals per reviewer
- Other metrics to be used to measure the progress of the review and their milestones
- Production due dates set forth by the document request or the court
Milestones may be tied to specific custodians who are important to the matter, or critical time periods, issues or concepts. Using the power of the review tool to manage the workflow by segregating documents along these lines can greatly increase the speed and efficiency with which the project is completed.
Another key to ensuring that the workflow proceeds smoothly is to determine who will create the work assignments and how and when the assignments will be disseminated to the review team members. Depending on how a project is staffed, work assignments may be created by the outsourced service provider, the review manager, a paralegal or someone in the organization’s litigation support department.
Review Techniques
Determining the format for the review should also be established during the initial planning stages. Knowing what formats you are able to produce in will help the attorney team during their meet and confer to ensure that they are not promising what cannot be accomplished.
A review may take many forms and utilize several different formats: manual paper review; ESI or paper review using an in-house review system; ESI or paper review using a hosted online system; ESI or paper review using temp attorneys or ESI or paper review using an outsourced review provider (domestic, offshore, or hybrid). The nature and needs of the matter as well as the following factors will help determine the format of the review:
- Agreements made by counsel during the meet and confer phase, or ordered by the court
- Discovery deadlines
- Case type
- Time and cost constraints for the matter at hand
- How the documents were collected
- Training required by each member of the review team
( Available resources to support the process
The focus of our discussion is on electronic discovery, so the format decisions will typically be in-house vs. online and native vs. TIFF/PDF. See Review - Selecting a Vendor for a more in-depth analysis of the decision factors and pros/cons to these types of review platforms.
Additional techniques to be employed during the course of the review may include:
Culling irrelevant documents
A paralegal or junior level associate may go through the repository to identify records known not to be relevant to the matter at all, suppressing them from further review, or deleting them from the database altogether. They may engage the help of a litigation support specialist or electronic discovery service provider to automate the identification of known irrelevant documents so that the reviewers don’t have to look at every single document that have all the attributes as those previously identified as irrelevant.
Many electronic discovery tools provide an index to all the documents in the collection, allowing for a review of file names, types, dates, etc. for the purpose of excluding information from being further processed. When reviewing ESI, there is a diminished need for bibliographic coding (the process of capturing objective data from documents, i.e. date, author, recipients, title etc.) and entering it into database fields to facilitate the search and retrieval of documents. If not reviewing ESI, this level of review can be conducted by document processors, case clerks, paralegals, or outsourced to a service provider. When paper documents are converted to images, this level of coding is often necessary to facilitate search and retrieval. There is some debate whether or not OCR technologies provide enough full-text search capability to eliminate this step. Most feel, however, that some objective coding, along with OCR text is the safest way to ensure effective searches.
When planning the review that has a paper component, it is important to address how much bibliographic coding is going to be required ahead of time. If the review team will be relying on a metadata and full-text search strategy, tests should be run on a selection of representative records to ensure its ability to meet the review objectives without further indexing. Certain documents lend themselves to an auto-coding process where the search engine has the ability to automatically recognize specific fields of information and will populate the appropriate field with very little, if any human intervention. How much coding is done, as well as the dependence on a degree of accuracy will depend again on volume, time constraints, and the case budget.
Identifying producible documents
After culling, suppressing irrelevant documents, and providing the necessary indexing to the document repository, the review team is ready to address the task of reviewing documents that need to be produced, (or that have been produced by opposing parties). In a large repository with several reviewers, this task is often divided in a logical way in order to meet discovery deadlines.
The documents may be simply divided up by volume such that each reviewer is reviewing a specific numbered range of records that don’t overlap with any other reviewers. In more complex scenarios, the review sets may be divided based on the source of the records, or a subset of records that meet search specifications for a particular key player or issue. The review assignments are then based on the importance of the source or key player, for example, with higher level attorneys performing the most critical key players’ records, and more junior level or paralegals reviewing less important ones. Many times the tool being used to conduct the review will dictate the best way to logically break the documents into smaller sets for review purposes. A firm understanding of the features and limitations of your review application is key to ensuring this workflow is done correctly.
To aid in an expedited privilege review, searches could be conducted for names of attorneys or law firms that would indicate a high probability of the result set being attorney/client privileged communication. The documents meeting the privileged search criteria could be assigned to an attorney whose task it is to review for privilege and mark the documents as such. Documents that are responsive to a request for production can be marked for production in a specific field, or with a simple tagging feature. If possible, it is best to identify which request the document is responsive to, as it may be required to provide this information to opposing parties.
Further, documents subject to production should be reviewed for confidentiality or privacy requirements. If there is specific information that needs to be redacted, such as social security numbers, names, addresses, and phone numbers, a team of trained redaction specialists can go through the database, reviewing for this kind of information, and apply the appropriate electronic redaction marks and confidentiality stamps. The team’s ability to perform electronic redactions is dependent on the platform and format of the documents being reviewed. The number of redactions and how they will be performed is an important piece of the planning process when selecting the review platform and format of the documents loaded onto it.
Handling foreign language documents
When planning a review, it is also imperative to know if foreign language documents will be included in the collection. If so, be sure to know your review technology's method for handling these documents. Are they able to “read” that information and search it using its English translation? Will you need to find temp attorneys who speak that language to do the categorization? Will you do some level of translation in order to use them throughout the lifecycle of the matter?
Additionally, it is not just "non-English" documents that are at issue when handling foreign language documents. ESI that is captured from a UK organization or email exchanges with English speaking non-US affiliates may pose issues related to the spelling of common words and should be taken into account when searching and key word filtering.
Document categorization
If information on key issues and players has been developed, the review team can capture documents that relate to key players and known issues in the case. Look-up tables can be developed to aid the reviewer to quickly pick from a pre-populated list of options or a database can be set up that contains a series of marks and/or tags in order to facilitate the categorization of these documents. Key players may also be identified by the bibliographic coding staff if they are asked to capture all names in a document. Technology can also be leveraged to provide an index of all the key words, revealing the names of key players that can quickly be drilled down on to review the documents containing the names of those key players. Often times, an index can reveal the name of a key player hidden in the metadata of an electronic document versus anywhere on the document itself. This is one argument for collecting and indexing as much information as you can in native format.
