How to Build an Effective Legal Ops Dashboard

A colleague I once had was fond of the phrase, “If you can’t measure it, you can’t improve it.” While there are of course exceptions to this rule, I think it’s an excellent starting point for explaining the goals, purpose, and design of a legal ops dashboard. 

A good dashboard is specific to the organization and its particular needs. Consider the problem, the data you have available, and who your stakeholders are. Put simply, not all dashboards are created equal, and you can’t always use the same dashboard for differing purposes, due to the nature of the data. Consideration of the actual problem is critical at the outset, because it’s entirely possible that the metrics you currently track don’t adequately identify the success or failure of that problem, and you may want to modify accordingly. Similarly, the most important attributes of your metrics are that they are actually measurable, and that they relate to the issues you are trying to keep track of and report off of. There should always be a reason for the metrics you track, and those reasons should be easily explainable. Finally, think about the metrics and the overall dashboard format as it will be consumed. If you’re presenting on a monthly basis to a highly technical audience of forensic analysts, the data they can consume will be very different from a room of Associate General Counsel. 

Now let’s explore some specifics as related to common reasons to use one, including typical metrics and cadences, as well as the most common missteps.

Assessing the Current Situation

If you or your organization are relatively new at building out a legal ops program, the best strategy for an initial legal ops dashboard is to design it in service of simply gaining a better understanding about the operations of your organization. Putting together a basic operations dashboard can be a huge help in assessing key areas of risks, costs, and opportunities for improvement. Key types of metrics to track in this scenario are: 

• Overall Case Metrics
  • Number of Custodians (on preservation, with collected data, under active review)
  • Number of Active Legal Holds
  • Average Length of Cases
  • Amount of Data in Review
• Data Governance
  • Legacy/Retiring Systems
  • Incoming Systems
  • Systems on Hold
• Costs
  • Number of Outsourced Cases
  • Size of Outsourced Cases
  • Average Length of Outsourced Cases
  • Monthly Service Provider/Outside Counsel Fees
    • By Case Type
    • By Firm
  • Internal System Costs (including Maintenance)

Now, these are common examples, but it’s critical when you’re doing initial assessment to select measures you already have – you can always measure additional items later, but even the act of identifying the information you have will help you in gap analysis. Another tip when assessing a current situation is that, if you do decide to make changes based off of your assessment and dashboard, start with quick wins! Once you show quick success, it’ll be easier to get buy in for fixing the trickier issues.

Managing Risk

You may be in a situation where you or the organization has already identified the areas of biggest risk, and legal operations has been tasked with addressing that area as a key objective within. In this case, a dashboard is likely to be highly focused on that key risk area. This could of course be related to any core risk, but the most common risks and associated metrics are spoliation and excessive data and associated risk.

• Spoliation/Data Loss
  If your challenges are more on the left of the EDRM, then you’re almost certainly going to want to focus on process management. This means some combination of measuring system effectiveness, adoption, and training programs. An example of this would be:
  • Training Programs 
    • Legal Hold training added to annual Compliance Training
    • Legal Hold Training added to new hire Training
  • Legal Hold Technology
    • Standardized Legal Hold Language
    • Custodian Compliance on Legal Holds
  • Systemized Preservation in Place
    • Automated Preservation of Core Assets (Office 365, Google Suite, etc.)

• Excessive Data Risk
  If your challenges are related to an excess of legacy system data, poor data purge processes, then you will want to be as specific as possible regarding systems and retention.
  • New Systems
    • Incoming Systems and Associated Risk
    • Accessibility of Data
    • Any Retention Policies
    • Types of Data/Users (including uniqueness of data)
  • Legacy Systems
    • Types of Data/Users (including uniqueness of data)
    • Accessibility of Data
    • Any Retention Policies
    • Planned Obsolescence 
  • Data Outside Standard Retention
    • Amount of Data
    • Time Frames/Keywords
    • Number of Custodians/Unique Custodians
  • Data Provided Externally
    • Which Firms Have Our Data
    • Volumes/Unique Data
    • Firm Destruction Policies

You might notice that I’ve called out uniqueness in a few areas. Identifying data that is held for a single case or single custodian can be a huge data volume reduction at a relatively low effort. If for example a legacy system is being held for a single custodian in a single case, then you may be able to simply collect that custodian’s data and then archive or delete the entire legacy system. This process can be a massive risk reduction in a very brief amount of time, without any ediscovery risks or concerns.

Measure Effectiveness of Changes

It’s common for legal operations to be tasked with large scale software or process change initiatives, some of which may take months to a year from start to finish. It is easy to get in the weeds on this kind of project, but putting a regular dashboard in place for ongoing reporting can dramatically help with everything from expectation setting to resource management. Try to keep the dashboard specific to the single initiative, and the measures specific to what is controllable as part of it. 

Unlike the other dashboards we’ve covered, the most critical part of a Change Management type dashboard isn’t the content, but the fact that you have it. A dashboard ensures that the stakeholders involved at a minimum have visibility into what the agreed upon purpose and success metrics of the project are, and ideally the stakeholders will all be aligned to that purpose and success. 

Another area where this dashboard will differ from other types is that it may be less about specific numbers and metrics, and more a report of metrics as measured against timelines and expectations. For example, a dashboard focused on the rollout of a new in house Review system might look something like this:

• Administrative Training
  • Expected Date
  • Actual Date
  • On Track Yes/No
  • Remediation Plan (if necessary)
• User Training
  • Expected Date
  • Actual Date
  • On Track Yes/No
  • Remediation Plan (if necessary)
• First Case In Application
  • Expected Date
  • Actual Date
  • On Track Yes/No
  • Remediation Plan (if necessary)
• Cost Effectiveness
  • Cost Per Case Prior to Roll-out
  • Expected Cost Per Case Post Roll-out
  • Actual Cost Per Case Post Roll-out

Stakeholder Visibility

Perhaps none of the above apply to you, and you’re instead in a situation where people across the organization simply don’t understand the purpose of Legal Ops, or don’t have a strong sense of what the overall scope of the legal department’s work is. When this is the case, a dashboard is typically straightforward, high level, and hard numbers focused.

• Legal Hold
  • Number of Legal Holds
  • Number of Custodians
  • Number of Unique Custodians
• In Place Preservation
  • Volume of Data Overall/by Matter
  • Released Custodians/Standard Retention
• Collected Data
  • Volume of Data Overall/by Matter
• Data in Review
  • Number of Reviewers
  • Volume of Data Overall/by Matter

You may also want to include information around any key projects and initiatives –

• Process/System Changes:
  • Standard Operating Procedures, New Technology Implementation, Additional Insourcing
• Cost Reduction: 
  • Overall Spend, Spend Per Matter, Anomalies
• Risk Considerations:
  • Areas Of Greatest Risk, User Access Audits, Retention Policies And Process – Released Emp Returning To Standard Retention, Termed Emp

In summary, an effective Legal Ops dashboard is one that is purpose built for your organization, your situation, and your stakeholders. You’ll have the most success if you build it out by starting with the data you have, and that you are able to get in a reasonably easy fashion on a regular basis. 

If I can leave you with one final thought, it’s that, regardless of your particular legal ops dashboard, once you have something reasonable in place, resist the urge to make changes too frequently. Month over month data isn’t useful if that metrics themselves are changed every time, so imperfect metrics tracked over time are better than perfect metrics that you can’t actually track because of constant change.