How to Effectively QC Check Your Discovery Productions

It’s a terrifying thought: you’ve spent days, weeks, or even months combing through a data set, and you finally get that production out the door, only to discover that you’ve inadvertently disclosed a number of documents that should have never seen the light of day. Just thinking about it should have your hair standing on end—and with good reason.

Releasing confidential information carries serious legal and financial consequences, not to mention the threat of public embarrassment. When Wells Fargo inadvertently released confidential information in response to a third-party subpoena, for example, it led to multiple legal proceedings, threats of a federal investigation, and headlines in the New York Times.

Inadvertent data disclosure could endanger both your client and your career, and a clawback agreement won’t always save you. But you don’t need to live in fear.

There’s good news. Performing QC, or quality control, checks throughout discovery can drastically reduce the risk of inadvertent data disclosure. It’s well worth the time to revisit your review before production, and the right eDiscovery software will make searching for and correcting reviewer mistakes easy and efficient.

The Basics of eDiscovery Quality Control

First things first: before producing a single document, parties should have an effective ESI agreement in place that both requires inadvertent productions to be returned and protects against privilege waiver. This agreement should include a traditional clawback agreement as well as, when in federal court, a plan to petition the court for a 502(d) order. Such orders provide that a party’s inadvertent disclosure of documents protected by the attorney-client privilege or work-product protection does not waive that privilege.
‍

If privileged data does get exposed, these agreements won’t keep opposing counsel from seeing the documents, but they will ensure that you get the privileged documents back, limiting damage to your case.

Once you’ve got those agreements squared away, it’s time to get down to the business of document review. Depending on the volume of the data and the length of the case, QC checks should be performed on a daily, weekly, or monthly basis. However, there’s no need to rehash each document. The objective here is to check your work efficiently, increasing accuracy without a dramatic increase in review time, and this can be accomplished with a limited QC check. Start with 10 percent of the data set, which is usually enough to catch a widespread issue, if one exists.

“However, there’s no need to rehash each document. The objective here is to check your work efficiently, increasing accuracy without a dramatic increase in review time.”

When a check “overturns” a document label (for example, changes a “not privileged” tag to a “privileged” tag), that change should be recorded, so the lead attorney can keep track of accuracy by reviewer. If there is an overturn pattern—for example, one reviewer is consistently mistagging privileged documents—you can expand your QC based on that information.

Four Effective Quality Control Approaches

During a QC check, you want to look for at least four things:

1. production size discrepancies
2. inconsistent tagging logic
3. substantive issues
4. privilege designation errors

Production Size Discrepancies:

This one is simple: make sure the actual number of documents in your production matches the number of documents you want to produce. If the data set includes multi-page image files (PDF or TIFF), for example, the number of images needs to match the document number. On the other hand, if you are producing single-page files, the number of images needs to match the number of pages produced, and so on. Verify these numbers before sharing your production. If you will be copying the production to media, by burning the files onto a CD or putting them on a USB drive, check these numbers before and after the transfer, to make sure all documents survived intact.

Inconsistent Tagging Logic:

To identify inconsistent tagging logic, look for obvious errors, such as non-responsive documents that have issue tags. In Logikcull, simply create a search that includes all issue-tagged documents, plus the “non-responsive” tag.

Any documents returned in that search may have been incorrectly tagged, and you can see which user tagged each item by looking at the document history record in the user activity panel. If you notice that one reviewer is consistently tagging incorrectly, run the search again but add “AND reviewed by [Reviewer Name]” to return all erroneous documents tagged by that person.

Substantive Issues:

Substantive issues concern document content, such as a document tagged “apple” that contains only information on oranges. Again, finding these review errors in Logikcull is relatively simple. To check for such errors, select a few issue tags and run a random keyword search (i.e. search for the “apple” tag AND NOT the word “apple”), then skim through these documents to make sure the keyword is not discussed in the abstract.

Privilege QC:

One of the most important QC checks is focused on preventing the release of privileged material. Logikcull makes it easy to both identify potentially privileged documents at the outset of your review and to QC for their production at the end stages.

For example, when data is uploaded to Logikcull, the platform will automatically identify potentially privileged documents by checking them against a database of law firm domains.
‍

‍

When it comes time to produce documents through Logikcull, the platform offers a second failsafe. If you have documents tagged as privileged or a variant thereof in your production set, Logikcull will alert you to how many documents in your download are tagged as privileged before you start your download.

For a belt-and-suspenders approach, you can also conduct further privilege QC checks. For example, to find privileged documents, search for the “privilege” tag and the key term “privilege*” with an asterisk, to pick up variations of the word (i.e. privileged). This search will return documents already tagged as privileged, so you can confirm that they are not responsive.

To check non-privileged documents, search for documents not tagged “privilege*” AND a string commonly privileged terms together (such as attorney names, titles, emails, or signature blocks,) with an OR in between each (i.e. attorneyname@firm.com OR Attorney OR Name). This will return all responsive documents that contain your selected privileged keywords, so you can see if you missed any privileged documents. Possible search terms include:

• Names and internet domains of in-house counsel and outside counsel, paralegals, and other legal staff
• Email addresses for in-house and outside counsel, paralegals, and other legal staff
• Legal terms, such as subpoena and deposition

With advanced search, these errors are easy to catch, but if you miss one, remember that Logikcull’s built-in QC controls are there to help. When you start a download, the platform automatically alerts you if you’ve included privileged files in your production, reducing the risk of inadvertent production and waiver.

A Final Production QC Checklist

In addition to the checks listed above, consider including the following in your final review, where appropriate:

• Production criteria QC: Have you determined your criteria for production? Create a saved search based on your finalized production criteria, factoring in all inclusion and exclusion criteria, then generate and apply a final production tag to your saved search to mark it as your production set.
• Previously produced documents: Avoid creating multiple production numbers for one document. With Logikcull’s advanced search builder, use the download doc ID field to search for previously produced items. You can also check the download filters carousel for any matches.
• Redacted documents: Verify whether you need to burn in redactions. If so, make sure to select “Yes, include redactions on images” when creating your production.
• Family inclusion: Verify whether you need to keep full families intact. If so, when tagging your production set with a final production tag make sure that you chose “select all” in your results pane and then select “include families”.
• Categorical review: Utilize category filters to check if produced documents fall within your tagging criteria.
• Duplicates: Check for coding inconsistencies between original and duplicate documents. In Logikcull, just utilize the “has duplicate” QC tag filter or MD5 hash field in order to isolate these documents for QC review.

Use QC Checks to Bring Some Peace of Mind

Discovery is risky business, but smart lawyers know how to protect themselves. While this post is by no means exhaustive, these QC tips can help you avoid the potentially catastrophic disclosure of privileged documents, so you can rest easy.

‍