Producing documents is perhaps the most sensitive event in the ediscovery lifecycle.
Not only do production errors necessitate duplicate work, more importantly, errors may expose you to the risks of violating a court order or leaking sensitive information to the opposing party. Thus, throughout my 13+ years working as a sole practitioner and project manager in the legal industry, I have developed several simple yet valuable ways to QC proactively for production errors.
While this post is by no means exhaustive, it provides a good starting point around best practices and hopefully helps those of you who are new to the world of ediscovery to jump start your quality control toolset.
Proactive ways to quality check your production set
To begin, an understanding of your production requirements drives all of your discovery decisions, including your quality control strategy. Therefore, production specifications should always be reviewed and thoroughly understood before running a production, to ensure the specifications make sense and any points of confusion are addressed in advance by the parties. The production specifications should also be used as the basis for a matter-specific Production Quality Control checklist used to confirm the specific production requirements are fulfilled after a production has been run. However, before running the final production, here are three proactive ways to quality check your production set:
1. QC of workflow
QC of your workflow should include a review of your static potential production population for tagging inconsistencies, incomplete families, redactions without redaction reason, and the use of DISCO’s search visualization tools to make sure you know exactly what is in the production. Begin by creating a review protocol with the feedback of your project manager. This will establish a set of tagging rules specific to your matter, which if not followed, will result in tagging conflicts. To note, working with your Project Manager to set up Review Stages and the associated review panel, will reduce the potential for tagging conflicts; however, prior to a production being run, any tagging conflicts need to be resolved to ensure the appropriate files are being produced.
2. QC of production decisions
After a production has been run, the initial and most important check which should be performed is to ensure the number of files produced matches the number of files identified for production. Depending upon the production specifications, a number of file counts may need to be checked.
- Image Files If the production specifications require multi-page image files (e.g., PDF or TIFF) for certain document types, the number of images needs to match the number of documents satisfying the document type requirement. If the production specs require single-page image files (e.g., PDF or TIFF), the number of images needs to match the number of pages being produced.
- Native Files If native files are being produced for all files not fully withheld or redacted, there needs to be a native file for each document in the production. Frequently only certain file types are required to be produced natively (e.g., audio, video, excel), for ease of review by the receiving party. If only certain file types are required to be produced natively, a native file must exist for only those file types within the production. Confirm natives are not being produced for fully withheld documents in the production.
- Searchable Text Files Production specifications may require that searchable text files be produced in conjunction with image files. As with image counts, the searchable text file count must match either the number of documents for multi-page image files or the number of pages for single-page image files, for those files with searchable text. During this quality check phase, the decision to produce an empty text file or not for documents or pages with no searchable text will impact whether your text file count needs to be adjusted down accordingly.
- Placeholders Ensure the number of documents requiring placeholders (i.e., native produced documents, fully withheld documents in families with a producible document) matches those with placeholders in the production.
- Final Production Media When a production is being copied to a drive or posted to an FTP, the file counts need to be verified again after all quality control checks have been performed to ensure all files have been copied or posted.
3. QC of production format
The third way to check a production after a production is run is to ensure the formatting of the production adheres to the production specifications and internal client decisions.
- Numbering Confirm the Bates prefix, starting number, and number of digits in the Bates number is correct.
- Stamping Confirm all stamp language is correct and applied to the appropriate number of documents.
- Sorting If the production specifications or the production at hand require a specific sort order, confirm the files are sorted correctly.
- Redacted and Withheld Files Check redacted and withheld text files to ensure proper status and ensure no Natives are included in the production.
- Redactions Ensure redaction text is correct and all redactions are burned into the appropriate locations in the documents. If the number of documents with redactions is overly burdensome to review all redactions, accepted statistical sampling procedures should be used to identify an appropriate random sample size for quality control review. DISCO’s Quality Control feature automates advanced sampling and statistical analysis for those without a deep understanding of statistics, while increasing efficiency for those with a higher comfort level. Our quality control process will automatically create a random sample of documents based on a pre-determined confidence interval and margin of error and provide a visual dashboard affording the ability to easily assess error rates and patterns of errors.
- Placeholders Check placeholder language to ensure text is correct.
- Data Load File Ensure the load file format is correct, open the load file, and ensure all required metadata fields are present and in the correct order. If the metadata file headers need to be renamed, ensure renaming is complete. Please note, DISCO’s custom loadfile feature easily allows for renaming the title of a metadata file header.
- Image Load File Ensure the image load file format is correct and if the production is not relatively small, use accepted statistical sampling procedures to identify an appropriate sample size and review the sample set to check if the images are clear and contain the correct text. To note, DISCO’s Quality Control feature can easily assist with identifying the appropriate random sample size for review.
Once you have completed your production quality control process and are satisfied your production is correct, ensure you are providing the requested number of correctly labeled copies, delivered in the requested manner, to the appropriate location whether that be an FTP or physical address.
Overall, I hope that the best practices above provide a strong starting point for your QC processes. These, along with your matter-specific Quality Control checklist, will go a long way in ensuring accurate, clean productions.