course, the answer is it depends, and it cannot be overemphasized that
this decision should be based on the legal judgment of the lawyer
managing the review. The most basic analysis would be that there are
(with 99% confidence) no more than 10,658 of the 730,000 unreviewed set
that are responsive. Using the metrics ascertained in the review to
provide the approximate number of documents that can be reviewed per
hour (that is, to review the set to get to 155,000 responsive), the
approximate cost of reviewing the additional documents is fairly easy to
quantify. For example, assume that a review group reviewed 50 documents
per hour, with an average hourly rate of $50 per hour. To review the
remaining 730,000 documents would then cost approximately $730,000. Much
harder to quantify, of course, is the potential “benefit” that (in all
likelihood the opposition would argue) the remaining review might yield.
If the entire amount in controversy is $100,000, the proportionality
analysis is “probably” straightforward (and in fact the entire scope of
this review would have been questionable).
However, a proportionality analysis may not be appropriate until all avenues of review have been exhausted except for a full linear review. That is, if keyword, date, custodian, or other metadata searches could reasonably target some or all of the remaining 10,658 responsive documents, those efforts should also be evaluated. One simple method is to use the 41 documents found in the second sample, and determine if these 41 documents suggest any other avenues by which more responsive documents could be identified. Similarly, but with more effort, information learned during the review of the 155,000 responsive documents may provide additional clues for searching the remaining corpus of unreviewed documents. A defensibility position needs to anticipate the argument that there is a “better” (and cheaper) alternative to a full linear review; namely that a targeted search would significantly reduce the cost component of a given proportionally analysis. Once those potential objections are addressed, counsel will at least have the ammunition necessary to defend the decision to stop the review.
And speaking of defensibility, it is important to document the decisions made during the workflow. Maintain records and lists of any keywords, custodians, date ranges, etc. used for culling decisions, what sample calculations and calculators were used and results, prevalence estimates found for each measured issue (e.g. privilege, responsiveness, issues), and alternative search strategies and results of each. With the combination of powerful technology such as DISCO AI and documented statistically-accepted methodologies, counsel will be able to maximize search potential while providing the client with the most cost-effective review.