2008-06-03 Search Protocol Sufficiency Challenges Recognized; Testability Required

In a May 29, 2008 opinion in Victor Stanley, Inc. v. Creative Pipe, Inc. 2008 WL 2221841, 3 (D.Md. 2008) Judge Grimm acknowledges that challenges to eDiscovery protocol adequacy may be made by a party. Of course, it does help know how to challenge, which in turn speaks to counsel core competency (assisted by expert testimony) in making the assertion.

Nevertheless, Judge Grimm sums it up in one sentence in noting that "all keyword searches are not created equal" and acknowledges that a test of any assertion of adequacy must be made.

Excerpt from the decision:

"First, the Defendants are regrettably vague in their description of the seventy keywords used for the text-searchable ESI privilege review, how they were developed, how the search was conducted, and what quality controls were employed to assess their reliability and accuracy. While it is known that M. Pappas (a party) and Mohr and Schmid (attorneys) selected the keywords, nothing is known from the affidavits provided to the court regarding their qualifications for designing a search and information retrieval strategy that could be expected to produce an effective and reliable privilege review. As will be discussed, while it is universally acknowledged that keyword searches are useful tools for search and retrieval of ESI, all keyword searches are not created equal; and there is a growing body of literature that highlights the risks associated with conducting an unreliable or inadequate keyword search or relying exclusively on such searches for privilege review. Additionally, the Defendants do not assert that any sampling was done of the text searchable ESI files that were determined not to contain privileged information on the basis of the keyword search to see if the search results were reliable. Common sense suggests that even a properly designed and executed keyword search may prove to be over-inclusive or under-inclusive, resulting in the identification of documents as privileged which are not, and non-privileged which, in fact, are. The only prudent way to test the reliability of the keyword search is to perform some appropriate sampling of the documents determined to be privileged and those determined not to be in order to arrive at a comfort level that the categories are neither over-inclusive nor under-inclusive. There is no evidence on the record that the Defendants did so in this case. Rather, it appears from the information that they provided to the court that they simply turned over to the Plaintiff all the text-searchable ESI files that were identified by the keyword search Turner performed as non-privileged, as well as the non-text searchable files that Monkman and M. Pappas' limited title page search determined not to be privileged."