Rantanen, Jason, Empirical Analyses of Judicial Opinions: Methodology, Metrics and the Federal Circuit (May 3, 2016). Connecticut Law Review, Forthcoming. Available at SSRN: http://ssrn.com/abstract=2774307

“Despite the popularity of empirical studies of the Federal Circuit’s patent law decisions, a comprehensive picture of those decisions has only recently begun to emerge. Historically, the literature has largely consisted of individual studies that provide just a narrow slice of quantitative data relating to a specific patent law doctrine. Even studies that take a more holistic approach to the Federal Circuit’s jurisprudence primarily focus on their own results and address only briefly the findings of other studies. While recent developments in the field hold great promise, one important but yet unexplored dimension is the use of multiple studies to form a complete and rigorously supported understanding of particular attributes of the court’s decisions. Drawing upon the empirical literature as a whole, this Article examines the degree to which the reported data can be considered in collective terms. It focuses specifically on the rates at which the Federal Circuit reverses lower tribunals — a subject whose importance is likely to continue to grow as scholars, judges, and practitioners attempt to ascertain the impact of the Supreme Court’s recent decisions addressing the standard of review applied by the Federal Circuit, including in the highly contentious area of claim construction. The existence of multiple studies purportedly measuring the same thing should give a sense of the degree to which researchers can measure that attribute. Surprisingly, as this examination reveals, there is often substantial variation of reported results within the empirical literature, even when the same parameter is measured. Such variation presents a substantial hurdle to meaningful use of metrics such as reversal rates. This article explores the sources of this variability, assesses its impact on the literature and proposes ways for future researchers to ensure that their studies can add meaningful data (as opposed to just noise) to the collective understanding of both reversal rate studies and quantitative studies of appellate jurisprudence more broadly. Although its focus is on the Federal Circuit, a highly studied court, the insights of this Article are applicable to virtually all empirical studies of judicial opinions.“