Edward Cheng on the Reference Class Problem in Mass Torts

Edward Cheng has sent me the following response to my post on his article A Practical Solution to the Reference Class Problem.

Many thanks to Alexandra for raising a number ofgood questions about the implications of my article.  I’ll try to address two of them here.

i) The question of value.

Oneundoubted limitation of the use of model selection methods (at least in theregression context) as a means for resolving reference class type problems isthat you need to have a measure of outcome. Thus, the ideas in the paper work well when what we want to predict isthe market value of a house or the pre-exposure risk of cancer.  Where they do not work straightforwardly areareas  determining commonality in classaction cases, because there, you really don’t have an obvious target forprediction

Onepossibility for analyzing commonality through this lens is to use clusteranalysis and the “cluster selection” tools that accompany them.  (Thanks go to Richard Nagareda for spurringthis idea.)  Cluster analysis is aboutfiguring out how to sensibly construct groups, and I think may be a fruitfulavenue.  More details to come as my workprogresses.

ii) Relevancy. 

Theother big issue that Alexandra raises is about the “relevance” of thepredictors.  How do we know that we’vegotten all of the important predictors, or put differently, how do we know whenour model is “right”?

Asa response, I have to admit that I am in many ways advocating for a far more practical and data-driven perspective than what we conventionally see in socialscience studies of law.  I think we needto view model selection methods as an attempt to make the best predictionsgiven the available data.  Take propertyvaluation for example – I’d argue that we’re not really interested in the truemodel of property valuation; all we want is a reasonably accurate prediction ofwhat the house would have sold for on the market.  Might we get greater accuracy ultimately ifwe understood the underlying phenomenon better? Possibly.  But until we do, Ithink the model selection methods are powerful ways of making do with what wehave.  And arguably, that’s what thelegal system does anyway.  We aren’t inthe business of ultimate truths.  We’rein the business of resolving cases based on the evidence at hand.

ADL