In connection with the Supreme Court’s health care decision, many commentators doubted whether judges’ questioning at oral argument can be used to predict outcomes. Yet recent research has shown that the emotional content of a judge’s questions does predict voting patterns at the Supreme Court. We applied this research to the Sixth Circuit by looking at the tenor of questions asked by judges at oral argument – and predicted case outcomes with uncanny accuracy.
First, a note on methodology. In January 2012, we asked an intern to record data about the questions asked by judges during three days of oral argument. She did not read the briefs, nor was she familiar with the law being discussed. Though she was very bright, she had no formal legal training. She recorded the number of questions asked by each judge at each stage, and then coded each question as either hostile, neutral, or helpful. Hostility was based on both tone of voice and whether the question challenged the party’s position. Based on only this data, we asked her to predict the outcome of each appeal. At no point did we consider the substance of the cases.
Almost all of the cases have now been decided, and this method predicted thirteen of fourteen cases correctly. The party that received more hostile questions invariably lost the appeal. In the one exception, the winning side had faced only one more hostile question than the other side (which we had noted as a close call). In two other cases, the data showed that two judges asked more hostile questions to one side, while the other judge directed his hostile questions to the other side. Both of those cases resulted in a dissent, and the data accurately predicted the vote of each judge.
These results were much stronger than we expected, and show that oral argument can be a good predictor of the outcome of an appeal. We don’t expect that this method will always be this accurate, as our sample was limited to just five oral argument sessions. But hostile questions can help attorneys evaluate their chances of success – and the desirability of settlement after oral argument. Our prior posts on oral argument are available here, here, here, and here.
