Posted on February 02, 2022

The paper studies how good machine predictions are compared with human decisions. Specifically, it analyses the case study of bail decisions judges have to make, based on their belief of what would the subject do if released.

The motivating research question is: “can machine predictions ever substitute human decisions?”; while the concrete operationalization is: “can gradient boosted decision trees be better at predicting defendants' actions if released than judges?”

They address the possible bias they could encounter of not having data of "whether jailed defendants would have committed crimes had they been released". On the other hand, among the weaknesses of this study is the fact that they are trying to generalize after only evaluating the New York City scenario where the decision to bail or not a subject depends only on the judge's consideration of the flight risk, and not the public safety risk. Also, if they have access to a national dataset, I wonder why they chose to show their main results only on the New York data.

Among the characteristics of big data, even though the authors specify as "large", the data is only about 1.4 million data points, of which they used around half of it, which is considerate, but not very big for today's standards.Also, the data is nonreactive as the facts that the records are kept, doesn't change the behavior of future subjects.

I like how the authors reflect on the fact that decisions are good or bad depending on the goal in question. I also appreciate how the authors explain the process with details. For instance, the fact that they first proved that the data could not be linearly fitted and that a ML model would help. Then, they explained that even though some metrics are commonly used to evaluate a model, different metrics help evaluate different scenarios. Moreover, I agree with the study on risk they authors perform. This gives a sense that the research was not based only on numbers, but to actually make a change in society: it's not about how many people should be held or not, but about how to reduce society's risk of being victims if the subjects are released. Finally, I think the authors did a thorough study, but more importantly, a very comprehensive methodology worthy of keeping in mind when doing our own research.