Why RAI’s are NOT A REASON to Oppose Criminal Justice Reform
One of the most consistent arguments from the left opposing this #cjreform legislation is that it utilizes a Risk Assessment Instrument.
Here are some answers to that argument
First, for those that don’t know, a Risk Assessment Instrument is an algorithmic predictive tool which predicts dangerousness, in this particular instance, based on a number of static factors which will be evaluated using that tool.
Here is the risk assessment conundrum in a nutshell
The comparison is not RAI’s against equality, they are the risk of bias from RAI’s against the risk of bias from HUMAN DISCRETION alone. Here is some evidence on that from a Carnegie Mellon academic:
Alexandra Chouldechova 2016 (Fair prediction with disparate impact: A study of bias in recidivism prediction instruments, Heinz College, Carnegie Mellon University 5000 Forbes Avenue, Pittsburgh, PA, USA)
“In closing, we would like to note that there is a large body of literature showing that data-driven risk assessment instruments tend to be more accurate than professional human judgements and investigating whether human-driven decisions are themselves prone to exhibiting racial bias We should not abandon the data-driven approach on the basis of negative headlines. Rather, we need to work to ensure that the instruments we use are demonstrably free from the kinds of quantifiable biases that could lead to disparate impact in the specific contexts in which they are to be applied.”
The irony here is that many of the same people who were, only a few months ago, railing against Attorney General Jeff Sessions having control a First Step Act risk assessment instrument have, apparently, no problem with a new Attorney General having NO risk assessment instrument to check his or her discretion.
In the context of the First Step Act, the alternative to the RAI is not fairness, it is the unfettered discretion of the Attorney General, the Bureau of Prisons, and correctional officers and prison administrators.
The thing to remember is that while nobody is getting would be getting out now using time-credits, but if they did and there was no risk assessment tool, the deciders would be the same people (but without a tool ).
I trust a transparent tool more than I trust CO fitness reports, wardens, and BOP administrators
I am not arguing that RAI’s are not worthy of suspicion, I am arguing they can’t be evaluated in a vacuum.
In addition, while there are few checks on the exercise of decision-making power for these people, there are MANY transparency and accountability measures built into the First Step Act’s Risk Assessment Tool.
- Consultation with an advocacy organization expert in risk assessments and with problems associated with risk assessment tools
- Publication of the tool
- Consultation with other government departments
- An audit of the tool
- An annual report to Congress on the tool
Thus, while a recent article in The Hill [by Clarence Wardell and Colleen Chien], for example, calls for this change (and a much more detailed data release):
“Lawmakers should add — or the DOJ should implement — provisions specifying that de-identified data, including metrics of interest to stakeholders will be aggregated and collected in a consistent and standardized way across systems and shared with researchers and relevant communities (along with details about the resulting system).”
The latest version of First Step Act answers this call, by requiring the tool to be published — and explicitly directing the BOP to correct demographic disparities explicitly:
“(4) on an annual basis, review, validate, and release publicly on the Department of Justice website the risk and needs assessment system, which review shall include…‘‘(D) statistical validation of any tools that the risk and needs assessment system uses; and ‘‘(E) an evaluation of the rates of recidivism among similarly classified prisoners to identify any unwarranted disparities, including disparities among similarly classified prisoners of different demographic groups, in such rates; ‘‘(5) make any revisions or updates to the risk and needs assessment system that the Attorney General determines appropriate pursuant to the review under paragraph (4), including updates to ensure that any disparities identified in paragraph (4)(E) are reduced to the greatest extent possible; and ‘‘(6) report to Congress in accordance with section 3634.”
These protections are critical and create the grounds for expansion. Don’t just take my word for it, here is what Congresswoman Sheila Jackson-Lee said recently:
“I am pleased that our diligent efforts in the House allowed a fruitful yield in the inclusion of sentencing reform to the First Step Act. I am also pleased that my amendment to create an Independent Review Committee that will oversee the implementation of the risk assessment tools and the bill generally, was also included in the final version of this bill. I look forward, in the next Congress, to expanding upon this preliminary progress,”
My point here is not suggest that Risk Assessment Instruments are not likely biased, they are (because they are created by human beings), but rather it is to suggest that, unfortunately this might be a zero-sum game favoring the use of technology as Julian Adler and Greg Berman explained in their recent book:
Julian Adler and Greg Berman 2018 (Start Here: A Road Map tio Reducing Mass Incarceration)
“Risk assessment is not an exact science. But, in the aggregate, it is massively superior to what constitutes business as usual at the moment: judges and prosecutors and other criminal justice officials making decisions about who should be in jail and who shouldn’t with little information beyond a rap sheet and their own instincts and inherent bias.”
I am guessing, but not entirely sure, this is the reason Sherrilyn Ifill wrote this in the NAACP Legal Defense Fund letter on the First Step Act in November
This does not mean we should refuse to question risk-assessment tools or refuse to try to find a better way beyond somewhere beyond algorithms. But let us also not forget that bad tools can be the best available option and that we can transform assessment tools to serve better purposes too, something the research agrees with too:
Seth J. Prins and Adam Reich, 2018 (Can we avoid reductionism in risk reduction? Theoretical Criminology 2018, Vol. 22(2) 258–278)
There is a long history to the idea that “nothing works” in the field of criminal justice that is beyond the scope of this analysis. The more relevant question here is whether the skepticism epitomized by Harcourt is warranted. Just as the proponents of risk assessment have, by and large, neglected the distinctions we draw above, so too have the most prominent critics of risk assessment neglected careful consideration of exactly what risk assessment is, what questions it can legitimately and appropriately answer, and how to implement “risk knowledge” when it is addressed to appropriate questions. Rather than embrace actuarial thinking uncritically, or reject it entirely, our analysis recommends a deepening engagement with the assumptions underlying risk assessment.
Just yesterday, a CBO report scoring the bill indicated that the First Step Act would result in release or release to supervision for 53,000 inmates over ten years, it seems incredibly questionable to me how — on the basis of concerns over a transparent risk assessment instrument — it makes sense to vote no and prevent these folks from reuniting with their families.
We should always remain vigilant, but this risk assessment instrument is entirely transparent.
Tell your Senators to VOTE YES on the First Step Act next week!
Josh is the co-host of the Decarceration Nation podcast and is a blogger and freelance writer who writes about criminal justice reform, television, movies, music, politics, race, ethics, and more.
