In late 1987, Sheri Costa drove her three young children from Alameda County to Fresno County to see her husband in jail. He had been arrested more than 30 days earlier in Fresno on charges of drug possession with intent to sell and distribute, and the price of his freedom was set: $100,000.
Costa paid a bail bondsman $10,000 in cash, a non-refundable deposit representing 10 percent of the bail. For the bail bondsman to pay the remaining $90,000, she also needed to promise property as collateral. But at 26, she didn’t own a home, so she convinced her friend’s parents to put up their homes.
Over the years, Costa has helped with bail payments for her nephew and neighborhood kids who have had run-ins with the law, on charges ranging from attempted murder to assault.
“The nice thing about having a big family is that you all get to split up the bail payments,” Costa said.
If California’s Senate Bill 10 passes this fall, families like Costa’s might be able to bring loved ones home without the financial burden of bail.
The proposed legislation would drastically reduce the use of money bail, and instead require all 58 California counties to use risk assessments to help judges decide when to release those charged with crimes.
The assessments ask questions to predict whether a defendant will show up to court or commit a crime if they are released. Many counties throughout the state have already begun to implement risk assessments in their pretrial systems.
Proponents of the bill see the proposed legislation as a step in the right direction for fixing an outdated and unfair bail system. But the use of risk assessments is not without controversy.
Some experts and community organizations suggest that SB 10 is too vague and doesn’t address key issues with the design and use of the required pretrial risk assessments.
On July 30, more than 100 organizations — including the American Civil Liberties Union, Color of Change, and the Electronic Frontier Foundation — released a signed statement arguing against the use of pretrial risk assessments because of the concern that they may exacerbate or perpetuate bias in the criminal justice system.
How that report will impact SB 10 will be seen after the state Legislature reconvenes later this month.
Counties that already use risk assessments implement them in myriad ways, according to details culled from over 40 public records requests and 25 interviews.
They use different tools, provide different information to their judges about defendants, and sometimes struggle to find resources to evaluate the tools’ effectiveness.
The differences between counties don’t mean officials are not being fair, experts say, but they do indicate a system that is fraught with misunderstandings over what risk assessment tools can do – and how they can be evaluated.
Using a “pretrial risk assessment tool” isn’t always high-tech.
Most mornings in Napa County, probation officers receive a list with the names of people arrested in the last 24 hours. They head downstairs to the jail and ask a few questions to those who are locked up.
Each answer to an officer’s questions is assigned a number. The officer adds up a defendant’s score — intended to predict whether that person will skip a court date or commit a crime if released — and writes an accompanying report for the judge, who will decide which defendants to release by the end of the afternoon.
Napa’s specific questionnaire, called the Ohio Risk Assessment System (ORAS) pretrial tool is one of many pretrial risk assessments used by counties across the state. San Francisco and Santa Cruz use the Public Safety Assessment (PSA), which was developed by the Laura and John Arnold Foundation. Some counties have worked with research organizations to design their own pretrial risk assessment tool.
Using different tools isn’t necessarily a problem, said Marie VanNostrand, the co-founder of Luminosity, a company that focuses on developing data-driven solutions for criminal justice problems.
“To me, it almost doesn’t matter what risk assessment is used, as long as it’s valid and it’s accurately classifying the risk in an unbiased way,” VanNostrand said.
That question of bias is central to the future of SB 10. The recent statement of concerns about pretrial risk assessments, which included several organizations that back the bill, argues that pretrial risk assessments could perpetuate racial bias and shouldn’t be adopted in jurisdictions that don’t already use them.
But SB 10 would require many counties in California to start using risk assessments for the first time.
The statement also hits on problems highlighted in several research papers. A study commissioned by the Laura and John Arnold Foundation found that one tool overestimated the likelihood of women to commit violent crime compared to men.
Another study found the same effect for a risk assessment used at the federal level. These risk assessments are designed to assign risk scores based on a person’s likelihood of committing a new violent crime, meaning that people with the same likelihood of committing a violent crime should have the same risk score.
But these studies found that among women and men with the same risk score, the women were on average less likely to commit violent crime than the men were — suggesting the women should have been given lower risk scores or the men higher ones.
In short, risk assessment scores can be biased against women by potentially rating them as higher risk than they actually are.
In Wisconsin and New Jersey, courts have tried to curtail this gender bias by allowing gender to be included as a factor used to calculate risk scores. They reason that including gender in the assessments may lead to more accurate ratings.
SB 10, meanwhile, stipulates that risk assessment tools must be “equally accurate across all racial groups, ethnic groups, and genders” to ensure that they are fair, but the proposed bill also bans the use of race, ethnicity and gender in deciding who is released or kept in jail.
While acknowledging the potential for bias, many are optimistic about risk assessments, arguing that they push the system in the right direction.
In response to the July 30 statement critiquing the use of pretrial tools, the justice reform organization Pretrial Justice Institute wrote that tools can have positive effects when used carefully.
“Is there ever going to be a perfect system where an algorithm can predict risk? No,” said Jeff Adachi, San Francisco’s public defender. “But from the data, this is a better way. The risk assessment tools, while not perfect, are better than simply taking a human being and saying ‘you decide.'”
SB 10 leaves many details of risk assessment implementation to each county. The same score — received through the same assessment — could have different results depending on how a county implements the tool.
Judges, who make the final decisions about release or detention, use the risk assessments as one factor in the process. They also rely on their own experiences and discretion when deciding who to release and who to detain.
VanNostrand, who implemented pretrial risk assessments in New Jersey, said that more research is needed on how scores should be translated into recommendations for judges.
“The risk assessment is research-based,” she said. “The recommendation usually is not.”
Another pressing question is whether judges will be required to follow the risk assessment recommendation. In San Francisco, Adachi said, judges often detain defendants for whom a risk assessment has recommended release.
It’s unclear what the final version of SB 10 will say about judges deviating from risk assessment recommendations. Deciding whether to regulate judge discretion means recognizing the “delicate balance” between the state’s legislative, judicial, and executive branches, state Sen. Robert Hertzberg, D-Van Nuys, said.
The issue of judicial oversight — and other aspects of implementation — could also affect counties that already use tools in myriad ways, from what information judges receive to how defendants get released.
Judges in Solano County receive a narrative about each defendant, while judges in Mariposa County get a standardized sheet with the defendant’s risk level, risk factors like previous offenses and sometimes the defendant’s income. It’s not yet clear whether these counties will have to change their practices in response to SB 10.
Many of these decisions around the implementation of risk assessments are political rather than scientific, said Sakira Cook, director of the Criminal Justice Program at the Leadership Conference on Civil and Human Rights, which authored the recent critique of the tools.
When these key implementation decisions are made, “the community must be at the table,” Cook said.
Tim Dowler, who oversees the Ventura County Probation Agency’s adult services bureau, wanted his county to pick a tool that had a history of effectiveness in California when they overhauled their pretrial program in 2011.
That means testing, or validating, whether the predictions about whether a person would show up for court were accurate, based on data about people accused of crimes in California.
“We knew right from the beginning — we were like, ‘What is validated in California?’ and couldn’t find one,” Dowler said.
The county picked the ORAS-pretrial tool, but vowed to evaluate the tool to make sure it was accurately predicting scores. This year, the county will go through the evaluation process with the help of the University of Cincinnati.
Every jurisdiction should follow Dowler’s instinct, said several experts, including VanNostrand. After implementing a tool, counties ought to collect data to test its effectiveness — and continually repeat the process.
But under SB 10, validation is suggested rather than explicitly required. The bill’s language around evaluation says counties should send data to the Judicial Council “when possible” and that tools should be “regularly validated and adjusted.”
Two tools used in many California counties, the ORAS-pretrial tool and a tool from Virginia, were developed based on analyses performed on people going before the court in other states. The “cutoffs” for these tools — the number of points defendants must receive to be considered low-, moderate- or high-risk — might not be the same across all populations.
Evaluating the risk assessment tools isn’t hard, said Edward Latessa, the creator of the ORAS-pretrial tool. Since so many people move through the system quickly, data can be collected about how well a risk assessment works in a year or less.
“There’s really no excuse for not validating the tool on your own population,” Latessa said.
But there are challenges to such evaluation. While some counties have evaluated their tools, others say cost or outdated technology make the process difficult.
In Mariposa County, the probation department’s case management system isn’t technologically compatible with the court’s case management system.
Officials in Mariposa County said they would have to compare data by hand. Monterey County’s Probation Department also cited technological barriers — alongside a lack of funding — as preventing the county from performing a full evaluation study.
Even with those problems, it’s possible that SB 10 will bring greater balance to the pretrial system.
“If someone is booked for the same offense, whether you can afford or can’t afford bail shouldn’t make a difference,” said Ryan Oliphant, assistant chief probation officer in Mariposa County.
For Sheri Costa’s family, the changes proposed by SB 10 didn’t come soon enough.
“We’ve never been fortunate to have pretrial as an option,” she said. “For us, it was always bail.”