Why California Backed Off Again from Ambitious AI Regulation
In summary
A bill mandating disclosure and appeals for a wide range of AI decisions was delayed until next year for the third legislative session in a row. Gov. Gavin Newsom is expected to consult on the measure.
After three years of effort to give Californians the right to know when AI is making consequential decisions about their lives and to appeal when things go wrong, Assemblymember Rebecca Bauer-Kahan announced that she and her supporters will have to wait again until next year.
The San Ramon Democrat revealed that Assembly Bill 1018, which cleared the Assembly and two Senate committees, has been designated a two-year bill, allowing more time for conversations with Gov. Gavin Newsom and over 70 opponents. This decision came in the final hours of the California Legislative session, which ends today.
Bill Overview
Her bill would require businesses and government agencies to alert individuals when automated systems are used to make important decisions about them, including for apartment leases, school admissions, hiring, firing, promotions, and disciplinary actions in the workplace. The bill also encompasses decisions made in education, health care, criminal justice, government benefits, financial services, and insurance.
Automated systems that assign people scores or make recommendations can prevent Californians from receiving unemployment benefits they’re entitled to, declare job applicants less qualified for arbitrary reasons unrelated to job performance, or deny health care or mortgages based on race.
“This pause reflects our commitment to getting this critical legislation right, not a retreat from our responsibility to protect Californians,” Bauer-Kahan said in a statement shared with CalMatters.
Legislative Context
California has introduced 30 new proposals to rein in AI, but political opposition in Washington D.C. complicates matters. President Donald Trump released an AI Action Plan in July advocating for deregulation at both federal and state levels. Earlier this year, Congress attempted and failed to pass a moratorium on AI regulation by state governments.
AB 1018 gives individuals the right to rectify errors made by automated systems within 30 days. It reiterates that algorithms must provide “full and equal” accommodations to all individuals, prohibiting discrimination based on characteristics such as age, race, gender, disability, or immigration status. Developers are required to conduct impact assessments to test for bias within their systems. Non-compliance can result in fines of up to $25,000 per violation.
Recent amendments exempt generative AI models from coverage under the bill, potentially limiting its impact on major AI companies and ongoing projects. The bill also postponed the developer auditing requirement until 2030.
Opposition and Lobbying
AB 1018 faced significant opposition from industry groups, big tech companies, healthcare providers, and venture capital firms. A coalition of hospitals and healthcare software companies urged lawmakers to reject the bill, arguing it could negatively affect patient care and increase costs.
Venture capital firm Andreessen Horowitz opposed the bill, claiming it imposes costs that extend beyond California’s borders. The Judicial Council of California expressed concerns that compliance with AB 1018 could threaten the judicial branch’s ability to use important risk assessment tools.
Support and Future Prospects
Supporters of AB 1018, including unions and various advocacy groups, continue to push for its passage. The bill’s anti-discrimination provisions are particularly significant as they counter the notion that automated systems should be exempt from scrutiny under discrimination laws.
Inioluwa Deborah Raji, an AI researcher, emphasized the necessity for transparency: “I need to know that an AI system was the reason I wasn’t able to rent this house. Then I can at an individual level appeal and contest.”
The struggle to pass AB 1018 reflects broader tensions in California regarding the regulation of AI technologies, with the ongoing battle between innovation and public protection remaining a central theme.
