The Office of Administrative Law approved the Civil Rights Council’s regulations on automated decision systems on June 27. They take effect on October 1, which gives employers a quarter to work out what they are already using.
Most of the commentary that will follow over the summer is going to be about artificial intelligence. That is the least useful way to read these rules. They are not really about AI. They are about who is responsible when a decision about a person is made partly by software, and the answer California has given is broader than most employers expect.
Start with the provision nobody is discussing
The regulations define an agent as any person acting on an employer’s behalf to exercise a function the employer would traditionally exercise, and they say expressly that this includes recruiting, screening, hiring, promotion, and decisions about pay, benefits or leave, including where those activities are carried out in whole or in part through an automated decision system. Then comes the sentence that matters: an agent of an employer is also an employer for purposes of the Fair Employment and Housing Act.
Read that twice. The vendor whose software ranks your applicants is not your supplier for FEHA purposes. It is an employer. Its conduct is within the statute, and so is yours, and the contract between you does not divide the liability the way the invoice divides the money.
The same treatment reaches employment agencies procuring applicants through an automated system, and staffing agencies, and any entity that evaluates a candidate’s history on an employer’s behalf. If you have been comforting yourself that the screening happens outside your walls, that comfort is gone on October 1.
What counts as an automated decision system
The definition is deliberately wide: a computational process that makes an employment decision or facilitates human decision making about one. The words “or facilitates” do most of the work. A tool that produces a ranking a human then approves is inside the definition, not outside it.
The regulations then name five things by way of example, and it is worth reading the list slowly, because employers who are certain they use no AI in hiring routinely use three of them.
- Computer-based assessments or tests, including questions, puzzles and games, used to make predictive assessments, to measure skills, dexterity or reaction time, or to measure personality, aptitude, attitude or cultural fit.
- Directing job advertisements or recruiting materials to targeted groups. This one catches almost everybody. If you have ever set an audience on a job advertisement, you have used an automated decision system within the meaning of these rules.
- Screening resumes for particular terms or patterns, which is what your applicant tracking system does by default.
- Analyzing facial expression, word choice or voice in an online interview.
- Analyzing candidate or employee data obtained from third parties.
Ordinary software is carved out: word processors, spreadsheets, databases, spam filters, spell check and the rest, provided they are not making a decision about an employment benefit.
Bias testing is not required, and that is the trap
A great deal of vendor marketing is about to tell you that California now requires bias auditing. It does not. What the regulations say is that evidence of anti-bias testing, or the lack of it, is relevant to a discrimination claim and to any defense against one, together with the quality, recency and scope of the effort, the results, and the employer’s response to the results.
That formula is repeated through the regulations, on national origin, sex, pregnancy, religious creed, disability and age. It creates a position that is more uncomfortable than a mandate would be. Not testing is admissible against you. Testing, finding a disparity, and doing nothing about it is worse than not testing at all, because “the response to the results” is expressly in issue.
An employer that commissions an audit without deciding in advance what it will do with an unwelcome finding has bought itself a document, not a defense.
Records: two years becomes four, and datasets are included
The retention period for employment records moves from two years to four, running from the making of the record or the personnel action, whichever is later. The list of what must be kept now names selection criteria and automated decision system data explicitly.
And automated decision system data is defined to include not only the outputs but the data used to develop or customize the system for your use. If a vendor trained or tuned a model on your historic hiring decisions, that training data is inside the retention obligation. Very few contracts signed before this year contemplate that, and a vendor that deletes it on a ninety-day cycle is creating an evidentiary problem that will be yours rather than theirs.
Disability is where this will actually be litigated
Two provisions deserve more attention than they are getting.
The first: a medical or psychological examination, or a disability-related inquiry, includes one administered through an automated decision system, and may include a test, question, puzzle or game likely to elicit information about a disability. Because such an examination is unlawful before a conditional offer, a gamified assessment that measures reaction time or attention has a real problem, and the problem exists whether or not anyone intended to learn anything about disability.
The second is one sentence long and disposes of a common defense: the use of an automated decision system, without additional process or action, does not constitute an individualized assessment. Anywhere the law requires you to consider a person individually, and California requires that in more places than most employers realize, the tool cannot be the assessment. It can inform one. Somebody still has to make it.
What to do with the quarter
- Inventory what you use. Not what you call AI. Every point between a job advertisement and an offer where software narrows, ranks, scores or targets. Include the advertising audience settings, the applicant tracking system’s keyword filters, any assessment, and anything your recruiter uses that you have never seen.
- Read the vendor contracts against the agent provision. Ask what data exists, how long it is kept, whether the vendor will produce it on request, whether any bias testing has been done, and what the results were. A vendor unwilling to answer the last two questions has told you something.
- Extend retention to four years now, including the records the vendor holds rather than you.
- Find every assessment that could elicit disability information and move it after the conditional offer, or remove it.
- Write down where the human judgment happens and make sure it is real. A reviewer who approves a ranked list without the ability to disagree with it is not an individualized assessment either.
Three months is comfortable for this. It is not comfortable for discovering in September that your recruiting agency has been running an assessment nobody in your organization has ever seen the questions to.
Sources
- California Civil Rights Department, Civil Rights Council rulemaking, including the final text and the notice of approval, calcivilrights.ca.gov.
- 2 CCR sections 11008, 11008.1, 11009, 11013, 11017.1, 11070, 11071 and 11072, as adopted and amended.
This describes general California employment requirements. It is not legal advice about your organization, and it does not account for a collective bargaining agreement, an industry-specific order, or a local ordinance that may change the answer.