To fight bias, first-of-its-kind New York law regulates tech-enabled employment decisions

The law, codified at N.Y.C. Admin. Code § 20-870 et seq., prohibits employers and employment agencies from using “automated employment decision tools” in New York City to screen candidates for employment or assess employees for promotions unless such tools have been subject to independent bias audits, the results of which must be summarized and posted publicly on the employers’ or employment agencies’ websites. The law also requires employers and employment agencies to provide candidates and employees with disclosures regarding the use of automated employment decision tools.

We explore the key concepts of the law in greater detail below.

Automated Employment Decision Tools

The law defines “automated employment decision tool” as “any computational process, derived from machine learning, statistical modeling, data analytics, or artificial intelligence, that issues simplified output, including a score, classification, or recommendation, that is used to substantially assist or replace discretionary decision making for making employment decisions that impact natural persons.” “Employment decisions” mean screening “candidates for employment or employees for promotion within the City.”

The definition of “automated employment decision tool” is quite broad. It may be interpreted to go beyond innovative employment tools, such as those leveraging machine learning and natural language processing, to encompass tools that have long been used in the employment context, such as credit scores. Turnkey software may constitute automated employment decision tools under the law, and employers and employment agencies may thus need to collaborate with their vendors to address compliance.

Bias Audits

Employers and employment agencies that use automated employment decision tools must conduct “bias audits” of such tools no more than one year preceding their use.

A “bias audit” is defined to mean an “impartial evaluation by an independent auditor” that must assess at minimum the tool’s disparate impact on persons of any component 1 category that employers must report to the EEOC pursuant to 42 U.S.C. 2000e-8(c). These categories include race/ethnicity, gender, and job category. “Disparate impact” is undefined under the law, though the term has received meaningful interpretation in other contexts and by companies. Beyond requiring that the audits be impartial and conducted by independent auditors, the law is silent on the specific requirements of bias audits.

The law does not require that automated employment decision tools be unbiased or otherwise “pass” an audit, though to the extent the audit results indicate that the tool has a disparate impact, employers and employment agencies should consider whether they can use the tool in a manner that conforms with state and federal anti-discrimination laws.

Employers and employment agencies must publish summaries of the results of their bias audits on publicly available pages of their websites, noting the date that the tool was distributed. 

Disclosures

Employers and employment agencies must also make certain disclosures to candidates and employees in the City who are to be evaluated by automated employment decision tools.

Candidates and employees must receive notice that the employer or employment agency will use automated employment decision tools in connection with the hiring or promotion process. The employer or employment agency must also describe the job qualifications and characteristics that the tool will use in the assessment. Candidates and employees must receive these notices at least 10 business days before the employer or employment agency uses the automated employment decision tool.

Candidates and employees must additionally receive notice that they can request an alternative selection process or accommodation. The law does not expressly require employers or employment agencies to accommodate such requests.

In addition, employers and employment agencies that use automated employment decision tools must either post the following disclosures online or provide the materials to candidates or employees within 30 days of receiving a written request: (i) information about the type of data collected for use by automated employment decision tools; (ii) the sources of such data; and (iii) the employers’ or employment agencies’ data retention policy.

Enforcement

New York City’s Corporation Counsel, and other persons designated by the New York City Law Department, can seek civil fines of up to $1,500 for each violation. Separate violations arise for each day that non-compliance occurs, and a separate violation occurs each time an employer or employment agency does not provide required notices to a candidate or employee.

Potential Rulemaking

Though not expressly indicated in the law, New York City agencies may have authority to promulgate regulations implementing the law. Indeed, the New York City Council page tracking the status of the law classifies the law as “Agency Rule-making Required.” In addition, N.Y.C. Admin. Code § 20-872 states that “[a]ny person that violates any provision of this subchapter or any rule promulgated pursuant to this subchapter is liable for a civil penalty.”

Employers and employment agencies subject to the law should be on the lookout for any related rulemaking activities from City agencies.

Next Steps

Employers and employment agencies that recruit or promote within New York City should evaluate the law’s applicability. If the law does not apply, it would be prudent to document internally the analysis underlying the determination. If the law applies, then the following actions should be taken in the lead-up to January 1, 2023:

Communicate with any third party providers of automated employment decision tools to confirm that the tools have undergone necessary audits and that information is collected and available to support providing required disclosures.

Where necessary to conduct a bias audit independently, engage independent and qualified auditors subject to appropriate contractual provisions. Work with the auditors to conduct the bias audits and prepare the publicly available summaries.

Prepare the necessary disclosures for candidates and employees, working with third parties as needed.

Update internal processes, including recruitment and promotion processes in New York City, as necessary to facilitate continued compliance with the law.

***

Employers operating in the State of New York will also be subject to new disclosure requirements for employee monitoring activities beginning in May 2022. Read more about the State-wide requirement here.

Authored by Kenneth Kirschner, W. James Denvil, Harriet Pearson, Filippo A. Raso, and Heather McAdams.