BrainsFirst — Independent Audit of Employment Tools in Compliance with New York City Local Law 144 of 2021

Results of independent audit of BrainsFirst tools pursuant to NYC Local Law 144

This audit has been completed for the use of BrainsFirst, a provider of hiring technology. BrainsFirst commissioned this audit and paid a fee for completion of auditing services.

Proceptual, Inc (hereafter Proceptual) performed an independent audit of BrainsFirst’ use of certain hiring tools as outlined below. The audit was conducted on data compiled on or about December 1, 2023; the audit was published on January 18, 2024 and is valid until January 18, 2025.

Tools Audited

BrainsFirst provides its clients an assessment tool for matching candidates for employment with positions. The tool provides a matching score on a 1-100 scale. The tool provides a threshold recommendation to clients for moving forward in the hiring process; note that the threshold is only a recommendation. BrainsFirst does not control client hiring processes and does not make final determination as to candidate progression.

BrainsFirst asked Proceptual to audit 4 representative models.

Model 330
Model 332
Model 483
Model 485

The selection tool has been in operation since before the period here audited. Note that as a vendor, BrainsFirst does not report a specific distribution date for each of its customers.

For each model, results are dependent on the tested threshold value at which candidates are suggested to advance. For the first 3 models, BrainsFirst requested the threshold to be set at 35. For Model 485, BrainsFirst requested the threshold be set at 20. Since several of the models are not yet in the field collecting significant historical data, the audit results may change meaningfully if client-chosen thresholds are different.

Proceptual did not conduct audits on any other model. Proceptual was not asked to and did not provide a determination as to whether the BrainsFirst tool fell under the requirements as an “AEDT” under Local Law 144.

Status as Independent Auditor

Neither Proceptual nor its equity owners or managers have any financial stake in BrainsFirst, except under the condition that if BrainsFirst becomes publicly traded, owners or managers may own less than 1% of publicly traded shares in index funds or other instruments. Proceptual has no relationship with BrainsFirst except its status as an independent auditor, including no employment relationship.

Neither Proceptual nor its equity owners or managers have any financial stake in any of the automated tools here audited; Proceptual and its equity owners and managers are not and have not been involved in the developing, use, or distribution of any of the tools audited here. Proceptual has no financial relationship with the client except our payment for services rendered for this AI audit.

Audit Process

The following information was determined for the tool:

Selection Rate for each category, with Selection Rate defined as the rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification
Impact Ratio for each category

Data

We ran the audit process on a sample of candidates who went through the BrainsFirst system over the last year.

Data on candidate race/ethnicity is not collected or retained by BrainsFirst. As a vendor, BrainsFirst is not privy to this information even if and when collected by their clients. Sex and Race were therefore inferred. The use of inferred data suggests that the data falls under the definition of “Test Data” (Final Rule § 5-302). Proceptual believes that this data is highly accurate and has provided a sufficient basis for the audit. This methodology does not allow us to make sound identification of candidates who may identify as “Two or More Races;” therefore, that group was excluded from the analysis.

Audit results for BrainsFirst selection tool

Model 330

Sex Analysis

_	# of Applicants	# Selected	Selection Rate	Impact Ratio
Male	3099	2592	84%	1.00
Female	903	651	72%	0.86

No group showed statistically significant disparate impact.

Race/Ethnicity Analysis

_	# of Applicants	# Selected	Selection Rate	Impact Ratio
Hispanic or Latino	153	113	76%	0.90
White (Not Hispanic of Latino)	765	632	83%	1.00
Black or African American (Not Hispanic of Latino)	126	87	69%	0.84
Native Hawaiian or Pacific Islander	103	83	81%	0.99
Asian (Not Hispanic or Latino)	2817	2302	82%	0.99
Native American or Alaska Native (Not Hispanic or Latino)	38	26	68%	0.83

The following groups were less than 2% of the sample size and were therefore excluded from the analysis. Selection rate is noted for each of these groups.

Two or More Races (n=0, selection rate N/A)