This audit has been completed for the use of Paletter, a provider of hiring technology. Paletter commissioned this audit and paid a fee for completion of auditing services.
Proceptual, Inc (hereafter Proceptual) performed an independent audit of Paletter’s use of certain hiring tools as outlined below. The audit was conducted on data compiled on or about February 5, 2024; the audit was published on March 8, 2024 and is valid through March 8, 2025.
Tools Audited
The following tools were audited on behalf of Paletter.
- Palatter AI soft skill recognition tool
This tool is provided as a service to Paletter’s customers. The tool reviews video interview footage and, on the basis of that footage, provides a set of scores designed to measure soft skills. The algorithm produces a composite “fit check” score. Paletter advised that the “fit check” score is the key metric used by their clients; we have therefore audited only the composite fit check score.
The selection tool is in pre-deployment and, to Proceptual’s knowledge, has not at the time of audit been deployed in the market. We therefore conducted our audit using test data provided by Paletter.
Paletter did not ask Proceptual to audit any other tool. Proceptual was not asked to and did not provide a determination as to whether the Paletter tool fell under the requirements as an “AEDT” under Local Law 144.
Status as Independent Auditor
Neither Proceptual nor its equity owners or managers have any financial stake in Paletter, except under the condition that if Paletter becomes publicly traded, owners or managers may own less than 0.1% of publicly traded shares in index funds or other instruments. Proceptual has no relationship with Paletter except its status as an independent auditor, including no employment relationship. Proceptual is not a law firm and has not provided legal advice.
Neither Proceptual nor its equity owners or managers have any financial stake in any of the automated tools here audited; Proceptual and its equity owners and managers are not and have not been involved in the developing, use, or distribution of any of the tools audited here. Proceptual has no financial relationship with the client except our payment of services rendered for this AI audit.
Audit Process
The following information was determined for the tool:
- Selection Rate for each category, with Selection Rate defined as the rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification. Note that the Paletter tool produces a data point that can be considered by a human in a hiring process; the tool does not automatically advance or deny a candidate
- Impact Ratio for each category
Vendor multi-employer audit justification
Paletter, a vendor of a HR tool, has provided this audit report for the use of its customers. According to § 5-302 of the Final Rule, use of one audit by multiple employers is acceptable if “such employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit or if such employer or employment agency has never used the AEDT.” Since the Paletter tool has not been deployed in the market yet, and no employer has used the tool in production at time of audit, the use of a single audit by multiple employers is acceptable.
Data
Paletter supplied Proceptual with test data. The test data included candidate-supplied sex and race/ethnicity identification. The audit was run with the complete set of test data available at time of audit.
Audit results for Paletter selection tool
Paletter’s tool provides a “fit score” for each candidate. The audit was run by comparing the median fit score for each candidate group against the top-scoring group in each category to determine Impact Ratio.
The data set available for audit is very small; at this time, with this data, conclusions as to potential disparate impact of the tool should not be drawn.
Sex Analysis
| _ | # of Applicants | # Selected | Selection Rate | Impact Ratio |
| Male | 20 | 7 | 35% | 0.49 |
| Female | 17 | 12 | 71% | 1.00 |
Race/Ethnicity Analysis
| _ | # of Applicants | # Selected | Selection Rate | Impact Ratio |
| Hispanic or Latino | 8 | 6 | 75% | 1.00 |
| Black or African American (Not Hispanic or Latino) | 25 | 11 | 44% | 0.59 |
| Asian (Not Hispanic or Latino) | 2 | 1 | 50% | 0.67 |
| Two or More Races | 2 | 1 | 50% | 0.67 |
The following groups were less than 2% of the sample size and were therefore excluded from the analysis. Selection rate is noted for each of these groups.
- White Not Hispanic or Latino (n=0, Selection Rate N/A)
- Native Hawaiian or Pacific Islander (n=0, Selection Rate N/A)
- Native American or Alaska Native (n=0, Selection Rate N/A)
Given the small sample size, no group shows statistically significant adverse impact (p<0.05).
Intersectional Analysis
| _ | _ | _ | # of Applicants | # Selected | Selection Rate | Impact Ratio |
| Hispanic or Latino | Male | 6 | 4 | 67% | 0.67 | |
| Female | 2 | 2 | 100% | 1.00 | ||
| Not Hispanic or Latino | Male | Black or African American | 11 | 2 | 18% | 0.18 |
| Asian | 1 | 0 | 0% | 0 | ||
| Two or More Races | 2 | 1 | 50% | 0.50 | ||
| Female | Black or African American | 14 | 9 | 64% | 0.64 | |
| Asian | 1 | 1 | 100% | 1.00 | ||
The following groups represent less than 2% of the overall sample size and were therefore excluded from the analysis. As required, their Selection Rate is noted.
- Native Hawaiian or Pacific Islander Male (n=0, Selection Rate N/A)
- Native Hawaiian or Pacific Islander Female (n=0, Selection Rate N/A)
- Native American or Alaska Native Male (n=0, Selection Rate N/A)
- Native American or Alaska Native Female (n=0, Selection Rate N/A)
- White Not Hispanic or Latino Male (n=0, Selection Rate N/A)
- White Not Hispanic or Latino Female (n=0, Selection Rate N/A)
- Two or more Races Female (n=0, Selection Rate N/A)
Given the small sample size, no group shows statistically significant adverse impact (p<0.05).
Employer Notification Requirements
As a vendor of the tool in question, Paletter itself is not required to provide or post notification requirements for its use. Proceptual has provided to Paletter guideline notification requirements that may, at Paletter’s discretion, be provided to Paletter’s employer customers.
Data Retention and Privacy Policy
Data from the audited tool may be retained indefinitely. Data for the tool is collected from applicant resumes. Paletter’s privacy policy can be found here. This data collection policy can be requested by at this address.
Disclaimers and Limitations
Proceptual is not a law firm and has not provided legal advice to Paletter.
Proceptual conducted its audit with information supplied by Paletter. We believe we were given access to proper and complete data for performing this audit.