This audit has been completed for the use of Mojohire, a provider of hiring technology. Mojohire commissioned this audit and paid a fee for completion of auditing services.
Proceptual, Inc (hereafter Proceptual) performed an independent audit of Mojohire’ use of certain hiring tools as outlined below. The audit was conducted on data compiled on or about September 1, 2023;
Tools Audited
The following tools were audited on behalf of Mojohire.
- Job matching tool
This tool is provided as a service to Mojohire’s customers. The tool matches candidates based on whole provided text data, considering every aspect of candidate’s experience in the matching process using Machine Learning techniques. The tool is not explicitly trained on any of the sensitive features, like sex, race, or ethnicity.
The selection tool has been in operation since before the period here audited. Note that, as a vendor, Mojohire does not report a specific distribution date for each of its customers.
Mojohire did not ask Proceptual to audit any other tool. Proceptual was not asked to and did not provide a determination as to whether the Mojohire tool fell under the requirements as an “AEDT” under Local Law 144.
Status as Independent Auditor
Neither Proceptual nor its equity owners or managers have any financial stake in Mojohire, except under the condition that if Mojohire becomes publicly traded, owners or managers may own less than 1% of publicly traded shares in index funds or other instruments. Proceptual has no relationship with Mojohire except its status as an independent auditor, including no employment relationship.
Neither Proceptual nor its equity owners or managers have any financial stake in any of the automated tools here audited; Proceptual and its equity owners and managers are not and have not been involved in the development, use, or distribution of any of the tools audited here. Proceptual has no financial relationship with MojoHire, except for MojoHire’s payment of services rendered for this AI audit.
Audit Process
The following information was determined for the tool:
- Selection Rate for each category, with Selection Rate defined as the rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification. Note that the Mojohire tool produces a data point that can be considered by a human in a hiring process; the tool does not automatically advance or deny a candidate
- Impact Ratio for each category
Vendor Multi-employer Audit Justification
Mojohire, a vendor of an HR tool, has provided this audit report for the use of its existing customers and prospective customers. According to § 5-302 of the Final Rule, use of one audit by multiple employers is acceptable if “such employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit or if such employer or employment agency has never used the AEDT.”
Mojohire has represented that the data set provided to Proceptual for completion of this audit satisfied the above requirement by including data from all of its existing customers as of the audit date.
Data
We ran the audit process on a statistically significant sample of candidates who went through the Mojohire system since the company has been operating. This is a subset of the total number of candidates from the Mojohire system. However, this is the complete set of data for which data sufficient to run the audit was available.
Data on candidate sex and race/ethnicity is not collected or retained by Mojohire. Sex and race were therefore inferred. The use of inferred data suggests that the data falls under the definition of “Test Data” (Final Rule § 5-302). Proceptual believes that this data is highly accurate and has provided a sufficient basis for the audit.
Audit Results for Mojohire Selection Tool
After conducting the analysis, Proceptual believes there is no evidence that the audited tool exhibits a statistically significant disparate impact.
Mojohire’s sorting tool produces a scaled score for candidates. The scaled scores are converted into “cold,” “warm,” or “hot” rating for each candidate. There is no specific percentile cutoff for these categories. Percentile matches are also not retained by Mojohire. We analyzed the number of applicants falling into each category (cold, warm, hot) and found that the majority of candidates fell into the “cold” category, and Mojohire confirmed that result is by design. Therefore, we established the “cold” tag as defining a candidate who was not “selected” for the purposes of the audit; conversely, candidates receiving the “warm” or “hot” tag were put into the “selected” category for this audit.
Sex Analysis
| | # of Applicants | # Selected | Selection Rate | Impact Ratio |
| Male | 3797 | 1646 | 43.35% | 0.96 |
| Female | 3046 | 1372 | 45.04% | 1.00 |
For 45 applicants, sex was unknown; those candidates were excluded from the analysis.
Race/Ethnicity Analysis
| _ | # of Applicants | # Selected | Selection Rate | Impact Ratio |
| Hispanic or Latino | 643 | 271 | 42% | 0.91 |
| White (Not Hispanic of Latino) | 3124 | 1439 | 46% | 1.00 |
| Black or African American (Not Hispanic of Latino) | 638 | 270 | 42% | 0.91 |
| Asian (Not Hispanic or Latino) | 1136 | 488 | 43% | 0.93 |
| Native American or Alaska Native (Not Hispanic or Latino) | 171 | 77 | 45% | 0.98 |
The following groups were less than 2% of the sample size and were therefore excluded from the analysis. Selection rate is noted for each of these groups.
- Native Hawaiian or Pacific Islander (n=84, selection rate 37%)
- Two or More Races (n=0, selection rate N/A)
No group showed statistically significant disparate impact.
For 1,137 candidates, race/ethnicity was unknown; those candidates were excluded from the analysis.
Intersectional Analysis
| _ | _ | _ | # of Applicants | # Selected | Selection Rate | Impact Ratio |
| Hispanic or Latino | Male | 336 | 144 | 43% | 0.90 | |
| Female | 307 | 127 | 41% | 0.87 | ||
| Non/Hispanic or Latino | Male | White | 1880 | 846 | 45% | 0.94 |
| Black or African American | 268 | 118 | 44% | 0.92 | ||
| Asian | 632 | 261 | 41% | 0.87 | ||
| Female | White | 1244 | 593 | 48% | 1.00 | |
| Black or African American | 370 | 127 | 41% | 0.86 | ||
| Native Hawaiian or Pacific Islander Male | 188 | 38 | 43% | 0.91 | ||
| Asian | 504 | 227 | 45% | 0.95 | ||
The following groups represent less than 2% of the overall sample size and were therefore excluded from the analysis. As required, their Selection Rate is noted.
- Native Hawaiian or Pacific Islander Male (n=52, Selection Rate 37%)
- Native American or Alaska Native Male (n=83, Selection Rate 47%)
- Native Hawaiian or Pacific Islander Female (n=32, Selection Rate 38%)
- Two or More Races Men (n=0, Selection Rate N/A)
- Two or more Races Women (n=0, Selection Rate N/A)
No group showed statistically significant disparate impact.
Employer Notification Requirements
As a vendor of the tool in question, Mojohire itself is not required to provide or post notification requirements for its use. Proceptual has provided to Mojohire guideline notification requirements that may, at Mojohire’s discretion, be provided to Mojohire’s employer customers.
Data Retention and Privacy Policy
Mojohire’s privacy policy can be found here. Data from the audited tool may be retained indefinitely. Data for the tool is collected from applicant resumes. This data collection policy can be requested by contacting Mojohire using this form.
Disclaimers and Limitations
Proceptual is not a law firm and has not provided legal advice to Mojohire.
Proceptual conducted its audit with information supplied by Mojohire. We believe we were given access to proper and complete data for performing this audit.