Explainability(Interpretability) in AI – Learn Why it Matters and What to Do About It

by | Jan 4, 2023 | Uncategorized | 0 comments

In the case of NYC Local Law 144, New York City is mandating employers to audit their automated employment decision tools (AEDTs).  Some of these tools may use AI, Machine Learning, or they might just use algorithms that help streamline the decision making process for your HR team.  There is logic behind what those tools are doing, and if that logic is flawed, it could be applying biases to your hiring process.

According to C3.ai, “explainability (also referred to as ‘interpretability’) is the concept that a machine learning model and its output can be explained in a way that ‘makes sense’ to a human being at an acceptable level.”

One classic example of how AI can apply bias in the screening process is an AI hiring tool that Amazon built.  According to a Reuter’s article on the subject, the historical high rate of men hired in the industry led the AI to prefer male candidates over female candidates, and downgraded applications that included the words “woman” or “women’s”.

How does this happen?  There are two ways:

  1. AI tools are “trained” with data sets.  That data can potentially lead AI to apply bias.
  2. AI tools use algorithms to do work.  Those algorithms could potentially lead AI to apply bias.

It is important to note that automated tools are becoming increasingly valuable.  Companies get around 250 applications for the average job post, and these tools save a tremendous amount of time.  In fact, according to SHRM around a quarter of employers are using automation or AI for hiring and another quarter are planning to.  Understandably, the explainability of these tools is of chief concern for these employers.

How do you ensure an AEDT has explainability?  First, by auditing these tools you can retrace outcomes from the data you have provided.  While you may not be able to retrace exactly how these tools’ algorithms work, you can begin to identify whether or not there is bias in the outcomes. 

Second, you can keep in mind that there is an entire study of Explainable AI (XAI) that is seeking to embed explainability into future AI tools.

Overall, even though legislation is rapidly emerging to usher in explainability, the outcomes from understanding the way these tools are working for your organization could be incredibly valuable.  Increasing diversity can have huge positive impacts, and knowing whether or not an AEDT is hurting your diversity initiatives is key.