Technological Disadvantage: When AI Discriminates

By Anthony May

In my last post in this series, I discussed the emerging use of artificial intelligence (“AI”) in the workplace and how employers utilize such technology to screen applicants and evaluate current employees. This post will delve into the negative repercussions of using such tools, including the discriminatory impacts of AI, and government action to curtail and ameliorate those effects.

As discussed in a recent Forbes article, when it comes to using AI in employment decisions, researchers are finding that there is “little accountability for how these products are built or tested.” Even the most well-intentioned algorithms can still exhibit biases due to unforeseen issues. For example, many AI tools are developed as “black boxes,” meaning that the machine-learning algorithms interpret data in ways that humans simply can’t understand. As explained in a 2018 Harvard Law Review article, the “complexity” of a system—meaning the algorithm’s structure—may involve such “a deep neural network . . . consist[ing] of thousands of artificial neurons working together in a diffuse way to solve a problem,” that humans have no idea what information the tool used or how it reached a certain decision. Similarly, machines can rely on “geometric relationships that humans cannot visualize,” resulting in “dimensionality” issues that prevent humans from understanding the machine’s outcomes. In layman’s terms, we don’t know what the machine is doing—a scary thought of which the world is beginning to take notice.

In September 2021, the United States-European Union Trade and Technology Council met for the first time to discuss a variety of global economic issues, including collaborating on a study that would examine the impact of AI on the future of workforces. That report stressed key concerns that the “greater adoption of AI-driven algorithms could potentially introduce bias across nearly every stage of the hiring process.” For example, the report highlighted a 2018 study that showed that STEM career adds designed to be gender neutral were nevertheless disproportionately displayed to male applicants because the cost of advertising to younger female applicants was higher. In that instance, the AI technology optimized cost efficiency at the expense of discriminating against female applicants. In another example, Amazon famously announced in 2018 that it would suspend all use of AI technology in its job recruitment after finding that the system it employed strongly discriminated against females.

But the discriminatory impacts of AI are not limited to sex. Persons with disabilities are frequently screened out or negatively impacted by AI in myriad ways. According to the Center for Democracy and Technology, the employment rate for persons with disabilities is about 37%, compared to the 79% employment rate for persons without disabilities. The lack of available employment opportunities can result in gaps on a person’s resumé through no fault of their own, but AI tools will automatically flag such gaps and screen out those applicants, preventing them from even being considered for a job.

Moreover, the Zoom-era in which we live has used AI in the form of video interviews to assess a person’s speech and mannerisms to determine whether the applicant’s physical manifestations would lead to success in a given position. These algorithms are based on “training data” from current employees. But when persons with disabilities apply and are interviewed, they can be excluded from consideration if the “training data” does not reflect current employees with similar disabilities. In other words, the technology is designed to assess based on ableist stereotypes of who can perform certain roles and not whether a person’s disability truly has any impact on their ability to perform the job. These concerns are only the tip of the iceberg and have caused both federal and state governments to act.

In January 2023, the United States Equal Employment Opportunity Commission (EEOC) published its Draft Strategic Enforcement Plan, announcing its target enforcement priorities for 2023 through 2027. The Plan highlights the EEOC’s intent to eliminate barriers to recruitment and hiring caused by “artificial intelligence or machine learning” that “adversely impact protected groups.” The EEOC’s focus on the discriminatory impacts of AI should come as no surprise to employers; it issued guidance in 2022 explaining how employers may be liable under the Americans with Disabilities Act if, among other things, their use of AI intentionally or unintentionally “screens out” an individual with a disability.

State and local governments are following the Feds’ lead. In 2021, New York City passed a landmark piece of legislation designed to prevent bias in employment decisions through the use of “automated employment decision tools.” Many business leaders have taken umbrage with the law, resulting in delays in enforcement until April 2023. While the law’s enforcement status is currently in limbo, it will be one of the laws to watch as legal developments in the intersection of AI and employment unfold.

In Maryland, the use of AI in employment interviews recently sparked privacy concerns, leading to the passage of a law under the Labor and Employment Article requiring employers to provide adequate notice and obtain consent before using facial recognition services during the hiring process. Many other states are beginning to respond to the emerging us of AI in employment, and we can expect to see a slew of legislation in the coming years.

We know that AI in the workforce is here for the long run. But that does not mean that an employer can use it to discriminate. I will be presenting more on this topic at the 2023 Maryland State Bar Association Legal Summit & Annual Meeting in June 2023. The next post in this series will discuss your rights to be free from discrimination in employment through the use of AI, the tools available to you to prevent such discrimination, and options for you to pursue legal recourse if you have been subjected to such mistreatment.  If you have questions about this matter or feel like you’ve been subject to discrimination in the hiring process, please contact us today for a consultation.