AI Hiring Tools Warned to Replicate Bias
AI-powered hiring tools risk replicating and exacerbating historical workplace biases if not carefully designed and audited. A new analysis from Feminist India warns that these systems can reinforce inequalities under the guise of automation. The report suggests that features for bias detection and transparent rationale are now baseline requirements for HR technology.
- A well-known example of AI hiring bias involved Amazon, which discontinued a recruiting tool after discovering it penalized resumes containing the word "women's" and downgraded graduates from two all-women's colleges. The system had been trained on a decade's worth of resumes submitted to the company, which were predominantly from men, teaching the AI that male candidates were preferable. - Bias can manifest in several ways within AI hiring tools, including historical bias from biased training data, algorithmic bias from skewed data sets, and interaction bias, where the AI's biases evolve based on user interactions. For example, an English tutoring company, iTutor Group Inc., used software that automatically rejected female applicants over 55 and male applicants over 60, leading to a $356,000 settlement with the U.S. Equal Employment Opportunity Commission. - Research from the University of Washington revealed significant racial and gender bias in how large language models rank resumes. One study found that AI models favored female candidates but disadvantaged Black male applicants, even with identical qualifications. Specifically, the models awarded higher scores to Black female candidates, followed by white female candidates, then white male candidates, with Black male candidates scoring the lowest. - The global market for AI in recruitment was valued at $590.50 billion in 2023 and is projected to reach $942.3 million by 2030. As of early 2024, 81% of surveyed companies plan to invest in AI-driven solutions for their recruiting processes. - In response to these issues, several jurisdictions have enacted regulations. New York City's Local Law 144 requires annual independent bias audits for automated employment decision tools. Similarly, Colorado's AI Act regulates "high-risk" AI systems used in significant employment decisions, and California has established rules requiring meaningful human oversight for any "automated-decision system" in hiring. - Lawsuits are increasingly targeting companies for biased AI hiring practices. For example, a lawsuit has been filed against Workday, alleging its AI-driven applicant screening system has a disparate impact on applicants based on race, age, and disability. Another lawsuit against Sirius XM Radio claims the company's AI tool discriminated against an applicant based on his race by relying on historical hiring data that perpetuated past biases. - To mitigate bias, experts recommend regular audits of AI hiring tools, using diverse and representative training data, implementing blind recruitment techniques that anonymize candidate details, and maintaining human oversight in the final decision-making process. Some companies are using tools like IBM's AI Fairness 360 to test for and mitigate bias in their systems.
