Artificial Intelligence: How Good Intent Can Still Lead to Unintentional Bias
Updated: Jan 10, 2022
We are still a long way away from terminators(1), but “artificial intelligence” is everywhere these days. Whether AI is helping your spam filter better screen phishing attempts from North Korea or you are using TurboTax to prepare your tax return, you have used AI as a consumer. But what about as an employer? Amazon has famously charged into the machine learning market with its AI bot Alexa, but Amazon less famously attempted to develop a machine learning system to screen job applications.
Amazon’s experimental computer models vetted potential applicants based on observations in resumes submitted to Amazon over the previous 10-years(2). Amazon’s efforts were a warning sign to other companies of the potential discrimination hazards involved with using computer models to streamline hiring practices, but the promise of AI’s ability to streamline HR processes remains. Companies are improving employment-related AI to improve quality analytics and decrease the time involved in the traditional hiring process. However, while such companies likely are not intentionally developing AI that would discriminate against minorities or women, this remains a risk for employers taking advantage of machine learning in the screening process, as shown by Amazon.
State and federal laws prohibit discrimination based on race, color, sex, national origin, or religion when hiring. The AI systems available today are not “intelligent” such that it can understand whether it is illegally discriminating against candidates. AI systems are simply told to review data sets with certain criteria highlighted as preferable and applying those criteria to future applicants. In Amazon’s case, this unintentionally led to gender bias. Therefore, employers using AI systems for hiring, promotion, or termination decisions, should consistently review those systems with their attorney to ensure the AI systems are not violating the law, either in theory or in practice.
Brettner Counsel attorney advertising materials prepared for informational purposes only. These materials are not legal advice and are subject to the disclaimer that can be found at https://www.bcfirm.law/privacy-policy. This information is not intended to create an attorney-client or similar relationship, and you should not rely on these materials alone to determine whether you need legal services or the counsel you should choose. Past performance is not a guarantee of future results or success. Please do not send us confidential information without entering into a written engagement agreement with us.
1. In reality, even cutting-edge AI today is not close to what is called Strong AI or Artificial General Intelligence, i.e., AI that are capable of human-level cognition or better. For the most part, AI programs in commercial use today either qualify as machine learning or rules-based knowledge-representation systems.