In business, AI is a big part of how decisions get made. And because bias and inaccurate information are known problems for AI, decision-makers often override automated recommendations.
Tech expert Omar Gallaga wrote about the findings of a UT-Austin researcher studying the impact of human involvement in AI-based decision-making.
Highlights from this segment:
– Researchers studied the role of AI in hiring decisions. The system read thousands of applicant biographies and was able to use keywords to classify applicants by their believed gender, among other factors. When presented with the results, human hiring managers, believing that AI was biased, often sought to override the system. Researchers found these overrides could lead to bad decisions, as they overcorrect the AI.
– If AI-based explanations of its decisions are incomplete, humans could end up using them to make bad overriding decisions.
– The researchers conclude that better training of AI is important, as well as realistic expectations of what a particular AI model can do.
