Symposium: Judgment & Hybrid Decision-Making: Human-AI delegation, oversight, and punishment - Talk: Human Bias in Oversight Increases Algorithmic Bias

Recommended citation: Dores Cruz, T. D., Starke, C., Katzke, T., Müller, E., Kwiatkowska, M., Lobel, O., Köbis, N., & Shalvi, S. (2025, September 4). Human Bias in Oversight Increases Algorithmic Bias, Subjective Probability, Utility & Decision Making, Lucca, Italy.

Abstract: Symposium

Artificial intelligence (AI) has radically changed the decisions that humans make and experience. Understanding human judgement and decision-making when faced with hybrid decision involving AI is therefore crucial. This symposium explores the dynamics of oversight, delegation, and accountability in hybrid decisions through three presentations and a discussion. First, Terence Dores Cruz examines how human oversight interacts with algorithmic bias, revealing that incorporating humans in the loop can exacerbate errors, with political biases influencing outcomes. This underscores the limitations of relying on human intervention to mitigate algorithmic bias. Second, Neele Engelmann examines the role of framing and transparency in ethical AI deployment. Findings indicate that moral framing—explicitly linking user actions to ethical implications—effectively reduces dishonesty, whereas transparency alone fails to deter unethical decisions in delegation. Third, Margarita Leib examines punishment and responsibility in hybrid decisions, focusing on how evaluators assign blame and mete out punishment for selfish behavior following AI advice. Results demonstrate that selfish behavior is punished more leniently when it aligns with AI’s advice, yet evaluators perceive decision-makers as less responsible when AI, rather than humans, provide guidance. Finally, Nils Köbis leads a discussion on the interplay of oversight, delegation, and punishment in hybrid decision-making to provide a research agenda for judgement and decision research on human-AI interactions. Taken together, this symposium aims to highlight critical challenges in human-AI interactions and provide insights that can help behavioral science to pave the way for responsible AI governance and ethical hybrid decision-making.

Talk A prominent policy used to reduce algorithmic bias is to integrate humans in decision-making processes involving algorithms, commonly referred to as “humans in the loop”. A major blind spot in using humans in such a way is the potential bias humans may introduce into the hybrid decision-making process. An empirical test of the extent to which humans mitigate, or conversely, introduce bias to the hybrid decision-making process is lacking. In incentivized behavioral experiment, we implemented several algorithmic variants to simulate public benefit classification decisions: one favoring local candidates, one favoring immigrant candidates, and one based purely on meritocracy. We stratified our sample to consist of an equal share of Democrats and Republicans to observe how personal and algorithmic bias interact (N = 2545; nobs = 50900). Contrary to expectations, including humans in the loop did not reduce error rates. In fact, error rates increased. We found political bias heavily influenced the quality of final decisions and that errors similarly increased for hybrid decisions regardless of the algorithm’s initial bias. Investigating responses to the algorithmic decisions showed that republicans exhibited a stronger tendency to (erroneously) reject immigrant candidates more and reject local candidates less. Democrats displayed the reverse, but differences were less consistent. This provides the first insights into human in the loop processes failing to mitigate bias. This opens discussion on the limitations of human oversight in algorithmic systems, where individuals trusted with selecting humans to be in the loop could introduce bias into final decisions, while the impression of an impartial procedure is maintained.