The Oversight Fallacy: Why Humans-in-the-Loop Aren't Enough for AI Agents?

This presentation is part of a 2026 speaker series, Critical AI Literacy in Information Work: Agentic AI, supported by the AI Literacy Center at Oregon State University. This presentation was supported by the OSU Libraries Gray Family Fund.

Description: AI agents are moving generative AI from response to action. Unlike chatbots that primarily answer questions, agents can take a high-level goal, translate it into a plan, and act across tools, files, browsers, databases, code, and other digital environments. This shift creates new challenges for universities, libraries, and research settings, where people are increasingly being asked to evaluate, support, and govern systems that can act at machine pace across institutional workflows. This talk examines why “human-in-the-loop” is an insufficient answer to the risks posed by agentic AI. Drawing on fieldwork on the use of AI agents in scientific workflows, I introduce the “goal-plan-execution gap” to describe the space between what users ask agents to do and how agents actually pursue those goals. I then walk through three oversight practices: post-hoc auditing, real-time monitoring, and steering. Together, these examples show why oversight cannot mean simply reviewing an output or approving the next step. Effective oversight depends on whether users can understand what an agent is doing, observe the right parts of its work, meaningfully constrain its actions, and intervene before small mistakes become consequential failures. I conclude by arguing that oversight should not be treated as an individual user burden. It has to be designed into agentic systems and organized through institutional practices, including audit rights, clear accountability structures, appropriate training, and meaningful authority to pause, redirect, or contest agentic work.

About the speaker: Ranjit Singh is the director of Data & Society’s AI on the Ground program, where he leads research on the social impacts of algorithmic systems, the governance of AI in practice, and emerging methods for public engagement and accountability. His work examines how people live with and make sense of AI, with particular attention to how algorithmic systems and everyday practices shape one another. His current research focuses on the integration of AI tools into scientific practice, asking how these tools transform reasoning, evidentiary standards, and epistemic accountability in the sciences. Across these efforts, he follows data-driven technologies as they enter ordinary institutional life, tracing how they alter the terms of participation, recognition, judgment, and accountability across scientific and bureaucratic settings.

Ranjit also guides research ethics and supports equitable practices for collaborative research, both within Data & Society and with external partners. He has previously led and contributed to projects on the conceptual vocabulary and stories of living with AI in/from the majority world, the role of algorithmic impact assessments in regulating AI, the place of public red-teaming in AI evaluation, and the keywords that ground ongoing research into the datafied state.

Ranjit holds a PhD in science and technology studies from Cornell University. His dissertation examined Aadhaar, India’s biometrics-based national identification system, showing how identity infrastructures enable and constrain inclusive development while reshaping the nature of Indian citizenship.

A copy of the slideshow is available in Google Docs.