AI Seminar 138: Jaime Fernández Fisac
From Prasad Tadepalli
views
From Prasad Tadepalli
Title: Safety Is Not a Tax: Scaling Autonomy Without Surrendering Guarantees
Abstract: The promise of robotic and AI systems hinges on their ability to operate safely in complex settings and around people. From delivery drones to generative AI assistants, today's systems remain far from the reliability we take for granted in bridges, elevators, and airplanes. As they enter public use, two questions become urgent: what assurances should society demand from them, and must those assurances come at the cost of capability?
This talk presents a unified safety foundation to equip autonomous systems with auditable assurances within a specified operational design domain. We first consolidate the guarantees of previously disparate safety filters—runtime schemes that monitor intended actions and intervene to preempt harm—under a general theorem that lets us combine their distinctive features without losing certifiability. This yields a simple recipe that combines adversarial learning with predictive runtime reasoning to scale these safeguards beyond classical limits, from legged locomotion to high-speed racing.
We then turn to long-tail safety in autonomous driving and show how game-theoretic planners can adaptively guard against rare critical events without becoming prohibitively conservative. The same principle reveals a structural safety–performance separation in reinforcement learning, enabling optimal performance with zero violations. Across these settings, formal and empirical results show that strong guarantees need not come at the cost of capability. Finally, we demonstrate how this reasoning can improve safety in frontier AI systems, from proactive guardrails that steer interactions away from downstream harm to alignment pipelines that neutralize hidden feedback loops rewarding the manipulation of human beliefs. The talk ends with a vision for general “safety compilers” that let designers and regulators automatically translate safety specifications into enforcement and evaluation mechanisms.