AIA Blueprint for Keeping Humans in Control of AI
New research teaches independent AI agents to ask for help — and builds oversight of models we don’t fully trust.
In Brief
● Stanford GSB researchers are building frameworks that keep AI under human control even as they become more autonomous.
● In a cooperative game, an AI agent learns when to check in with a person and when to act alone.
Another framework sets up effective oversight of an untrusted model by pools of human or AI supervisors.
William Overman began his PhD program at Stanford Graduate School of Business in an auspicious moment: just two months before ChatGPT publicly launched in November 2022, exploding the widely held understandings of what machines are capable of.
Even as Overman began enlisting AI for his research, he grew wary of where the technology was headed. “This isn’t only about the apocalyptic potential of what could happen; I’m also thinking a lot about the future of human flourishing,” Overmanopen in new window says. He fears that misaligned AI could overstep its bounds — not necessarily maliciously — and inflict subtle, yet real, harms on people.
“To prevent that, we need to get this right,” Overman says. “We must set up the proper interactions and training and incentives for these AI agents and models. It’s critical to think about shaping all of that now, so that these tools help make life better for us, not worse.”
To think through the AI architecture that could help to ensure this balance, Overman enlisted his technical toolkit, calling on game theory, reinforcement learning, and causal inference. He partnered up with Mohsen Bayati, his advisor and a professor of operations, information, and technology at Stanford GSB.
Overman and Bayati have coauthored two recent papers focused on a single question: As AI systems become capable of acting on their own, how do humans stay meaningfully in control?
___________________________________
“We must set up the proper interactions and training and incentives for these AI agents and models. It’s critical to think about shaping all of that now.
Author Name”
— William Overman
___________________________________
The papers approach the question from two directions. One demonstrates how to design a relationship between a person and an AI agent from scratch, so the agent’s drive toward independence never comes at the human’s expense. The other paper offers a more practical framework that can be used with an AI system a user didn’t build and doesn’t fully trust.
