Will We Know If AI Takes Over? Q&A with Benjamin Boudreaux

Let’s get into that, because your real contribution here was to model these trends mathematically. What was your approach?
We were focused on collective agency, how we as humans make decisions together. We built our model on social choice theory, which is the mathematics of how groups come together and make decisions. Then we started to add AI to those decisionmaking groups. We could look at how many humans are in these decisive coalitions, how many AIs, and how that changes over time. We could model these little perturbations and see what happens. And in that way, we found that we could begin to track the erosion of human agency.

Your modeling pointed to an end state, where humans have lost agency and cannot get it back. How would this become irreversible?
There are a few ways. First, in this hypothetical end state, the AIs are in control. They would have to decide to give power back to humans, and that might not happen. Second, we’d be much more dependent on AIs. Even if there were an opportunity to take back control, we might not have the skills or expertise anymore to do it. And finally, once AIs are embedded in all of these different decisionmaking structures, it would just be very costly and complex to remove them.

How can the government and other groups use your modeling to avoid a future like that?
Whenever a new AI model comes out, there are all these different assessments that happen. Could it contribute to a cyberattack? Does it increase biological risks? But we haven’t had any way to assess what happens to human agency when we deploy these models in different decisionmaking contexts. We’re working now to develop what we call agency audits. The government or companies and researchers could use these audits to assess what might happen to human decisionmaking with each new model that comes out.

Has this research changed how you personally use AI?
You know, in many ways, I feel like my agency has grown with AI. I can code up apps now or build simulations, things I never thought I’d be able to do. But I try to be much more deliberate about how I use it. I lay out my own ideas first, before I ask the model anything, because I worry that whatever it hands me, I’ll tend to accept. I sometimes treat the very convenience as a warning sign. The smoother it feels to just go with a recommendation, the more I think it might be part of the problem.

All of this is a practice that I definitely haven’t perfected. But the idea is that maybe I can use AI and get the benefits, while also preserving my own capacity to set and act on my values.

Doug Irving is a communications analyst at RAND. This interview is published courtesy of RAND.