AIEnsuring Human Control over AI-Infused Systems
Human control over technology was a concern thousands of years ago when early humans sought to ensure safe use of fire. Later, control over horse-drawn wagons and eventually steam engines led to debates about how to make the most of their benefits while limiting dangers. Now questions of control are central in the design of AI-infused technologies, for which some advocates envision full machine autonomy while others promote human autonomy.
Human control over technology was a concern thousands of years ago when early humans sought to ensure safe use of fire. Later, control over horse-drawn wagons and eventually steam engines led to debates about how to make the most of their benefits while limiting dangers. Now questions of control are central in the design of AI-infused technologies, for which some advocates envision full machine autonomy while others promote human autonomy(Shneiderman 2020).
Spirited conference discussions demonstrate the complexity of designing for different people working in varied contexts and carrying out diverse tasks (Schmidt et al. 2021). Designers must make difficult decisions about user interfaces and the underlying machine and deep learning algorithms. Human control trade-offs are especially relevant in consequential applications such as finance, law, and business, and in life-critical systems such as medicine, transportation, and defense (Fischer 2017). In successful systems, people can choose how much automated help they want—for example, they can decide whether to use the automated parking feature of modern cars.
I suggest four types of control that give designers greater flexibility in their projects and give people greater flexibility in how they accomplish their tasks:
1. Degree of control is more helpful in guiding design than the idea of levels of automation. The latter, which has dominated discussions for 4 decades, assumes that as automation increases, user control decreases—a zero-sum situation. But programmable thermostats are a form of automation that enables homeowners to better control room temperature, and cruise control enables drivers to better manage their driving speed.
2. With shared control people control certain system features while the computer carries out other tasks. For example, digital camera users determine the composition, zoom, and moment they take the photo, but are usually happy to have the camera’s AI algorithms control the aperture, color balance, and focus while reducing hand jitter. Similarly, GPS navigation users specify the start and destination, transportation method, and time of departure, and then machine learning algorithms offer possible routes based on traffic conditions, leaving the user to choose routes that are shorter, avoid certain roads or regions, or are more scenic.
