Pressure mounts to keep human control over killer robots

Human Rights Watch coordinates the Campaign to Stop Killer Robots, which believes that work in the Convention on Conventional Weapons forum should lead to new international law prohibiting fully autonomous weapons. The Convention on Conventional Weapons preemptively banned blinding lasers in 1995.

“Humans should retain control of weapons systems and individual attacks, not only of overall operations,” said Docherty, who is also a lecturer at the Harvard human rights clinic. “Mandating human control would help avoid threats to the fundamental moral principles over the decision to use force.”

HRW says that fully autonomous weapons would lack the human capacity to feel empathy, which can act as a key check on killing, the organizations said. Ceding human control over decisions about who lives and who dies would also deprive people of their inherent dignity, as inanimate machines can neither truly comprehend the value of human life nor the significance of its loss.

Mandating meaningful human control would close the accountability gap that would be created by the use of fully autonomous weapons. It would ensure that someone could be punished for an unlawful act caused by the use of the weapon. With a legal requirement for human control, a commander could be held criminally liable for using any weapon without such control.

Meaningful human control over the use of weapons is also consistent with and promotes compliance with the principles of international humanitarian law, notably distinction and proportionality. Distinction requires the ability to understand an individual’s behavior and proportionality requires human judgment to weigh civilian harm and military advantage. Human control is also crucial to upholding human rights law. Two UN special rapporteurs said in February that: “Where advanced technology is employed, law enforcement officials must remain personally in control of the actual delivery of use of force.”

Since the international debate over full autonomy in weapons systems began at the Convention on Conventional Weapons meeting in 2013, almost thirty countries have specifically addressed the concept of human control in their statements, largely characterizing it as meaningful, appropriate, or effective. Many of these countries explicitly support requiring human control and most have called for more in-depth discussions of the topic.

HRW notes that the concept of meaningful human control is also gaining currency outside of the Convention on Conventional Weapons. For example, a 2015 commentary on the right to life issued by the treaty body of the African Charter of Human and People’s Rights found that: “Any machine autonomy in the selection of human targets or the use of force should be subject to meaningful human control.”

Countries participating in the Geneva meeting will not make any formal decisions. The meeting aims to build a common base of knowledge about technical, ethical, legal, operational, security, and other concerns relating to the weapons. However, they “may agree by consensus on recommendations for further work for consideration” by the treaty at its Fifth Review Conference, a multilateral meeting held every five years, in December 2016.

Human Rights Watch is a co-founder of the Campaign to Stop Killer Robots. Mary Wareham, advocacy director for the Human Rights Watch arms division, serves as the campaign’s global coordinator. The international coalition of more than sixty nongovernmental organizations is working to preemptively ban on the development, production, and use of fully autonomous weapons.

“Countries should not only dedicate more time to considering concerns about killer robots but also commit to pursue a timely and tangible outcome,” Docherty said. “Substantive work will only be possible if countries adopt policy and legislative measures to retain human control of weapons before the technology advances too far.”