Losing control: The dangers of killer robots
Human control also promotes compliance with international law, which is designed to protect civilians and soldiers alike. For example, the laws of war prohibit disproportionate attacks in which expected civilian harm outweighs anticipated military advantage. Humans can apply their judgment, based on past experience and moral considerations, and make case-by-case determinations about proportionality.
It would be almost impossible, however, to replicate that judgment in fully autonomous weapons, and they could not be preprogrammed to handle all scenarios. As a result, these weapons would be unable to act as “reasonable commanders,” the traditional legal standard for handling complex and unforeseeable situations.
In addition, the loss of human control would threaten a target’s right not to be arbitrarily deprived of life. Upholding this fundamental human right is an obligation during law enforcement as well as military operations. Judgment calls are required to assess the necessity of an attack, and humans are better positioned than machines to make them.
Promoting accountability
Keeping a human in the loop on decisions to use force further ensures that accountability for unlawful acts is possible. Under international criminal law, a human operator would in most cases escape liability for the harm caused by a weapon that acted independently. Unless he or she intentionally used a fully autonomous weapon to commit a crime, it would be unfair and legally problematic to hold the operator responsible for the actions of a robot that the operator could neither prevent nor punish.
There are additional obstacles to finding programmers and manufacturers of fully autonomous weapons liable under civil law, in which a victim files a lawsuit against an alleged wrongdoer. The United States, for example, establishes immunity for most weapons manufacturers. It also has high standards for proving a product was defective in a way that would make a manufacturer legally responsible. In any case, victims from other countries would likely lack the access and money to sue a foreign entity. The gap in accountability would weaken deterrence of unlawful acts and leave victims unsatisfied that someone was punished for their suffering.
An opportunity to seize
At a UN meeting in Geneva in April, Ninety-four countries recommended beginning formal discussions about “lethal autonomous weapons systems.” The talks would consider whether these systems should be restricted under the Convention on Conventional Weapons, a disarmament treaty that has regulated or banned several other types of weapons, including incendiary weapons and blinding lasers. The nations that have joined the treaty will meet in December for a review conference to set their agenda for future work. It is crucial that the members agree to start a formal process on lethal autonomous weapons systems in 2017.
Disarmament law provides precedent for requiring human control over weapons. For example, the international community adopted the widely accepted treaties banning biological weapons, chemical weapons and landmines in large part because of humans’ inability to exercise adequate control over their effects. Countries should now prohibit fully autonomous weapons, which would pose an equal or greater humanitarian risk.
At the December review conference, countries that have joined the Convention on Conventional Weapons should take concrete steps toward that goal. They should initiate negotiations of a new international agreement to address fully autonomous weapons, moving beyond general expressions of concern to specific action. They should set aside enough time in 2017 – at least several weeks – for substantive deliberations.
While the process of creating international law is notoriously slow, countries can move quickly to address the threats of fully autonomous weapons. They should seize the opportunity presented by the review conference because the alternative is unacceptable: Allowing technology to outpace diplomacy would produce dire and unparalleled humanitarian consequences.
Bonnie Docherty is Lecturer on Law, Senior Clinical Instructor at Harvard Law School’s International Human Rights Clinic, Harvard University. This article is published courtesy of The Conversation (under Creative Commons-Attribution/No derivative).
