Fighting the rise of the machines

Mary Wareham wants to be clear: This isn’t about the Terminator.

At the same time, she knows a flashy name will get more attention: Hence, the Campaign to Stop Killer Robots.

{mosads}“This is about framing a future threat. One way in which we’re trying to do that is not by using images of the Terminator and talking about far-off depictions of military application of artificial intelligence,” Wareham said in a recent interview with The Hill. “We’ve been more interested in looking at existing weapons systems today because there are many now that have got some autonomy in them.”

Wareham is the global coordinator for the group, which since 2013 has brought together several nongovernmental organizations with the goal of pre-emptively banning fully autonomous weapons and ensuring humans retain control over critical functions of weapons systems and the decision to launch an attack.

“So the Campaign to Stop Killer Robots, we don’t make any apologies for using that as the title of our campaign,” she said. “Killer is a weapons system and autonomous is a robot, and it would have probably never gotten off the ground if we had called this thing the campaign to stop lethal autonomous weapons systems. So it did capture people’s imagination.”

What at first blush sounds like science fiction is increasingly becoming a technological reality. The military has tested fully autonomous drones, submarines and trucks in recent years. An October 2016 test that aired on “60 Minutes” showed a swarm of 103 small drones getting into formation, reacting to their environment and otherwise operating without any human help.

Wareham pegs her interest in arms control back to her days growing up in Wellington, New Zealand, in the 1980s, joining protests against nuclear weapons.

“French nuclear testing in the Pacific, the impact of American nuclear testing, the British nuclear testing in Australia — as a country in the Pacific, that alarmed us,” she recalled. “And there was a concerted effort in the 1970s, which kind of led to New Zealand’s nuclear-free legislation” in 1987.

In 1996, she moved to the United States to help the coordinator of the International Campaign to Ban Landmines, Jody Williams, believing the work would take decades. The group achieved its goal, an international treaty banning
anti-personnel landmines, a year and a half later.

In 1998, Wareham started working for Human Rights Watch to set up a civil society verification system for the landmines treaty, and has been working on and off for the group ever since, including currently as advocacy director of the arms division.

In that role, she continues overseeing adherence to treaties on banned weapons and in recent years has been documenting the use of landmines, cluster munitions, chemical weapons and incendiary weapons in Syria.

The Campaign to Stop Killer Robots first took shape in 2012, she said, and officially launched with a couple dozen nongovernmental organizations in April 2013. Autonomous weapons were always on the agenda for her division, and roboticists and artificial intelligence experts had been contacting her and her colleagues, so they decided it was finally time to do something.

The campaign’s work includes pushing governments for a ban at several Convention on Certain Conventional Weapons meetings. Wareham counts the existence of such meetings as an early win for the campaign.

The international process isn’t perfect. She said it goes too slowly and is too modest in its goals. But it does give the campaign a chance to meet with officials from countries such as Russia and China that are otherwise hard to reach.

“I did have a long talk with the Russian delegation last November, and their argument was less with the substance on killer robots,” she said. “It was more with the priority. Their delegation said that the weaponization of outer space is a more important issue.”

The campaign’s contact with U.S. officials also tends to be limited to those multilateral settings, she said, but the U.S. delegation is always willing to sit down.

Wareham gives credit to the United States for being one of the only countries with a policy on autonomous weapons. The Pentagon policy, which was first imposed in 2012 and was renewed last year, says humans must retain judgment over the use of force even in autonomous and semi-autonomous systems.

“Is it a perfect policy? No. Some of the members of the campaign are upset that it green lights the development, while curbing the use,” Wareham said. “But we still point to that policy as one of the few in the world. … You could ask a lot of questions about, is the Trump administration having a negative impact on our work? And in that case, I couldn’t really see their involvement in that policy review process.”

Asked about nongovernmental organizations that want to ban autonomous weapons, the director of the Department of Defense (DOD) office at the helm of many artificial intelligence efforts pointed to that policy.

“Where it gets tricky is making the lethal force decision, and we have a pretty clear DOD policy that says we’re not going to put a machine in charge of making that decision,” Defense Advanced Research Projects Agency (DARPA) Director Steven Walker told reporters last week, in response to a question from The Hill. “And I think that’s the right policy, and every uniformed, four-star general that I’ve heard talk about it agrees with that. So I don’t think at least the U.S. is ever going to put a machine in charge of making that lethal force decision.”

Walker also said the technology still has a long way to go.

“We did the DARPA Robotics Challenge, which I think showed how far away we are from robots being able to do military missions,” he said. “At least on the ground. The ground is complicated, the perception you have to have on the ground.”

This year, Wareham said, one of the campaign’s biggest goals is to expand its global reach. Among those efforts, she highlighted a manga comic recently published by one of the campaign’s nongovernmental organizations in Japan and being sold to kids.

She also hopes to identify new civil society contacts to work with and encourage them to take steps such as issuing briefing papers, holding events and reaching out to their own governments.

“I talked to one government diplomat — not from the U.S. but another country — a couple of weeks ago, and he said, ‘We don’t have a policy on this yet because nothing bad’s happened, Mary,’ ” she recalled. “And then you get some other countries who are like, ‘We haven’t gotten a policy on this yet because we’re waiting for the big guys to decide.’ ”

But to Wareham, the choice is clear: ban autonomous weapons now or go down a path that leads to an arms race akin to the nuclear arms race.

“A lot of people dismissed us early on, but now they see that we’re not going anywhere,” she said. “We’re not disappearing. We’re definitely sticking with this, and the sense of urgency just is growing and with it, the expectation that the countries are going to do something.”