Recent reports of large language models developing emergent capabilities beyond their programming parameters suggest that once artificial intelligence (AI) is fully integrated with real-time data sets, the temptation for it or its human masters to use it to control, manipulate and suppress people may become irresistible. Totalitarian governments have been early adapters of digital surveillance technologies because of the ability it gives them to entrench their power. Will democratic nations be able to withstand the temptation to follow?
In recent books like “We Have Been Harmonized” and “Surveillance State,” the authors describe how the Chinese Communist Party uses elaborate surveillance systems composed of more than 300 million facial recognition cameras, mobile phone applications, GPS services, internet gating mechanisms, and a range of human overseers to collect and analyze data on the movement, activities, thoughts and patterns of citizens. Often packaged as ways to enhance societal safety and security, these technological tools are rapidly facilitating repressive forms of government that can be used to both entitle or punish citizens based on their adherence to authoritarian rules. The brutalization of classes of people such as Chinese Uyghurs is a clear example of the misuse of such power.
There is no hiding from the kind of ubiquitous digital surveillance that governments can impose, and there is little chance of defeating it once it is embedded throughout society. Even the most well-intentioned surveillance program may naturally lead to the next step — government intervention before opposition has a chance to occur — in the name of public safety.
Edward Snowden believed that the National Security Agency was on that path, leading him to choose a self-destructive course of disclosing classified materials. Even Hollywood seems to have anticipated these kinds of repressive scenarios. “Person of Interest (2011) and Mr. Robot (2015) graphically portrayed environments where machines had effectively taken over society as governments and corporate interests competed to control them.
Governments are not the only ones that can be tempted by the ways that digital technologies can be deployed. Shoshanna Zuboff introduced us to the topic in her 2019 book, “The Age of Surveillance Capitalism.” Surveillance capitalism may have started with companies like Facebook and Google, but as technology gets cheaper and the stakes grow, the pernicious use of technology will become commonplace as unethical players use it to harvest, digest and sell even more parts of our lives for profit.
In a recent article in The Atlantic, titled “The Rise of Techno-Authoritarianism,” executive editor Adrienne La France explains how Mark Zuckerberg was able to launch Facebook: as he said, “dumb f**ks” trusted him. Zuckerberg could not have been more prophetic in offering us a view of the destructive and addictive technologies that could seduce unsuspecting people into giving away large portions of their lives. LaFrance characterizes the behavior of those who run companies engaged in such authoritarian technology as hypocritical, greedy and status-obsessed people whose power to reengineer society now exceeds both that of Wall Street and Washington.
We are perched on a frightening precipice. Technology can lead to unimagined enhancements of the human condition, but it can also lead to unimaginable manipulation and control in the hands of power-hungry governments and corporate executives. For that reason, regulation of the elements, products and people that seek to compete in this new world is an unsavory but necessary choice. Ultimately, I prefer decisions about my life to be made by short-sighted, venal elected officials who can be un-elected, rather than technocrats who view us as faceless data sets who we have no access to or power over.
It is clear that we are only at the front end of a revolution that is hurtling toward new levels of machine consciousness resembling the way humans learn, analyze and make judgments. As that happens in a relatively uncontrolled environment, we can envision a time when we — the humans — lose control, and machines begin to make the decisions. After all, isn’t that what the human masters who programmed these machines would do?
The potential for new kinds of authoritarianism and human tyranny, thanks to the perverted use of new technologies, is one of the most frightening challenges facing humans today, largely because it will make them less relevant to the future of the world. Understanding this, why wouldn’t governments and malevolent people seek to control machines that can control us?
The temptation to use tech tyranny may be overwhelming and may already be written into our futures. That script will only be altered if we choose to identify the risks, agree on how to oversee them, and find capable leaders to referee the process. I don’t see any hint of those leaders coming forward, and the ones we have seem only to understand how to create slogans that pit us against each other. I wish our children and grandchildren the best of luck.
Thomas P. Vartanian is the executive director of the Financial Technology & Cybersecurity Center and the author of “The Unhackable Internet.”