With the arrival of Ursula von der Leyen and the “geopolitical Commission” focused on the twin strategic priorities of digital and green, the Brussels bubble has been continuously abuzz with the trendiest of catchphrases: artificial intelligence, and its impact on humankind.
Any new technology since the steam engine has shaped society. In the 18th and 19th century, the industrial revolution ripped through the established order and transformed fundamentally how we did things without much foresight as to consequences. In 2020, we have a duty to better guide societal evolution instead of building the digital society out of inertia and path-dependency. We have a monumental opportunity to get artificial intelligence right from the start, and to leverage it to build the society of tomorrow in a more conscientious fashion. For this, we need to make the right choices now and to open up avenues for making the right choices in the future.
What I feel is missing in the debates in Brussels and elsewhere, is a vanguard vision for artificial intelligence — a vision that projects us decades into the future and gives us a sense of direction. Without dismissing the important conversations around AI that are on the agenda today, we need to start thinking of the “why” of tomorrow to complement the “what” and the “how” of today. We need to start preparing for successfully dealing with the systemic transformations AI will bring to our societies, and nowhere is this more pressing than in the way the State and its institutions function. That, to my mind, is a global conversation we should be having, and the EU can lead the way in planning for the transformation of the State in line with the AI revolution, and through this, become a truly geopolitical superpower as it aspires to be.
My premise is simple: the modern state, as shaped after the Peace of Westphalia, risks becoming the most helpless and likely future victim of artificial intelligence. With a slow, ossified, often inefficient and ineffective bureaucracy and administration, the state has but a single advantage in the face of all other structures that organize human activity: its size and its monopolies on providing security and — to varying extents — social services, as well as on writing and enforcing rules for how society and economy work. But its monopoly on power, which allowed it to withstand massive shocks throughout history (from wars to large-scale rise of disruptive technologies) can make things turn terribly wrong when it comes to AI, and not only in the EU but anywhere else.
Artificial intelligence, alongside the great opportunities it brings for humans, comes loaded with the means for the demise or the perversion of the state. Suprastatal structures, such as the EU, are just one step removed from the direct threat states face, but the danger they face is of the same nature.
A challenge from market forces
The first threat to the state’s existence comes from market forces — a challenge to its economic viability. As data roams (almost) freely in the hands of private companies, and as AI algorithms are being developed around the world and are improving efficiency in every sector of the economy imaginable, challenges to the state’s already inefficient delivery of core services are increasing. When AI-powered private health becomes cheaper and more effective than state provided healthcare, how will the state justify its expenditure on health? When private AI-powered education — personalized, cutting edge, data-driven, digital-age-ready — becomes the norm, how will the state compete? When, ultimately, private AI-assisted security makes citizens feel safe and protected, how will the state justify its existence? When all services the state provides will be better provided by an AI-powered private sector, what will give the state the authority to make and enforce laws and rules? And when private companies begin reinventing rules and providing alternative systems of behavior “governance” that become more important to citizens than the rules of the state, how will the state maintain its monopoly on enforcement, arbitration, and the dispensation of justice?
Before the rise of artificial intelligence and the data economy, citizens entrusted the state with exclusive competences because the state was the only one large enough to provide certain services. The state was the only institution large enough to collect taxes, it was the only institution large enough to administrate natural monopolies, and it was the only institution large enough to maintain a standing army to counter other states’ standing armies.
Enter data, the new fuel — or even future currency — of the digital economy. And on this data, specifically because of the state’s inherent vulnerability to perversion from collecting data and building AI, the private sector has a cvasi-monopoly. The tables are turned: private companies are now increasingly more effective at providing what the state was historically entrusted to provide. And while there is yet no company that can match a modern state (even if because the state still holds a citizens-entrusted monopoly on making and enforcing rules), the trend is here, at our doorstep. As more and more of human activity becomes data-driven, data-fueled, and data-dependent, how can the state adapt, so that it can compete and survive?
The threat of state perversion
The second threat AI brings to the very existence of the state comes from the opposite direction: it offers the right set of tools for the state to acquire incredible power over its citizens. The more the state collects, stores, and processes increasingly sophisticated and detailed data on their citizens with the help of increasingly sophisticated AI, the more the institutional setup that ensures its democratic — or, arguably, any type of — functioning becomes prone to unraveling. Massive advances in one functional area — such as, for example, the ability to predict every single citizens’ potential criminal behavior or, oppositely, to identify and reward every citizen’s law-abiding behavior — shift the institutional balance of power and alter the power competition between institutions. Resource allocation between institutions gradually shifts to institutions that are more effective in providing for the “wellbeing” of the citizens and, in time, the state transforms into a giant population control machine, continuously and more and more intrusively optimizing the lives of its citizens. This is but one scenario, which does not even need a totalitarian state to come to pass. Nor does the state — or its leaders — need to consciously choose to “control” citizens. Once the state becomes capable to profile every single one of its “customers” — behaviorally, psychologically, politically, and in a myriad other ways — there is no turning back. The technology allows it, for the first time in human history. And, incidentally, the state is since times immemorial the default repository of the personal data of its citizens. So the temptation is there.
By 2030, the EU must define a new rulebook, fit for the digital age, by which the state needs to play in dealing with citizens’ data and in deploying AI. Brussels is rightfully worried by facial recognition technology, predictive policing, social scoring, and other blatantly dangerous technologies that could be misused by public authorities, especially when these technologies combine with potential biases and discrimination. But the road to Big Brother needs not be a straight line. We need to think carefully and to anticipate any possible negative outcomes from using citizens’ personal data and placing powerful AI in the hands of state authorities, even in the most benign use cases. For this, the state itself needs to be transformed, so that it is resilient to perversion even if endowed with dangerous technological tools. We need to reinvent the checks-and-balances and rewrite the social contract for the digital age so that the state survives and continues to serve its purpose.
Fake news and disinformation
The third challenge to the state brought about by artificial intelligence comes from without — the direct attack on truth brought about by AI-powered disinformation and fake-news. Once truth unravels social order unravels as well, and the state is threatened. While propaganda, disinformation, mass manipulation, and fake news are all as ancient as human civilization, AI acts as a catalyst that amplifies these threats beyond the containment threshold.
This is an existential challenge, first and foremost, to democratic systems, because democracy is fueled by truth: citizens need to know the truth in order to vote, and the vote is the fundamental building block of democratic systems. The democratic state is bound to unravel once third parties (whether states or non-state actors, international or domestic) can sway the results of elections through powerful AI-enabled tools. We are not yet there, but the progression is also not linear: once such mass manipulation tools become powerful enough that they are beyond containment, the failure of the democratic state will be abrupt, not incremental.
Social order is relevant for the stability of all societies, not just democracies. AI-powered fake news and disinformation can also bring to their knees non-democratic states, including those very states who are now building and using such tools to advance their international agenda. And, in the future, authoritarian states might be even more vulnerable to this threat than democracies. As democratic societies develop antibodies to deal with such direct attacks on their democratic functioning, authoritarian states are more likely to deploy mass-manipulation tools internally in order to control their citizens and to project a preferred version of reality. But when the proliferation of powerful disinformation AI will make it accessible to smaller actors, including domestic challengers, maintaining a fabricated version of the truth and continuously fine-tuning it will become increasingly difficult.
The stability of the state, democratic or not, depends on truth or, at the very least, on the stability of the local “truth”. The progress of society, including the digital transformation, must be fueled by truth, because scientific progress can only happen when facts remain unchanged. And the rules-based international order is fueled by truth, because rules and decisions in international fora rely on facts and the truth. AI-powered tools that can manipulate populations on a large scale and change the truth to influence mass-scale outcomes are a threat to all states and a threat to the future global stability.
Transforming the state to withstand the challenges of AI
The answer to all three challenges — the increasing competition from private companies in providing fundamental services, the state’s vulnerability to perversion from within if misusing technology, and the challenge to social order posed by AI-powered mass manipulation tools — is in the reinvention of the state as the safest and most trusted digital platform for its citizens. But how?
First, the state needs to be the first to provide citizens a secure, trusted, and inalienable mechanism to own and use their personal data, including in the interaction with the state itself. The Commission’s “secure European e-identity”, foreshadowed by Ursula von der Leyen in her State of the Union address, is a step in the right direction, but it needs a boost in magnitude and ambition. Looking at 2030, the EU must provide its states and therefore its citizens with a solution to not just the ownership and the use of their personal data, but to its “monetization”. If people are given a mechanism through which they can stand to gain quantifiable state-backed benefits from their data, perhaps to the same extent to which the state guarantees currency as a carrier of economic value, market forces will prop-up the state in the uneven competition with the private sector. The Googles and the Amazons of the world will have to adapt to increasing demands for value from the data they use, and the state will have a newfound role in protecting its citizens’ rights.
Second, the state needs to transform itself so as to become a de facto (and powerful) standard-setter in the way personal data is used. The state needs to become itself a data-agnostic digital platform, in which citizens plug their data to access services or to derive economic benefits. The state needs to become the selfless and trusted repository of its citizens data, just like a bank or a safehouse. Rules must be rewritten and the functioning of the states’ institutions must be rebalanced so as to guarantee citizens the inviolability of their data. And, just like the current state pays workers for their time and skill through direct hires and public tenders, so should it set the tone for the digital future by offering citizens direct economic value in those instances when it does need to use, temporarily and for limited and clearly-defined purposes, their personal data.
Third, the state needs to open itself up for everybody to benefit — citizens and industry alike — thus setting both transparency and information standards. By opening the increasingly massive amounts of industrial data it generates, the state can be a standard-setter and a catalyst for the data-driven industrial transformation, fueling innovation and progress. Allowing industry to benefit from public data and from co-creating data with public authorities, encouraging and facilitating the sharing and common use of industrial data, and relying on the natural monopoly on certain types of data to set standards, the state can fuel growth and also become a trusted platform for its economic actors, not just for its citizens. And by opening up its data and acting as a neutral actor in an ecosystem fueled by data and information, the state can also guarantee a consistent and empirics-driven version of “the truth” that is harder to undermine, boosting its own immunity to disinformation and manipulation attacks from without.
Given the economies of scale and interoperability requirements for such transformative changes to the very foundations of the state, the European Union has a crucial role to play here. The traditional “state” will become increasingly insignificant in a digital economy where data flows at incredible speeds all across the globe. No one state can manage such transformation on its own, and no one state would benefit from it, should it go at it alone. While according to the principle of subsidiarity member states will ultimately be responsible for many of the genetic changes the digital age requires of them to ensure their survival, the Union is in the perfect position to provide the ultimate transformation handbook. By 2030, we need to have the handbook ready — and with it, a new set of rules for the role of the Union in the digital transformation.
