Game Theory: Opening Questions

Veritasium’s video on game theory provided an excellent overview of the key concepts of game theory, especially with regards to how repeated games change the success rates of different strategies. In each game, each player is presented with the opportunity to defect or cooperate. In a single game with no potential for information sharing, the Nash equilibrium — the point where there is no incentive to change one’s strategy — is always to defect. However, as demonstrated by Axelrod’s work, over repeated games, a predictable, retaliatory but forgiving, tit-for-tat strategy seems to be most successful. Axelrod’s work provides a powerful explanation for why cooperation has emerged as a rational strategy, across societies and species.

The first area of questions that come to my mind is to do with information and the role it plays in repeated games. The video explains that information and memory (the ability to consider information from the past in one’s decision-making) are central to the rational basis for cooperation. With the tit-for-tat strategy, the decision to defect or cooperate is based on past information of one’s opponents actions. Without this information and memory, the game can no longer be viewed as a continuous repeated game, but rather, a series of single games in which the optimal strategy is always to defect (since that is always the optimal strategy in a single game). This raises the set of questions of how information is both shared and retained by players in repeated games. On the geopolitical level, the Soviet Union and the United States were able to gain information on the decision-making of the other side through formal conversations and negotiations, as well as through classified intelligence. It is worthwhile considering the implications of asymmetric information in repeated games — what would have happened if the United States had intelligence on Russia’s nuclear program, but not the other way around?

Another set of questions that came to mind pertained to the rewards, or payoffs, for each player. In the video, each player is treated as equals and payoffs are symmetric (you can reverse the decisions, and the payoffs also get reversed). While this might be an accurate model for human beings, or possibly animals in general, it is far from accurate in other domains. For example, on the geopolitical level, payoffs are highly asymmetric. Take the tariff policy of the Trump administration for example. If we were to treat tariffs as a form of defection by the United States, countries, wielding far less economic power and leverage than the United States, are incentivised to try to forgive. This is because the payoff for mutual defection is far worse and the payoff for mutual cooperation is far better compared to the respective payoffs for the United States. Beyond this, there are a host of other questions to consider when payoffs are asymmetric and how that changes the strategies of players.

The last set of questions that came to my mind was the definitions of cooperation and defection. In Axelrod’s repeated game experiments, and in the foundational concepts of game theory explained in the video, decision-making is both binary and discrete. A player can either choose to cooperate or defect, and there are no options in between. However, in the real world, there are a spectrum of choices for any given decision-maker that not only range from cooperation to defection, but can actually involve both cooperation and defection. Once again, returning to geopolitics, an argument could be made that defection can produce mutual benefits for both sides through the gains associated with competition. Take the space race for example, while both the United States and the Soviet Union were ‘defecting’ in their technological race across a number of areas, undoubtedly leading to a handful of negative outcomes, the technological advancements made as a consequence of this mutual defection has helped advance society and improve the lives of countless individuals. There may be a similar story with the current AI race between China and the United States — mutual defection, resulting in serious technological competition, is likely to both greatly benefit and harm the world in a myriad of ways. An interesting area to explore further would be the modelling of decisions in such a way that is useful for game theoretic scenarios, which would likely entail breaking an action/decision down into components of defection, cooperation, and potentially other areas.

On the call for a moratorium on AI development

One of many pieces details the need and the call for a moratorium on the development of AI. I have two points I would like to make with regards to this kind of argument. The first is that I am somewhat skeptical of the claims being made by leaders within the AI space regarding the potential of AI to transform everything we know and understand. It is not that I think AI will not have a significant impact, I certainly believe it will. I simply do not trust the statements made by CEOs and leaders within the AI space, especially the leaders of the AI labs, such as Sam Altman, Dario Amodei, Demis Hassabis, and others. While some of these individuals may have good, and even honest, intentions, their primary responsibility is to deliver growth, investment, and attention to the products for which they lead the development. By repeatedly stating the power, potential, and even danger, of their AI systems and products, they attract large amounts of attention and investment, which serve their aims of developing those products further. It is for this reason that I consider statements around AI safety from the leaders of the AI industry to be untrustworthy. I believe that we must push for independent bodies, such as the AI Security Institute in the United Kingdom, to conduct trustworthy research on the impacts and risks of AI.

The second point that I would make in regards to the moratorium on the development of AI is that it is not only unrealistic, it is undesirable. The piece references other international agreements such as the chemical weapons convention that 98% of countries agree to and proposes that something similar could be developed with AI. There are two key problems with this suggestion. First, while a 98% agreement rate is a success for chemical weapons, it would be a disaster for AI. One of the countries that has not signed the chemical weapons convention is North Korea. If AI ends up being as powerful as many within the industry predict, it would be disastrous for an undemocratic and illiberal country like North Korea to gain this devastating technological advantage over other countries. We can return to game theory here: if North Korea defects on extremely powerful and dangerous AI, then we are rationally incentivized to defect as well. It would be interesting to explore the geopolitics of AI safety from a game theory perspective in greater detail.

The second problem with the piece calling for a moratorium on the development of AI is that history tells us that it is not possible. Humanity has undergone several technological waves, from the invention of fire to the development of the steam engine, and despite the efforts of many, these technological waves have never been stopped fully. The industrial revolution is a great example. Many warned of the economic inequality, suffering, and decline in living standards that the industrial revolution would create, and many tried to stop it. Both the domestic and geopolitical incentives meant that civil society and the government failed to hold back the technological wave of the industrial revolution. Although I have not explored this area enough to explain why the prevention of technological waves is necessarily impossible, I would be very curious to apply game theory to the various incentives associated with technological innovation and competition to understand it better.