Shall We Play a Coordination Game?

“The seasons change; and both of us lose our harvests for want of mutual confidence and security.” – David Hume, A Treatise on Human Nature As I expounded before, security should be treated as a product – as “something created through a process that provides benefits” to the organization. Every product has a purpose, something it is trying to help its users accomplish. If security is a product, what is its purpose? What is it trying to help its users – the organization – accomplish? Without this purpose, security can become aimless – falling into the wretched trap of “security for its own sake.” If we treat security instead as a business enabler, what results? In many tech organizations, the most critical business enabler is software delivery performance. Therefore, security should cooperate with relevant stakeholders who focus on software delivery performance, especially the engineering organization (colloquially known as the DevOps function). As is well-discussed in the industry, the relationship between security and DevOps is typically described as fraught, icy, or adversarial – a far cry from cooperative, let alone collaborative. There are cultural reasons for this, but I will not be covering them in this post. Instead, I am going to draw on behavioral economics, looking both to cooperation games within game theory and the concept of moral hazard as a lens through which we can better understand the security and DevOps relationship. So, shall we play a coordination game? Let’s dive in. Barriers to Cooperation Cooperation Games Most relationships in life can be considered through the concept of “games,” behavioral relationships between decision-makers involving certain rules or conditions. It is worth exploring the different potential attributes of games as a backdrop for how to think about the game infosec plays with its DevOps peers. Games can involve cooperation or non-cooperation. Non-cooperative games involve competition between the game’s players, without any sort of external authority to enforce cooperation between the players – resulting in no chance for alliance. The game between attackers and defenders can be considered a non-cooperative game. Cooperative games, unsurprisingly, involve cooperation rather than competition. Players can form coalitions to coordinate their strategies and share potential payoffs1. One of the more famous cooperation games is the Prisoner’s Dilemma, a non-zero-sum cooperation game in which two prisoners must make the decision to confess or stay silent.

What is a non-zero-sum game? In zero sum games, the total payoffs for all players in the game add up to zero. That is, one player’s gain will equal the other player’s loss exactly. Few real-world scenarios involve zero-sum games, but the game poker is an example of one. Non-zero-sum games are thus games in which the total payoffs for all players do not add up to zero – that the gain by one player does not result in an equivalent loss by the other player. Free-trade is an example of a non-zero-sum game in which all players can benefit in a win-win scenario. The aforementioned Prisoner’s Dilemma is non-zero-sum, as it can result in a win-win or lose-lose scenario. Information is an essential component of every game, as information is at the heart of strategic interaction – particularly information regarding other player’s decision-making in the game. In perfect information games, all players know all decisions previously made by the other players. As you might suspect, real-life rarely allows such omniscience, outside of games like tic-tac-toe or chess. Imperfect information is common to our existence, wherein players cannot see all prior decision-making by the other players within the game. Complete and incomplete information is another informational characteristic of games. In a game with complete information, players understand the potential payoffs, risk tolerances, strategies, and player “types” among other players. Again, complete information is largely unrealistic in the real-world. Instead, games with incomplete information are most common in human interaction, wherein players cannot discern other players’ preferences, motivations, and other strategic information. Although it may scandalize true game theorists, for perspicuity’s sake, I will summarize these information-based characteristics into the concept of information asymmetry – that players possess relevant information to which the other players do not have access. Typically, information asymmetry is analyzed through the lens of transactions, though I argue it all comes back to decision-making between relevant players. I believe one can view DevOps and infosec’s relationship as a coordination game with information asymmetry. I do not believe it is a non-cooperation game, as there is ample room for infosec and DevOps to form a coalition, and there is potential for the organization to serve as an external enforcer of cooperation.

There is also information asymmetry, as neither DevOps nor infosec has perfect insight into the other’s decision-making process nor access to all relevant information (much of which might be tacit and considered tribal knowledge). If we proceed with the assumption that DevOps and infosec’s relationship is a coordination game, then our goal is to understand how to encourage cooperation to maximize the collective payoff. This results in a crucial question: what leads to coordination failures? One impetus is strategic uncertainty among players, that they do not realize their objectives are aligned. Perhaps obviously, misalignment of objectives can also present friction in cooperation games. Thus, mechanisms are needed either to facilitate cooperation – in the event of aligned preferences – or enforce cooperation – in the event of less aligned preferences. Interestingly, empirical evidence suggests that humans are far more cooperative by nature than traditional game theory predicts. In fact, the wealth of data points showing cooperation within games far exceeding predicted values has led scholars to delve into the nature of cooperation, seeking an answer to why this pattern holds. There is a split among those who seek to prove that cooperation still fits within the confines of rational behavior, and those who argue that cooperation represents more of a strategic irrationality – the philosophical nuance of which I will not cover here. Despite humans being cooperative by nature, there are potential wrinkles that can come when decision-making power and information possession are unequal – known as moral hazard, which we will explore presently. Moral Hazard Moral hazard results when someone increases their risk exposure because they are protected from risk impact in some way. It is related to the principal-agent problem, in which someone (the agent) can make decisions on behalf of another person (the principal), leading to the potential for self-interested actions by the agent that are not in the interests of their principal. Information asymmetry exacerbates moral hazard issues, since one party possessing information the other party cannot access creates an incentive for exploitation. A tangible example of moral hazard is in insurance. If you received an insurance policy with full coverage for all potential losses due to security incidents, theory suggests you will have less of an incentive to invest in a strong security program. The insurance provider may assume you will maintain your current level of protection, but you possess information secret to them – that you have no intention of doing so.

Thus, you are willing to accept more risk because you are protected from the risk impact, potentially empowering you to pursue a #yolosec strategy2. Looking to infosec and DevOps, there is the potential for moral hazard given the team structure at most organizations. DevOps can increase their exposure to security-related risk because they are not held accountable for it, making the infosec team bear the risk impact. DevOps can likewise possess hidden motivations or take hidden action from the vantage point of the security team, as not all efforts to secure systems will be observable by infosec. This makes it difficult for infosec to manage the organization’s risk exposure – and thus the ability to manage the risk impact they experience. I believe moral hazard goes both ways in this relationship, however. Infosec can also increase risk exposure for the rest of the organization – but risk of a different kind. Part of infosec’s infamy is due to creating policies or implementing tools that add risk to their colleagues’ workflows, such as a salesperson wrestling with a clunky VPN when trying to close a deal with a customer. Because infosec is shielded from the risk resulting from friction to their organizational peers’ workflows, they are more willing to add such risk in the name of their own priority: security. Moral hazard is particularly relevant when considering conflicting goals. Very recent research suggests that decision-makers resolve goal conflicts on behalf of others differently than they would for themselves, potentially resulting in undesirable outcomes for the recipients of the decision. But, first, what defines goal conflict? It is when achieving one goal prohibits or discourages the achievement of another goal. Unless there is the option to satisfy both goals (more on this later), people will only “activate” the higher priority goal. One study examined moral hazard and goal conflict through the lens of health practitioners choosing between curative care or palliative care for their patients3. Health practitioners typically prioritize the goal of curative care over palliative care, believing the two are in conflict. What makes this context interesting is widespread evidence that these two goals are not actually in conflict. Palliative care can extend lifespan; but at the very least, it is not shown to interfere with curative care. Why do health practitioners believe these goals are in conflict, and why do they opt for curative care rather than improving patient comfort and pain reduction?