Soatok’s Informal Guide to Threat Models - Dhole Moments

After a long day of exhausting conversations about Hybrid Post-Quantum Cryptography, random jackasses trying to play gotcha with endpoint attacks against end-to-end encrypted messaging apps, and message board discussions in the wake of dumb politicians pushing more “age verification” bullshit on us all, it’s become abundantly clear to me that the phrase “threat model” is a foreign concept to most people.

Except, y’know, as a buzzword.

Art by Embyr.For context, this was commissioned during the era of anti-vaccine losers claiming to “do their own research” briefly co-opting the word “threat model” as a buzz word.I just still find it kind of funny even without this context.

To be up front: If you’re here looking for an academic resource with over 100 citations on how to write a formal threat model document for your new startup which involves multiple blockchains, this probably isn’t the gay furry blog for you. Maybe start with STRIDE and system theory.

But if you’re looking to build an intuition for what questions a good threat model should answer, and you’re starting from zero, you’re probably in the right place.

So let’s talk about threat modeling.

Threat Modeling For Neophytes

Their name is Neophyte, if you didn’t get the joke.Art: Harubaki

At a high level, don’t overthink this too much.

While a threat model is a formal cybersecurity process that some infosec folks actually specialize in, you can run informal threat models in the design and architecture phases of developing a new product or service and no one can stop you. You might just end up with a better result.

A threat model should, at minimum, answer these basic questions:

What are we even protecting to begin with?

If you can’t answer this, you have a lot of ground work to do.

Who/what wants to harm what we’re protecting?

Hackers, activists, cyber-stalkers, social media harassment networks

Natural disasters / bad karma

Underpaid and overworked employees who get fed up

Idiotic legislatures paid by large corporate lobbyists to pass stupid laws that hurt everyone

Nation State Adversaries!!!!1oneon

How might (2) attack (1)?

Attack scenarios go here

Murphy’s Law goes here

What will we do to prevent (3) from happening?

Murphy’s Law also goes here!

And, like, okay. If you can check those off, you can call your document a threat model in some sense.

However, this is often useless in practice because some crucial details are omitted.

How are the assets (1) related / connected?

Think in graphs, not lists.

Not all targets are equal value.

What assumptions are we making, especially with (4) and (5)?

I’ll say more about this below.

What threats are we deliberately not addressing?

You literally cannot address every possible attack that any person will ever imagine in the unforeseeable future, so don’t pretend to.

Too many people take assumptions (6) for granted, ironically, but it’s incredibly important to be as clear about what your assumptions are.

If one of your assumptions is wrong, then your model is incomplete (at best), or your list of accepted risks (7) needs to be reconsidered.

For example: The Invisible Salamanders attack breaks abuse reporting in some end-to-end encrypted messaging designs, but only if you introduce abuse reporting.

The attack is possible because one of the assumptions that went into the AEAD schemes in question (AES-GCM, ChaCha20-Poly1305) is that there is only one valid key for a given message. The second you introduce multiple valid keys for a given message (or confused deputies for that matter), you’ve gone outside the security guarantees of your algorithm–which, as an attacker, makes for a fun trick.

Being clear about your assumptions allows you to identify your own unknown unknowns. You don’t have to be perfect.

In fact: Threat models are supposed to be living documents, not point-in-time snapshots. Update them whenever you deem appropriate.

How to Get Started

If you’re looking to do threat modeling professionally, you probably want to read the Threat Modeling Manifesto. Here’s how I approach it.

First, write down the 7 items above in a format that you can vaguely copy and paste rapidly. You’re going to need it.

Next, map out (on a large piece of graph paper, preferably–or the digital equivalent) the components of the system you’re designing or analyzing.

If any widget directly talks to, depends on, or interacts with another widget, you need that relationship drawn in whatever convention is most useful to you.

Once this is setup, you want to draw a box around the entire graph, and then pretend you’re playing Fortnite: Every so often, the box gets smaller and focused more on each individual component. Each iteration, note all the inputs and outputs to each component, and try to answer as many of the 7 items as you can.

Repeat until you’ve drilled down as far as your abstraction allows you, then brainstorm what assumptions you have about the layers you aren’t drilling deeper into.

What you’re doing is starting at the highest level and working your way down into more specific pieces.

Your database probably doesn’t depend on the security of X25519 the same way that your load balancer does. But your database also probably shouldn’t have an RSS feed built into it either. Take note of inappropriate relationships and aim to sever them if you can.

Example: My Own Work

You may or may not already be aware that I’m working on delivering key transparency to the Fediverse. The work is being tracked on publickey.directory if you’re curious about the state of it after this blog post goes live.

This work began with a specification, which includes a prominently featured threat model.

The threat model is organized into the following sections:

Assumptions (stated up front)

Assets

Actors (both attackers and people we want to protect), given role names

The risks, which have one of four statuses attached

Prevented by design: Attack simply won’t work lol 😀

Mitigated: Attacks shouldn’t succeed, unless an assumption is wrong. Most interesting for researchers to focus on.

Addressable: There’s a way to mitigate the risk, but it requires effort or care. Operators should be aware of this.

Open: This is a risk we cannot or will not mitigate. These are the attacks that will succeed.

This threat model isn’t perfect, of course. I didn’t perfectly relate the assets and actors to each other in a human-readable graph. There might be blind spots in the risks section I never considered. I might have forgotten to write down some assumption that matters for the security of the system.

If you can look at my project’s threat model and see its shortcomings, you probably understand the assignment well enough to write your own.

But enough thinly-veiled shameless self-promotion. You won’t learn much only looking at my example of fur-in-the game. We also need an example of a bad threat model doc, and boy howdy do I have one ready.

Bad Example: Matrix’s Threat Model

I’ve picked on Matrix before (twice), so if you’ve read those blogs, this won’t be news to you.

This is Matrix’s threat model (latest, v1.18):

9. Security Threat Model

9.1 Denial of Service

The attacker could attempt to prevent delivery of messages to or from the victim in order to:

Disrupt service or marketing campaign of a commercial competitor.

Censor a discussion or censor a participant in a discussion.

Perform general vandalism.

9.1.1 Threat: Resource Exhaustion

An attacker could cause the victim’s server to exhaust a particular resource (e.g. open TCP connections, CPU, memory, disk storage)

9.1.2 Threat: Unrecoverable Consistency Violations

An attacker could send messages which created an unrecoverable “split-brain” state in the cluster such that the victim’s servers could no longer derive a consistent view of the chatroom state.

9.1.3 Threat: Bad History

An attacker could convince the victim to accept invalid messages which the victim would then include in their view of the chatroom history. Other servers in the chatroom would reject the invalid messages and potentially reject the victims messages as well since they depended on the invalid messages.

9.1.4 Threat: Block Network Traffic

An attacker could try to firewall traffic between the victim’s server and some or all of the other servers in the chatroom.

9.1.5 Threat: High Volume of Messages

An attacker could send large volumes of messages to a chatroom with the victim making the chatroom unusable.

9.1.6 Threat: Banning users without necessary authorisation

An attacker could attempt to ban a user from a chatroom without the necessary authorisation.

9.2 Spoofing

An attacker could try to send a message claiming to be from the victim without the victim having sent the message in order to:

Impersonate the victim while performing illicit activity.

Obtain privileges of the victim.

9.2.1 Threat: Altering Message Contents

An attacker could try to alter the contents of an existing message from the victim.

9.2.2 Threat: Fake Message “origin” Field

An attacker could try to send a new message purporting to be from the victim with a phony “origin” field.

9.3 Spamming

The attacker could try to send a high volume of solicited or unsolicited messages to the victim in order to:

Find victims for scams.

Market unwanted products.

9.3.1 Threat: Unsolicited Messages

An attacker could try to send messages to victims who do not wish to receive them.

9.3.2 Threat: Abusive Messages

An attacker could send abusive or threatening messages to the victim

9.4 Spying

The attacker could try to access message contents or metadata for messages sent by the victim or to the victim that were not intended to reach the attacker in order to:

Gain sensitive personal or commercial information.

Impersonate the victim using credentials contained in the messages. (e.g. password reset messages)

Discover who the victim was talking to and when.

9.4.1 Threat: Disclosure during Transmission

An attacker could try to expose the message contents or metadata during transmission between the servers.

9.4.2 Threat: Disclosure to Servers Outside Chatroom

An attacker could try to convince servers within a chatroom to send messages to a server it controls that was not authorised to be within the chatroom.

9.4.3 Threat: Disclosure to Servers Within Chatroom

An attacker could take control of a server within a chatroom to expose message contents or metadata for messages in that room.

(Yes, I excerpted the whole section in the scrolling box above.)

A few things you might notice, scrolling through this:

This is just a list of different attack types.

There is no list of assumptions.

There is no list of assets, nor their relationships to other assets.

The list of attacks is woefully incomplete.