Threat Modeling, part 1

One of the requirements for designing software at Microsoft these days is that every (and I do mean EVERY) feature in every product needs to have a threat model.

Threat modeling isn't new, it's been a part of good design for years.  Heck, the IETF has a threat model for the entire internet :) 

Note that I didn't say good security design, IMHO, threat modeling isn't necessarily about security design.  Threat modeling's really all about understanding your design, at a level that design documents don't necessarily cover.  

Threat modeling is almost more of a discipline than an analysis tool- if it's done right, it enables you to gain a much deeper understanding of your components and how they interact.  And by applying that methodology, you learn a great deal about your component.

We're currently running through the threat modeling process in my group.  Since I'm the security nut on my team, I'm the one who volunteered to run the process.  And it's really been an interesting exercise..

The big thing that writing the threat model for your feature (or component, or protocol, or whatever) is that it really forces you to understand the interactions between the various pieces of your feature.

When threat modeling, you need to concentrate on a bunch of things.  First, there's your assets (also known as protected resources).  All threats relate to assets that need to be protected.  If you don't have assets, you don't have threats.  But sometimes the things that are your assets can be quite subtle.  For example, the audio samples being sent to a sound card might be an asset (especially if the audio samples came from DRM'ed files).  So might the privileges of the LocalSystem (or the current user) account.  The contents of a file on the disk might be an asset for a filesystem, similarly, the contents of an email message is an asset for an email system (like Exchange).  Understanding your assets is key - assets tend are attacked in different ways, the attack strategies mounted against an email message probably won't work for transient data like an audio sample.

The next thing you need to determine is the entry points to your components.  The entry points to your components are what the attacker is going to use to gain access to your component.

Related to entry points, one other thing that you need to determine are the trust boundaries in your component.  A trust boundary is a boundary (physical or virtual) across which there is a varied level of trust.  For example, when you RPC from one machine to another (or one process to another), that forms a trust boundary.  Another example of a trust boundary is the network - almost by definition, "the wire" is a trust boundary.

For some systems, there are no real trust boundaries - an interactive application running as the user that only outputs data may have no trust boundaries.  Similarly, an API that processes data and returns it to the caller may have no trust boundaries.

Related to trust boundaries are trust levels - trust levels indicates how much you "trust" a portion of the system.  For instance, if a user is an administrator, they are trusted to do more than normal users.  When data flows between entities whose trust level is different, by definition, there is a trust boundary between those two entities.

Once you've identified your entry points, assets, trust boundaries, and trust levels, the next major part of a threat model is the data flow diagrams.  A data flow diagram indicates the flow of data into and out of the various parts of your components.

All of this is the up-front work.  It lays the foundation for the "meat" of the threat model, which, of course are the threats.  I'll write more about threats tomorrow..

One of the things that I realized while writing down all the stuff above is that getting all this stuff written down provides a really cool backup for your design documents.  Much of the time, design documents don't necessarily include all the interactions of the various pieces of the system (often they do, but...).  Threat modeling forces you to write those aspects of the design down.  It also forces you to think about (and write down on paper) the design of your component in terms of the DATA manipulated by your component, instead of the CODE that makes up your component.

 

Btw, Frank Swiderski and Window Snyder have an excellent book on threat modeling, it's a bit dry reading, but it's really invaluable for learning about the process.  Microsoft has also provided a threat modeling tool (written by Frank) here that can be used to help guide you through the process of developing the threat model.  Microsoft's been working hard internally at making the threat modeling process require less effort - the ultimate goal of the team is "Do the DFD, turn the crank!".

There are also lots of fascinating resources on the web available, including Dan Epp's article here, and Krishnan Ranganathan's threat model checklist here.  In addition, Michael Howard's written a bunch about TM, here and here.

Edit: Included some additional information and links for Michael Howard's TM articles.

Edit2: Corrected possesive on Dana Epp's name :)

Comments

• Anonymous
  January 13, 2005
  What level of granularity is a "feature" defined at? Is it every exposed function in a library? Is it every button in a GUI? How do you relate those to a class's private data and public member functions? Does it change for every type of programming being done?

• Anonymous
  January 13, 2005
  I've been on the road for the last year teaching a workshop on .NET security programming best practices to ISVs.
  In the first part of the workshop, I spend a good 4 hours reviewing Thread Modelling and even demo Frank's tool.
  What's amazing is the collective yawn of cynisism that spreads through the entire room. It's gotten so bad that I now save these discussions for the end of the workshop, and I've reduced my TM content.
  What I have discovered is that while most programmers and architects are whole-heartedly behind threat modelling, most project managers and business managers don't understand why it is necessary. Most developer project managers believe that security is up to the "networking" folks and don't budget the time up front for security threat modelling. As a result, it never gets done.
  
  I think selling threat modelling to programmers is the wrong audience.
  It seems that we need to be evengelising threat modeling in MSF to project management folks rather then programmers.

• Anonymous
  January 13, 2005
  Good post! I'm trying to get people to understand that TM should be part of any design. Hey, even really small designs would benefit from a TM. For some reason people think that any TM would take months to complete and involve a whole team of security people. I guess my view is that you only need a small TM for a small design. Hey, surely a small TM is better than none! =)
  
  Thanks for the links!

• Anonymous
  January 13, 2005
  The comment has been removed

• Anonymous
  January 13, 2005
  The comment has been removed

• Anonymous
  January 13, 2005
  the band plays in dealing truth to the liars & criminals universally.

• Anonymous
  January 14, 2005
  The comment has been removed

• Anonymous
  January 14, 2005
  Absolutely Jeremy, you're totally correct - I tend to get caught up in my world without seeing the bigger picture.
  

• Anonymous
  January 26, 2005
  MORE PICTURES!!!!!!

• Anonymous
  February 27, 2006
  Way back in the day, back when we were doing the initial threat models for the vista audio stack, I wrote...

• Anonymous
  August 30, 2007
  About 2.5 years ago, I wrote a series of articles about how we threat model at Microsoft, about 18 months

• Anonymous
  June 09, 2009
  PingBack from http://cellulitecreamsite.info/story.php?id=8825