Facts
This is a short introduction to BSIMM. For the full and unexpurgated model in all its glory, see http://www.bsimm2.com.
What is BSIMM?
BSIMM (pronounced “bee simm”) is short for Building Security In Maturity Model. The BSIMM is a study of real-world software security initiatives organized so that you can determine where you stand with your software security initiative and how to evolve your efforts over time.
Why software security?
Software security is about building software to be secure even when it is under attack. As we have learned from years of practicing network security, protecting software is much easier if the software is built with security in mind. Furthermore, security is a property and not a thing, so software security involves much more than simply adding security features like passwords or crypto to software.
Who needs that?
Organizations that depend on software to work (and that's pretty much everybody these days) need software that doesn't leak millions of identity records, call election results into question, gin up huge legal liabilities, or allow secrets to fall into the wrong hands. The only way to make software trustworthy is to build security in. In short, everyone who relies on software needs the BSIMM.
Aren’t there lots of maturity models out there? What makes BSIMM different?
We built the BSIMM entirely from observations we made by studying real software security initiatives. The BSIMM does not tell you what you should do; instead, it tells you what everyone else is actually doing. This approach stands in sharp contrast to “faith-based” approaches to software security.
Who did you study?
BSIMM2 describes the software security initiatives at thirty well-known companies. The full public list of participants is here. BSIMM2 represents the work of 635 people whose firms have a collective 130 years of experience working on software security. On average, the target organizations have practiced software security for four years and five months (with the newest initiative being three months old and the oldest initiative being fourteen years old in September 2009). We are immensely grateful to everyone who participates in the BSIMM, including those who can't be named. Thanks guys.
Who’s actually responsible for this stuff inside these companies?
The thirty executives in charge of the software security initiatives we studied have a variety of titles, including: Director of IT Security and Risk Management, Director of Application Controls, Product Security Manager, Sr. Manager of Product Security, SVP of Global Risk Management, Global Head of Information Security Markets, and CISO. We observed a fairly wide spread in exactly where the SSG is situated in the firms we studied as well. In particular, 7 SSGs exist in the CIO’s organization, 7 exist in the CTO’s organization, 6 report to CSO’s, 3 exist in either the General Counsel’s office or the Office of Compliance and Risk Management, and 2 report directly to the founder or CEO. Five of the companies we studied did not specify where their SSG fits in the larger organization.
What's in the BSIMM?
BSIMM2 describes 109 activities organized in twelve practices according to the Software Security Framework. During the study, we kept track of how many times each activity was observed in the thirty firms. Here are the resulting data (to interpret individual activities, download a copy of the BSIMM document which carefully describes the 109 activities).
When we describe an activity, we give the objective, a description, and a real example to illustrate how at least one of the organizations made it happen. The example is never the only way to conduct a given activity, but we think it's helpful for understanding software security reality.
So now I have to go do 109 security activities? Sounds like a lot. I hope you're not expecting a big "thank you" at this point.
Never fear! There's probably no organization that needs to carry out all 109 activities. We found a surprising amount of common ground between the financial services organizations, ISVs, and technology companies we studied, but all initiatives are by no means identical, and every organization is at least a little bit different. You wouldn't implement a carbon copy of a friend's financial plan, would you? Don't expect to lift their software security initiative either. Use the BSIMM as a source of ideas and general guidance—as a trail guide rather than as a cookbook.
Why are some activities highlighted?
The fifteen highlighted activities are ones we observed most often. We describe them in some detail in an article titled What Works in Software Security.
I’m more of a visual person. Draw a picture for me?
To give you some idea of analysis capabilities provided by the BSIMM, here are three “spider graphs” showing average maturity level over some number of organizations for the twelve practices. The first graph shows data from all thirty BSIMM firms. The second graph shows data from the top ten firms (as determined by recursive mean score). The third graphs shows data from the financial services vertical (twelve firms) and independent software vendors (seven firms) charted together. These charts are created by noting the highest level of activity in a practice for a given firm, and then averaging those scores for a group of firms, resulting in twelve numbers (one for each practice). The spider chart has twelve spokes corresponding to the twelve practices.
One obvious comparison would be to chart your own maturity high water mark against these averages to see how your program stacks up. Note that in all of these charts, level 3 (outside edge) is considered more mature than level 0 (inside point). The average was computed using the “high water mark” method for assessing level.
So is everybody in the study equally good at software security?
Not by a long shot. By computing a recursive mean score for each firm in the study, we can also take a look at relative maturity and average maturity for one firm against the others. To date, the range of observed scores is [9, 74]. Of particular note to statisticians, recursive mean scores fall into a standard Gaussian distribution. We are pleased that the BSIMM study continues to grow (the data set has tripled since initial publication). Note that we have already reached a sample size of thirty firms, yielding statistically significant analysis capabilities.
We have created balanced scorecards for all thirty BSIMM2 participants and can create the same for other organizations.
Is BSIMM a standard?
BSIMM is not a standard like Control Objectives for Information and related Technology (COBIT) or the Official Rules of Table Tennis (ping-pong). Instead BSIMM describes the set of activities practiced by 30 of the most successful software security initiatives in the world. In that sense, it is a de facto standard because it's what organizations actually do. You could say we discovered it rather than dreamed it up.
What should I do with BSIMM?
If you don't have a software security initiative, you need one. You can use BSIMM to get started. BSIMM can help you figure out how many people you'll need in your software security group, what those people should do first, and what kinds of things they'll probably be thinking about in a few years. If you already have a software security initiative, you can use BSIMM to learn where you stand and make plans for the future.
That's so cool! Who do I pay? How much does this cost?
BSIMM is free. As in, have at it. We've released BSIMM under a creative commons license. You can take it and use it as fodder for your own internal documents or use our data set to make a model of your very own. If you do those things, you are required to tell people where the material came from. In other words, point back to the BSIMM.
Maybe you could shed some light on who "we" is, Mr. first-person-plural?
We are software security experts Gary McGraw, Brian Chess, and Sammy Migues.
What is an SSG? Do I have to have one?
All thirty BSIMM participants have an internal group devoted to software security—the Software Security Group (SSG). We have never observed an organization carrying out the activities in the BSIMM successfully without an SSG. SSG size on average is 21.9 people (smallest 0.5, largest 100, median 13) with a “satellite” of others (developers, architects and people in the organization directly engaged in and promoting software security) of 39.7 people (smallest 0, largest 300, median 11). The average number of developers among our targets was 5061 people (smallest 40, largest 30,000, median 3000), yielding an average percentage of SSG to development of just over 1%.
Though none of the thirty SSGs we examined had exactly the same structure (suggesting that there is no one set way to structure an SSG), there are some commonalities we observed that are worth mentioning. At the highest level of organization, SSGs come in three major flavors: those organized according to technical SDLC duties, those organized by operational duties, and those organized according to internal business units. Some SSGs are highly distributed across a firm, and others are very centralized and policy-oriented. If we look across all of the thirty SSGs in our study, there are several common “subgroups” that are often observed. They are: people dedicated to policy/strategy and metrics; internal “services” groups that (often separately) cover tools, pen testing, and middleware development/shepherding; incidence response groups; groups responsible for training development and delivery; externally-facing marketing/communications groups; and, vendor-control groups. For more about this issue see the informIT article You Really Need an SSG.
In the numbers reported above, we noted that percentage of SSG to development of just over 1% in the thirty organizations we studied. This was a number that we originally uncovered in the initial BSIMM work (a study of nine companies). Much to our surprise it has held steady as we collected data from twenty-one more firms. That means one SSG member for every 100 developers. The largest SSG was 2.6% and the smallest was 0.05%.
If BSIMM is so important, how has the world gotten along without it for so long?
For many software makers (including ISVs, banks, healthcare providers, governments, etc), software security has been a real concern only for ten years or less. The collective "we" are just now reaching the point where enough experience has been accumulated to compare notes and talk about what works at a macro level. Secure programming, penetration testing, and the like have been topics for a while now, but the best methods for organizing humans into software security initiatives take longer to emerge.