Defining risk levels in Software Development

Risk can be defined many ways in software development. The team itself can add risk when team members do not communicate effectively or choose poor development methods. The build process can add risk when people submit changes late into the development cycle or the review process for new submissions is ill-defined. Lack-of-prioritization or poor understanding of needs can lead to integration of unnecessary changes. This blog is an attempt to address only the code-writing aspect of risk.

First of all, there is always risk any time you add or modify existing software. Many software developers make the claim "I'll just slip in this minor change and it has no risk of breaking anything." With that statement, begins the long march toward finding the bug, coding, testing, and integration. I've certainly broken many builds in my time as an engineer and giving a clear indication of risk to management is nigh impossible.
Many times, as we approach alpha or beta in a project, we are told to only include changes that are low-risk. Without any real metric, we can only use our best judgment as to what is risky or not and even very good engineers get this wrong. My goal here is to define a simple metric by which to measure risk level.
My metric here is defined in a non-TDD environment. TDD changes these numbers, dramatically lowering the risk levels in most cases. These numbers do make a few assumptions and those are:
* working with an existing code base
* multiple programmers working within this code base (more than three)
* a strong dependency on working builds by the rest of the team
The numbers here are a simple ratings system meant to rate the risk of breaking the build by any given check-in that you may be performing. These are based on my personal experience, a bit of intuition, and in a few cases, generalizing some things that vary a great deal.
Here are my risk levels:
1) Same software, modifications to constants still within range. Level one are always very minor, change no behavior, change no loading, does not include additional data, no new constants, variables, etc. Removing dead code (commented out).
2) Same software but actual lines of code modified. This means the rework of one or two functions to add clarity, reveal a latent bug, or to optimize. Removing unused code, unused member variables, and general cleanup.
3) Same software, but major refactor. Consolidation of some common code into a common function called from different places. Adding another function invocation from another location. Changing of a basic type from int to float or something similar.
4) Minor structural changes. Adding files to a project. Changing a project's settings, adding support for additional platforms, etc. Adding enumerations that will be used. Changing ifs to switches or vice versa. Supporting new conditions in an if-else statement. Adding validation code on startup.
5) New basic behaviors with no large changes to existing behavior. New messages added. New queries added. These new behaviors are not coupled to existing functionality. Integrating new files into a project. Moving functionality into or from a base-class. New GUIs inevitably fall into this category.
6) Behavioral changes. Any concurrency changes. Refactoring a class or a major section of code. Callbacks added when a file-write completes, when a db query finishes, etc. Adding or removing coupling between classes, functions, or some hybrid. Adding virtual function overrides. Replacing existing functions with template functions to support cross-platform or cross-build behaviors. Updating a third-party library (this can vary wildly depending on the vendor).
7) Removing existing functionality that is in use or major new behavior. Refactoring classes into a new hierarchy. Introducing polymorphism where it did not exist before.
8) Rewriting a system. New concurrency model. New DB model. Complete rewrite of existing system. The breaking apart of an existing system into small subsystems. Introducing concurrency or removing it, adding callback or signaling, making software work on a new platform, integrating a new third-party library, introducing a new asset system, removing a legacy coupling from existing systems.
As a simple guideline, you can call anything less than level four low-risk, 4-5 medium, 6-7 high, and 8 very high. Each of these ratings simply speak to the likelyhood of unintended breakage or behaviors including side-effects, knock-ons, and misunderstood requirements.
Low risk items here have almost no chance of breaking the build. Medium items are possible to break the build and usually adding a code review is enough to prevent any serious damage. High risk items are those on which no code review can gauge the probable outcome and even unit testing is likely to miss many bugs. Level 8 risk is guaranteed to break something. In fact level 8 risk rarely can be done without some down time while the project recovers from all of the new bugs introduced.
When giving your final assessment, always consider consider CRI:
Composite Risk Index = Impact of Risk event x Probability of Occurrence
C = l * p. This can mean modifying the above numbers. It is best to consider both Impact which is what my system attempts to gauge, and PoO separately so that others can judge the real risk.