Pages

Showing posts with label Boston Matrix for Quick Triage of Use Cases Based on Risk Exposure. Show all posts
Showing posts with label Boston Matrix for Quick Triage of Use Cases Based on Risk Exposure. Show all posts

Tuesday, January 5, 2016

Boston Matrix for Quick Triage of Use Cases Based on Risk Exposure - Part II

In my book, Use Case Levels of Test, I look at an an extension of the product's operational profile to provide a quantitative way to compare risk exposure across all the use cases of a use case diagram. In the last post I discussed a quick and easy, less formal way to triage a set of use cases in terms of risk that works great for example in test team planning workshops using the white board: The Boston Matrix.

In this post I'll work through an example, evaluating the risk exposure of the use cases for a new public library book management system, an example that runs throughout my book.



Use case diagram for library management system
Use case diagram for library management system
To the left is the use case diagram for which I'll be building a Boston Matrix to assess the risk of each use case. In alphabetical order the use cases are (a fuller description of each is given in the book):

  • Add Books to Library Database
  • Cancel Book Reservations
  • Do Book Search
  • Lend Books
  • Manage Book Transfers
  • Remove Books from Library Database
  • Reserve Books
  • Return Books



 

Working Through the Matrix: Team Alignment

There's no one right way to fill  out a Boston Matrix, especially in a workshop setting. But here's one suggestion, and how I'll proceed here. Pick one of the two axis -- frequency of use, or severity of failure -- then have your team start by splitting the set of use cases based on that single criterion. In the example below, I've chosen to split the use cases based on frequency of use.

Start of Boston Matrix for analysis of use case risk exposure
Start of Boston Matrix for analysis of use case risk exposure
 You may well look at how I've split the use cases based on frequency of use and say, "No way, that doesn't look right!". And such disagreements in a workshop setting are the basis for discussion and discovery. As I noted in the previous post, the process of having your team work through the Boston Matrix to triage the use cases will undoubtedly lead to "Ah, Ha!" moments of discovery and team alignment. As discussions arise, be prepared to capture the issues and questions that arise.

To illustrate, here's a dialogue between team members typical of those that might arise in a workshop:
Joe: "No way, that split doesn't look right!"
Sue: "Why do you say that?"
Joe: "Because XYZ .."
Sue: "Oh, well I didn't know you were assuming XYZ .. is that a valid assumption?"
Joe: "Well I think so because I spoke with the customer last week and they said ABC .."
Sue: "Ah, Ha!, well, I'm glad you brought that up, I didn't know that; it's good we all know that now!"
The team has just reached alignment on what was previously an unshared assumption!

 

Determining Severity of Failure

Having split the use cases into two groups based on frequency of use, I'll next split the use cases into two groups based on severity of failure. Again this is just how I'm working the example; in a workshop setting there are any number of ways you could have your team approach this, for example using a group decision making technique such as dot-voting / multi-voting tailored to the Boston Matrix.

You may find the severity of failure harder to pin down than frequency of use. Here are some factors to consider. First, there is the matter of the unit of measure for severity. Common units of measure for severity are cost, lost time (e.g. system downtime), and for safety-critical systems, deaths and/or injuries.

And given any one of these units of measure – cost, lost time, etc.. – you have to also decide what it is that needs to be measured. For example for cost, is it the cost to repair a failure; or the cost of lost revenue due to a failure; perhaps both?

Finally, as Musa et al [1]  point out, the severity of failure depends a lot on whose perspective you choose to measure it from. A defect / failure that is relatively inexpensive to correct from a development standpoint can be catastrophic to a customer, and vice versa.

Having a team work through questions such as these will be a fruitful exercise in identifying issues, and getting the team on the same page.

 

Finalizing the Matrix

The completed Boston Matrix is shown below. Having triaged the use cases by risk exposure, I now have a better idea of the biggest bang for the buck in terms in terms of test design, and test execution: Lend Books and Return Books being the riskiest of the bunch; frequently used, with high severity of failure.

What, you don't agree? Well then, we have just identified a misalignment in our priorities for test design. See? The process works great!

A Boston Matrix for analyzing risk exposure of use cases of library management system
A Boston Matrix for analyzing risk exposure of use cases of library management system


[1] John Musa, Anthony Iannino, and Kazuhira Okumoto, Software Reliability: Measurement, Prediction, Application. McGraw-Hill, 1990



Tuesday, December 29, 2015

Boston Matrix for Quick Triage of Use Cases Based on Risk Exposure


In the last post I talked about how to apply the concept of risk exposure to the data of your system or application. The risk exposure of an event is the likelihood that the event will actually happen, multiplied by the expected severity of the event. Each time your customer runs a use case, there is some chance that they will encounter a defect in the product. For use cases, failure is the "event" whose risk exposure we'd like to know. What you’d like to have is a way to compare the risk exposure from use case to use case so you can work smarter, planning to spend time making those use cases with the highest risk exposure more reliable.

In my book, Use Case Levels of Test, I look at an an extension of the product's operational profile to provide a quantitative way to compare risk exposure across all the use cases of a use case diagram. But in this post we'll look at a quick and easy way to do this that works great for example in test team planning workshops using the white board.

 

The Boston Matrix

The Boston Matrix is said to have been created by the Boston Consulting Group (hence the name) as a tool for product portfolio analysis.  It was originally presented as a way to categorize a company's product portfolio into four quadrants based on two scales: current market share and potential future market growth. The idea is to identify your product "stars" (large market share with potential for even more growth), "dogs" (low market share, not much potential for growth), etc. It's a way to triage a product portfolio for future development (spend less on dogs, more on stars).

But the Boston Matrix is a great tool for quickly categorizing any set of things. Below is a Boston Matrix set up for triaging use cases based on their risk exposure, with the horizontal axis representing how frequently you expect a use case to be used, and the vertical axis representing the severity of failures in a use case.

A Boston Matrix for analyzing risk exposure of use cases
A Boston Matrix for analyzing risk exposure
The idea is to assign each use case for which you need to design tests into one of four quadrants. The upper right quadrant represents those use cases that are both frequently used, with high severity failure and hence have the biggest bang for the buck in terms of making sure they are thoroughly tested and reliable. Conversely the lower left quadrant represents use cases that are infrequently used, and if they fail, the severity is low. When test design time is at a premium these use cases are the ones to test lighter.

As I noted above, this is a great tool for test team planning workshops. As with any workshop, an important part of working through assigning use cases to each quadrant will be the “Ah, Ha!” moments that occur. As a test designer, be prepared to capture ideas, issues, assumptions, notes and questions that will inevitably arise.

In the next post we'll work through an example, evaluating the risk exposure of the use cases for a new public library book management system.

Pages