The Failure of Risk Management. Douglas W. Hubbard

Читать онлайн книгу.

The Failure of Risk Management - Douglas W. Hubbard


Скачать книгу
developed with rigorous scientific testing or mathematical proofs. So we can't be certain that everyone answering the surveys identified in chapter 2 is really using a valid standard to rate his or her success. But even if risk managers had some uniform type of professional quality assurance, surveys of risk managers would still not be a valid measure of risk management effectiveness. That would be like measuring the effectiveness of aspirin by a survey of family practice doctors instead of a clinical trial. What we need are objective measures of the success of risk management.

      Recall from chapter 1 that risk can be measured by the probability of an event and its severity. If we get to watch an event over a long period of time then we could say something about how frequent the event is and the range of possible impacts. If a large retailer is trying to reduce the risk of loss due to shoplifting (an event that may occur more than a hundred times per month per store), then one inventory before the improved security efforts and another a month after would suffice to detect a change. But a risk manager isn't usually concerned with very high-frequency and low-cost events such as shoplifting.

      In a retailer such as Target or Walmart, theft should be so common that it becomes more of a fully anticipated cost than a risk. Similarly, the “risks” of running out of 60W incandescent bulbs or mislabeling a price on a single item are, correctly, not usually the types of risks we think of as foremost in the minds of risk managers. The biggest risks tend to be those things that are more rare but potentially disastrous—perhaps even events that have not yet occurred in this organization.

      If it is a rare event (such as many of the more serious risks organizations would hope to model) then we need a very long period of time to observe how frequent and impactful the event may be—given we can survive long enough after observing enough of these events. Suppose, for example, a major initiative is undertaken by the retailer's IT department to make point-of-sale and inventory management systems more reliable. If the chance of these systems being down for an hour or more were reduced from 10 percent per year to 5 percent per year, how would they know just by looking at the first year? And if they did happen to observe one event and the estimated cost of that event was $5 million, how do we use that to estimate the range of possible losses?

       The big experiment

       Direct evidence of cause and effect

       Component testing

       Formal errors

       A check of completeness

       Answering the right question

      The Big Experiment

      The most convincing way—and the hardest way—to measure the effectiveness of risk management is with a large-scale experiment over a long period tracking dozens or hundreds of organizations. This is still time-consuming—for example, waiting for the risk event to occur in your own organization—but it has the advantage of looking at a larger population of firms in a formal study. If risk management is supposed to, for example, reduce the risk of events that are so rare that actual results alone would be insufficient to draw conclusions, then we can't just use the short-term history of one organization. Even if improved risk management has a significant effect on reducing losses from various risks, it may take a large number of samples to be confident that the risk management is working.

      Of course, it would seem unethical to subject consumers to an experiment with potentially dangerous health effects just to test different risk management methods. (Patients in drug trials are at least volunteers.) But if you could conduct a study similar to what was just described, the results would be fairly good evidence that one risk management method was much better than the other. If we did the math (which I will describe more later on as well as show an example on the website www.howtomeasureanything.com/riskmanagement) we would find that it would be unlikely for this result to be pure chance if, in fact, the probability of the events were not different. In both groups, there were companies that experienced unfortunate events and those that did not, so we can infer something about the performance of the methods only by looking at the aggregation of all their experiences.

      Again, this is the hard way to measure risk management methods. The best case for organizations would be to rely on research done by others instead of conducting their own studies—assuming they find the relevant study. Or, similar to the insurance industry study, the data are all historical and are available if you have the will to dig all of it up. Fortunately, there are alternative methods of measurement.

      Direct


Скачать книгу