The Failure of Risk Management. Douglas W. Hubbard

Читать онлайн книгу.

The Failure of Risk Management - Douglas W. Hubbard


Скачать книгу
on a scale of 1 to 5. For example, if the manufacturer already produces a similar but not identical drug, it might get a low risk score of 2 on the indicator called proven technical proficiency. If it was inspected by and got a positive evaluation from the Chinese health agency, but was not yet inspected by the Food and Drug Administration, then it might get a 4 on the formal inspections indicator.

      Then these scores were each multiplied by a weight of somewhere between 0.1 and 1.0 and then all of the weighted scores were totaled. The total of the weighted score might be 17.5 for one outsourcing strategy, 21.2 for another, and so on. The team that chose the scores also chose the weights and, again, it was based only on subjective judgments. The team further separated the resulting scores into various stratifications of risk that would, apparently, have some bearing on the decision to use a particular China-based source for a drug. For example, risk scores of over 20 might mean “extremely high risk: Find an alternative”; 10 to 19 might mean “high risk: Proceed only with increased quality assurance”; and so on.

      When the presenter had finished, I was expected to provide my two cents on the method. I decided I could neither endorse nor reject the approach outright. To be perfectly fair, neither position could yet be positively justified at that point without knowing a few more details (although there is a good chance it shared the flaws of many weighted scores, which I discuss later). I simply asked, “How do you know it works?” This is the most important question we could ask about a risk analysis and risk management approach. Once I knew the answer to that question, then I could legitimately take a position.

      Then I pointed out that in the design of processes in drug production, once they had thoroughly reviewed the literature on a topic, no doubt they would design empirical tests of various components in the process and measure them in a way that would satisfy the peer-reviewed journals and the FDA inspectors alike. Again, this same philosophy can apply to risk.

      In fact, a much more sophisticated method is often already used to assess a different risk in the drug industry. Stop-gate analysis (also variously referred to as phase-gate and stage-gate analysis) is used to determine whether a candidate for a new product should advance from formulation to animal testing, then from animal testing to human trials, until finally the company decides whether to go to market. Many drug companies use proven statistical methods at each step in the stop-gate analysis. But, somehow, none of the basic concepts of stop-gate analysis were built on to assess the risks of outsourcing production to China.

      I was already fairly sure that they had no objective measure for the effectiveness of this method. If they had known to create such measures, they would probably have been inclined to create a very different approach in the first place. When it came to designing a method for assessing and managing risks, these scientists and engineers developed an approach with no scientific rigor behind it. Although the lack of such rigor would be considered negligent in most of their work, it was acceptable to use a risk assessment method with no scientific backing at all.

      The presenter and the audience felt that the weighted scoring method they described was something close to “best practices” for the industry. When I asked, nobody in the room claimed to have an approach that was any more sophisticated. Most had no risk analysis at all for this problem.

      Fortunately for the company that was presenting its risk management solution, it had not yet seen the worst-case scenarios that might result from unsound risk analysis. But with an entire industry approaching the outsourcing problem with either unscientific risk analysis methods or none at all, the worst case was inevitable. Just a few months after the conference, another major drug company using similarly subjective risk management methods on this problem would discover exactly how much was being risked by the outsourcing decisions (and the meager risk analysis applied to it).

      Baxter International, Inc. was receiving reports of dangerous adverse reactions to its Chinese-manufactured blood-thinning drug called heparin. To its credit, by mid-January 2008, Baxter had voluntarily recalled some lots of the multidose vials of the drug. By then, the FDA was considering a mandatory recall but had not yet done so because they believed other suppliers might not be able to meet demand for this critical drug. The FDA reasoned that this additional risk to patients requiring heparin therapy would be higher. (I have no idea how much risk analysis went into that decision.)

      The risks of outsourcing drug production to China always were high, and the fact that some firms were at least attempting to develop a risk management method—regardless of its effectiveness—indicates that the industry was at least aware of the risk. The FDA is entrusted to inspect the operations of any drug manufacturer selling products in the United States, including foreign-based factories but, by March 2008, the FDA had inspected just 16 of the 566 Chinese drug manufacturers. Most drugs used in the United States are now produced overseas and most of those are from China. The scale of the problem easily justifies the very best risk analysis available.

      Obviously, we can't be certain with only this information that the industry's lack of more sophisticated risk management for overseas drug manufacturing was the direct cause of the heparin incident. If the industry had used more sophisticated methods, such as it already uses for stop-gate analysis, we could not be certain that some similar problem would not still have occurred. And, because the entire industry was unsophisticated in this area of risk management, there is certainly no reason to single out Baxter as a particularly bad example. This anecdote, by definition, is merely a single sample of the types of events that can occur and, by itself, is not sufficient to draw scientifically justified conclusions.

      For any risk management method used in the pharmaceutical industry or any other industry, we must ask, again, “How do we know it works?” If we can't answer that question, then our most important risk management strategy should be to find a way to answer it and adopt a risk assessment and risk mitigation method that does work.


Скачать книгу