Randomised phase III studies should be designed to find out whether a new drug or treatment makes a meaningful difference in patients' survival or quality of life, according to a commentary published online December 3rd in The Journal of the National Cancer Institute. Instead, most trials now are designed to detect a statistically significant difference between treatment and control groups, which may not be clinically meaningful, write Dr Alberto Ocana, and Dr Ian F. Tannock of Princess Margaret Hospital in Toronto.

Regulatory agencies such as the US Food and Drug Administration (FDA) and the European Medicines Agency (EMEA) approve drugs usually based on statistically significant results of randomised phase III trials comparing a new, investigational drug with standard treatment. Ocana and Tannock note that pharmaceutical companies have typically sponsored clinical trials that are large enough to detect statistically significant differences in survival. But these differences are often trivial, they say. For instance, the trial that led to approval of erlotinib (Tarceva) for pancreatic cancer found that patients who took the drug had a median survival just 10 days longer than patients in the control group. However, the difference was statistically significant, and the drug was approved.

The authors write that pharmaceutical companies look for a difference in survival outcome between two groups of a trial that is "not usually the minimal difference in overall survival or progression-free survival that is clinically important, but more likely the minimal difference that is feasible to detect, considering the limits on the sample size and hence the cost of the trial."

The authors argue for another approach: "Ideally, trials should be designed [to detect] the minimum clinically important difference, taking into account the tolerability and toxicity of the new treatment, that would persuade oncologists to adopt the news treatment in place of the standard treatment" and investigators should try to reach at least a clinically important difference that was specified in the protocol.

They also suggest that the FDA and EMEA "should define what constitute a positive trial based on the concept of establishing a meaningful clinical benefit for patients similar to those included in any given trial."

In an accompanying editorial, Dr J. Jack Lee of the M.D. Anderson Cancer Center, writes that this "excellent commentary" calls for a new drug approval paradigm and challenges both the medical and statistical communities to find a better way to assessing whether a drug really works. Lee goes on to argue for the adoption of the Bayesian approach in contrast to the more conventional frequentist approach. "Statistics in medicine has passed through its infancy and childhood. As it moves into its adolescence, the growing pains of reconciling frequentist and Bayesian views continue," he writes. In his view, though, the "roadblocks" of the Bayesian approach, namely the notion of subjectivity and computation difficulty, have been overcome.

"The Bayesian approach is complementary to and can provide a superior alternative to the frequentist paradigm," Lee writes. "I encourage medical researchers to have an open mind, learn more about Bayesian methods, and apply them to provide a more accurate statistical assessment of the results in clinical trials."

Source: The Journal of the National Cancer Institute