The European Association for Cancer Research, 7-10 July 2012, Barcelona, Spain

Constructing successful biomarker clinical trials

Dr Lisa McShane – National Cancer Institute, Bethesda, USA

Dr McShane, you’re a biostatistician from the NCI in Bethesda. You gave a very interesting presentation on, from a statistical point of view, some of the problems with biomarker research. Can you summarise this for us?

Sure. I think that we’ve had a lot of difficulty translating interesting exploratory findings about biomarkers into clinical tests that would actually be useful for managing the care of cancer patients. A lot of the issues really revolve around statistical approaches to the analysis of biomarker data. We’ve grown to learn over the last several decades how to do cancer therapy trials well, things like having randomised controlled clinical trials, and for biomarkers there hasn’t been that same appreciation that you need to have rigorous study designs with adequate sample size; you need to have control groups and this has really led to a lot of confusion in the literature. You’ll find thousands and thousands of biomarker articles published that produce significant p-values, in other words, correlations between a biomarker and some clinical endpoint, but those interesting results never really go that next step, so they turn into tests that can be used for the cancer patients to improve the outcome for cancer patients.

So, some of the things that I discuss in my talk are why it actually can take a larger study to validate the worth of a biomarker for clinical use than for a therapy and this is because there are additional considerations that you have to take into account such as how frequent is the biomarker quote positive. So if you’re developing a new therapy that is going to, you think, work better in patients whose tumours express a particular biomarker, if you have a biomarker that is expressed in 50% of the patients versus 10% of the patients, it will take a very different sized trial to establish the worth of that biomarker. Another thing that people often do is that they think that the way to establish a biomarker as useful for determining who will benefit from a new therapy is that they’ll study only patients who receive the new therapy and they’ll say if the patients who have the biomarker positive get a better outcome, live longer or take a longer time before they recur, they think that that establishes the worth of the biomarker for selecting which patients should receive the therapy. It turns out that you can mislead yourself doing that because it could be that the biomarker is what we call prognostic, meaning that it actually doesn’t matter which therapy the patient gets, that those patients are going to do better. Maybe it’s a biomarker that indicates a patient’s tumour will be more indolent and therefore they’re going to do better regardless. So that would be an example of where it’s important to have the involvement of a statistician to say, you really need to have a control group where the patients are receiving either no systemic therapy or a standard therapy and if you see that biomarker makes the same distinction in that group, we’d say that biomarker is not really necessarily helpful for determining who should get the therapy, it’s just a biomarker that says these patients are going to do better than these other patients.

So those are the kinds of mistakes that people make routinely in biomarker analyses and other types of issues that we’re encountering more and more now are the use of biomarkers that are really high dimensional biomarkers, and by that I mean things like gene expression, microarrays, where you might be measuring expression level of, say, 10,000 genes at one time. What are some of the pitfalls in that kind of research? We’ve gotten very good at building information systems, data systems, to capture  that kind of high dimensional data and we’ve gotten very good at developing computational algorithms to develop mathematical models that would, say, take that large amount of information and predict which patients will do better in general or on a particular therapy. But what people have not appreciated is that all these other statistical design considerations, as I mentioned earlier, things like having proper control groups.

The other downside of having more biomarkers is you actually increase the chance of finding false positives so we have to always be vigilant about that and figure out what are appropriate statistical approaches to use to really sort through the findings to determine what is spurious and what is a real believable finding. So I’ve dealt a lot with that recently too and basically people get into trouble a lot faster when they have high dimensional biomarker data than when they have single biomarkers and we’ve already gotten ourselves into plenty of trouble even with single biomarkers.

So there’s really a lack of people who are trained to think in terms of the biomarker questions instead of the therapy questions and that holds true both for the statisticians, we have many statisticians who are absolutely wonderful at designing therapy trials but the additional issues you have to think about for biomarkers not all statisticians are familiar with. So the types of scientists who do the early biomarker research tend to be different types of scientists who do the early work leading to new therapies. The new therapy work may be done by, say, pharmacologists and basic scientists who understand the mechanisms of actions of drugs whereas the biomarker folks may be pathologists or basic scientists who really understand the cancer biology. I find that often that group of researchers is not accustomed to thinking in the same terms as, say, clinicians who are familiar with running clinical therapy trials. So there’s an education that has to occur on many fronts, including the statisticians, that the design considerations for therapy trials are somewhat different than the design considerations for studies that are hoping to validate the worth of a biomarker. In fact people had made predictions as we particularly were coming up with these high dimensional biomarkers like the gene expression profiling that we would be able to do trials more easily and with smaller numbers of patients when we had all this massive amount of data. In fact, it’s been just the reverse, that it takes more studies to narrow down from the larger number of biomarkers to what’s actually the important set of biomarkers and then to do a trial to confirm the value of a biomarker, typically that kind of trial is larger, sometimes 2-4 times larger than a trial to assess the benefit of a therapy. So some examples of recent trials that were specifically designed to validate the worth of a biomarker based test, in each of these two cases it was the high dimensional marker based on gene expression and microarray data or gene expression data, the one was using a different technology, those trials were like 10,000 patients which is much larger than our typical therapy trials. So I think that there’s a real education process that has to go on, people have to learn to think a little bit differently about how to tackle the biomarker based questions.

Fortunately you’re in NCI NIH so you’re in a very good position to interact with these labs, I think a lot of the biomarker research will be coming out of these labs.

That’s right, so I do interact with a lot of lab people as well as clinicians and even in our statistics group at NCI we specialise. I’m known as one of the biomarker statisticians and we have people who specifically specialise in the therapy trials and it often takes a team of statisticians just to address some of these big trials that are trying to validate the worth of a biomarker test.

Thank you.

Thank you.