A Matter of Trust

The high stakes and uncertainty involved in trials has led to numerous concerns about how economic incentives may generate conflicts of interests for investigators. But the need for a vaccine to treat SARS-CoV-2 means that the integrity of trials is arguably more important than ever before.

Contrary to previous concerns about the potential for results to be manipulated, a study published in PNAS reports some reassuring findings about the integrity of clinical trials (1). The paper focused on comparing the distribution of results in phase II and III trials registered on ClinicalTrials.gov. “Previous studies that analyzed results published in academic journals showed that results tend to bunch right above the threshold for significance. This led some to question whether investigators were selective in reporting their results,” says study co-author Marco Ottaviani, Professor of Economics at Bocconi University, Italy. “But our study found no glaring evidence of manipulation of results to clear the hurdle.”

Ottaviani’s analysis also revealed subtle patterns across trials. Small companies, when compared with big pharma, were less likely to terminate a project with phase II results under the threshold and were also more likely to report a higher share of significant results in phase III. “Smaller companies are often built around one or two projects – the failure of which could result in their commercial death. Managers of smaller companies, who often hold substantial financial shares of the company themselves are, therefore, more reluctant to give up, even if their phase II results are poor,” says Ottaviani. But with little definitive evidence to support this explanation, the researchers are digging deeper to understand the influence of economic incentives on investigators’ decisions to accurately disclose study results.

Ottaviani believes that asking investigators to make all their results and study protocols public using clinical trial databases is an important step towards transparency. “Even though FDA legislation mandates disclosure for a large number of trials and prescribes fines for noncompliance, to our knowledge, these rules have not been enforced,” he says. “In recent years, however, the industry has become more aware of the importance of transparency in clinical research – motivating many large companies to establish internal disclosure policies.” In his opinion, small companies and public research institutions – seemingly under less scrutiny than their larger counterparts – need to follow suit with tougher sanctions to hold them to account.

Though the Bocconi University researchers believe that improved transparency can positively impact clinical trial integrity, they realize there are still many open questions. “Some analysts argue there are drawbacks to transparency and that imposing disclosure policies could make investigators reluctant to carry out trials, but this has not yet been investigated empirically,” says Ottaviani. “It would be interesting to find out whether there is any truth to this claim.”

Though there is much for the researchers to explore, they are confident that, for the most part, safe and effective vaccines will make it to market for COVID-19. Ottaviani says “When we began our research, we were concerned that drugs and vaccines that do more harm than good could be prematurely approved and distributed to millions of people. We cannot take data integrity for granted, but we can rely on the discipline imposed by ClinicalTrials.gov.”