RCTs in Orthopaedics: Where We've Been, Where We Are, and Where We Need To Go
RCTs in Orthopaedics: Where We've Been, Where We Are, and Where We Need To Go
Mohit Bhandari, MD, PhD, FRCSC
Danny Whelan, MD, FRCSC, MSc
University of Toronto
Dr. Whelan discussed the current trends and quality of randomized controlled trials (RCTs) in orthopedic research. Our discussion uncovered 4 core themes. These themes and their supporting insights are highlighted below.
1. The state of RCTs in orthopedics has changed in the last 15 years.
Dr. Whelan compared two articles published by Dr. Bhandari examined the evidence from 1988-2000, and himself along with Dr. Bhandari, examining the evidence from 2000-2015, which evaluated the quality of RCTs published in the Journal of Bone and Joint Surgery. Since 2000, there have been more, higher quality RCTs being published. However, there is a trend towards smaller, single-centre RCTs. This trend towards smaller RCTs may also be impacting results, as fewer published RCTs have shown positive results, which may be due to studies being underpowered.
This is our most recent data where you see there's a gradual increase over time from 1988 to 2013, that randomized trials are increasing and that's great news. We're up to over 40 or 50 or even 60 published in a year in some years in the Journal of Bone and Joint Surgery, our single biggest journal, which I think is really encouraging. The other encouraging thing is that the quality of the randomized trial seems to be increasing as well.
This is where the trends that are a bit disturbing come in, they were less often collaborative. If we look at the 1988 to 2000 era, they were multi-centered in 67 percent of the randomized trials. So, 67 percent of the 72 randomized trials published between 1988 and 2000 were multi-centered and only 30 percent of the 280 plus that were done in 2001 to 2013 were multi-centered. That's really unfortunate when you think about it, these trials are going to be smaller, these trials are definitely going to be smaller if they're not multi-centered and they're not likely to be positive.
2. Do trials being conducted address questions that lead to meaningful change?
Drs. Whelan and Bhandari agree that smaller trials may not address questions that will lead to meaningful change in practice and improvement in patient outcomes. This may also lead to spending resources and money on studies which do not advance science. One hurdle identified is the role of academic advancement, where being the primary investigator on a small, single centre study can help with advancement more than being a collaborator on a large, multicentre trial.
Why are we spending so much money to do conduct studies that ultimately are not moving the needle? It's a really big issue.
It's a huge issue actually that we're not advancing science and I think that's particularly the most important point. But on a, perhaps smaller and more focused level, if we look at what it does to the patient's confidence in us, that patients are being enrolled in trials and perhaps subjected to treatments or subjected to the vagaries of doing research filling out forms, etc. when really when you look at the studies that they're involved in those studies are destined to not be successful in determining an outcome or an appropriate outcome. Moreover, one of the bigger problems we're facing in health care systems is that perhaps payers and people who help to reimburse us for what we do are going to look at say you know, you guys are really not showing us that the treatments you're providing are helping people.
[…] If they were to include another center they wouldn't get the prestige of having done and designed and completed that trial so they were resistant to including other centers because they wanted, they needed that for themselves and the institution wanted the credit for doing the randomized trial.
3. There is a need for larger trials, and more collaboration.
Drs. Whelan and Bhandari discussed that while trials and trial quality are increasing, the next step is to increase collaboration within the community. An increase in collaboration will help to remedy the influx of small, single centre trials and hopefully lead to more impactful research.
[In reference to meta-analysis] […] We are starting to see research moving to the combination of results of case series because there aren't enough randomized trials on many topics. We need more trials. We need more people doing trials so we can actually use meta-analysis more powerfully.
Even within our country there are randomized trials that go on in one center that is 200 kilometers from me and some of the people I work with are resistant to get involved in that because they say, you know ‘well we didn't design that and that's not that's not our work, we should we be able to do something too’. The truth is, well we're not and they're doing a good job of it so why would we not collaborate with that group, and that's the barrier we're having right now.
4. The emergence of big data is a potential new source of better data.
The emergence of big data was discussed, and it was agreed that large registries and data sets have great value. While traditionally these are not viewed as favourably as RCTs, these large datasets, if performed correctly, are potentially more impactful than very small RCTs.
The real problem with small randomized trials is that they can lead to misleading estimates. People are more likely to believe a randomized trial of 25 patients randomized and discount a large observational study of 30 000 patients, saying well -- that's not level one evidence.
Questions and Answers:
Dr. Bhandari: We must align the system to allow surgeons to collaborate, rather than penalize them. The argument is, if you're the lead author in a single center study, even if it's an underpowered small study with the branding of an RCT, it's still much better for your career than participating in transformational trial in which you are part of a group authorship.
Dr. Whelan: And much better for your institution. So, the model of academic advancement that we have right now may be some of what we're seeing in the data that we present.
Dr. Bhandari: What is your take [on big data], is it better in your mindset, you know a 50 000 patient registry where you're doing a series of analytics that are not necessarily hypothesis-driven data collection?
Dr. Whelan: It happens, as you refer to earlier, and it's something that's close to both of us and that we really believe in. There has to be a signal even for the smallest interventions we do, it's just a matter of tuning your receiver to capture it. Across 50 000 patients it doesn't matter how one surgeon does it versus another surgeon, or how active one patient is versus another patient, if you have 50 000 patients that's likely going to wash out in the volume of patients you have and I really think there's great value in registries and in large data collection provided everyone buys in and everybody sees the value in that.
Dr. Whelan summarized the state of RCTs in orthopedics. He highlighted that while there has been an increase in the number and quality of these studies, there is still much work to be done. The increase in number and quality has been met with a trend towards smaller, single centre trials which more often than not show no significant results. This may not be due to there being no difference between the interventions, but potentially due to studies being underpowered. This highlights the need for increased collaboration within the orthopedic community in order to be able to conduct large, multi-centred trials which will be able to tackle questions that can truly advance science in the field and lead to better decision making to improve patient outcomes.
How to Cite:
Danny Whelan. RCTs in Orthopaedics: Where We've Been, Where We Are, and Where We Need To Go. OE Perspectives. 2020; 1(6):1.