Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that enables research into pragmatic trials. It shares clean trial data and ratings using PRECIS-2 permitting multiple and varied meta-epidemiological studies to compare treatment effects estimates across trials that have different levels of pragmatism, as well as other design features.
Background
Pragmatic studies are increasingly recognized as providing real-world evidence for clinical decision making. The term "pragmatic", however, is used inconsistently and its definition and evaluation require further clarification. Pragmatic trials must be designed to inform policy and clinical practice decisions, rather than confirm an hypothesis that is based on a clinical or physiological basis. A pragmatic trial should try to be as close as possible to real-world clinical practices that include recruiting participants, setting, design, delivery and execution of interventions, determination and analysis results, as well as primary analyses. This is a significant distinction from explanation trials (as described by Schwartz and Lellouch1) that are intended to provide a more complete confirmation of an idea.
Truly pragmatic trials should not conceal participants or clinicians. This can result in an overestimation of the effect of treatment. The trials that are pragmatic should also try to attract patients from a variety of health care settings, to ensure that the results can be compared to the real world.
Furthermore the focus of pragmatic trials should be on outcomes that are crucial to patients, like quality of life or functional recovery. This is particularly relevant in trials that require invasive procedures or have potentially harmful adverse impacts. The CRASH trial29, for instance, focused on functional outcomes to compare a 2-page case-report with an electronic system for the monitoring of patients in hospitals suffering from chronic heart failure. In addition, the catheter trial28 focused on urinary tract infections caused by catheters as the primary outcome.
In addition to these features pragmatic trials should reduce the procedures for conducting trials and data collection requirements in order to reduce costs. Finaly, pragmatic trials should aim to make their results as relevant to real-world clinical practice as is possible. This can be accomplished by ensuring their primary analysis is based on an intention-to treat approach (as defined in CONSORT extensions).
Despite these requirements, a number of RCTs with features that challenge pragmatism have been incorrectly self-labeled pragmatic and published in journals of all types. This could lead to false claims about pragmatism, and the usage of the term should be standardised. The creation of the PRECIS-2 tool, which provides an objective and standard assessment of pragmatic features, is a good first step.
Methods
In a pragmatic research study, the goal is to inform policy or clinical decisions by demonstrating how an intervention can be integrated into routine treatment in real-world situations. Explanatory trials test hypotheses concerning the cause-effect relationship within idealised environments. Consequently, pragmatic trials may have less internal validity than explanatory trials and may be more susceptible to bias in their design, conduct and analysis. Despite these limitations, pragmatic trials can provide valuable information to decision-making in the context of healthcare.
The PRECIS-2 tool evaluates the degree of pragmatism in an RCT by assessing it on 9 domains that range from 1 (very explicative) to 5 (very pragmatic). In this study the areas of recruitment, organization, flexibility in delivery, flexible adherence and follow-up received high scores. However, the primary outcome and the method of missing data was scored below the pragmatic limit. This suggests that it is possible to design a trial with good pragmatic features without harming the quality of the results.
It is difficult to determine the level of pragmatism that is present in a trial because pragmatism does not have a single characteristic. Certain aspects of a study may be more pragmatic than others. Furthermore, logistical or protocol modifications during the course of a trial can change its score on pragmatism. Koppenaal and colleagues discovered that 36% of the 89 pragmatic studies were placebo-controlled, or conducted prior to licensing. Most were also single-center. Thus, they are not as common and are only pragmatic when their sponsors are accepting of the absence of blinding in these trials.
Another common aspect of pragmatic trials is that the researchers attempt to make their findings more valuable by studying subgroups of the sample. This can result in imbalanced analyses and less statistical power. This increases the possibility of missing or misdetecting differences in the primary outcomes. This was a problem during the meta-analysis of pragmatic trials due to the fact that secondary outcomes were not corrected for covariates' differences at the baseline.
Furthermore, pragmatic trials can also have challenges with respect to the gathering and interpretation of safety data. It is because adverse events are usually self-reported and are susceptible to delays, errors or coding differences. It is therefore important to enhance the quality of outcomes ascertainment in these trials, in particular by using national registries rather than relying on participants to report adverse events on a trial's own database.
Results
While the definition of pragmatism does not require that all trials are 100 percent pragmatic, there are some advantages to including pragmatic components in clinical trials. These include:
Enhancing sensitivity to issues in the real world as well as reducing cost and size of the study and allowing the study results to be more quickly implemented into clinical practice (by including routine patients). However, pragmatic trials have their disadvantages. The right amount of heterogeneity, like could help a study extend its findings to different patients or settings. However the wrong type of heterogeneity could reduce the sensitivity of an assay and, consequently, lessen the power of a trial to detect minor treatment effects.
Many studies have attempted classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 developed an approach to distinguish between explanatory trials that confirm a clinical or physiological hypothesis and pragmatic trials that inform the choice of appropriate therapies in the real-world clinical setting. The framework was comprised of nine domains, each scored on a scale of 1 to 5, with 1 indicating more explanatory and 5 indicating more practical. The domains included recruitment, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 was based on a similar scale and domains. 프라그마틱 불법 and colleagues10 created an adaptation of the assessment, dubbed the Pragmascope, that was easier to use for systematic reviews. They discovered that pragmatic reviews scored higher across all domains, however they scored lower in the primary analysis domain.
This distinction in the primary analysis domains could be explained by the way that most pragmatic trials approach data. Certain explanatory trials however, do not. The overall score for pragmatic systematic reviews was lower when the domains of management, flexible delivery and following-up were combined.
It is important to remember that the term "pragmatic trial" does not necessarily mean a poor quality trial, and indeed there is an increasing rate of clinical trials (as defined by MEDLINE search, but this is neither sensitive nor specific) that use the term "pragmatic" in their title or abstract. These terms may indicate a greater appreciation of pragmatism in titles and abstracts, but it's unclear whether this is evident in the content.
Conclusions
As the importance of real-world evidence grows popular the pragmatic trial has gained popularity in research. They are clinical trials randomized that evaluate real-world alternatives to care rather than experimental treatments under development. They have patient populations that more closely mirror those treated in routine care, they use comparisons that are commonplace in practice (e.g., existing drugs), and they depend on the self-reporting of participants about outcomes. This method could help overcome the limitations of observational studies which include the biases that arise from relying on volunteers, and the limited availability and the variability of coding in national registry systems.
Pragmatic trials offer other advantages, such as the ability to draw on existing data sources and a higher probability of detecting meaningful differences from traditional trials. However, these tests could still have limitations which undermine their validity and generalizability. The participation rates in certain trials could be lower than expected due to the health-promoting effect, financial incentives or competition from other research studies. Practical trials are often restricted by the need to recruit participants on time. Certain pragmatic trials lack controls to ensure that the observed differences aren't caused by biases that occur during the trial.
The authors of the Pragmatic Free Trial Meta identified RCTs published up to 2022 that self-described themselves as pragmatic. They assessed pragmatism using the PRECIS-2 tool that includes the eligibility criteria for domains as well as recruitment, flexibility in adherence to interventions, and follow-up. They discovered that 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or higher) in at least one of these domains.
Trials with a high pragmatism score tend to have more expansive eligibility criteria than traditional RCTs which have very specific criteria that are unlikely to be found in the clinical setting, and comprise patients from a wide range of hospitals. According to the authors, can make pragmatic trials more relevant and useful in the daily practice. However, they cannot guarantee that a trial is free of bias. The pragmatism is not a fixed attribute the test that does not have all the characteristics of an explicative study can still produce valuable and valid results.