Top

28-03-2025 | Commentary

Interpreting the meaningfulness of treatment effects estimated in parallel groups designs: comment on Trigg et al.

Auteur: Kevin Weinfurt

Gepubliceerd in: Quality of Life Research

Abstract

Draft guidance from the U.S. Food and Drug Administration states that one can interpret a treatment effect on a clinical outcome assessment–based endpoint when expressed as some difference between group means. Recently, Trigg et al. examined different approaches for deriving thresholds for interpreting such between-group differences. In this commentary, I make several observations to advance further discussion around this issue. Some key points are (1) rather than “between-group difference,” specify the level at which you wish to infer a treatment effect: population or individual; (2) points of reference may be different for interpreting individual- and population-level treatment effect estimates; (3) who provides input and what types of anchor variables are used to generate points of reference might differ for interpreting individual- versus population-level estimates of treatment effect; and (4) in a parallel groups design, meaningful within-patient change is not especially relevant for understanding the meaningfulness of a treatment effect.

Note that in a parallel groups design, the estimated treatment effect—whether marginal or conditional—will always be a kind of average (known technically as an expectation). For example, a marginal estimate from a linear model provides the expectation for an entire population with the same overall characteristics (demographic, clinical, etc.) as the trial sample. A conditional estimate from a linear model with eight baseline covariates will provide the expected average treatment effect for all patients with a specific combination of values for the covariates.

To avoid the connotation that there is a single, precisely known “threshold,” I will use “point of reference” to refer to the value of a score difference that might be viewed by patients as meaningful.

Technically, the treatment effect is evaluated in terms of a COA-based endpoint, which may or may not be the same as the COA score at a fixed time point or a change in COA score from baseline to a fixed time point. For simplicity here, I am assuming that the COA score is the same as the trial endpoint, but the argument would apply to any type of COA-based endpoint.

This illustration was inspired by Sect. 8.2.11 in Senn [16].

Note that while Trigg et al.’s simple example compared mean change-from-baseline scores between study groups, a real trial analysis would use some model (e.g., ANCOVA) to adjust for the baseline COA score to improve the efficiency of the analysis [17].

U.S. Food and Drug Administration. Patient-Focused Drug Development: Incorporating Clinical Outcome Assessments Into Endpoints For Regulatory Decision-Making. Draft Guidance. April 2023. Available at: https://www.fda.gov/media/166830/download. Accessed March 5, (2025).

Coon, C. D., & Cappelleri, J. C. (2016). Interpreting Change in Scores on Patient-Reported Outcome Instruments. Ther Innov Regul Sci, 50(1), 22–29. https://doi.org/10.1177/2168479015622667CrossRefPubMed

Coon, C. D., & Cook, K. F. (2018). Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores. Quality Of Life Research, 27(1), 33–40. https://doi.org/10.1007/s11136-017-1616-3CrossRefPubMed

Terwee, C. B., Peipert, J. D., Chapman, R., Lai, J. S., Terluin, B., Cella, D., Griffiths, P., & Mokkink, L. B. (2021). Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures. Quality Of Life Research, 30(10), 2729–2754. https://doi.org/10.1007/s11136-021-02925-yCrossRefPubMedPubMedCentral

Trigg, A., Ayasse, N. D., & Coon, C. D. (2025). Conceptualizing meaningful between-group difference in change over time: a demonstration of possible viewpoints. Quality Of Life Research, 34(1), 151–160. https://doi.org/10.1007/s11136-024-03798-7CrossRefPubMed

Weinfurt, K. P. (2019). Clarifying the Meaning of Clinically Meaningful Benefit in Clinical Research: Noticeable Change vs Valuable Change. Journal Of The American Medical Association, 322(24), 2381–2382. https://doi.org/10.1001/jama.2019.18496CrossRefPubMed

Barrett, B., Brown, D., Mundt, M., & Brown, R. (2005). Sufficiently important difference: expanding the framework of clinical significance. Medical Decision Making, 25(3), 250–261. https://doi.org/10.1177/0272989x05276863CrossRefPubMed

Barrett, B. (2013). Sufficiently important difference: concepts, caveats, and challenges. Medical Decision Making, 33(6), 869–874. https://doi.org/10.1177/0272989x13476764CrossRefPubMed

U.S. Food and Drug Administration. Adjusting for Covariates in Randomized Clinical Trials for Drugs and Biological Products. Guidance for Industry. May 2023. Available at: https://www.fda.gov/media/148910/download

10.

Remiro-Azócar, A., Heath, A., & Baio, G. (2021). Conflating marginal and conditional treatment effects: Comments on Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study. Statistics In Medicine, 40(11), 2753–2758. https://doi.org/10.1002/sim.8857CrossRefPubMed

11.

Morris, T. P., Walker, A. S., Williamson, E. J., & White, I. R. (2022). Planning a method for covariate adjustment in individually randomised trials: a practical guide. Trials, 23(1), 328. https://doi.org/10.1186/s13063-022-06097-zCrossRefPubMedPubMedCentral

12.

Wei, J., Xu, J., Bornkamp, B., Lin, R., Tian, H., Xi, D., Zhang, X., Zhao, Z., & Roychoudhury, S. (2024). Conditional and Unconditional Treatment Effects in Randomized Clinical Trials: Estimands, Estimation, and Interpretation. Stat Biopharm Res, 16(3), 371–381. https://doi.org/10.1080/19466315.2023.2292774CrossRef

13.

FDA-NIH Biomarker Working Group (2016). BEST (Biomarkers, EndpointS, and Other Tools) Resource. Available at: https://www.ncbi.nlm.nih.gov/books/NBK326791/. Accessed March 5, 2025.

14.

Abugov, R., Clark, J., Higginbotham, L., Li, F., Nie, L., Reasner, D., Rothmann, M., Yuan, X., & Sharretts, J. (2023). Should responder analyses be conducted on continuous outcomes? Pharmaceutical Statistics, 22(2), 312–327. https://doi.org/10.1002/pst.2273CrossRefPubMed

15.

Cook, K. F., Kallen, M. A., Coon, C. D., Victorson, D., & Miller, D. M. (2017). Idio Scale Judgment: evaluation of a new method for estimating responder thresholds. Quality Of Life Research, 26(11), 2961–2971. https://doi.org/10.1007/s11136-017-1625-2CrossRefPubMed

16.

Senn, S. (2021). Statistical Issues in Drug Development (3rd ed.). Wiley.

17.

Vickers, A. J., & Altman, D. G. (2001). Statistics notes: Analysing controlled trials with baseline and follow up measurements. Bmj, 323(7321), 1123–1124. https://doi.org/10.1136/bmj.323.7321.1123CrossRefPubMedPubMedCentral

Titel: Interpreting the meaningfulness of treatment effects estimated in parallel groups designs: comment on Trigg et al.
Auteur: Kevin Weinfurt
Publicatiedatum: 28-03-2025
Uitgeverij: Springer New York
Gepubliceerd in: Quality of Life Research
Print ISSN: 0962-9343
Elektronisch ISSN: 1573-2649
DOI: https://doi.org/10.1007/s11136-025-03952-9

Bohn Stafleu van Loghum

Welkom bij Erasmus MC & Bohn Stafleu van Loghum

Registreer

Login

Interpreting the meaningfulness of treatment effects estimated in parallel groups designs: comment on Trigg et al.

Abstract

Bohn Stafleu van Loghum

Welkom bij Erasmus MC & Bohn Stafleu van Loghum

Registreer

Login

Deel dit onderdeel of sectie (kopieer de link)

Abstract

Log in om toegang te krijgen