A Framework For Determining Meaningful Effects in Sport and Exercise Science

Does It Matter?

This framework is intended to help structure the reasoning behind evaluations of meaningful effects in sport and exercise science. It highlights questions that may need to be considered when evaluating whether an observed effect is meaningful. It does not generate thresholds or determine what action should be taken, but helps users think through the evaluation process.

Click on a box for its definition and guiding question. Choose a scenario to see a worked example for each element.

Worked Example
MEANINGFUL Does it matter? WORTHWHILE Is it worth it? value-judgement value-judgement PERSPECTIVE FRAME OF REFERENCE SPECIFICATION Variable · Measure · Outcome · Target Quantity GOAL WILLINGNESS Approaches & Methods SESOI set threshold a priori ACCEPTABLE COST set threshold a priori Meaningful? Worthwhile? Appraisals Reliability · Construct Validity · Causal Link OBSERVED EFFECT measured OBSERVED COST measured Expected Benefit

A framework for determining meaningful effects in sport and exercise science.

The framework separates two evaluative questions, represented by two panels: Meaningful? (Does it matter?) and Worthwhile? (Is it worth it?). Both panels are framed by Perspective, Frame of Reference, and Specification. In the Meaningful panel, a value judgement (pink) leads from the Goal to the SESOI, the meaningfulness threshold. The dotted arrow labelled Expected Benefit connects the Goal in the Meaningful panel with Willingness in the Worthwhile panel. It shows that the expected benefit of an effect in relation to the Goal informs Willingness to bear the specified cost. A corresponding value judgement leads from Willingness to Acceptable Cost. Approaches & Methods are used to operationalise both thresholds. Each is set a priori and independently of the corresponding observed quantity. Observed Effect and Observed Cost enter their respective evaluations through Appraisals (Reliability, Construct Validity, and Causal Link), which qualify the interpretation that each quantity can support. In each panel, the appraised quantity is compared with its threshold. The block arrow indicates a one-way logical dependency: the Worthwhile? evaluation is conditional on a positive Meaningful? conclusion.

SESOI: Smallest Effect Size of Interest