A major replication study raises some uncomfortable questions about the evidence behind our training advice. Here's what it means for runners, coaches and anyone trying to get faster.
We have all seen it happen.
A new study tells us that a particular training session improves running economy. Another claims that a specific strength exercise increases performance. Somewhere else, researchers have discovered the optimal interval duration, the perfect recovery period or a new way to improve endurance.
The temptation is to take these findings and immediately incorporate them into our training.
But how much confidence should we actually place in a single published study?
A paper published in Sports Medicine in June 2025, titled Estimating the Replicability of Sports and Exercise Science Research, provides an important reason to pause before taking research findings at face value.
The results are concerning. They also have some very practical implications for how we should approach training.
The problem with sports science research
Science is a process of testing ideas, challenging assumptions and gradually building a more accurate understanding of the world. A researcher conducts an experiment, publishes the results, and other researchers attempt to reproduce those findings using new data. If the same effect appears repeatedly, our confidence in the original finding increases.
If it doesn't, we have a reason to investigate further.
This process is called replication, and it is fundamental to scientific progress. However, sports and exercise science has historically struggled with replication and it’s not the only scientific discipline where this is the case. Many studies use relatively small sample sizes, and research findings can be affected by statistical uncertainty, selective reporting and publication bias.
Publication bias is particularly important. Studies reporting statistically significant or exciting results may be more likely to get published than those finding no meaningful effect. Over time, this can create a published research landscape that gives us an overly optimistic impression of how consistently particular interventions work.
That's a problem when athletes and coaches are using those findings to make decisions.
The study: how much sports science actually replicates?
Murphy and colleagues set out to provide an initial empirical estimate of the replicability of applied sports and exercise science research. They screened 9,385 studies, initially identifying 638 abstracts that met their selection criteria. Following further assessment, 587 articles were included in the pool from which studies could be selected for replication. The researchers ultimately analysed 25 replication studies.
These were drawn from applied sports and exercise science research published in journals between 2016 and 2021, covering areas including biomechanics, physiology, psychology, nutrition and injury prevention.
The replication studies attempted to repeat the original research and assess whether the findings held up when new data were collected.
The researchers used three criteria to classify a study as successfully replicated:
The replication had to achieve statistical significance.
The effect had to be in the same direction as the original study.
The effect size had to be statistically compatible with the original finding.
Only studies meeting all three criteria were classified as demonstrating robust replicability.
Just 7 out of 25 studies (28%) met all three criteria.
That is a pretty uncomfortable finding and it's important to understand what it does and doesn't mean. It does not mean that 72% of sports science research is definitively false. A study failing to meet the replication criteria does not automatically prove that its original finding was wrong. There are several possible explanations, including statistical uncertainty, differences between study populations and experimental conditions, and methodological limitations. But it does mean that we should be cautious about treating individual published findings as established facts.
The size of the effect matters too
Perhaps even more interesting than the replication rate was what happened to the reported effect sizes. An effect size describes the magnitude of an observed difference or relationship. In practical terms, it helps us understand how big an effect might be, rather than simply whether a statistical test considers it significant.
The researchers found that:
88% of replication effect sizes were smaller than those reported in the original studies.
The median reduction in effect size magnitude was 75%.
Imagine reading a study that reports a substantial improvement in performance following a particular training intervention. You might reasonably conclude that the intervention could make a meaningful difference to your training. But if subsequent research finds an effect that is considerably smaller, the practical value of that intervention may be very different from what the original paper suggested.
The authors also noted that 64% of replication effect sizes were both smaller than the original estimates and statistically incompatible with them. This matters because an exaggerated effect size can influence future research, meta-analyses and practical recommendations. It can also create unrealistic expectations about what an intervention is likely to achieve.
The question we need to ask of new findings is how much does it actually work, and how confident can we be in that estimate?
Does this mean we should ignore sports science?
The wrong response to this research would be to dismiss sports science altogether. Replication is not a mechanism for declaring every original study either right or wrong. It is a way of building a more reliable body of evidence. A failed replication can be informative.
Perhaps the original effect only occurs in a particular population. Perhaps it depends on training status, the intervention's duration, the intensity of the exercise or the way the experiment was conducted. These are all useful pieces of information.
The authors specifically caution against treating one divergent replication as definitive proof that an original finding is incorrect. Instead, they argue that we should focus on the accumulation of evidence. This is not a novel argument, and is how scientific research is supposed to function.
One study might suggest that a particular intervention is beneficial. A second might find a smaller effect. A third might find no meaningful difference. Rather than picking whichever result supports our existing beliefs, we should consider the quality, consistency and context of the evidence as a whole.
So when should you actually change your training?
This is where things become practical. If you're a runner, you probably don't have the time or resources to wait for multiple large-scale replication studies before making every training decision.
There is a difference between making a major change to your training programme and experimenting with a relatively small adjustment. We recommend you consider three things before incorporating a new research finding.
1. Look for independent replication
Has the finding been observed by other researchers, using new participants and data? A result that has been independently replicated is generally more convincing than one that has only appeared in a single experiment. Ideally, look for several studies pointing in the same direction.
2. Consider the quality of the evidence
Not all studies are created equal. Look at the sample size, the population being studied, the intervention, the duration and the outcomes being measured. A study involving 12 recreational runners over two weeks may not tell you much about how an intervention will affect an experienced athlete over a full training cycle.
3. Consider the cost and risk of experimenting
This is perhaps the most useful distinction for athletes.
If a potential training adjustment is inexpensive, low-risk and easy to reverse, you may decide that experimenting is worthwhile even when the evidence is still developing. Low-cost does not automatically mean low-risk. Adding a demanding interval session, increasing weekly mileage or introducing additional strength training may have consequences for fatigue and injury risk.
The potential benefit needs to be weighed against the demands it places on your training. The bigger the intervention, the stronger the evidence you should ideally want before making the change.
How should you incorporate new research into your training?
Even when the evidence looks convincing, there is still the question of implementation. One of the biggest mistakes athletes make is changing too many things at once.
They read about a new interval protocol, change their strength programme, increase their mileage and adjust their recovery routine. A few weeks later, their performance improves or deteriorates and it can be difficult to understand what caused it.
A better approach is to treat training changes as controlled experiments.
Step 1: Identify one specific change.
Be clear about exactly what you're changing and why.
Step 2: Establish a baseline.
Understand what your current training and performance look like.
Step 3: Make the change gradually.
Avoid unnecessarily disrupting the rest of your training.
Step 4: Monitor the response.
Pay attention to performance, perceived effort, fatigue and recovery.
Step 5: Evaluate whether it is actually helping.
Don't confuse novelty with progress.
Give the intervention an appropriate opportunity to work, consider the wider context and be prepared to reverse the change if it isn't delivering the expected benefit.
What this means for runners
The biggest lesson from this research isn't that sports science is useless but that we need to be more selective about how we use it. A published study is a piece of evidence, not a training instruction.
Good coaching requires more than finding interesting studies and copying their conclusions.
It requires understanding the athlete, considering the wider body of evidence, accounting for training load and recovery, and evaluating whether a particular intervention is actually producing the intended outcome.
This is how The Running Algorithm works. It learns what works for you over time, and as that changes, your training is continually optimised.
The original research
Murphy, J. et al. (2025). Estimating the Replicability of Sports and Exercise Science Research. Sports Medicine.
Published online: 16 June 2025.
DOI: https://doi.org/10.1007/s40279-025-02201-w
A note on interpreting the findings: This was an initial estimate based on 25 replication studies selected from applied sports and exercise science research published in Q1 journals between 2016 and 2021. The results should not be interpreted as a representative estimate of the proportion of all sports science research that is false or unreliable.
