Individual Measurement of Performance Change in Sports

The dynamic planning of training load and the effects on individual change of endurance performance are of great importance not only from the perspective of sports medicine and training science to achieve performance-oriented design of training plans, but also for leisure and health-orientated sports for follow-up and diagnosis of individual load and demand optimization. In “classical group designs”, it is implicitly assumed that individual changes in endurance performance can be represented sufficiently accurately in controlled group designs (see Figure 1, right side, for a simple example) where effect size and confidence intervals for the effect size are calculated in addition to the deductive statistical testing (4).

For example, in a randomized and controlled training study with N=20 participants conducted by the same researchers, it could be demonstrated that a regime of endurance training, comparable between individuals (load: min 1.5 to max. 2.5 Watts/kg body weight; demand: between min. 65% to max. 90% maximum heart rate), led on average to significant improvements (both statistically and in practice) in “physical working capacity” or to the PWC performance at a heart rate of 150 beats per minute (t[19]=-3.08, p<0.001, d=0.96 [KI90%:0.2-1.8]). However, individual scrutiny of the data is advised (see Figure 1, left), both generally also individually due to the relatively large confidence interval for the effect size. For subject 3, deterioration in performance by 0.12 Watts/kg can be seen, whilst the performance change of subjects 1 and 2 is comparable to the group results, with an increase in the individual PWC performances (see Table 1). Subject 1 improves on starting performance by 0.50 Watts/kg, and subject 2 shows an improvement of 0.78 Watts/kg in comparison with starting performance. When interpreting the different improvements, it should be emphasized, however, that both subjects were classified under the same assessment category (i.e. the “+” category) with their final performance scores on the basis of standard PWC150values, despite having different improvement gradients and different starting values (difference between starting performance values of 0.26 Watts/kg) (see Table 1). On the basis of this categorization, an accurate assessment of the differences in the starting performance scores, and particularly of a significant individual performance change between pre-test and post-test scores cannot be considered to be sufficiently reliable.

The calculation of effect sizes for the group comparisons also proves to be unsatisfactory for individual performance changes, given that the effect sizes always also include subjects who showed no improvement or even showed deterioration, and the representation of the average effect can be substantially distorted. Group studies involving larger group sizes elucidate the problem: During a Heritage study, for example, the individual increases in VO2max varied between 0 and 1000ml/min, but the effect size (effect size was estimated ex-post from the official data) d=1.9 (KI90%:1.8-2.0) implied a significant improvement in performance in practical terms for the N=720 participants (2). Contrasting with this, only a trivial training effect of d=0.4 (KI90%:-0.01–1.0) becomes evident after a 2-week controlled endurance training plan followed by healthy male and female subjects (1), despite 90% of individuals showing at least a moderate improvement in VO2peak (7). A reason for the lack of adaptation seen in individual subjects could be due to collecting data only on target values. In this regard, measurement of several other parameters pertaining to different adaptation and measuring levels was proposed, as not all parameters react in the same way to endurance training in individual cases. However, this multivariate approach does not solve the problem of different levels of adaptation according to each individual, but rather exacerbates the problem, as the individuality for each parameter, e.g. the heterochronicity of adaptations, as well as time-dependent and effect-dependent interactions would need to be taken into account. For this reason, vis-a-vis empiric “shotgun designs”, parameters selected justified on a theoretical-content basis which can explain the different levels of adaptation according to each individual, seem more promising. Methods for quantification and prediction of individual performance changes, such as methods using single-case diagnosis, offer the opportunity to make differentiated and specific statements for each individual subject regarding the individual point in time, as well as the magnitude of any statistically and practically significant training effect.

The determination of PWC is used as an established measuring instrument in the field of leisure and health-orientated sport, yielding training and treatment recommendations on the basis of tables of standard values (21, 27). PWC₁₅₀ is the denotation used to describe performance on a bicycle ergometer at a heart rate of 150 beats/min. In order to make a meaningful comparison between all individuals possible, the performance capacity assessment is carried out relative to body mass (9, 15). The load schema compiled (amongst others) by the Federal Committee of Performance Sports of the German Sports Federation [Bundesausschuss Leistungssport des Deutschen Sportbundes] for bicycle ergometry (BAL schema), or the guidelines from the German Cardiology Society [Deutschen Gesellschaft für Kardiologie] (DGK, 2000) (14, 31) serve as the basis for selection of starting load levels, as well as duration of each load and loads for each subsequent level. During the stress tests as carried out, the physical performance capacity of students of sport science who had undergone moderate endurance training were selected for determination of PWC₁₅₀ according to the BAL schema (17).

The concept of the Reliable Change Index (RCI) takes into account (as do all other concepts) the fact that changes cannot be 100% free of measurement errors, and are calculated from the difference between pre-test and post-test values, which are relativized for the standard error differential (see Table 2). For calculation of standard error, the standard error of measurement is once again employed, taking a reliability value into account, e.g. the reliability from repeating the test, or the split-half reliability method (3). The RCI determined on this basis represents a z-value considering a confidence level or confidence interval determined a priori, which can be compared to the critical z-value from the standard normal distribution. Where the calculated RCI is greater than the critical z-value, an improvement or deterioration can be said to be clinically significant or meaningful in practical terms, with the change not being a result of lacking measurement accuracy (3).

During diagnosis and assessment of individual training effects, one is predominantly concerned with ensuring that no practically significant training effects are being overlooked, and practically insignificant training effects are not being overestimated. In order to minimize this risk, an 80% or 90% confidence level, and with this a greater α-error, should be selected for the determination of the confidence intervals, (and also for the RCI) (6, 11, 21). There are two possibilities available for determination of the confidence interval width (80 or 90%); backing up individual results with the aid of a standard error of measurement or standard estimation error (3). The basis for this is provided by the equivalence and regression hypotheses. The equivalence hypothesis is preferred for application-oriented problems, and assumes that the measurement error remains constant, i.e. is of the same magnitude for each person, and that the feature characteristics vary. On the other hand, the regression hypothesis assumes that the true value for an individual must be estimated from the observed values. Based on the assumption that the observed values for subjects represent a good approximation to the true values, the statistical validation of individual values is carried out using the standard error of measurement (equivalence hypothesis) (3). As it is assumed that an improvement in performance would result over the course of the systematic training program, one-sided hypothesis testing is applied. For the (practical) sports-orientated issue of individual training effects, a confidence level of 90% is established (6, 11, 21).

Further, to generate the confidence intervals, reliability estimates of the measured values are necessary. The reliability estimates are carried out in this example on the basis of a data set of a total of 84 measured values (PWC150), which were drawn from a previously-published study on PWC150(16). To determine the reliability, the entire data set was classified into two testing halves according to the odd-even method, by odd and even sequence numbers (26, 30). The subsequent strong parallel correlation is calculated on a pre-to-post-test basis, and the reliability estimated to be ρtt=.94. This estimate of measurement accuracy provides a fundamental basis for evidence of individual effects using the RCI over the course of the training process (3).

Table 2 gives an overview of characteristic values and formulas which were used as the basis for RCI calculations and to determine the confidence intervals (which were ascertained using Konfi 2.4, a freeware program for psychometric single-case diagnosis) (3, 25).

For the overall sample (N=20), subjects are sports students of ages ranging from 23-27 years (M=25.3; SD=1.3 with weight M=76.8; SD=6.5 kg). Taking into account the performance requirements placed on students, the PWC150was selected as the submaximal performance test (see introduction).

During a 90-day training study, in which the load level (watts/kg body mass) and duration of load for the test subjects were identical, the bicycle ergometer was used for training during daily training units in accordance with the duration method of maximum 65 and minimum 25 minutes with a peak load of 75% and a minimum of 50% of the maximum performance, with submaximal performance tests (PWC150) carried out at defined points in time. Regarding questions relating to specific points in time or to different adaptation reactions on an individual level over the course of the examination, the changes were specifically analyzed at around 3 and 8 weeks. At these points in time, as well as both before and after the examination period, the performance development was verified using sports medicine methodology, and the training load was adjusted for a constant individual stress, using exhaustion tests as the basis (lactate performance diagnostics at the Occupational Medical Center of the company Infraserv GmbH & Co Hoechst KG Frankfurt).

For the three subjects, the data over the entire examination period were recorded with very few missing data (less than 2%), such that the following targeted expectations should be verified on the basis of the empirical characteristic values:
- Is there a clinically significant or practically meaningful improvement in performance?
- At what discrete point in time over the training process can a clinically significant or practically meaningful improvement in performance be expected?

In the first instance, confidence intervals were determined for the individual performance changes, as were difference values from pre-test to post-test, in which the true value for subjects is to be expected with a given probability. Using this, an initial individual performance estimate is made possible in comparison with the group mean value and the standard deviation. Figure 2 shows how the performance changes of the three selected subjects from figure 1 could be classified within the interindividual comparison of mean performance change of the group. Independent of the fact that subject 3 (S3) demonstrates above-average performance in the post-test, he was not able to improve on the performance achieved in the pre-test. Subject 1 (S1) was able to significantly improve performance with respect to the pre-test, but despite this remains in the area relating to average performance. The above-average increase in performance of subject 2 (S2) similarly shows average performance in the post-test.

To provide an answer regarding the question of a clinically significant or practically significant training effect from the pre-test to the post-test, in the next step the RCI is calculated. The calculated index (RCI=3.28) for subject 1 is greater than the critical z-value (z1-a=1.28; α=10%) and indicates that the changes from pre-test to post-test are not to be traced back to lacking accuracy of measurement, and as such can be interpreted as practically meaningful improvements in performance (see calculation example in Table 3).

In the post-test, subject 2 achieves a value of 2.46 Watts/kg. In this instance too, the calculated RCI=5.12 is greater than the z-value (z1-a=1.28; α=10%), indicating that the change from pre-test to post-test is to be interpreted as an individual improvement in performance and thus as a practically meaningful training effect.

For subject 3, the RCI values confirm that no practically significant improvement in performance was achieved over the examination period (see Table 4), such that the other questions for examination shall only be followed up for subjects 1 and 2.

The following table provides an overview of the individual RCI change scores at the points in time when measuring was carried out (henceforth, ‘time of measurement’ [T]) as indicated in the training process, and shows the practically significant improvements in performance at the different measuring time points in relation to the achieved value in the pre-test for subjects 1 and 2.

When considering the changes with respect to each respective previous time of measurements [T] (see Table 6), i.e. from pre-test to T24, from T24 to T60, and from T60 to T90, an additional practically significant improvement can also be seen between the second (T24) and third (T60) time points for subject 2, in addition to the first practically significant improvement seen from pre-test to measurement time point after 24 days.

In figure 3, the PWC150values at the individual time of measurements (T) as well as the confidence intervals for the individual Ts over the examination period are represented for both of the subjects exhibiting practically significant performance improvements. The dotted guidelines allow clear identification of the different adaptation characteristics of the two subjects.

In summary, it may be stated that subject 1 reacts with a practically significant improvement after just eight weeks (Figure 3, left side) when participating in endurance training over 90 days (using maximum performance to put results into context), whereas subject 2 demonstrates a practically significant improvement in performance at both just three weeks and at the 8- week time points over the examination period (Figure 3, right side).