Current Considerations and Future Directions of Psychometric Training Monitoring of Recovery-Stress States

The present article aims at providing an overview of basic requirements and considerations in the area of psychometric training monitoring in elite sports. Theoretical foundations and practical demands on the selection and implementation of psychometric instruments are summarized and exemplified by the assessment of recovery-stress states. Most importantly, validated and evidence-based questionnaires should be preferred over self-built scales to obtain meaningful data.

Additionally, current trends and approaches of individualized data analyses to guide an informed decision are discussed. Further research on the applicability of statistical strategies for psychometric data is required. Digital and automatic processing of the data may reduce participant burden, while control over the algorithm and inferences should be maintained by human beings.

In essence, psychometric monitoring of the recovery-stress state may serve as supplement to training and regeneration management, provided that the aims and procedures are made transparent, proportionate and reasonable.

KEY WORDS: Sport Psychology, Questionnaire, Regeneration

Dieser Artikel gibt einen kurzen Überblick über grundlegende Voraussetzungen und Überlegungen zur Anwendung psychometrischer Verfahren im Trainingsmonitoring im Leistungssport. Sowohl theoretische Grundlagen als auch praxisrelevante Anforderungen sind bei der Auswahl geeigneter Verfahren sowie bei deren Implementierung zu berücksichtigen. Dies wird beispielhaft anhand der Erfassung des Erholungs-BeanspruchungsZustands skizziert. Zu betonen ist, dass wissenschaftlich validierte und evidenzbasierte Fragebögen gegenüber selbstentwickelten Skalen vorzuziehen sind, um aussagekräftige Daten zu erhalten.

Aktuelle Trends und Möglichkeitenstatistischer Methoden zur individualisierten Analyse und Interpretation werden aufgezeigt. In Bezug auf deren Anwendung bei psychometrischen Daten besteht allerdings noch weiterer Forschungsbedarf. Eine digitalisierte und automatisierte Datenverarbeitung kann zu einer besseren Akzeptanz der Beteiligten beitragen, wobei zu beachten ist, dass die spezifischen Algorithmen sowie die Schlussfolgerungen nach wie vor unter menschlicher Kontrolle bleiben sollten.

Grundsätzlich kann ein psychometrisches Monitoring des Erholungs-Beanspruchungs-Zustands eine nützliche Unterstützung im Trainings- und Regenerationsmanagement darstellen, sofern dessen Ziele und Verfahrensweisen transparent, verhältnismäßig und nachvollziehbar sind.

SCHLÜSSELWÖRTER: Sportpsychologie, Fragebogen, Regeneration

Considering past and recent developments in the field of sport and exercise science in high-performance management in elite sports, two main conclusions can be drawn. First, optimal training management ought to be balanced in terms of training load and recovery periods, which should be supported by monitoring systems (3, 12). Second, it is widely accepted that the psychological perspective plays an important role in the response to training loads and in the prevention of overload symptoms (20, 29). As highlighted by a group of experts on a symposium on issues of recovery and performance in sport, interdisciplinary approaches are strongly recommended (11, 12). Ideally, practitioners and staff members receive abundant support by an interdisciplinary team of sport scientists. Sometimes, however, these interdisciplinary perspectives have to be applied by a single person. Either way, relevant information of each perspective needs to be combined and aggregated to analyze the full situation and to reach an evidence-based decision. In addition, parameters need to be measured efficiently in terms of time and resources and the feedback needs to be quick and straightforward. Regarding the athletes’ perception of their current recovery-stress state and their individual response to external training loads, psychometric assessments can be considered as established and useful source of information. Nevertheless, scientific requirements should be considered in order to choose and implement the appropriate tool.

A recent review showed that athlete self-report measures that assess the recovery-stress or mood states are reliable instruments to detect (early) signs of overload and mal-adaptive changes in response to prolonged intensified training (29). An overview of available measures is provided by two recent articles (22, 27). Most important aspects in terms of relevant considerations and quality criteria shall be illustrated by the description of the Acute Recovery and Stress Scale (ARSS) and the Short Recovery and Stress Scale (SRSS) (14, 15).

First, in order to provide valid and reliable data, questionnaires should be developed based on a sound theoretical background. One possible consideration may be the ‘scissors’ model’ as a basic general approach, according to which recovery and stress describe two separate states as well as processes within a continuum which are interrelated in response to the accumulation of demands (10). As long as these states are balanced within the individual boundaries of capacities, a state of high stress and low recovery can be tolerated simultaneously. Nevertheless, increased stress result in increased recovery requirements, and an ongoing recovery debt may cause severe psychological and physiological consequences (20). Moreover, recovery is not simply the absence of stress (16) and these states cover several dimensions, such as physical, psychological, emotional and social (10). Therefore, the ARSS covers multiple facets of recovery and stress that are assessed based on 32 adjectives. Most importantly, this item list was generated through several developmental processes including expert surveys (i.e., athletes and coaches) and measurements among the target group. This approach enhanced content validity as well as practical meaningfulness. Consequently, in the setting of sport practice, a psychometric measure should be able to capture the targeted constructs which are reflected by real-world scenarios and accepted by the users. In order to be feasible in regular and continuous assessments, the duration of answering and evaluating the questionnaire should be as short as possible (9). Once the psychometric criteria of the ARSS were ascertained (e.g., construct validity, reliability, sensitivity to change), the SRSS was developed to serve as an economic tool for high-frequency and long-term applications (15). As the 32 items of the ARSS are summarized to eight scales, these superordinate constructs serve as items of the SRSS, whereby the corresponding adjectives are used as additional descriptors.

It is important to point out that another theoretical basis may be chosen depending on the monitoring purpose and the targeted dimensions. For instance, the discussed criteria are applicable to related psychological constructs such as mood and emotions which may be assessed via the Profile of Mood States (19) and the Emotional Recovery Questionnaire (17), respectively. Overall, theory-based and scientifically proven instruments should be preferred over self-built customized scales without a ridged validation process. While customized scales seem to provide higher feasibility (e.g., no need to purchase) and are adapted to the specific needs, their actual usefulness may not be supported by empirical evidence. Lack of sound theoretical foundations and standardized guidelines regarding application and interpretation are critical drawbacks which need to be considered. Ideally, the relevant quality criteria are documented in a proper user manual which provides clear instructions and recommendations for the application and interpretation. Furthermore, tools can be considered as established when their applicability has been shown in different settings. Numerous publications may serve as one indicator of scientific as well as practical evidence and support for the validity of a tool (27).

Once questionnaire data have been assessed, they probably need to be processed (e.g., aggregated) and prepared for the data analysis and interpretation. Researchers and practitioners are strongly advised to follow the manual’s or published guidelines for each instrument to yield valid and meaningful results. For instance, single items should not be analyzed out of their context when they were developed to be aggregated to a scale (7). In general, the most obvious way to interpret psychometric data is to compare the obtained values with the corresponding normative data typically coming along with validated questionnaires. For state-oriented measurements that are designed to assess rapid changes over time, this is somewhat impossible and in some cases it is explicitly not intended (for instance (13, 15)). A simple solution is to compare different groups of athletes or to analyze repeated measures over the course of time.

In the social sciences there is a general debate about the appropriate ways to analyze Likert-type rating scales in terms of parametric vs. non-parametric methods (5). However, some of the common concerns (e.g., a single Likert-type item is not suitable to measure a concept, or information of that item can only be treated as ordinal data) have recently been pointed out as myths (33). For instance, individual items may be treated as continuous when the numerical response format contains at least five categories (7), and parametric tests such as t-tests and analysis of variance maintain essential validity even when their parametric assumptions are not strictly met (33). Nevertheless, the question is whether tests based on group statistics provide meaningful information that is relevant for the individual at all. As is the case in elite sports, findings of group data that are based on averages are seldom applicable to the specific needs and circumstances of an elite athlete (26).

The difficulty of applying large-scale study designs is illustrated by a pilot prospective study on the feasibility of investigating the role of psychological states on injury prediction in circus artists, whereby 1000 participants would be needed to be assessed daily for a period of 250 days to yield a 61% statistical power (31). Indeed, that study constitutes an interesting benchmark in terms of the combined use of trait and state measures and participatory research approach (e.g., iterative process including the stakeholders’ most salient theoretical models as well as their acceptability and comprehension of the questionnaires) to deal with the predictability of certain psychological states as risk factors for injury among those athletes who get injured. Yet, monitoring training and performance in athletes requires an individual approach to guide timely decision-making processes (3).

Methods and statistics of single-case research designs, such as graphical approaches to analyze stability and variability of baselines and between as well as within certain phases seem to be useful (2). Moreover, several statistical strategies exist (e.g., descriptive statistics, Z-scores, effect size, individual coefficient-of-variation, meaningful change; for a comprehensive overview see McGuigan (18)). However, these approaches are typically applied to the analysis and interpretation of physiological and performance parameters. Currently, there are no general guidelines and recommendations for the use of psychometric data.

The following description of possible statistical approaches of training and exercise sciences is to be considered as an invitation to contemplate adopting these methods for psychometric data rather than a clear recommendation, as their transferability needs to be tested yet. Differences between physiological and psychometric measures need to be taken into account which may limit the applicability. For instance, subjective ratings reflect the perception and interpretation of physiological processes which may be intentionally or unconsciously biased by the individual. As a consequence, data may be manipulated (although physiological measures can be subject to measurement errors causing noise in the data, as well). Furthermore, while physiological parameters are typically measured on continuous metrics, the above mentioned issues regarding Likert-type rating scales have to be kept in mind. In particular, reporting habits may differ between athletes (e.g., range of values; definition of the individual ‘normal’ or ’optimum’) (27). This underlines the adherence to the guidelines and the correct use of the instruments. At the same time, the athlete’s buy-in needs to be ascertained which may involve clear instructions, education about the usefulness of the monitoring, and data security (28).

In several team and aerobic-oriented sports, heart rate measures are established in monitoring routines using heart rates at rest, or during (submaximal) performance and subsequent recovery. Despite the differentiation between acute and chronic responses to training load, several methodological considerations and statistical approaches need to be considered such as the measure’s typical error, the within-individual coefficient-of-variation, and magnitude-based inferences to detect meaningful and worthwhile changes. These are summarized by two recent reviews (4, 30). For the monitoring of muscle recovery, a Bayesian approach has been applied recently to provide individualized reference ranges of biomarkers (i.e., creatine kinase and urea), which are incorporated in an individualized algorithm. Based on a relevant reference population combined with repeated individual measures of defined recovered and non-recovered time points, Hecksteden et al. identified cut-off values from posteriori distributions (8). Graphical representations offer informative and intuitive insights of the gradual development of those individualized reference ranges. As the applicability to the discrete nature of Likert-type rating scales requires closer investigation, another possibility may be the application of statistical control charts which visualize collected data of a certain period of time with respect to an individual’s control line (average) and control limits (e.g., 1.5 or 2 standard deviations of the mean) (24). As it involves several measurements of an individual, outliers below/above those limits can be identified.

Overall, training monitoring has evolved as a complex issue that requires certain statistical skills to guide evidence-based decision-making processes. To identify important trends in the data of an individual athlete, especially those deviating from normal or expected patterns, is one of the major challenges in current sport science and practice. This is highlighted by the commentary of four scientific practitioners who support the usefulness of statistical approaches, such as magnitude-based inference, statistical process control or mixed models (analyzing pooled and non-pooled data) (32). Despite the statistical approach, contextualization is another crucial consideration regarding the interpretation and inference of the data. Figure 1 and 2 depict an overview of the eight SRSS items combined with the internal training load (6) over a 28-day period for an individual combat sport athlete. Within this timeframe, the athlete did not use the entire range of the rating scales, yet a certain variability becomes apparent. For instance, it can be seen that the highest training load was perceived on day 19 and 20 (T19, T20), while the lowest score of Physical Performance Capability and the highest score of Muscular Stress occurred one day before (T18, Figure 1). At the same time, scores of Emotional Balance increased from 2 to 5 (Figure 2). Graphical, descriptive illustrations, like this example, may facilitate communication between the athlete and relevant staff members to serve as starting point for intervention.

Apart from the theory-driven and evidence-based application of (psychometric) monitoring tools, an essential criterion for success is the athlete buy-in and adherence. Therefore, it is important to ascertain an understanding of the purpose and benefits of training monitoring in athletes, for instance, through an explicit introduction and education session. Difficulties in establishing the habit of regularly completing the monitoring in their normal training and the lack of feedback on their data are major reasons for poor buy-in (23). Practical implications for the implementation of a monitoring system are summarized in table 1 which are further extended by Saw et al. (27, 28) and Neupert et al. (23). In terms of the usability of monitoring systems, online platforms and digital applications have increased in popularity and are becoming a booming market, as these facilitate data collection, processing as well as providing quick visualizations. According to Barboza et al., necessary features to increase end-user satisfaction involve mobile friendly interface (e.g., answer in a few clicks, no need to type much), timely feedback, and fusion of information from different sources (1). From the coaches’ and staff’s perspective, traffic-light systems are certainly an appealing option to support the interpretation of the data due to its visual intuitive signaling (i.e., green, amber, red) (25). However, at this point of time, this practice is problematic among psychometric data, when there are no clear documented guidelines and, especially, when not properly put into context or applied by unskilled users (27). Furthermore, it has to be considered that providing misinformation may cause an escalation of symptoms, especially when bodily reactions are ambiguous (21). More research is needed, before such a traffic-light system seems appropriate.

In sum, this short review emphasizes the choice of validated standardized psychometric tools over un-validated custom measures. Depending on the monitoring purpose, researchers and practitioners need to take several considerations into account before implementation. In addition to evidence-led decisions, this involves the suitability regarding the characteristics of the target group (e.g., age, type of sports, time of season). While various statistical approaches are available to aggregate and to analyze the data, definite rules and cut-offs do not exist. Researchers and practitioners are encouraged to explore the most promising strategies that may support decision-making processes in the daily business. Digital and automatic processing of the data may reduce participant burden, while control over the algorithms and inferences should be maintained by human beings. As long as the overall aims and mechanisms are made transparent, proportionate and reasonable to the stakeholders, psychometric monitoring may be a useful complement to the training and regeneration management in elite sports.

Conflict of Interest
The authors have no conflict of interest.