Towards Sideline Testing of Neuro-Ophthalmological Function in Football (Soccer): Reliability and Effects of Repetitive Headers on a possible Neurophysiological Biomarker

Problem: Recent research indicates an increased risk of neuropathologic changes in former elite contact team sports athletes possibly due to repetitive mild traumatic brain injury. Impaired neuro-ophthalmological function has been linked to mild traumatic brain injuries which can be detected using eye tracking. The aim of this project was to assess the reliability of a novel eye-tracking device using virtual reality (VR), and to analyze the acute effects of repeated headers on the neuro-ophthalmological function of competitive football players.
Methods: A reliability study with 50 healthy participants (26.7 years, 70% females) and an interventional cross-over study with 50 competitive football players (23.9 years, 50% females) were conducted. Overall, 29 parameters from 7 different VR eye-tracking tasks were computed.
Outcomes: The reliability assessment revealed poor reliability for most parameters (75% poor, 22% moderate, 3% good). The four most reliable parameters were analyzed in the interventional study and did not reveal significant differences (directional error: p=0.39, gain of the first saccade: p=0.56, latency of the first sac-cade: p=0.59, gaze velocity: p=0.73) between a header-focused training session and one without opponent contact or headers.
Conclusion: Our results show that technical challenges still exist to provide sideline availability of an objective neuro-ophthalmological screening in near real-time. Further research is needed to provide insight into the acute and long-term effects of repeated headers in football.

KEY WORDS: Eye-Tracking, Concussions, Heading, Oculomotor Assessments, Virtual Reality (VR)

Cognitive impairments resulting from sport-related concussions have become a topic of interest, recently pushed into the spotlight due to both media attention and scientific studies. The current discourse gained attention from recent studies that have demonstrated an increased likelihood of suffering from neurodegenerative diseases among American football (17), rugby (23), and football (soccer) players (15, 24), compared to the general population. In men’s football, outfield players had a higher risk compared to goalkeepers, which has been discussed in light of the more frequent sport-related concussions and headers (15, 24). This was substantiated by a study which revealed an association between heading frequency and risk of cognitive impairments in former soccer players (9). It is debated in team contact sports whether sport-related concussions and mild traumatic brain injuries are the principal risk factors for negative long-term effects from head impacts (7, 11, 27). Repetitive head impacts are a potential mechanism leading to neurodegeneration (15). In contact sports, many head impacts occur. Most concussions occur in men´s rugby during matches (3.89 per 1000 match hours or 3.00 per 1000 athletic exposures) (22).
For a prompt clinical evaluation of a sport-related concussion on the sideline, a reliable and quick assessment is of high importance. Symptoms can be manifold and may show affection of cognition, balance, and eye-movement (15, 28). The mechanisms underlying sport-related concussions are multi-factorial (26). It has been suggested that axonal damage at the brainstem can elicit eye movement abnormalities resulting in delayed response to stimuli and during pursuit tasks (18). The neuro-ophthalmological function is a pivotal marker for brain damage as it seems sensitive to traumatic and mild traumatic brain injuries (1, 7, 30). The impairment of neuro-ophthalmological function in patients following a concussion correlates with the severity of concussion symptoms (5, 12, 19, 29). To measure neuro-ophthalmological function, eye tracking is a common method. Technical advances have come to a combination of eye tracking with three-dimensional virtual reality (VR) glasses. This might enhance the precision of the captured data outside of a laboratory setting, as it can reflect a more natural environment and allows for greater freedom of movement during the measurement, thereby facilitating greater immersion (6). New technical innovations are crucial prerequisites in the development of biomarkers for diagnosing (sports-related) concussions (20). However, to date, neither the reliability of such a system is known, nor is the influence of game-specific head impacts, such as typical headers in football.
Thus, the aim of the present project was a) to investigate the reliability of a novel eye-tracking device integrated into VR glasses and b) to investigate the acute effects of headers on the neuro-ophthalmological function.

The project included two studies: One investigating the reliability of a new measurement system and the second analyzing the effects of a trainings session with header play on neuro-ophthalmological function. Ethical approval for the project was granted by the university’s ethics committee (MSH-2021/128).

Study 1: Reliability Assessment
Recruitment and Study Protocol
For the first study, a convenience sample of 50 healthy participants (35 women, 15 men, mean age 26.7 years) completed three identical video-based eye-tracking assessments in virtual reality over two consecutive days. Participants were recruited from the university employees and students. Participants were divided into two groups. The assessments for the first group were conducted in the sequence of morning, evening, and morning. The second group completed the assessments in the sequence of evening, morning, and evening. All participants gave written informed consent to participate in the study.

Data Acquisition
Neuro-ophthalmological functions were recorded using a VR eye-tracking system (200Hz, eyeTrax GmbH & Co. KG, Osnabrück, Germany). Different eye movement tasks were displayed in a three-dimensional virtual reality. In total, 15 different tasks were presented with a total duration of 6:30 minutes. The set of tasks comprised following a ball that changed position in the VR environment either abruptly (saccades) or in a circular or sinusoidal pattern (smooth pursuit). Further tasks involved changes in gaze direction opposite to the ball’s position (antisaccades) as well as self-paced changes of the gaze direction (self-paced saccades).

Data Processing
Data processing was performed both manually and automatically. The manual evaluation was carried out by two raters using eyeTraxAnalytics (version 2.1, eyeTrax GmbH & Co. KG, Osnabrück, Germany). Raters were not blinded as the software provided a video feed of the participants for documentation purposes. The manual data processing required about 2 hours per measurement. Therefore, an algorithm was developed for automated evaluation (3). This algorithm, in turn, is based on a previously published algorithm (21) which automatically classifies eye movement into saccades, fixation and smooth pursuit from the eye position timeseries data which were obtained from the eye tracking software. A total of 36 parameters were determined from the test data using manual analysis, and 29 parameters using automated analysis. The automatically computed parameters were directional error, latency of the first saccade after stimulus, velocity of the saccades, gain of the saccades, saccade count, and phase lag. Depending on the task, these parameters were either calculated for the first saccade after the stimulus or for all saccades. In the case of the smooth pursuit tasks, directional error and latency were not calculated. We calculated the median value for all parameters from multiple stimuli or repetitions (horizontal biflicker: six stimuli; vertical biflicker: three stimuli; anti-biflicker: 82 stimuli; self-paced saccades (horizontal & vertical): ten seconds; smooth pursuit – sine: 3 repetitions; smooth pursuit – circle: four circles in 16 seconds).

Statistics
For each parameter intraclass correlation coefficients (ICC (2,1)) were calculated for the values of the different assessments, and Bland-Altman plots were created for the same. The ICC values were classified according to Koo & Li (14) as excellent (>0.90), good (0.75-0.90), moderate (0.50-0.75), and poor (<0.50). The statistical procedures were conducted using R (R Version 4.2.2 (2022), Statistical Computing, Vienna, Austria) through the Python (python.org, Version 3.9) library rpy2 (rpy2, Version 3.4.5).

Study 2: Heading Intervention
Recruitment and Study Protocol
For the intervention study, 25 female and 25 male football players (n=50) were recruited from competitive football clubs in the Hamburg Football Association. The average age of the football players was 23.9±5.3 years without significant difference between sexes (p=0.25). The study was conducted in a cross-over design according to a 2x2 study protocol with a baseline measurement (see figure 1). Within both teams, participants were randomly divided into two groups by drawing numbered vests they were to carry during the sessions from a bag. Both groups simultaneously completed two different football training sessions and exchanged the training content in the second round, so that both groups completed two training sessions each. One training session focused on headers, while the other training session excluded both header play and body checks. Both training sessions took place outdoors on natural grass under dry weather conditions in October 2022. The duration of the first training session was 20 minutes, followed by a 40-minute wash-out window. Then the second 20-minute training session ensued. All participants gave written informed consent to participate in the study, in accordance with the Declaration of Helsinki.

Data Acquisition
Measurements were taken with the same VR eye-tracking system presented in Study 1, conducted in a sports hall adjacent to the training ground. Three identical tests were performed for each person: A baseline measurement before the start of the sports units, a second test immediately after the first training session, and a third measurement directly after the second training session (figure 1). The task order was consistent. Additionally, the training sessions were filmed using two cameras (60Hz, GoPro Hero 5, GoPro Inc., San Mateo, CA, USA; 30Hz, Pixellot Air, Pixellot Ltd., Petah Tikva, Israel) to evaluate and classify the headings.

Data Processing
Eye-tracking data was automatically processed as presented in the reliability study. The evaluation of the frequencies and classifications of the headers was carried out by two independent investigators according to an internationally standardized header protocol (2). Blinding was not possible for this purpose.

Statistics
Analysis of Covariance (ANCOVA) was performed to identify the potential effects of the header intervention on the four parameters that had demonstrated the highest reliability in Study 1 (table 1). As covariates, the respective values of the baseline measurement were included. Additionally, Analysis of Variance (ANOVA) was performed to determine general differences between the three measurement points. In case of a significant result, post-hoc t-tests with Bonferroni correction were conducted. The significance level was set at p<0.05 for all statistical tests. All statistical tests were carried out using Jamovi for Windows (the jamovi project, Version 2.3). We report our findings according to the CONSORT statement extension to randomized crossover trials (8).

Reliability Assessment
Due to multiple difficulties with the measurement system (e.g., corrupt database files, operating errors), there was an unexpectedly high loss of data. Of the 150 measurements carried out, only 114 (76%) were available for analysis. The reliability of the manually processed data showed mostly poor ICC values for both intra-rater (50.0% poor), intra-day (93.1% poor), and inter-day (88.9% poor) as well as interrater reliability (100% poor).

The automatically processed data, too, mainly showed poor reliabilities (75% of the parameters). Only four parameters showed at least moderate reliability in at least two of the three measurement point comparisons: directional error (ICCs: 0.66, 95%CI: 0.60 to 0.68), latency of the first saccade (0.45, 95%CI: 0.51 to 0.80), and gain of the first saccade (0.72, 95%CI: 0.74 to 0.66) of the anti-biflicker task, and gaze velocity of the smooth pursuit task (0.69, 95%CI: 0.39 to 0.60, see table 1). These parameters were considered for the analysis of the following interventional study.

The Bland-Altman plots also revealed large biases and limits of agreement between the respective sessions. As an example, figure 2 illustrates the Bland-Altman plots for the parameter “gain of the first saccade” of the anti-biflicker test for the three session measurement comparisons.

Header Intervention
All 50 soccer players performed the entire protocol. During the 20-minute header training session, male football players performed approximately 37% more headers with an average of 20.9±4.2, compared to the female football players who performed an average of 15.3±3.3 headers. The vast majority of these were intentional headers (99.0%). Additionally, the most common headers were played after the ball had travelled airborne less than five meters (82.9%). The most frequent head region that was used to play the ball was the front (96.7%).

Figure 3 displays the results of the four evaluated parameters. The results of the covariance analyses showed no statistically significant differences between the two training formats for the four examined parameters (ANCOVA directional error: p=0.39, gain of the first saccade: p=0.56, latency of the first saccade: p=0.59, gaze velocity: p=0.73). Furthermore, it was apparent that only the values of the latency of the first saccade following both training sessions (Header: 223±29.2ms; Without Header: 220±34.0ms) deviate statistically significantly from the baseline measurement (252±69.7ms) (post-hoc t-test: Baseline-Heading: p=0.003, Baseline-Without Header: p=0.005).

The study showed overall poor reliability for most parameters analyzed by a new neuro-ophthalmological test system and no effects of repetitive header play on four reliably measured neuro-ophthalmological function parameters in male and female football players.

Poor Reliability of the VR Eye-Tracking Device
Overall, the reliability of the neuro-ophthalmological test system using VR glasses was mostly poor, both in manual and algorithmic evaluation. The reliability of a new test system can be poor for several reasons. To eliminate potential error sources introduced by human factors, we developed an automatic algorithm using a well-established eye movement classification algorithm (21). However, the reliability was only slightly improved (in terms of ICC values) from the manual to the automatic processing. Even though the development of the test system was not the goal of this project but rather its scientific evaluation, a possible source of error seems to have been the automatic detection of the pupil in relation to the eyeball. When investigating the cause of the poor reliabilities, we encountered many files in which this detection apparently failed (see figure 4c as an example). In turn, these files also showed noisy eye movement signal data (figure 4d). As pupil recognition is a proprietary algorithm of the manufacturer of the test system, no further optimization could be carried out at this stage. Further signal processing for automated classification of eye movement into saccades, fixation, or smooth pursuit is dependent on sufficient signal quality. If this is not given, reliable parameter computation cannot be expected.

Effects of Repeated Headers on Eye-Tracking Parameters
The test-retest reliability refers to the consistency of the results when the test is repeated at different times. If a test system does not show stable reliability over time, then intervention effects also cannot be reliably measured. Overall, four outcomes from two tests in at least two cases proved to be moderately reliable and were further investigated in the intervention study. These outcomes were not affected by a training session with an average of 18 headers. This may be due to the headers having no influence on neuro-ophthalmological function (13), but also because the impacts of the mostly intentionally headers from close distance (<5m) may not have been high enough to affect neuro-ophthalmological function (4, 10).

Even though the currently reliable parameters were not affected by the header training session tested in this study, the VR glasses can currently not be recommended for field diagnostics of sports-related concussions due to the high processing effort, the data losses, and the overall mostly poor reliability. To this end, the processing effort should be significantly reduced, and greater stability of data processing should be enabled. Moreover, data are currently missing as to whether the parameters shown to be reliable in this project are also reliable in patients with diagnosed sports-related concussions and whether they are affected by a sports-related concussion.

Implications for Sideline Diagnosis of Concussions
In the discussion on sideline diagnosis of concussions, it’s crucial to clarify that the primary goal of such assessments is not to establish a definitive diagnosis but to conduct an objective screening to quickly identify players who may need further evaluation. In the recently published consensus statement on sports-related concussion (20), the recommendations for field diagnostics at the sideline have been changed compared to the previous consensus statement (16). Players suspected of having a sports-related concussion are to be removed and thoroughly examined using the Sports Concussion Assessment Tool 6 (SCAT-6). Nevertheless, the role of biomarkers and current developments in technology are emphasized, particularly in assessing recovery and planning return to sports (20). For such a scenario, an application of neuro-ophthalmological testing in a calmer and more laboratory-like setting with measurement systems like the one used in this project might be possible. However, for both scientific monitoring and clinical implementation, reliability must be ensured, which is currently not the case for the system examined in this project.

The presented project had strengths and limitations. Technological advances need to be evaluated by rigorous research. In this project, we planned a project with a novel measurement system that has not been previously evaluated in research. Unfortunately, the reliability of most parameters was poor but changing the measurement system during the conduct of the project was discussed but not executed due to time constraints. Furthermore, we tested a training session specifically designed using headers. However, most headers were played intentionally from a short distance with the forehead. A specific setting with more frequent longer distance headers would not be feasible from a participant safety point of view. Therefore, it is not possible to generalize the findings to game situations and other header types that may also be more relevant for a sideline evaluation.

In this project report, we demonstrate the technical challenges in bringing an objective neuro-ophthalmological function assessment towards the sideline. A novel eye tracking system with a virtual reality goggle used in this project did not show satisfactory reliability for a wide range of established eye movement parameters. Furthermore, no changes were detected in a 20-minute header training intervention compared to a non-contact and non-header play training session. Our findings should encourage future researchers to thoroughly evaluate the basic quality criteria of novel measurement systems before using them for the analysis of changes in neuro-ophthalmological functions.

Conflict of Interest
The authors have no conflict of interest.

Achknowledgements
We would like to thank Stephan Kerber (Hamburg Football Association) for his help in planning and coordinating this project.

Funding
This project was funded by the German Federal Institute for Sports Science (Bundesinstitut für Sportwissenschft, BISp, grant number: 070116/21-23).

Ethical Approval
The study was reviewed and approved by the MSH Medical School Hamburg Ethics Committee (Institutional Review Board: MSH-2021/128).

Data Sharing
The code developed in the project will be openly shared: github.com/MedicalSchoolHamburgBiomechLab/NEO_Kopfball. All accompanying data and technical appendix are available upon reasonable request from first author at dominik.fohrmann@medicalschool-hamburg.de

A reliability study with 50 healthy participants (26.7±6.1 years, 70% females) and an interventional cross-over study with 50 competitive football players (23.9±5.3years, 50% females) were conducted. Overall, 29 parameters from 7 different VR eye-tracking tasks were computed.
The reliability assessment revealed poor reliability for most parameters (75% poor, 22% moderate, 3% good). The four most reliable parameters were analyzed in the interventional study and did not reveal significant differences between a header-focused training session and one without opponent contactor headers.