Sport-related concussions are a significant public health problem, and concussion management is one of the most controversial issues in sports medicine. The latest international consensus statement on concussion in sport advises that players not return to play on the same day they were injured and that they not return until they are asymptomatic. But it can be difficult to make an accurate on-field diagnosis of concussion and to determine when an athlete’s symptoms have completely resolved.
Computerized neuropsychological assessment devices (CNAD), software and mobile applications for evaluating memory, reaction time, and other brain functions, have been widely adopted by school districts and professional sports leagues as a means of assessing cognitive function after concussion and informing decisions on return to play. The most widely used CNAD in North America is ImPACT (Immediate Post-Concussion Assessment and Cognitive Testing); ImPACT’s marketing information says it is used by “more than 7,000 high schools, colleges, sports medicine centers, and professional teams throughout the world,” including most National Football League teams, and all Major League Baseball, Major League Soccer, and National Hockey League teams. Internationally, professional sporting leagues have also mandated the use of CNADs.
The manufacturers of ImPACT and another widely used assessment device, CogState/Axon, recommend that athletes obtain a baseline assessment prior to the beginning of a sports season. The baseline assessment is later compared to a post-concussion test to determine if an athlete’s cognitive functioning has returned to normal. But in a practice known as “sandbagging,” athletes intentionally attempt to reduce their baseline scores in order to circumvent the guidelines. A lower baseline score may mean that a poor post-concussion score won’t reveal impairment and keep the injured athlete from being allowed to return to play. Quarterback Peyton Manning famously claimed he sandbagged a preseason CNAD test in 2011. Sandbagging raises important concerns about the reliability of CNADs.
In a recent New York Times article, “Flubbing a Baseline Test on Purpose is Often Futile,” physicians interviewed considered the chances of getting away with sandbagging to be small. But a 2012 study found that 11 percent of student athletes who were instructed to attempt sandbagging were successful at evading the “validity factors” built in to ImPACT. Those students were not instructed on how to fool the test, but were simply told to attempt to do so. A 2013 study found that the test missed 35 percent of sandbaggers who were coached on how to avoid detection, and 30 percent of others. If up to 35 percent of sandbaggers can slip through the net, then sandbagging may not seem “futile” to those motivated to try it.
The physicians interviewed in the Times article who expressed confidence in the tests are specialists who can spot sandbaggers. But the reality is that most concussed athletes are not evaluated by a neuropsychologist. They are high school football players assessed by athletic trainers, coaches, or their own family physicians. CNAD baseline tests are designed to be administered by anyone.
In their joint position paper on CNADs, the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology warn that effort and motivation by the person being examined can substantially influence neurocognitive test scores and produce variance “greater than that attributable to injury severity or other variables more directly related to pathophysiology.” While competent use of CNADs can improve access to neuropsychological testing, they warn that “expertise in the interpretation of computerized tests requires advanced knowledge of testing theory and the complex interaction of multiple factors that can affect performance on cognitive tests.”
Athletes often take these tests multiple times, so they are not naïve to the process. It’s likely that at least some are coached on how to take the test, perhaps by other athletes. More research is needed to explore the prevalence of test coaching, sandbagging strategies, and how often sandbaggers get flagged in the real world.
There are no validated biomarkers, blood tests, x-rays, or brain scans that can diagnose a concussion. The risks of premature return to sports after concussion include increased susceptibility to additional concussions, slower recovery, more severe concussion symptoms, and enhanced risk of post-concussion syndrome, all of which can have negative effects on athletic and academic performance. When athletes return to play based on test results of suspect validity and reliability, they are at risk of these harms. When athletes (or in the case of youth athletes, their parents/guardians) make decisions based on misinformation about the “protection” offered by CNADs, they cannot provide meaningfully informed consent to undertake the risks of sports participation.
Improperly used CNADs, both for baseline and post-concussion assessment, create the illusion that effective, scientifically valid measures are being deployed to combat the sport concussion problem. It’s a dangerous illusion that doesn’t serve the interests of athletes at risk of concussion.
L. Syd M Johnson is an assistant professor of philosophy and bioethics at Michigan Technological University. Brad Partridge is a National Health and Medical Research Council Postdoctoral Research Fellow in Public Health with the Neuroethics Group at University of Queensland Center for Clinical Research.