Alexander M. Crizzle, PhD, MPH1,2, Nadia Mullen, PhD3, Diane Mychael, BSc(OT)4, Natasha Meger, BMR(OT), CDRS5, Ryan Toxopeus, MSc1, Carrie Gibbons6, Simeon Ostap, MSc3, Sacha Dubois, MPH3,6,7, Michel Bédard, PhD3,6,7
1School of Public Health, University of Saskatchewan, Saskatoon, SK, Canada
2School of Public Health and Health Systems, University of Waterloo, Waterloo, ON, Canada
3Centre for Research on Safe Driving, Lakehead University, Thunder Bay, ON, Canada
4St. Joseph’s Health Centre Guelph, Guelph, ON, Canada
5Saskatchewan Health Authority, Saskatoon, SK, Canada
6Centre for Applied Health Research, St. Joseph’s Care Group, Thunder Bay, ON, Canada
7Human Sciences Division, Northern Ontario School of Medicine, Thunder Bay, ON, Canada
DOI: https://doi.org/10.5770/cgj.24.444
ABSTRACT
Background
Studies have reported poor sensitivity and specificity of the Screen for the Identification of Cognitively Impaired Medically At-Risk Drivers, a modification of the DemTech (SIMARD-MD) to screen for drivers with cognitive impairment. The purpose of this study was to determine whether the SIMARD-MD can accurately predict pass/fail on a road test in drivers with cognitive impairment (CI) and healthy drivers.
Methods
Data from drivers with CI were collected from two comprehensive driving assessment centres (n=86) and compared with healthy drivers (n=30). All participants completed demographic measures, clinical measures, and a road rest (pass/fail). Analyses consisted of correlations between the SIMARD-MD and the other clinical measures, and a receiver-operating-characteristic (ROC) curve to determine the predictive ability of the SIMARD-MD.
Results
All healthy drivers passed the road test compared with 44.2% of the CI sample. On the SIMARD-MD, the CI sample scored significantly worse than healthy drivers (p < .001). The ROC curve showed the SIMARD-MD, regardless of any cut-point, misclassified a large number of CI individuals (AUC=.692; 95% CI = 0.578, 0.806).
Conclusions
Given the high level of misclassification, the SIMARD-MD should not be used with either healthy drivers or those with cognitive impairment for making decisions about driving.
Key words: SIMARD-MD, cognitive impairment, driving performance, older drivers, dementia, comprehensive driving evaluation, sensitivity, specificity
Approximately one quarter of the Canadian population will be aged 65 years and older in the next 10–20 years.(1) The importance of driving a personal vehicle for maintaining independence and autonomy, along with increasing life expectancy, has resulted in seniors keeping their driving licences well into their 80s.(2) However, with increasing age also comes an increase in age-related medical conditions, such as cognitive impairment/dementia, that can impact the ability to drive safely.(3–4) Given that projections in Ontario alone show there will be close to 100,000 drivers with dementia by 2028,(5) there is increasing pressure for health-care professionals and licensing authorities to screen for drivers with cognitive impairment/dementia who may be unfit to drive.
Cognitive impairment (CI) is present when a person has difficulties with memory, learning new skills or things, and concentration or decision-making that can impact instrumental activities of daily living such as driving. Compared to healthy older drivers, research shows that drivers with dementia are more likely to fail a road rest(6–10) and have a 2–3 times higher crash risk.(11–12) However, some studies show that drivers with early stage dementia can continue to drive safely,(13–14) although the progression of cognitive impairment/dementia will likely require eventual licence revocation.
Identifying when drivers with cognitive impairment/dementia have become unsafe remains a challenge for health-care professionals. One tool in particular, the Screen for the Identification of Cognitively Impaired Medically At-Risk Drivers, a modification of the DemTech (SIMARD-MD), has been controversial in its recommendation for use by frontline clinicians. The SIMARD-MD was designed to identify cognitively-impaired drivers whose driving skills may have declined to an unsafe level.(15) As indicated in the title of the instrument, the SIMARD-MD is derived from the DemTect.(16) The DemTect was developed to measure cognitive impairment and includes five sections: word list recall, a number transcoding task, a semantic verbal fluency task, reverse digit span, and delayed recall of the word list. However, the DemTect was not developed to identify unsafe drivers.
While the authors of the SIMARD-MD asserted it has a “high degree of accuracy” and “can be used for immediate decisions in the clinical setting”(15) to identify unsafe drivers, the presumed value of the tool has been questioned since the manuscript was published.(17) In the initial study with a sample of 146 cognitively impaired seniors, the authors found that the SIMARD-MD predicted 86% and 84% to fail and pass a road test, respectively.(15) A validation study with 192 cognitively impaired seniors similarly found that the SIMARD-MD predicted 80% and 87% of those predicted to fail and pass a road test, respectively.(15)
However, these numbers indicate a substantial number of false-positive and false-negative test results. Furthermore, the sample included a number of cognitively healthy drivers—introducing spectrum bias—and, in reporting the results above, the authors did not include drivers identified as “indeterminate” and requiring further testing (about 50% of the sample).(18) Ignoring half of one’s sample in determining the predictive value of a test renders the results meaningless within a screening context. A stronger presentation of their data would have included a receiver-operating characteristic (ROC) curve to allow the determination of optimal cut-points and the associated sensitivity and specificity. In a different study, Bédard and colleagues(19) concluded that the SIMARD-MD was vulnerable to an education bias, and may create a structural inequity by failing to account for education in the interpretation of the test results. Moreover, Wernham and colleagues(20) found no association between SIMARD-MD scores of clients with cognitive impairment and the clinical impressions of their geriatricians regarding fitness-to-drive, further questioning the validity of the test results.
These studies are particularly relevant given that the SIMARD-MD has been suggested to health-care professionals as a tool to determine fitness-to-drive in Canada and appropriate for routine administration in the family physician office setting once patients reach age 70.(21) Yet, despite the availability of the SIMARD-MD and strong claims about its value, there is a paucity of research on its clinical utility. There are legitimate concerns regarding the predictive ability of the SIMARD-MD towards pass/fail outcomes on road tests and its value within a clinical setting.
Hence, the objective of the present study was to evaluate the utility of the SIMARD-MD as a screening tool in a sample of CI older adults and in a sample of cognitively healthy older adults. Specifically, we compared SIMARD-MD scores between CI drivers and healthy controls, examined the extent to which the SIMARD-MD correlated with other common clinical measures, and determined whether SIMARD-MD scores differentiated between pass/fail outcomes on the road test in CI drivers. Additionally, we determined the ability of the SIMARD-MD to accurately predict pass/fail outcomes in CI drivers.
This study received ethics approval from the University of Waterloo, University of Saskatchewan, and Lakehead University. Data from comprehensive driving evaluations, performed by an occupational therapist, were collected retrospectively from one health centre in southwestern Ontario and from one health centre in Saskatchewan. Comprehensive driving evaluations consist of a clinical battery of tests assessing vision, motor control, and cognition, followed by a road test.
Given that comprehensive driving assessment centres typically test drivers referred for medical reasons, cognitively healthy drivers were recruited to provide a comparison group. Cognitively healthy participants were recruited from a city in northwestern Ontario through posters or direct telephone/e-mail solicitation of individuals who had previously consented to be contacted for driving research. These participants provided informed consent and completed a clinical assessment and a road test.
During the study time period there were 201 drivers from southwestern Ontario and 183 drivers from Saskatchewan who were referred for a comprehensive driving evaluation (CDE) either by the licensing authority or a physician. Of these drivers, 88 had cognitive impairment and/or dementia and were referred by a physician.
Thirty cognitively healthy participants were recruited and met the following conditions: 1) were community dwelling; 2) aged 70 years or older; 3) fluent in English, 4) possessed a valid driver’s licence, and 5) drove at least once per week. The exclusion criterion was a Standardized Mini-Mental State Examination (SMMSE) score of less than 24 (no participants scored below this threshold). All cognitively healthy participants received a $10 gift card for their participation.
Drivers provided basic information about their driving history. Participants also completed the SIMARD-MD, the Trail A and B tests, and the Useful Field of View (UFOV) test.
The SIMARD-MD test includes two subtests assessing immediate recall and one subtest measuring delayed recall, two tasks that measure the ability to remember words, one task involving converting numbers to words, and one task where the participant names objects that can be purchased at a supermarket within a one-minute time frame. Scores can range from 0 to 130; higher scores indicate better cognitive abilities. The SIMARD-MD score is classified into one of three categories: ≤30, 31–70, and >70. According to the test developers, a score of ≤30 predicts that the person will fail a road test, from 31 to 70 it is uncertain whether they will pass or not, and those who score higher than 70 are predicted to pass a road test.(15) The Trail Test A and B tests are used to assess psychomotor speed and divided attention, respectively.(22) Individuals with longer completion times on the Trails B, in particular, have greater odds of failing a driving test (OR = 2.5, 95% CI: 1.0–5.9).(23) The UFOV(24–25) includes three distinct parts measuring (in milliseconds) processing speed, divided attention, and selective attention, and is associated with on-road performance; higher scores indicate poorer performance.(26) Based on the completion times of the three UFOV sub-tests, a composite score from 1 to 5 is produced; higher scores indicating greater risk for poorer driving performance (1 = very low risk, 2 = low risk, 3 = low–moderate risk, 4 = moderate–high risk, and 5 = high risk).
The road test for the CI sample required participants to drive for 45–60 min, depending on traffic patterns. The test was conducted during the day in good weather conditions and included driving on residential, suburban, urban and expressway sections. The occupational therapist determined participants’ driving performance and provided ratings for pass, fail, or fail with lessons and retest. The two fail categories were collapsed and a binary variable was produced for statistical analyses (pass vs. fail). Pass/fail determinations were made using a weighted scoring system based on a total of 95 manoeuvres. A percentage score, based on the correct number of manoeuvres performed on different driving tasks (yielding, signalling, vehicle position, lane maintenance, scanning, gap acceptance speeding), was calculated. Any driver scoring less than 47.5 (or correctly performing less than 50% of all manoeuvres) failed the road test. Additionally, any driver who made hazardous driving errors (e.g., running a stop sign, crashing) received a fail outcome.
A driving instructor with 20 years of experience, from a driving school approved by the Ministry of Transportation of Ontario (MTO), conducted the road tests in cognitively healthy participants. The instructor was blind to the results of the clinical tests. The road test and scoring approach were designed to meet the requirements of the MTO for driver licensing examinations. Each road test was conducted using the same vehicle with dual pedals, using a standardized circuit taking approximately 45 min. Participants received a score out of 100; higher scores indicated better driving performance. A score of 70 or greater was considered a “pass”; scores below were deemed a “fail”. However, serious errors (e.g., which may have resulted in a crash) would have resulted in an automatic “fail” regardless of the final score. All drivers were tested during the day in good weather conditions. For all drivers, the driving instructor also provided an assessment on whether they would benefit from additional training.
Data (demographic information, clinical scores, and road test results) were entered into a SPSS database (version 26.0) by graduate research assistants. Data entry was monitored by the research team to ensure data completion and accuracy. Descriptive statistics were calculated for demographic information (e.g., age and gender), clinical measures, and the road test. Continuous variables are shown using the mean and standard deviation, while categorical variables are presented using frequencies and percentages. Pearson correlations were conducted to determine associations between SIMARD-MD scores and the other clinical measures (Trails A and B, UFOV). Independent t-tests or Chi-squares tests were used to examine the association between clinical scores and pass/fail outcomes on the road test. Two-tailed significance tests with alpha ≤.05 were used for all analyses.
A ROC curve was used to determine the validity of the SIMARD-MD against the pass/fail road test results. The ROC curve is a graphical representation of various cut-off points (based on the SIMARD-MD scores), each measuring the rate of true positives (sensitivity) vs. the rate of false positives (1-specificity), at different cut-off points for the SIMARD-MD. The AUC is considered an index of the overall predictive utility of a screening test and ranges from 0 to 1.0 (perfect prediction), where .50 represents chance discrimination, and .70 to .90 is considered an acceptable magnitude.(27) The ROC curve also captures the predictive value of a positive test (PPV), the predictive value of a negative test (NPV), and error (1-sensitivity + 1-specificity; essentially false negatives + false positives). A ROC curve was only performed on participants with CI due to the genuine uncertainty regarding driving safety for these individuals, and to prevent the introduction of spectrum bias with the inclusion of cognitively healthy drivers.(28) Using SPSS, ROC curves and AUC estimates, 95% confidence intervals, and p values were generated. We also determined the specificity, sensitivity, PPV, NPV and error rate at selected cut-off points.
Individuals with CI ranged in age from 45 to 94 years (mean 75.2, SD = 10.2); 77% were men. Three quarters of the sample had less than grade 12 education (75%), with the remaining 25% having completed college or university. Comorbid medical conditions most commonly reported were hypertension (23.5%), arthritis (13%), diabetes (10.3%), depression (11.7%), and stroke/TIA (7.4%). The cognitively healthy group ranged in age from 70 to 87 years (mean age of 75.6, SD = 5.1); 56.7% were men.
Prior to the comprehensive driving evaluation (CDE), the CI sample reported a history of driving ranging from 29 to 73 years. About 12% of the sample reported being involved in a crash and 8.4% receiving citations in the past two years. A physician referred all CI individuals for a CDE; five of whom had previously failed a first CDE. Prior to the CDE, the healthy participants reported a history of driving ranging from 39 to 71 years.
As shown in Table 1, individuals with CI, compared to cognitively healthy participants, had poorer results on the Trail A and B tests, had poorer processing speed, divided and selective attention on the UFOV, and had worse scores on the SIMARD-MD. When the UFOV and SIMARD-MD were categorized, a significantly greater proportion scored in the impaired ranges.
TABLE 1 Comparison of clinical measures between CI and Control participantsa
Table 2 shows the correlations between the clinical measures for the CI sample. Almost all clinical measures were significantly correlated with each other. The SIMARD-MD was weakly associated with the Trail Making A test and moderately associated with the Trails Making B test, as well as all three UFOV subtests.
TABLE 2 Correlations between clinical measures in CI drivers (N=86)
All cognitively healthy participants passed the road test, although 24 (80%) were considered candidates who would benefit from additional training (given the development of bad habits over time, this is possibly true of the general population). Associations between pass/fail outcomes on the road test and SIMARD-MD mean and risk index scores could not be calculated as no participant failed the road test. However, according to the SIMARD-MD scores, 21 of these participants were predicted to pass the road test, eight were in the “indeterminate” category and would require further testing, and one was predicted to fail the road test.
Table 3 compares scores on the clinical measures and pass/fail outcomes on the road test for the CI group. Of the 86 CI drivers, 48 (55.8 %) failed the road test. The SIMARD-MD mean scores were significantly poorer in those who failed the road test (t = 3.31, p = .001); however, there were no significant differences between SIMARD classifications between those who failed and passed the road test (p = .053). Participants with CI who were older and who performed more poorly on the Trail B test and all three UFOV subtests, were significantly more likely to fail the road test.
TABLE 3 Characteristics of CI drivers based on road test results (N=86)a
Figure 1 shows the ROC curve for the CI sample. The AUC is = 0.692, 95% CI = (0.578, 0.806), p = .003. Of the 48 individuals who failed the road test, 26 scored ≤30, 20 scored in the indeterminate range (31–70), and 2 scored > 70. Of the 38 who passed the road test, 11 scored ≤30, 23 scored in the indeterminate range (31–70), and 4 scored > 70. When examining individual cut-points on the SIMARD-MD, none provided both good specificity and sensitivity. For example, using a score of ≤30 provided a specificity of .71, a sensitivity of .54, a PPV of .70, and a NPV of .55. Similarly, using the cut-point of ≤70 provided a specificity of .60, a sensitivity of .67, a PPV of .11, and a NPV of .96. These findings suggest that the SIMARD-MD score of ≤30, which is used to identify unsafe drivers, and scores of >70, which is used to identify safe drivers, misclassify a large number of individuals.
|
|
||
|
FIGURE 1 ROC curve based on SIMARD-MD scores predicting pass/fail on the road test in drivers with CI. Specificity is defined as the probability that a person who passes the road test (a true negative) has a negative result on the SIMARD-MD (score of ≤ 30, ≤ 40, ≤ 50, etc). Sensitivity is defined as the probability to obtain a positive test on the SIMARD-MD (score of ≤ 30, ≤ 40, ≤ 50, etc) when a person fails the road test (true positive). |
||
Although the SIMARD-MD mean scores were statistically different between those who passed and failed the road test, the SIMARD-MD was not predictive, at an individual level, of the pass/fail outcome on the road test. Using the selected cut-points on the SIMARD-MD to predict pass/fail on the road test resulted in a high degree of error, regardless of the cut-point chosen, reflecting the typical trade-off between specificity and sensitivity. This resulted in a large number of misclassifications. For example, 29% of participants who scored 30 or below on the SIMARD-MD passed the road test. Scoring in the indeterminate range was near a 50% chance of passing/failing the road test, similar to the findings of prior studies.(18,20) The high misclassification error rate, regardless of the cut-point used, is further evidenced by the limited AUC (.692). The data show that the SIMARD-MD does not have the ability to accurately distinguish between safe and unsafe drivers, and should not be used as a sole screening tool in drivers with cognitive impairment.
The SIMARD-MD was moderately correlated to other measures typically used in comprehensive driving evaluations. Mean scores on the SIMARD-MD were negatively and moderately associated with the Trails B and all three UFOV subtests, as expected. This highlights some degree of overlap between the characteristics measured by the SIMARD-MD and those measured by other tools associated with driving performance. However, the data also show that the SIMARD-MD does not perform any better than other common clinical tests in differentiating between pass/fail outcomes in CI drivers. Specifically, the SIMARD risk indices did not significantly distinguish between CI drivers who passed or failed the road test, whereas the UFOV risk index and Trail Making B test did.
Consistent with other studies, we found that drivers with CI perform significantly worse on measures of visual and cognitive attention compared to healthy participants. Individuals with CI took on average almost 5 min to complete the Trails B and more than two thirds were classified as moderate-to-high risk on the UFOV (Risk Index 3 and higher). Individuals with CI who failed the road test, compared to those who passed, were significantly older and scored more poorly on Trails B, the UFOV, and the SIMARD-MD. However, almost 40% of participants who passed the road test were classified as moderate-to-high risk on the UFOV (i.e., Risk Index of 3 and above) and took 4 min to complete the Trails B, on average. These findings also show that other common measures, such as the Trails B and UFOV, can also misclassify those who should pass or fail a road test.
Similarly, in the cognitively healthy group where all drivers passed the road test, the SIMARD-MD would have incorrectly classified 30% of the participants (eight as indeterminate and one as predicted to fail the road test). This is a good example of problems arising with positive test results in low prevalence situations; when the prevalence of a condition is low, as it was with the absence of unsafe drivers in the cognitively healthy group, the number of false positives (i.e., those scored as indeterminate or fail on the SIMARD-MD when, in fact, they are fit to drive) generally increases. Using a test that has been developed to detect a given condition will not have the same psychometric properties when used with a population that has “different amounts of the trait in question”.(29) Given that the SIMARD-MD was developed with a sample where half of the participants were deemed unsafe, using it in a clinical setting where there are few unsafe drivers (e.g., a family practice office setting) would result in many false positives. This would lead to an overly large number of drivers requiring further testing, costing millions of dollars in additional road tests that would not be necessary, causing undue stress, and possibly even leading to safe drivers losing their driving privilege.
A limitation of our study was that we did not control for education. While there are no guidelines on the SIMARD-MD for education adjustment,(15) a prior study found that drivers with post-secondary education scored 8.19 points higher on the SIMARD-MD than those who did not have post-secondary education.(19) In our study, 75% of CI individuals did not have post-secondary education; these individuals may have been at greater risk of being labelled as indeterminate or likely to fail a road test than participants with more education. Nevertheless, our goal was to test the SIMARD-MD and score it as proposed by its authors. Given our results, it is doubtful that it would perform much better even if there was a small adjustment for education.
We found that using the SIMARD-MD resulted in a large number of false positives and false negatives. Further, given that approximately 50% of the participants fell into the indeterminate range and would require further assessment, it is unclear what value the test really provides. We conclude first that the SIMARD-MD should not be used as a routine test in a family practice office setting. Second, where there are concerns about someone’s ability to drive safely, it must not be used as the sole determinant for driving recommendations, if at all.
This study was funded by the University of Waterloo’s CIHR Research Incentive Fund (RIF) (PI: Crizzle).
The authors declare that no conflicts of interest exist.
1 Statistics Canada [Internet]. Population projections for Canada (2018–2068, Provinces and Territories (2018–2043). Catalogue number 91-520-X. Ottawa, ON: Statistics Canada; 2019 [cited 2020 Jan 24]. Available from: https://www150.statcan.gc.ca/n1/pub/91-520-x/2019001/hi-fs-eng.htm
2 Turcotte M. Profile of seniors’ transportation habits [Internet]. Statistics Canada Catalogue no 11-008-X. Ottawa, ON: Statistics Canada; 2012 [cited 2020 Jan 24]. Available from: http://www.statcan.gc.ca/pub/11-008-x/2012001/article/11619-eng.pdf
3 Carr DB, Ott BR. The older adult driver with cognitive impairment. “It’s a very frustrating life.” JAMA Network. 2010;303(16): 1632–1641.
4 Ott BR, Daiello LA. How does dementia affect driving in older patients? Aging Health. 2010;6(1):77–85.
5 Hopkins RW, Kilik L, Day DJA, et al. Driving and dementia in Ontario: a quantitative assessment of the problem. Can J Psychiatry. 2004;49(7):434–438.
6 Barco PP, Baum CM, Ott BR, et al. Driving errors in persons with dementia. J Am Geriatr Soc. 2015;63(7):1373–1380.
7 Davis JD, Papandonatos GD, Miller LA, et al. Road test and naturalistic driving performance in healthy and cognitively impaired older adults: does environment matter? J Am Geriatr Soc. 2012;60(11):2056–2062
8 Ott BR, Heindel WC, Papandonatos GD, et al. A longitudinal study of drivers with Alzheimer disease. Neurology. 2008;70(14):1171–1178
9 Jones Ross RW, Scialfa CT, Cordazzo STD. Predicting on-road driving performance and safety in cognitive impaired older adults. J Am Geriatr Soc. 2015;63(11):2365–2369.
10 Piersma D, Fuermaier ABM, De Waard D, et al. Prediction of fitness to drive in patients with Alzheimer’s dementia. PLoS ONE. 2016;11(2):e0149566.
11 Chee JN, Rapoport MJ, Molnar F, et al. Update on the risk of motor vehicle collision or driving impairment with dementia: a collaborative international systematic review and meta-analysis. Am J Geriatr Psychiatry. 2017;25(12):1376–1390.
12 Man-Son-Hing M, Marshall SC, Molnar FJ, et al. Systematic review of driving risk and the efficacy of compensatory strategies in persons with dementia. J Am Geriatr Soc. 2007; 55(6):878–884.
13 Wadley VG, Okonkwo O, Crowe M, et al. Mild cognitive impairment and everyday function: an investigation of driving performance. J Geriatr Psychiatry Neurol. 2009;22(2):87–94.
14 Duchek JM, Carr DB, Hunt L, et al. Longitudinal driving performance in early-stage dementia of the Alzheimer type. J Am Geriatr Soc. 2003;51(10):1342–1347.
15 Dobbs BM, Schopflocher D. The introduction of a new screening tool for the identification of cognitively impaired medically at-risk drivers. The SIMARD—a modification of the DemTech. J Prim Care Commun Health. 2010;1(2):119–127.
16 Kalbe E, Kessler J, Calabrese P, et al. DemTect: A new, sensitive cognitive screening test to support the diagnosis of mild cognitive impairment and early dementia. Int J Geriatr Psychiatry. 2004;19(2):136–143.
17 Hogan DB, Bédard M. Papers that might change your practice: review of the introduction of a new screening tool for the identification of cognitively impaired medically at-risk drivers. Can Geriatr J. 2011;14(2):51–54.
18 Bédard M, Weaver B, Man-Son-Hing M, Classen S, Porter M. The SIMARD screening tool to identify unfit drivers: Are we there now? J Prim Care Community Health. 2011;2:133–135.
19 Bédard M, Marshall S, Man-Son-Hing M, et al. It is premature to test older drivers with the SIMARD-MD. Accid Anal Prevent. 2013;61:317–321.
20 Wernham M, Jarrett PG, Stewart C, et al. Comparison of the SIMARD MD to clinical impression in assessing fitness to drive in patients with cognitive impairment. Can Geriatr Soc. 2014;17(2):63–69.
21 Government of British Columbia. Ministry of Public Safety and Solicitor General Office of the Superintendent of Motor Vehicles. 2010 BC guide in determining fitness to drive. (No. Version 1.5). Vancouver, BC: Ministry of Public Safety and Solicitor General Office of the Superintendent of Motor Vehicles; 2010. [cited 2020 Jan 24]. Available from: https://www2.gov.bc.ca/assets/gov/driving-and-transportation/driving/roadsafetybc/medical-fitness/medical-prof/2010-guide-in-determining-fitness-to-drive.pdf
22 Lezak M, Howieson DB, Bigler ED, et al. Neuropsychological assessment, 3rd ed. New York: Oxford University Press; 1995.
23 Classen S, Horgas A, Awadzi K, et al. Clinical predictors of older driver performance on a standardized road test. Traffic Inj Prevent. 2008;9(5):456–462.
24 Ball K, Owsley C. The useful field of view test: a new technique for evaluating age-related declines in visual function. J Am Optometr Assoc. 1993;64(1):71–79.
25 Edwards JD, Vance DE, Wadley VG, et al. Reliability and validity of useful field of view test scores as administered by personal computer. J Clin Experim Neuropsychol. 2005;27(5):529–543.
26 Clay OJ, Wadley V, Edwards JD, et al. Cumulative meta-analysis of the relationship between useful field of view and driving performance in older adults: current and future implications. Optom Vis Sci. 2005;82(2):724–731.
27 Streiner DL, Cairney V. What’s under the ROC? An introduction to receiver operating characteristics curves. Can J Psychiatr. 2007;52(2):121–128.
28 Byszewski A, Molnar FJ, French Markley V, et al. Driving and dementia toolkits for health professionals and for patients and caregivers. CGS J CME. 2012;2(3):10–13.
29 Streiner DL. Diagnosing tests: using and misusing diagnostic and screening tests. J Person Assess. 2003;81(3):209–219.
Canadian Geriatrics Journal, Vol. 24, No. 1, March 2021