Philip Donald St John , MD MPH CCFP FRCPC 1,2, Frank Joseph Molnar , MSC MD CM FRCPC 3
1 Section of Geriatric Medicine, Max Rady College of Medicine, University of Manitoba, Winnipeg, MB
2 Centre on Aging, University of Manitoba, Winnipeg, MB
3 Division of Geriatric Medicine, University of Ottawa, Ottawa, ON
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial No-Derivative license (https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits unrestricted non-commercial use and distribution, provided the original work is properly cited.
DOI: https://doi.org/10.5770/cgj.25.525
ABSTRACT
Background
The Ottawa 3DY (O3DY) is a simple measure of cognition.
Objectives
1) To determine if the O3DY predicts mortality; and 2) To compare the discrimination of the O3DY to the Mini-Mental State Examination (MMSE) and Modified MMSE (3MS).
Methods
Analyses of a population based cohort study of 1,751 participants aged 65+; conducted in 1991/2 with follow-up over five years. The O3DY, age, sex, education, comorbid conditions, the MMSE, and the 3MS were measured: 4.5% of the participants had missing data for the O3DY; 42.8% were considered as positive (one or more errors), and 52.7% were considered as negative (no errors). Logistic regression models were constructed with the outcome of death at time 2. A Receiver Operator Curve (ROC) was constructed and the Area Under the ROC (AUROC) was calculated using a c-statistic.
Results
The unadjusted odds ratio (OR) and 95% confidence interval (CI) for mortality was 1.96 (1.56, 2.47); and the adjusted OR was 1.33 (1.02, 1.72). The AUROC was 0.66 for the 3MS, 0.65 for the MMSE, and 0.60 for the O3DY.
Conclusions
The O3DY predicts mortality over a long time frame, although the discrimination is less than that of longer measures of cognition.
Key words: cohort study, cognitive screening test, Ottawa 3DY, mortality
Impaired cognition is common in older adults, and predicts multiple adverse outcomes. Measures of cognition which have previously been reported to predict these adverse outcomes are: clinical diagnoses of dementia,(1) mild cognitive impairment (MCI),(2) cognitive impairment no dementia (CIND);(3) detailed neuropsychiatric tests,(4) and poor performance on cognitive status tests which require in person assessments(5,6) [such as the Folstein Mini-Mental Status Examination (MMSE) and the modified MMSE (3MS)]. Most older adults also value cognition highly. While current guidelines caution against routine population-based screening for cognitive impairment,(7) a high clinical index of suspicion is appropriate. In certain settings, such as those presenting with memory complaints, as part of comprehensive geriatric assessment, and in high prevalence settings, case finding of cognitive impairment may be appropriate. A wide variety of instruments are available to assess cognition. There has long been interest in simpler measures of cognition which can be rapidly completed in clinical encounters. With the advent of the Covid-19 pandemic, interest has increased in the use of brief cognitive measures which minimize contact time, and which can be completed without the use of potential fomites (such as pencils and paper). Some of these measures could also potentially be administered remotely by telephone or via teleconferencing software. The evidence regarding remote assessments is evolving rapidly. Currently, it appears that virtual assessments may be acceptable to some patients and caregivers,(8) and that some measures—such as the Telephone Interview for Cognitive Status (TICS)(9)—have acceptable diagnostic accuracy compared to in-person instruments, though clearly more research is needed.
The Ottawa 3DY (O3DY) is a simple test of cognition derived from items of the 3MS which do not require pen and paper, and are short and easy to administer (date, day of the week, spelling WORLD backwards, and the year).(10) It has been validated in the Canadian Study of Health and Aging (CSHA) data,(10) and subsequently studied in emergency department settings, where is has good sensitivity for detecting cognitive impairment, as well as acceptable specificity for detecting delirium(11–15)—with one study showing moderate sensitivity and specificity.(16) It has been studied in French as well as English.(17) The O3DY has not been shown to predict adverse outcomes over long time frames.
We have conducted an analysis of the Manitoba Study of Health and Aging (MSHA), a population-based cohort study, to determine if the O3DY predicts adverse outcomes over a five-year time frame. The specific objectives are:
To determine if the O3DY predicts mortality over a five-year period after adjusting for confounding factors; and
To compare the discrimination of the O3DY to the MMSE and 3MS in predicting death.
The MSHA is population-based cohort study conducted in Manitoba, in conjunction with the CSHA.(18,19) The CSHA was a large epidemiological cohort study of those over 65 years old to determine the prevalence, incidence, and outcomes of dementia. Manitoba expanded the CSHA sampling frame to include all regions in the province, including rural areas, and supplementary questions were added. The original sampling frame was from a list provided by Manitoba Health, the provincial ministry of health. Because health-care coverage is universal in Manitoba, this represents a comprehensive representative sampling frame.(20–23) The sampling was stratified by region, with representation from the entire province; 56% lived in Winnipeg. The sampling frame was determined to oversample the oldest population, with a sample ratio of 2 for those aged 75 to 85 and 2.5 for those over 85, relative to those aged 65–75. Persons residing in institutions (nursing homes and chronic care hospitals) did not undergo the screening interview. Initially, 2,890 persons were selected, of whom 1,751 were enrolled. The initial time of assessment was 1991/2 with follow-up in 1996/7.
At both time points, participants who were living in the community were interviewed in their own homes by trained interviewers. Data gathered included age, sex, educational level (years of education), and a list of chronic conditions, including self-reported memory. The Modified Mini-Mental State Examination (3MS)(24) was used as the screening test for cognitive impairment and dementia. The 3MS extends the MMSE(25) to include additional items on date and place of birth, animal naming, similarities, and a second recall task, and expands the total score from 30 points to 100 points. The 3MS was conducted in a manner that allowed calculation of the MMSE.(26) The O3DY can be derived from the 3MS.
Death was ascertained at an interim contact at two years and at follow-up at five years. Data on death were considered at the time of the time 2 screening interview, at the time 2 clinical interview, and at the end of the entire time 2 period in 1997. We consider death at the time of the time 2 clinical interview. There were no missing data for death by 1997. Long-term care use—defined as nursing home residence or living in a chronic hospital—was ascertained at the time 2 screening interview. There were seven people missing data on long-term care use. There were 100 participants who entered long-term care and subsequently died whom we considered within the “dead” category, and 111 participants living in long-term care at the time 2 screening interview. Those who were waiting for long-term care in acute hospitals, and those who had been accepted for long-term care and were waiting in the community, were considered to be living in the community. For those participants who were still in the community, the assessment process and interview questions were similar to those at time 1.
The study adheres to the Declaration of Helskinki, and consent was obtained from the participants, or an appropriate proxy. The original study, as well as these analyses, received ethical approval from the Research Ethics Board of the Bannatyne Campus of the University of Manitoba.
We calculated the O3DY from the 3MS. The primary outcome was death at the end of the follow-up period. The secondary endpoint was death or long-term care admission. We conducted bivariate analyses with the O3DY as a binary variable. For categorical variables, we used chi-square tests, and for continuous variables, we used Student’s t- tests (assuming unequal variance). As a supplementary analysis, we considered the O3DY as a categorical variable (0 to 4 points). We also examined each item (orientation to date, day of the week, and year, as well as the ability to spell WORLD backwards) as a predictor variable. We then constructed logistic regression models with death as our primary outcome and the O3DY as the variable of interest. We included potential confounding variables as covariates in the logistic regression model. Finally, we constructed Receiver Operator Curves (ROC) for the effect of the O3DY on mortality, and calculated a c-statistic for the area under the ROC for the O3DY, the MMSE, and the 3MS. For the secondary outcome of long-term care use, we did not construct regression models. All analyses were conducted in SPSS Version 25 (IBM SPSS Statistics, Armonk, NY).
There were 1,751 participants who were enrolled in 1991/2. Of these, 79 were excluded due to missing data; 77 of these were missing data on spelling WORLD backwards. Most of the participants with missing data had low levels of literacy and were unable to spell WORLD forwards. Overall, 4.5% of the participants had missing data for the O3DY; 42.8% were considered positive (errors), and 52.7% were considered negative (no errors). The characteristics of the sample are shown in Table 1. Those who scored positive on the O3DY were older, had lower educational attainment, and were more likely to be male. Unsurprisingly, they also had lower mean scores on the overall MMSE and 3MS. When considering each item, 70.4% spelled WORLD backwards correctly, 96.1% accurately identified the year, 94.2% accurately identified the day of the week, and 76.1% accurately identified the date. If the O3DY was considered as a categorical score, 0.7% scored 0; 2.3% scored 1; 9.3% scored 2; 30.6% scored 3 and 52.7% scored 4; with 4.5% missing.
TABLE 1
Baseline characteristics of the sample
Over the course of the study, 417 of the participants died: 29.6% of those who scored positive vs. 17.7% who scored negative and 40.5% of those with a missing 03DY ( p < .001, chi square test.) The effect on mortality appeared to be a gradient across the score of the O3DY, if we considered the score as a categorical variable; as did the effect on long-term care admission (Figure 1).
|
|
||
|
FIGURE 1 Ottawa 3DY score and outcomes five years later |
||
In logistic regression models (Table 2), the O3DY predicted mortality, even after adjusting for a variety of potential confounding factors. Male sex, older age, and self-reported heart disease, stroke, diabetes and cancer also predicted death. The lack of an effect of education should be viewed with caution, since in this data set, education, and cognition are correlated, and the effect of education may be mediated through higher cognitive test scores.
TABLE 2
Results of logistic regression models for five year mortality; the odds ratios (ORs) and 95 per cent confidence intervals (95% CI) are shown
We considered the O3DY as a continuous score from 0 to 4, and constructed ROC curves for the O3DY, the MMSE, and the 3MS. All three scores predicted mortality with a moderate degree of discrimination; the c-statistic for the 3MS was 0.66; the MMSE was 0.65; and the O3DY was 0.60. Both the 3MS and the MMSE offered superior discrimination compared to the 03DY.
We have analyzed a prospective population-based cohort study and found that the O3DY predicts mortality, even after adjusting for potential confounding factors. This is a simple, easily administered test which can be conducted without physical contact, which may be useful in some clinical settings. There remain some advantages to considering more detailed cognitive tests in other settings. First, the percent of the population who scored positive on the O3DY was quite high. The intent of the O3DY is to identify those who require more in-depth assessment, and this may be a high percentage of individuals in a general population of older adults, and would likely be even higher in clinical settings. Second, the discrimination of the 03DY is less than the MMSE or the 3MS. This may be due to the expanded scoring within the normal range, which may be prognostically important. None of the tests used alone is discriminative enough to be used alone to predict death. On the other hand, all the cognitive scores are surprisingly accurate predictors of five-year mortality, given their brief and simple nature.
There are some limitations to our study. First, there is some overlap of participants in the MSHA with the CSHA from which the O3DY was derived. This overlap is one-quarter of the total number of participants who participated in both studies. Second, the data set is old. While the association between cognition and death has not likely changed in the intervening time, the baseline mortality rate in Canada has fallen since the time of the study. Third, Manitoba in general, and the participants in the MSHA in particular, had lower educational attainment than the Canadian population and the CSHA participants. Since the time of the MSHA, educational attainment has increased. This may limit generalizability. On the other hand, more modern populations may have fewer missing items as literacy has increased. Fourth, while there were no missing variables for vital status at time 2, there were missing items in our data set—notably on spelling WORLD. These participants had a high mortality rate, and data were not missing at random. Strengths of the study include a large representative sample of older adults using standardized measures gathered in a systematic manner.
Our findings are important for several reasons. First, our findings underscore the importance of cognition as an important prognostic marker.(1,6) Despite being recognized in Hippocrates time,(27) many currently used prognostic indicators do not consider cognition. Second, even brief bedside measures of cognition predict adverse outcomes. While more detailed measures offer more information, even these three basic measures alone offer surprisingly accurate information. More studies in both population-based studies and in clinical samples are warranted to determine if our findings are consistent in other places and times and in other cultures. We do not advocate using cognitive test scores alone to predict death, as their discriminative accuracy is modest. But cognition—even crudely measured—should be one consideration in predicting risk for future mortality.
The O3DY should be studied further for use in the Covid-19 era. As well, its accuracy if administered by telephone or video link as well, as in primary care settings, should be studied. This would allow brief cognitive assessments to be conducted while potentially minimizing viral transmission risk.
We would like to thank the investigators, staff, and participants of the MSHA. Dr. Ian McDowell provided helpful comments on the manuscript.
Dr. St John is an unremunerated member of the Board of Age and Opportunity (Manitoba), and has received speaking fees from the Regional Geriatric Program of Eastern Ontario. Dr. Molnar is the originator of the O3DY; he developed it and first published it. Neither author has a financial interest in the O3DY. The O3DY is intended for open use, and Dr. Molnar does not hold copyright. Drs. St John and Molnar have been colleagues and friends for many years.
The Manitoba Study of Health and Aging (MSHA) was funded primarily by Manitoba Health, with additional funding provided through the Canadian Study of Health and Aging by the Seniors Independence Research Program of the National Health Research and Development Program of Canada (Project No. 6606-3954-MC[S]). The Manitoba Study of Health and Aging (MSHA-2) was funded primarily by Manitoba Health’s Health Communities Development Fund, with additional funding provided through the Canadian Study of Health and Aging by the Seniors Independence Research Program of the National Health Research and Development Program of Health Canada (Project No. 6606-3954-MC[S]). No endorsement from these agencies is implied.
1 Todd S, Barr S, Roberts M, Passmore AP. Survival in dementia and predictors of mortality: a review. Int J Geriatr Psychiatry. 2013;28(11):1109–24. Epub 2013/03/26. doi:10.1002/gps.3946.
2 Bennett DA, Wilson RS, Schneider JA, Evans DA, Beckett LA, Aggarwal NT, et al. Natural history of mild cognitive impairment in older persons. Neurology. 2002;59(2):198–205. Epub 2002/07/24. doi:10.1212/wnl.59.2.198.
3 Tuokko H, Frerichs R, Graham J, Rockwood K, Kristjansson B, Fisk J, et al. Five-year follow-up of cognitive impairment with no dementia. Arch Neurol. 2003;60(4):577–82. Epub 2003/04/23. doi:10.1001/archneur.60.4.577..
4 Rostamian S, de Haan S, van der Grond J, van Buchem MA, Ford I, Jukema JW, et al. Cognitive Function in Dementia-Free Subjects and Survival in Old Age: The PROSPER Study. Am J Med. 2019;132(12):1466–74. Epub 2019/06/23. doi:10.1016/j.amjmed.2019.06.001.
5 Neale R, Brayne C, Johnson AL. Cognition and survival: an exploration in a large multicentre study of the population aged 65 years and over. Int J Epidemiol. 2001;30(6):1383–88. Epub 2002/02/01.
6 St John PD, Montgomery PR, Kristjansson B, McDowell I. Cognitive scores, even within the normal range, predict death and institutionalization. Age Ageing. 2002;31(5):373–78. Epub 2002/09/21.
7 Owens DK, Davidson KW, Krist AH, Barry MJ, Cabana M, Caughey AB, et al. Screening for cognitive impairment in older adults: US Preventive Services Task Force recommendation statement. JAMA. 2020;323(8):757–63. Epub 2020/02/26. doi:10.1001/jama.2020.0435.
8 Gosse PJ, Kassardjian CD, Masellis M, Mitchell SB. Virtual care for patients with Alzheimer disease and related dementias during the COVID-19 era and beyond. CMAJ. 2021;193(11):E371–E77. doi:10.1503/cmaj.201938.
9 Watt JA, Lane NE, Veroniki AA, Vyas MV, Williams C, Ramkissoon N, et al. Diagnostic accuracy of virtual cognitive assessment and testing: systematic review and meta-analysis. J Am Geriatr Soc. 2021;69(6):1429–40. doi:https://doi.org/10.1111/jgs.17190.
10 Molnar FJ, Wells GA, McDowell I. The derivation and validation of the Ottawa 3D and Ottawa 3DY three-and four-question screens for cognitive impairment. Clin Med Insights: Geriatr. 2008;2:1.
11 Yadav K, Boucher V, Carmichael PH, Voyer P, Eagles D, Pelletier M, et al. Serial Ottawa 3DY assessments to detect delirium in older emergency department community dwellers. Age Ageing. 2020;49(1):130–34. Epub 2019/11/23. doi:10.1093/ageing/afz144.
12 Ollerton RL. Sensitivity of the Ottawa 3DY Scale for assessment of cognitive impairment of older emergency department patients. Ann Emerg Med. 2018;71(1):152–53. Epub 2017/12/23. doi:10.1016/j.annemergmed.2017.08.016.
13 Wilding L, Eagles D, Molnar F, O’Brien JA, Dalziel WB, Moors J, et al. Prospective validation of the Ottawa 3DY Scale by geriatric emergency management nurses to identify impaired cognition in older emergency department patients. Ann Emerg Med. 2016;67(2):157–63. Epub 2015/11/27. doi:10.1016/j.annemergmed.2015.09.008.
14 Carpenter CR, Bassett ER, Fischer GM, Shirshekan J, Galvin JE, Morris JC. Four sensitive screening tools to detect cognitive dysfunction in geriatric emergency department patients: Brief Alzheimer’s Screen, Short Blessed Test, Ottawa 3DY, and the Caregiver-Completed AD8. Acad Emerg Med. 2011;18(4):374–84. Epub 2011/04/19. doi:10.1111/j.1553-2712.2011.01040.x.
15 Carpenter CR, Banerjee J, Keyes D, Eagles D, Schnitker L, Barbic D, et al. Accuracy of dementia screening instruments in emergency medicine: a diagnostic meta-analysis. Acad Emerg Med. 2019;26(2):226–45. Epub 2018/09/18. doi:10.1111/acem.13573.
16 Barbic D, Kim B, Salehmohamed Q, Kemplin K, Carpenter CR, Barbic SP. Diagnostic accuracy of the Ottawa 3DY and Short Blessed Test to detect cognitive dysfunction in geriatric patients presenting to the emergency department. BMJ open. 2018;8(3):e019652. Epub 2018/03/20. doi:10.1136/bmjopen-2017-019652.
17 Bedard C, Boucher V, Voyer P, Yadav K, Eagles D, Nadeau A, Et Al. Validation of the O3DY French Version (O3DY-F) for the Screening of Cognitive Impairment in Community Seniors in the Emergency Department. J Emerg Med. 2019;57(1):59–65. Epub 2019/04/27. doi:10.1016/j.jemermed.2019.02.007.
18 CSHA Working Group. The Canadian Study of Health and Aging: risk factors for Alzheimer’s disease in Canada. Neurology. 1994;44(11):2073–80. Epub 1994/11/01.
19 CSHA Working Group. Canadian Study of Health and Aging: study methods and prevalence of dementia. CMAJ. 1994;150(6):899–913. Epub 1994/03/15.
20 St John P, Montgomery P. Does self-rated health predict dementia? J Geriatr Psychiatry Neurol. 2013;26(1):41–50. Epub 2013/03/05. doi:10.1177/0891988713476369.
21 St John PD, Montgomery PR. Cognitive impairment and life satisfaction in older adults. Int J Geriatr Psychiatry. 2010;25(8):814–21. Epub 2010/07/14. doi:10.1002/gps.2422.
22 St John PD, Tyas SL, Montgomery PR. Cognition, even in the normal range, predicts disability: cross-sectional and prospective analyses of a population-based sample. Int J Geriatr Psychiatry. 2015;30(10):1008–16. Epub 2015/02/03. doi:10.1002/gps.4254.
23 Tyas SL, Manfreda J, Strain LA, Montgomery PR. Risk factors for Alzheimer’s disease: a population-based, longitudinal study in Manitoba, Canada. Int J Epidemiol. 2001;30(3):590–97. Epub 2001/06/21.
24 Teng EL, Chui HC. The Modified Mini-Mental State (3MS) examination. J Clin Psychiatry. 1987;48(8):314–18. Epub 1987/08/01.
25 Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatric Res. 1975;12(3):189–98. Epub 1975/11/01. doi:10.1016/0022-3956(75)90026-6.
26 McDowell I, Kristjansson B, Hill GB, Hebert R. Community screening for dementia: the Mini Mental State Exam (MMSE) and Modified Mini-Mental State Exam (3MS) compared. J Clin Epidemiol. 1997;50(4):377–83. Epub 1997/04/01. doi:10.1016/s0895-4356(97)00060-7.
27 St John PD, Montgomery PR. Utility of Hippocrates’ prognostic aphorism to predict death in the modern era: prospective cohort study. BMJ. 2014;349:g7390. Epub 2014/12/17. doi:10.1136/bmj.g7390.
Canadian Geriatrics Journal, Vol. 25, No. 1, March 2022