Andrew J. Larner, MD, PhD
Cognitive Function Clinic, Walton Centre for Neurology and Neurosurgery, Liverpool, United Kingdom.DOI: https://doi.org/10.5770/cgj.23.405
ABSTRACT
Background
The Mini-Addenbrooke’s Cognitive Examination (MACE) is a recently described brief cognitive screening instrument.
Objective
To examine the test accuracy of MACE for the identification of dementia and mild cognitive impairment (MCI) in a cohort of older patients assessed in a neurology-led dedicated cognitive disorders clinic.
Methods
Cross-sectional assessment of consecutive patients with MACE was performed independent of the reference standard diagnosis based on clinical interview of patient and, where possible, informant and structural brain imaging, and applying standard clinical diagnostic criteria for dementia and MCI. Various test accuracy metrics were examined at two MACE cut-offs ( ≤ 25/30 and ≤ 21/30), comparing the whole patient cohort with those aged ≥ 65 or ≥ 75 years, hence at different disease prevalences.
Results
Dependent upon the chosen cut-off, MACE was either very sensitive or very specific for the identification of any cognitive impairment in the older patient cohorts with increased disease prevalence. However, at both cut-offs the positive predictive values and post-test odds increased in the older patient cohorts. At the more sensitive cut-off, improvements in some new unitary test metrics were also seen.
Conclusion
MACE is a valid instrument for identification of cognitive impairment in older people. Test accuracy metrics may differ with disease prevalence.
Key words: cognitive screening, dementia, mild cognitive impairment, Mini-Addenbrooke’s Cognitive Examination, older people, screening
Age is the most important risk factor for the development of cognitive decline and dementia. Various guidelines and recommendations addressing the use of cognitive screening instruments (CSIs) in older adults have been published. The Canadian Task Force on Preventive Health Care strongly recommended against screening asymptomatic older adults (≥ 65 years), but indicated that consideration should be given to cognitive assessment if patients had signs and symptoms of impairment or if family members of patients expressed concerns about possible cognitive decline.(1) As to which CSI(s) might be used for this purpose, the US Preventive Services Task Force reported that several brief instruments can adequately detect dementia, but found no empirical evidence that such screening improved outcomes.(2) The Alzheimer Association mentioned 15 different tools which might be used, but recognized that no optimal CSI was suitable for all patient populations and settings.(3) In the United Kingdom, the Alzheimer’s Society produced a “practical toolkit” to assess cognition in older people, recommending different CSIs in different settings: for memory clinics, the specific recommendations were the Montreal Cognitive Assessment (MoCA) and the third iteration of the Addenbrooke’s Cognitive Examination, ACE-III.(4)
The Mini-Addenbrooke’s Cognitive Examination (MACE) is a relatively recently described screening instrument, derived from ACE-III by Mokken scaling.(5) MACE comprises tests of attention, memory (7-item name and address), verbal fluency, clock drawing, and memory recall (score range 0–30). The index study identified two cut-offs, one with high sensitivity (≤ 25/30) and one with high specificity (≤ 21/30).(5) Independent studies of MACE have reported its utility for identification of cognitive impairment in various clinical settings.(6–9)
Pragmatic screening test accuracy studies (DTAS) are required to inform the choice of suitable CSIs.(10) The aim of this study was to examine the screening utility of MACE for dementia and mild cognitive impairment (MCI) versus subjective memory complaint in older people (≥ 65 and ≥ 75 years) (i.e., in samples enriched for those at greatest risk of cognitive impairment and dementia). This affords an opportunity to observe how the examined metrics vary (i.e., how the test performs) with changing disease prevalence, for although within a given population sensitivity and specificity are properties of the test, their values change in different populations.
Some preliminary (two-year) data from this study have appeared as part of a broader study examining the utility of various CSIs in older patients (aged ≥ 65 years) seen in a dedicated cognitive disorders clinic.(11)
The dataset from a pragmatic prospective screening test accuracy study examining MACE in consecutive new patients referred over the period June 2014 to December 2018 to a neurology-led dedicated cognitive function clinic based in a regional neuroscience centre(9) was re-interrogated. The clinic operates no age-related exclusion criteria. Data from patients aged ≥ 65 and ≥ 75 years, as well as the whole cohort, were examined. Subjects gave informed consent and the study protocol was approved by the institute’s committee on human research.
Cross-sectional assessment of all patients comprised semi-structured patient history enquiring about cognitive symptoms and functional performance, with collateral history where possible; neuroradiological examination (brain CT in all patients; interval MR imaging in some cases); and formal neuropsychological assessment in some cases. Administration of MACE occurred on the same day as, but separate from, the cross-sectional assessment. Standard diagnostic criteria for dementia (DSM-IV)(12) and MCI (Petersen et al.)(13) were used; absence of dementia or MCI was categorized as subjective memory complaint (SMC). Criterion diagnosis (reference standard) was by judgment of an experienced clinician based on diagnostic criteria, but blinded to MACE scores in order to avoid review bias. STARDdem guidelines for reporting diagnostic test accuracy studies in dementia were observed.(14)
Using the two MACE cut-offs (≤ 25/30, more sensitive, and ≤ 21/30, more specific) from the index study,(5) in order to avoid any possible introduction of bias,(15) standard summary measures of discrimination(16) were calculated in each of the three cohorts (whole, ≥ 65 years, and ≥ 75 years) for any cognitive impairment (dementia plus MCI) versus SMC. These measures were sensitivity, specificity, positive and negative predictive values (PPV and NPV), correct classification accuracy (CCA), positive and negative likelihood ratios (LR+, LR-) classified as per Jaeschke et al.,(17) and positive and negative clinical utility indexes (CUI+, CUI-) classified as per Mitchell.(18)
In addition to these standard summary measures, “number needed” metrics were also calculated: to diagnose (NND) = 1/Y, where Y = Youden index (Y = sensitivity + specificity − 1);(19) to predict (NNP) = 1/PSI, where PSI = predictive summary index (PSI = PPV + NPV − 1);(19) and to misdiagnose (NNM) = 1/(1 - CCA).(20) New unitary metrics for DTAS,(21) as used in the main study,(9) were also calculated, namely the likelihood to diagnose or misdiagnose (LDM = NNM/NND or NNM/NNP), the summary utility index (SUI = CUI+ + CUI-), and the number needed for screening utility (NNSU = 1/SUI), and classified as previously.(9,21)
Over one-third of the patients were aged ≥ 65 years (n = 287; 38%), whereas less than one-fifth were aged ≥ 75 years (n = 119; 16%). As anticipated, the prevalence of any cognitive impairment (dementia plus MCI) was higher in the older age groups compared to the whole cohort (Table 1).
Standard summary measures of discrimination for the identification of dementia plus MCI versus SMC at the MACE ≤ 25/30 cut-off (Table 2) showed the test was very sensitive (>0.95) in all three patient cohorts (whole vs. ≥ 65 years vs. ≥ 75 years cohorts) with very large negative likelihood ratios. The positive predictive value (0.589 vs. 0.859 vs. 0.957), correct classification accuracy (0.685 vs. 0.854 vs. 0.950), post-test odds (1.432 vs. 6.088 vs. 22.0), and positive clinical utility index (0.570 vs. 0.827 vs. 0.948) all increased with increasing prevalence of cognitive impairment, whilst negative predictive value and negative clinical utility index both decreased.
TABLE 2 Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% CI) using MACE cut-off ≤ 25/30 in whole cohort vs. cohorts of older patients (aged ≥ 65 and ≥ 75 yrs)
Examining the “number needed” and unitary metrics (Table 3), there was an increase in the number needed to misdiagnose with increasing prevalence of cognitive impairment (3.17 vs. 6.85 vs. 20.0), and this was reflected in the increasing values of the likelihood to be diagnosed or misdiagnosed (LDM) measure. SUI and NNSU remained relatively constant, and adequate, throughout.
TABLE 3 Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of “number needed” and unitary measures of discrimination using MACE cut-off ≤ 25/30 in whole cohort vs. cohorts of older patients (aged ≥ 65 and ≥ 75 yrs)
Standard summary measures of discrimination for the identification of dementia plus MCI versus SMC at the MACE ≤ 21/30 cut-off (Table 4) showed the test was more specific than sensitive in all three patient cohorts (whole vs. ≥ 65 years vs. ≥ 75 years cohorts) with increasing positive likelihood ratios. As observed with the ≤ 25/30 cut-off, once again the positive predictive value (0.737 vs. 0.975 vs. 0.989), correct classification accuracy (0.767 vs. 0.780 vs. 0.807), post-test odds (2.798 vs. 39.0 vs. 89.0), and positive clinical utility index (0.546 vs. 0.708 vs. 0.793) all increased with increasing prevalence of cognitive impairment, whilst negative predictive value and negative clinical utility index both decreased.
TABLE 4 Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of standard summary measures of discrimination (with 95% CI) using MACE cut-off ≤21/30 in whole cohort vs. cohorts of older patients (aged ≥65 and ≥75 yrs)
Examining the “number needed” and unitary metrics (Table 5), there was discrepancy in NND (decreased) and NNP (increased) with increasing prevalence of cognitive impairment and, hence, no consistent pattern in values of the likelihood to be diagnosed or misdiagnosed (LDM) measure. SUI and NNSU remained relatively constant, and adequate, throughout.
TABLE 5 Diagnosis of dementia plus MCI vs. no cognitive impairment (SMC): comparison of “number needed” and unitary measures of discrimination using MACE cut-off ≤21/30 in whole cohort vs. cohorts of older patients (aged ≥65 and ≥75 yrs)
MACE proved acceptable to patients in the clinic setting, and was quick and easy to administer and score, as previously reported.(9) The data indicate that, at both MACE cut-offs examined, test PPV, CCA, post-test odds, and CUI+ improved in the older patient cohorts with increased prevalence of cognitive impairment, with some decline in NPV and CUI-. At the more sensitive MACE cut-off there was increase in the values of LDM with increasing prevalence of cognitive impairment, and the values of SUI and NNSU remained adequate at both cut-offs.
Potential limitations of the study include those related to all clinic-based studies, such as the inevitable selection bias. Most of the patients seen in this clinic are referred direct from primary care settings, have not been previously seen by any specialist (general physician, geriatrician, or neurologist), and have not been administered any cognitive screening instrument prior to referral.(22) Older patients seen in a neurology-led clinic may differ from those seen in geriatric or old age psychiatry memory clinics, for example in terms of comorbidities. Clearly further studies of MACE in these settings are required, and no definite comment can be made about test utility for the oldest old patients. The chosen age cut-offs (≥ 65 and ≥ 75 years) were arbitrary. One consequence of this choice was the relatively small number of SMC patients in the ≥ 75 years cohort (n = 8), a limitation reflected in the wide confidence intervals of some of the standard test metrics. Another possible limitation was examining only two MACE cut-offs, as per the index paper,(5) in order to avoid any possible introduction of bias,(15) although a previous analysis looked systematically at a range of cut-offs.(9)
Notwithstanding these limitations, the data presented suggest MACE may be a valid instrument, meaning that it does identify what it claims to identify, namely individuals with any cognitive impairment (dementia plus MCI), and it may do this effectively in cohorts of older people in whom the prevalence of cognitive impairment is higher. Whereas the content-related (construct) validity of MACE was established in the index study,(5) the current study examines criterion-related (concurrent) validity using standard diagnostic criteria (DSM-IV for dementia, Petersen for MCI).
The findings confirmed those of the preliminary report,(11) and greatly extended these, by examining two MACE cut-offs, two age cut-offs, and more summary measures in a larger patient cohort. The results suggest that in clinical practice either MACE cut-off may be chosen, dependent upon exact clinician requirements, since both increase PPV. The ≤ 25/30 cut-off has greater sensitivity, inevitably with more false positives, and ≤ 21/30 has greater specificity, inevitably with more false negatives. Avoidance of the latter (missed cases) is usually priority for clinicians.
The combination of this patient performance measurement with an informant interview, as per the recommendations of the Alzheimer Association(3) and the International Association of Gerontology and Geriatrics,(23) may be worth further investigation.
The author declares that no conflicts of interest exist.
1 Canadian Task Force on Preventive Health Care, Pottie K, Rahal R, Jaramillo A, et al. Recommendations on screening for cognitive impairment in older adults. CMAJ. 2016;188(1):37–46.
2 Lin JS, O’Connor E, Rossom RC, et al. Screening for cognitive impairment in older adults: a systematic review for the US Preventive Services Task Force. Ann Intern Med. 2013; 159(9):601–12.
3 Cordell CB, Borson S, Boustani M, et al. Alzheimer’s Association recommendations for operationalizing the detection of cognitive impairment during the Medicare Annual Wellness Visit in a primary care setting. Alzheimers Dement. 2013;9(2):141–50.
4 Alzheimer’s Society. Helping you to assess cognition. A practical toolkit for clinicians. London, UK: Alzheimer’s Society; 2015.
5 Hsieh S, McGrory S, Leslie F, et al. The Mini-Addenbrooke’s Cognitive Examination: a new assessment tool for dementia. Dement Geriatr Cogn Disord. 2015;39(1–2):1–11.
6 Matias-Guiu JA, Fernandez-Bobadilla R. Validation of the Spanish-language version of Mini-Addenbrooke’s Cognitive Examination as a dementia screening tool. Neurologia. 2016; 31(9):646–48.
7 Hobson P, Rohoma KH, Wong SP, et al. The utility of the Mini-Addenbrooke’s Cognitive Examination as a screen for cognitive impairment in elderly patients with chronic kidney disease and diabetes. Dement Geriatr Cogn Dis Extra. 2016;6(3):541–48.
8 Miranda DDC, Brucki SMD, Yassuda MS. The Mini-Addenbrooke’s Cognitive Examination (M-ACE) as a brief cognitive screening instrument in Mild Cognitive Impairment and mild Alzheimer’s disease. Dement Neuropsychol. 2018;12(4):368–73.
9 Larner AJ. MACE for diagnosis of dementia and MCI: examining cut-offs and predictive values. Diagnostics. 2019;9(2):E51.
10 Larner AJ. Diagnostic test accuracy studies in dementia. A pragmatic approach, 2nd edition. London, UK: Springer; 2019.
11 Wojtowicz A, Larner AJ. Diagnostic test accuracy of cognitive screeners in older people. Prog Neurol Psychiatry. 2017; 21(1):17–21.
12 American Psychiatric Association. Diagnostic and statistical manual of mental disorders, 4th edition, text revision (DSM-IVTR). Washington, DC: American Psychiatric Association; 2000.
13 Petersen RC, Smith GE, Waring SC, et al. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999; 56(3):303–08.
14 Noel-Storr AH, McCleery JM, Richard E, et al. Reporting standards for studies of diagnostic test accuracy in dementia: the STARDdem Initiative. Neurology. 2014;83(4):364–73.
15 Davis DH, Creavin ST, Noel-Storr A, et al. Neuropsychological tests for the diagnosis of Alzheimer’s disease dementia and other dementias: a generic protocol for cross-sectional and delayed-verification studies. Cochrane Database Syst Rev. 2013;28(3):CD010460.
16 Bolboaca SD. Medical diagnostic tests: a review of test anatomy, phases, and statistical treatment of data. Comput Math Methods Med. 2019;2019:1891569.
17 Jaeschke R, Guyatt G, Sackett DL, et al. Users’ guide to the medical literature. III. How to use an article about a diagnostic Test B. What are the results and will they help me in caring for my patients? JAMA. 1994;271(9):703–07.
18 Mitchell AJ. Sensitivity × PPV is a recognized test called the clinical utility index (CUI+). Eur J Epidemiol. 2011;26(3):251–52.
19 Linn S, Grunau PD. New patient-oriented summary measure of net total gain in certainty for dichotomous diagnostic tests. Epidemiol Perspect Innov. 2006;3(1):11.
20 Habibzadeh F, Yadollahie M. Number needed to misdiagnose: a measure of diagnostic test effectiveness. Epidemiology. 2013;24(1):170.
21 Larner AJ. New unitary metrics for dementia test accuracy studies. Prog Neurol Psychiatry. 2019;23(3):21–25.
22 Larner AJ. Dementia in clinical practice: a neurological perspective. Pragmatic studies in the Cognitive Function Clinic, 3rd edition. London, UK: Springer; 2018.
23 Morley JE, Morris JC, Berg-Weger M, et al. Brain health: the importance of recognizing cognitive impairment: an IAGG consensus conference. J Am Med Dir Assoc. 2015;16(9):731–39.
Canadian Geriatrics Journal, Vol. 23, No. 2, June 2020