George A. Heckman, MD, MSc, FRCPC1,2, Veronique M. Boscart, RN, PhD1,3, Bryan B. Franco, BSc Candidate2, Loretta Hillier4,5, Lauren Crutchlow, MA Candidate6, Linda Lee, MD, MCISc(FM), CCFP, FCFP1,7, Frank Molnar, MSc, MDCM, FRCPC8,9,10,11, Dallas Seitz, MD, PhD12, Paul Stolee, PhD2
1Schlegel – University of Waterloo Research Institute for Aging, University of Waterloo, Waterloo, ON, Canada
2School of Public Health and Health Systems, University of Waterloo, Waterloo, ON, Canada
3Conestoga College’s School of Health & Life Sciences and Community Services, Schlegel Villages, Kitchener, ON, Canada
4Specialized Geriatric Services, St. Joseph's Health Care London, London, ON, Canada
5Parkwood Institute, London, ON, Canada
6University of Western Ontario, London, ON, Canada
7Department of Family Medicine, McMaster University, Kitchener, ON, Canada
8University of Ottawa, Ottawa, ON, Canada
9Division of Geriatric Medicine, The Ottawa Hospital, Ottawa, ON, Canada
10Ottawa Hospital Research Institute, Ottawa, ON, Canada
11Bruyere Research Institute, Ottawa, ON, Canada
12Department of Psychiatry, Queen’s University, Kingston, ON, CanadaDOI: http://dx.doi.org/10.5770/cgj.19.233
ABSTRACT
Background
Primary care-based memory clinics (PCMCs) have been established in several jurisdictions to improve the care for persons with Alzheimer’s disease and related dementias. We sought to identify key quality indicators (QIs), quality improvement mechanisms, and potential barriers and facilitators to the establishment of a quality assurance framework for PCMCs.
Methods
We employed a Delphi approach to obtain consensus from PCMC clinicians and specialist physicians on QIs and quality improvement mechanisms. Thirty-eight candidate QIs and 19 potential quality improvement mechanisms were presented to participants in two rounds of electronic Delphi surveys. Written comments were collected and descriptively analyzed.
Results
The response rate for the first and second rounds were 21.3% (n = 179) and 12.8% (n = 88), respectively. The majority of respondents were physicians. Fourteen QIs remained after the consensus process. Ten quality improvement mechanisms were selected with those characterized by specialist integration, such as case discussions and mentorships, being ranked highly. Written comments revealed three major themes related to potential barriers and facilitators to quality assurance: 1) perceived importance, 2) collaboration and role clarity, and 3) implementation process.
Conclusion
We successfully utilized a consultative process among primary and specialty providers to identify core QIs and quality improvement mechanisms for PCMCs. Identified quality improvement mechanisms highlight desire for multi-modal education. System integration and closer integration between PCMCs and specialists were emphasized as essential for the provision of high-quality dementia care in community settings.
Key words: quality improvement, primary care, chronic disease management, dementia, quality indicators, system integration
The rising prevalence of Alzheimer’s disease and related dementias (henceforth referred to as dementia) is emerging as a leading global health system challenge.(1) Effective early diagnosis and management models are required to mitigate its impact on patients, caregivers, and health-care systems.(2,3) Enhancing primary care capacity is seen as essential towards achieving this goal.(4–10) However, dementia care predominantly resides with geriatric specialists, who are in short supply in Canada and elsewhere, delaying access to care.(11)
To enhance the care of persons with dementia, many communities are creating programs to assess persons presenting with cognitive concerns. Many of these have been established as primary care-based memory clinics (PCMCs).(4–7,9,10,12,13) Initial evaluations suggest that PCMCs can provide timelier assessment, lead to a high degree of satisfaction among referring physicians, patients, and caregivers, and streamline access to specialists.(11–13) In order to retain the fidelity of such programs and consistency with initial training and practice guidelines, and thus prevent ‘practice drift’ and ensure ongoing high quality of care, quality assurance frameworks are needed.(9,12,14)
Quality assurance is a process to ensure that care provision meets established standards.(15) Quality Indicators (QIs), based on best practices, define achievable benchmarks and a quality assurance framework facilitates practice improvement through targeted, educational quality improvement mechanisms.(16,17) To our knowledge, there is no quality assurance framework specific to dementia care in primary care-based settings. This paper describes the results of a consensus approach to identify QIs and quality improvement mechanisms in an Ontario-wide network of interprofessional PCMCs, and identify potential barriers and facilitators to the implementation of a quality assurance framework.(11,12,18)
A Delphi technique was deployed to obtain agreement from PCMC clinicians and dementia specialists on preferred QIs and quality improvement mechanisms.(19–23) The Delphi technique is an iterative consensus process, wherein surveys are used to solicit opinions from groups, and responses summarized and redistributed in a subsequent round for consideration. We identified 38 candidate QIs and quality improvement mechanisms for dementia care by reviewing existing clinical guidelines and quality indicator and improvement compendiums developed with standardized methods (Table 1).(16,24–33,34) Respondents were asked to rate the QIs and quality improvement mechanisms using a continuous integer 9-point scale, with 1 representing the least important and 9 the most important. Written comments were solicited and professional information collected to characterize respondents.
TABLE 1 Candidate quality indicators
Links to the web-based survey were electronically distributed to all PCMC clinicians (n = 283) and Ontario specialists through the Ontario Medical Association sections of geriatric medicine (n = 123) and neurologists (n = 134), and the Canadian Association of Geriatric Psychiatrists (n = 305). Two reminders to complete the survey were sent by e-mail. After each round, QIs and improvement measures in the lower two tertiles of agreement (i.e., with mean ratings less than 7) were excluded, and those remaining were reviewed by the authors guided by respondent comments.(35) QIs and quality improvement mechanisms deemed redundant or containing duplicate themes were combined or amended with attention to preserving their intent and conciseness. The authors maintained an audit trail of changes and decision-making points. Data from the preceding round, including number of respondents, rating means, and standard deviations, were included in the subsequent round.
Descriptive statistics were compiled after each round. Student’s t-test was used to identify significant differences between PCMC clinicians and specialists. SPSS version 23.0 (IBM Corp.) was used, with two-sided p values of < .05 as the threshold for statistical significance. Two authors (GH, VB) independently analyzed all written comments using descriptive content analysis(36) and incorporated the feedback into the presentation of QIs and quality improvement mechanisms in the second Delphi round.
Two survey rounds were conducted between April and June 2014. In the first Round, 842 surveys were distributed with a response rate of 21.3%. The majority of respondents were physicians, and nurses and other health professionals equally represented the remainder of respondents (Table 2). Respondents had an average of 14.6 ± 10.1 years of clinical practice with older adults. Among PCMC respondents, 31% were from urban centres, 42% rural settings, and 25% from mixed urban/rural populations. In contrast, 73% of specialists worked in urban settings, and 24% served mixed populations. Only 11 neurologists responded to the first survey, and their specialty was not included in Round 2.
TABLE 2 Respondent characteristics for both Delphi rounds
In Round 2, 690 surveys were distributed with a response rate of 12.8%. The majority of respondents (60.2%) were physicians. Respondents had an average of 17.42 ± 10.08 years of clinical practice with older adults and practice settings were similar to Round 1. Respondent characteristics are shown in Table 2.
Of the initial 38 candidate QIs, only 14 remained after two Delphi rounds. A third Delphi round was not conducted due to the substantial drop in response rate between Rounds 1 and 2. Table 3 presents the results of the consensus process and Table 4 presents the final list of quality indicators. Of 19 candidate quality improvement mechanisms, 10 were ultimately selected. Quality improvement mechanisms characterized by specialist integration, including case discussions, shared care, observerships, and mentorships, ranked highly (Table 5). Other preferred quality improvement mechanisms included standardized electronic charting forms, self-directed learning activities, and interactive programs. Survey respondents recommended that between 10–30% of patients seen in a PCMC also be reviewed by a specialist.
TABLE 3 Summary of Quality Indicator selection process
TABLE 4 Final Quality Indicators (FQIs) and operational definitions
TABLE 5 Ranking of Quality Improvement mechanisms
Descriptive content analysis identified three themes related to potential barriers and facilitators to the establishment of a quality assurance framework in PCMCs: 1) its perceived importance, 2) collaboration and role clarity, and 3) the implementation process.(36)
Almost all respondents mentioned the need to maintain high-quality care through ongoing, targeted training on pertinent clinical knowledge and consistency of approach among PCMC teams. Most recognized a formal quality assurance framework and individual QIs as essential.
“Great indicators and very necessary” [PCMC clinician, Round 1].
“Our challenge is to maintain evidence based salience, which ultimately facilitates improved quality of life for persons with dementia and their family” [PCMC clinician, Delphi Round 2].
“All are relevant quality indicators for a PCMC” [Specialist, Round 2].
However, a few respondents were unsure why a quality assurance framework was needed at all. Some assumed that formal training should be sufficient to ensure quality. Others seemed unfamiliar with the purpose of quality assurance and construed the idea of tracking QIs as supplemental work.
“Is the intent of the questions to assess what we are doing now or [what] we think should be the standard?” [PCMC clinician, Round 1].
“This is the Canadian Consensus Guideline that every doc should know” [Specialist, Round 1; emphasis added by authors].
A second theme pertained to the operationalization and implementation of the quality assurance framework within the context of PCMCs, and identified a lack of clearly delineated responsibilities among referring clinicians, PCMCs, and specialists. Respondents perceived this as problematic because absence of clarity impedes the capture of clinical documentation relevant to QI measurement.
“Not clear on the team role with caregiver and patient care plan. Information is sent back to referral physician with recommendations for caregiver and patient mostly related to change in medication or treatment, lifestyle modification and support resources” [PCMC clinician, Round 2].
This lack of clarity was considered most problematic with regard to patient follow-up.
“Patients with dementia with great plans should not need constant surveillance and follow up BUT... This is not my experience... they appear to often benefit from a watchdog team to ensure their decisions are being carried out.” [Specialist, Round 1].
Similar concerns were raised regarding the management of comorbidities.
“… [My] assumption is that management of comorbidities is the primary care physician’s domain. Recommendations are made for lifestyle changes, however [congestive heart failure] and diabetes management are only commented on, suggestions are made to the referring physician” [PCMC clinician, Round 2].
It was not clear which health care provider should conduct a physical examination, leading to gaps in assessment.
“Referring Family Physicians are expected to have completed a complete physical including appropriate neuro exam prior to referral” [PCMC clinician, Round 1].
“My biggest concern is that the PCMC team assumes that a proper physical and neurologic exam has been done by the referring source” [Specialist, Round 1].
Respondents provided several comments on potential barriers to the implementation process of a quality assurance framework in PCMCs. One of the main obstacles was the perceived burden of documentation required to implement QIs.
“[It will take a lot of work to do the] searching and documenting as a group of patients needs a process to set up how to search, then doing the search” [PCMC clinician, Round 2].
Integrating QIs into existing electronic medical records was touted as a solution, though potentially a resource-intensive one.
“Searches are limited by consistency of nomenclature in the chart. Making up stamps to collect this information is doable but takes time for someone to make the [standardized template] and then to test it” [PCMC clinician, Round 2].
A second issue identified in relation to the implementation of the quality assurance framework is the need to establish benchmarks to properly interpret QI scores. Respondents suggested that specialist involvement with PCMCs could help define these benchmarks. However, access to specialists within a PCMC was often seen as insufficient.
“I think one of the most important quality indicators is a comparison of the PCMC performance versus the specialist, done on both patients referred to the specialist by the PCMC, and unselected patients seen in the clinic that would not have been referred to the specialist” [Specialist, Round 1].
Access to other health care professionals, within PCMCs and in the community, was identified as important for quality care, though access was not perceived as uniform.
“Having a pharmacist on our team is a great asset.... The local [pharmacists] volunteer their time to assist us at our once monthly day long clinics” [PCMC clinician, Round 2].
“The [Social Work] and [Occupational Therapy] members of the team do an excellent job…the [Alzheimer’s Society] representative has been excellent as well when given an opportunity” [PCMC clinician, Round 2].
An important finding is the relative lack of importance ascribed to end-of-life planning. While respondents agreed on its importance, many expressed that such planning discussions were not appropriate for patients with less advanced disease.
“Palliation and end of life discussion is usually not appropriate at time of our memory clinic initial or follow-up assessment since our patients are not that advanced. Family physician team will do this in following up patient” [PCMC clinician, Round 1].
Lastly, many respondents expressed a preference for quality improvement mechanisms and learning opportunities characterized by active engagement. They also identified specialists as agents to promote care quality, particularly in the context of a shared care approach.
“This [case review] could be done at the same time, as the specialist mentoring the clinic is in a clinic day with the team” [PCMC clinician, Round 1].
Overall, respondents recognized the need for a quality assurance framework for dementia care. Barriers to the implementation of this framework included role confusion among stakeholders and limited resources. Integrated electronic data collection was considered a facilitator for quality assurance.
This study used a consultative process among primary and specialty providers to identify a core group of QIs and preferred quality improvement mechanisms for dementia care. Quality assurance can be an effective method to ensure fidelity to best practices and maintain a high standard of care quality.(37–41) Selected QIs address care processes, assessment and reassessment, medication review and management, investigations, non-pharmacological management, pharmacological management, and managing concomitant conditions.(42) Selected quality improvement mechanisms emphasize a desire for multimodal education and closer collaboration with specialists. Assessing care quality and publically reporting the findings are increasingly commonplace. However, quality assurance still mainly focuses on specific care sectors or episodes, rather than on overall care for conditions that require coordination and collaboration across multiple sectors.(43) In that context, study participants identified important hurdles to its implementation.
Critical considerations include educating clinicians about the role of quality assurance, improving role clarity among providers involved in dementia care, expanding access to allied health professionals, and creating standardized electronic medical record templates to simplify QI documentation. Quality assurance must not create additional burden on clinicians often working with limited resources and time, a burden that could also impact the quality of the clinician-patient interaction. Greater integration of specialists within PCMCs was acknowledged as important for care quality. Respondents proposed that 10–30% of patients seen in PCMCs be reviewed with a specialist, preferably in a shared care approach. Closer integration of urban specialists with rural PCMCs represents a tremendous opportunity to extend quality dementia care, particularly to rural patients. However, integration of specialists within PCMCs remains suboptimal. To optimize specialist integration, it will be necessary to identify barriers (e.g., allocation of funding) and facilitators (e.g., resources for coordination, electronic medical records, telemedicine). Another important finding was the relatively low rankings of QIs related to end-of-life planning with dementia patients. In addition to potential discomfort among clinicians to address such issues, this finding may also stem from the understandable desire to maintain hope and engagement early in the course of the illness, though undue delays may leave patients and caregivers less able to meaningfully participate in such discussions.
The most important identified barrier to the implementation of the quality assurance framework is the need for greater clarity on responsibilities of referring family physicians, PCMC clinicians, and specialists, particularly with respect to follow-up, physical examination, and care of complex comorbidities. Quality assurance often targets relatively simple conditions, such as hypertension, or restricts its scope to specific aspects, processes or locations of care.(38–41) In contrast, dementia, like all major chronic conditions, follows a course of progressive decline punctuated by increasingly frequent health complications (related to dementia itself, as well as to exacerbations of concurrent comorbidities), multiple care transitions, progressive caregiver stress and health service utilization, and ultimately death. Optimal dementia care thus requires a systems approach of integration and coordination.(44,45) Addressing greater role clarity among all dementia stakeholders, a task that with proper resources could be coordinated from within PCMCs, requires immediate attention, in order to ensure a stable clinical infrastructure that is able to safely and effectively address the needs of these patients, wherever they may be and whenever they arise.(46,47) Until such integration is achieved, optimal quality dementia care will remain difficult to attain.
The results of this study should be understood in light of several limitations. First, only two Delphi rounds were conducted because the response rate fell markedly after the first. However, the response rate remained above what is considered appropriate for Delphi surveys, and ratings of individual QIs, except those related to physical examination, remained stable between rounds.(48,49) A second limitation is that the survey was solely distributed within the previously described network of PCMCs,(11–13) though this model is widely implemented across Ontario. As the QIs were selected by both primary and specialist providers, they are likely applicable to other dementia care settings. Third, participation of neurologists in the first round was low and their response was not solicited in the second round. The relatively lower participation of neurologists in this work, compared to geriatricians and geriatric psychiatrists, is notable and requires further investigation. Fourth, allied health providers were under-represented and, given their importance in dementia care, additional work is required to further understand and develop their role in an integrated system of dementia care. Fifth, patient and caregivers were not surveyed regarding what they consider to be important QIs. Finally, candidate QIs and quality improvement mechanisms were not identified through a systematic literature review, but the use of published guidelines and compendiums likely identified the most important elements of quality dementia care, which would thus have remained highly ranked.
While this study has identified QIs and quality improvement mechanisms to assess care quality for dementia, findings underscore the importance of system integration for the provision of quality dementia care, with specifically defined and mutually understood roles among stakeholders and, where necessary, the reallocation of existing resources to support this approach to care.(50) Understanding and overcoming system barriers to dementia care integration is an urgent priority. An approach whereby clear roles are negotiated among dementia stakeholders can provide sufficient flexibility to meet regional needs (especially in rural areas where access to specialists is more limited), foster more effective collaboration and accountability, and thus facilitate the delivery and measurement of care quality for dementia. Within that context, proper field-testing, validation, and evaluation of selected QIs and quality improvement mechanisms can then be conducted. This work has significant implications on the organization of care for aging patients with complex conditions. Primary and specialist providers share the responsibility of providing and supporting integrated dementia quality care. As such, the care of persons with cognitive impairment would be enhanced by the development of a practical and realistically feasible quality assurance framework, under whose umbrella both PCMC and specialist services are integrated, and which ensures high fidelity to intended design and best practices, and thus maintains a high level of care quality.(51)
This study was approved by the University of Waterloo’s Office of Research Ethics. The authors would like to acknowledge financial support from the Alzheimer Society of Canada, the Canadian Institutes of Health Research (CIHR), and the Schlegel-University of Waterloo-Research Institute for Aging.
VB receives salary support from CIHR, Conestoga College, and Schlegel Villages. DS served on an advisory board for Eli-Lilly in 2013. No other conflicts of interest to declare.
1 Prince M, Bryce R, Albanese E, et al. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement. 2013;9(1):63–75.
2 Wimo A, Jonsson L, Bond J, et al. The worldwide economic impact of dementia 2010. Alzheimers Dement. 2013;9(1):1–11.
3 Connell CM, Janevic MR, Gallant MP. The costs of caring: impact of dementia on family caregivers. J Geriatr Psychiatry Neurol. 2001;14(4):179–87.
4 Aminzadeh F, Molnar FJ, Dalziel WB, et al. A review of barriers and enablers to diagnosis and management of persons with dementia in primary care. Can Geriatr J. 2012;15(3):85–94.
5 Callahan CM, Boustani MA, Unverzagt FW, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA. 2006;295(18):2148–57.
6 Greening L, Greaves I, Greaves N, et al. Positive thinking on dementia in primary care: Gnosall Memory Clinic. Community Pract. 2009;82(5):20–23.
7 McCarten JR, Anderson P, Kuskowski MA, et al. Finding dementia in primary care: the results of a clinical demonstration project. J Am Geriatr Soc. 2012;60(2):210–17.
8 Moore A, Patterson C, Lee L, et al. Fourth Canadian Consensus Conference on the Diagnosis and Treatment of Dementia: recommendations for family physicians. Can Fam Physician. 2014;60(5):433–38.
9 Morgan DG, Crossley M, Kirk A, et al. Improving access to dementia care: development and evaluation of a rural and remote memory clinic. Aging Mental Health. 2009;13(1):17–30.
10 Vickrey BG, Mittman BS, Connor KI, et al. The effect of a disease management intervention on quality and outcomes of dementia care: a randomized, controlled trial. Ann Intern Med. 2006;145(10):713–26
11 Lee L, Hillier LM, Stolee P, et al. Enhancing dementia care: a primary care-based memory clinic. J Am Geriatr Soc. 2010;58(11):2197–204.
12 Lee L, Hillier LM, Heckman G, et al. Primary care-based memory clinics: expanding capacity for dementia care. Can J Aging. 2014;33(3):307–19.
13 Lee L, Kasperski MJ, Weston WW. Building capacity for dementia care: training program to develop primary care memory clinics. Can Fam Physician. 2011;57(7):e249–52.
14 Jolley D, Benbow SM, Grizzell M. Memory clinics. Postgrad Med J. 2006;82(965):199–206.
15 Kates N, Hutchison B, O’Brien P, et al. Framework for advancing improvement in primary care. Healthcare Papers. 2012;12(2):8–21.
16 Jamtvedt G, Young JM, Kristoffersen DT, et al. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care. 2006;15(6):433–36.
17 Kiefe CI, Allison JJ, Williams OD, et al. Improving quality improvement using achievable benchmarks for physician feedback — a randomized controlled trial. JAMA. 2001;285(22):2871–79.
18 Lee L, Weston WW, Hillier LM. Developing memory clinics in primary care: an evidence-based interprofessional program of continuing professional development. J Contin Educ Health Prof. 2013;33(1):24–32.
19 Campbell SM, Braspenning J, Hutchinson A, et al. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002;11(4):358–64.
20 Fleuren M, Wiefferink K, Paulussen T. Determinants of innovation within health care organizations: literature review and Delphi study. Int J Qual Health Care. 2004;16(2):107–23.
21 Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. J Adv Nurs. 2000;32(4):1008–15.
22 Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–80.
23 Reid N. The Delphi technique: its contribution to the evaluation of professional practice. In: Ellis R, editor. Professional competence and quality assurance in the aring profession. New York: Chapman & Hall; 1988.
24 Draskovic I, Vernooij-Dassen M, Verhey F, et al. Development of quality indicators for memory clinics. Int J Geriatr Psychiatry. 2008;23(2):119–28.
25 Feil DG, MacLean C, Sultzer D. Quality indicators for the care of dementia in vulnerable elders. J Am Geriatr Soc. 2007;55(Suppl 2):S293–S301.
26 Feldman HH, Jacova C, Robillard A, et al. Diagnosis and treatment of dementia: 2. Diagnosis. CMAJ. 2008;178(7):825–36.
27 Hogan DB, Bailey P, Black S, et al. Diagnosis and treatment of dementia: 4. Approach to management of mild to moderate dementia. CMAJ. 2008;179(8):787–93.
28 Kroger E, Tourigny A, Morin D, et al. Selecting process quality indicators for the integrated care of vulnerable older adults affected by cognitive impairment or dementia. BMC Health Serv Res. 2007;7:195.
29 Perry M, Draskovic I, van Achterberg T, et al. Development and validation of quality indicators for dementia diagnosis and management in a primary care setting. J Am Geriatr Soc. 2010;58(3):557–63.
30 Wenger NS, Solomon DH, Amin A, et al. Application of assessing care of vulnerable elders-3 quality indicators to patients with advanced dementia and poor prognosis. J Am Geriatr Soc. 2007;55 Suppl 2:S457–S463.
31 Odenheimer G, Borson S, Sanders AE, et al. Quality improvement in neurology: dementia management quality measures. J Am Geriatr Soc. 2014;62(3):558–61.
32 Greenhalgh T, Robert G, Macfarlane F, et al. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.
33 Mollon B, Chong J, Jr., Holbrook AM, et al. Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials. BMC Med Inform Decision Making. 2009;9:11.
34 McDonald KM, Sundaram V, Bravata DM, et al. Care coordination. In: Shojania K, McDonald K, Wachter R, Owens D, eds. Volume 7. Closing the quality gap: a critical analysis of quality improvement strategies. Rockville, MD: Agency for Healthcare Research and Quality; 2007.
35 Tu JV, Abrahamyan L, Donovan LR, et al. Best practices for developing cardiovascular quality indicators. Can J Cardiol. 2013;29(11):1516–19.
36 Cavanagh S. Content analysis: concepts, methods and applications. Nurse Res. 1997;4(3):5–13.
37 Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academies Press; 2001.
38 Goldenberg L, Trachtenberg J, Saad F. The importance of quality indicators: a call to action. Can Urol Assoc J. 2009;3(6):435–38.
39 Greenfield D, Pawsey M, Hinchcliff R, et al. The standard of healthcare accreditation standards: a review of empirical research underpinning their development and impact. BMC Health Serv Res. 2012;12:329.
40 Jaffe MG, Lee GA, Young JD, et al. Improved blood pressure control associated with a large-scale hypertension program. JAMA. 2013;310(7):699–705.
41 Munday GS, Deveaux P, Roberts H, et al. Impact of implementation of the Surgical Care Improvement Project and future strategies for improving quality in surgery. Am J Surg. 2014;208(5):835–40.
42 Donabedian A. Evaluating the quality of medical care. Milbank Q. 2005;43(3).
43 Ontario Ministry of Health and Long-Term Care. Excellent Care for All Act [About webpage]. Toronto, ON; 2010. www.health.gov.on.ca/en/pro/programs/ecfa/legislation/act.aspx. Accessed December 2015.
44 Brand C, Scott I, Greenberg P, et al. Chronic disease management: time for consultant physicians to take more leadership in system redesign. Intern Med J. 2007;37(9):653–59.
45 Scott IA. Chronic disease management: a primer for physicians. Intern Med J. 2008;38(6):427–37.
46 Heckman GA. Integrated care for the frail elderly. Healthcare Papers. 2011;11(1):62–68.
47 Heckman GA, Molnar FJ, Lee L. Geriatric medicine leadership of health care transformation: to be or not to be? Can Geriatr J. 2013;16(4):192–95.
48 de Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–43.
49 Hung HL, Altschuld JW, Lee YF. Methodological and conceptual issues confronting a cross-country Delphi study of educational program evaluation. Eval Program Planning. 2008;31(2):191–98.
50 Heckman G, Hillier L, Manderson B, et al. Developing an Integrated system of care for frail seniors. Healthcare Management Forum. 2013;26(4):200–08.
51 Lee L, Heckman G, McKelvie R, et al. Physicians’ perceptions of capacity building for managing chronic disease in seniors using integrated interprofessional care models. Can Fam Physician. 2015;61(3):e148–e57.
Canadian Geriatrics Journal, Vol. 19, No. 4, December 2016