Evaluation of the Public Health Agency of Canada's Chronic Disease Prevention Activities 2009-2010 to 2014-2015

Prepared by
Evaluation Directorate
Health Canada and the Public Health Agency of Canada

January 2015

Table of Contents

• Executive Summary
• Management Response and Action Plan
• 1.0 Evaluation Purpose
• 2.0 Program Description
  • 2.1 Program Context
  • 2.2 Program Profile
  • 2.3 Program Logic Model and Narrative
  • 2.4 Program Alignment and Resources
• 3.0 Evaluation Description
  • 3.1 Evaluation Scope, Approach and Design
  • 3.2 Limitations and Mitigation Strategies
• 4.0 Findings
  • 4.1 Relevance: Issue #1 - Continued Need for the Program
  • 4.2 Relevance: Issue #2 - Alignment with Government Priorities
  • 4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities
  • 4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)
  • 4.5 Performance: Issue #5 - Demonstration of Economy and Efficiency
• 5.0 Conclusions
  • 5.1 Relevance Conclusions
  • 5.2 Performance Conclusions
• 6.0 Recommendations
• Appendix 1 - References
• Appendix 2 - Logic Model
• Appendix 3 - Summary of Findings
• Appendix 4 - Role of Stakeholders
• Appendix 5 - Evaluation Description
• Appendix 6 - Stakeholder Mapping
• Appendix 7 - Description of Performance Measurement Data

List of Tables

• Table 1: Program Resources ($)*
• Table 2: Limitations and Mitigation Strategies
• Table 3: Project Profile - Multi-sectoral Partnerships Approach Projects (2012-2014)
• Table 4: Web analytics for Data Cubes: 2011-2012 to 2013-2014
• Table 5: Surveillance reports produced: 2009-2014
• Table 6: Direct and Indirect Costs of Cardiovascular Diseases, Diabetes, Cancer and Chronic Obstructive Pulmonary Diseases (2008)
• Table 7: Variance Between Year-End Budgets and Expenditures - 2009-2010 to 2013-2014

Executive Summary

This evaluation covered the healthy living and chronic disease prevention activities (herein referred to as chronic disease prevention) of the Public Health Agency of Canada (the Agency) from April 2009 to August 2014. The evaluation was undertaken in fulfillment of the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009).

Evaluation Purpose and Scope

The purpose of the evaluation was to assess the relevance and performance of the Agency's chronic disease prevention activities. The evaluation was also designed to assist senior management in program planning and decision making.

The evaluation covered activities funded through the Integrated Strategy on Healthy Living and Chronic Disease (ISHLCD), the Canadian Strategy on Cancer Control, the Canadian Breast Cancer Initiative, the Lung Health Program, and the Automated External Defibrillators initiative. It focused on the chronic disease prevention activities carried out by the Centre for Chronic Disease Prevention and excluded the ISHLCD investments in mental health, the Joint Consortium for School Health, investments in neurological diseases, the Federal Tobacco Initiatives, and food security activities.

Program Description

The Public Health Agency of Canada, through the Centre for Chronic Disease Prevention, conducts a range of chronic disease prevention activities which focus on upstream prevention through a common risk and protective factor approach. These include conducting surveillance, developing and managing knowledge products and platforms, funding interventions that focus on common risk factors through the Multi-sectoral Partnerships Approach to Promote Healthy Living and Prevent Chronic Disease (herein referred to as the Multi-sectoral Partnerships Approach), and providing policy leadership to national fora such as the Public Health Network's Healthy People and Communities Steering Committee, and supporting the Canadian Task Force on Preventive Health Care (CTFPHC).

Many other players are working to prevent chronic disease in Canada. Provinces and territories, other federal government departments, non-governmental organizations, local community groups, as well as academics and researchers, all conduct key activities in this area.

The Agency allocated approximately $46 million for these activities in 2013-2014.

CONCLUSIONS - RELEVANCE

Continued Need

There is a demonstrated need for stakeholders to continue to address chronic diseases. Chronic diseases remain a public health issue, and prevention is still needed given that the rate at which new cases of some chronic diseases are diagnosed continues to rise. Public health research has shown that common risk factors, such as unhealthy diet and lack of physical activity, contribute to chronic disease and that obesity increases the risk of many chronic diseases. The Agency has been able to adapt to the changing environment by shifting its activities to chronic disease prevention, with a focus on addressing common risk factors.

Alignment with Government Priorities

The federal government and the Public Health Agency of Canada continue to identify chronic disease prevention and obesity as priorities. In the past five years, numerous federal/provincial/territorial and international declarations and priority-setting documents have been signed, which are focused on the prevention of chronic disease through addressing obesity, common risk and protective factors, and health disparities.

Activities are well aligned with federal government priorities in the areas of chronic disease prevention and obesity. For example, the Centre for Chronic Disease Prevention has focused its activities in the prevention of chronic disease through a common risk factor approach. There may be opportunities to clarify, through the Centre for Chronic Disease Prevention policy refresh exercise, how the population health approach will assist in reducing health disparities.

Alignment with Federal Roles and Responsibilities

A clear federal role in the area of chronic disease prevention was identified, specifically in conducting surveillance and research, providing leadership, supporting community-based initiatives, identifying best practices, and supporting the development of screening guidelines. The current chronic disease prevention activities conducted by the Public Health Agency of Canada align with the federal public health role and responsibilities in addressing chronic disease.

The federal public health role does not appear to duplicate the role of other stakeholders. In fact, roles are more inclined to complement each other. However, external key informants - representing provinces and territories, non-governmental organizations, or local public health authorities - had difficulty in understanding the differences between the chronic disease prevention and healthy living roles played by the Agency and other federal government departments, especially Health Canada and Heritage Canada (i.e., Sport Canada).

CONCLUSIONS - PERFORMANCE

Achievement of Expected Outcomes (Effectiveness)

In the past five years, there has been limited evidence of the integration of activities (i.e., science and surveillance informing policy and programming) within and outside of the Centre for Chronic Disease Prevention. A few recent initiatives, such as the Surveillance and Epidemiology Division's Emerging Issues Forum and the Partnerships and Strategies Division's Ideas Forum, are great mechanisms to encourage all areas of the Centre, and the HPCDP Branch, to learn from each other, and come together to address common issues, when appropriate.

In the area of collaboration, the evaluation found that the Centre collaborates frequently with stakeholders to advance projects and priorities (e.g., obesity/healthy weights), and these activities are well appreciated; however, external key informants felt that improvements could be made, especially in terms of clearly communicating the Agency's priorities.

Over the past five years, a wide range of community-based projects have had an impact on the lives of Canadians. Although still relatively new, the Agency's new grants and contributions program, the Multi-sectoral Partnerships Approach, is demonstrating that common risk factor approaches can be adopted by funded projects, private sector partners are committed to the projects, and there is early indication that behaviour changes are occurring as a result of funded initiatives. However, there is an opportunity to clarify how the multi-sectoral partnerships' approach aligns with a population health approach.

Knowledge products have been developed in the past five years, and key platforms continue to be managed. Although the Centre's newly implemented Digital Strategy aims to increase and facilitate access, challenges remain. For example, many internal and external key informants felt that, in general, products and platforms were not promoted to intended audiences. Issues have been raised with the quality of some of the information included in one of the knowledge platforms, the Canadian Best Practices Portal.

The evaluation found that stakeholders use surveillance knowledge products frequently. Although there is limited information on the uptake of other knowledge platforms (i.e., the Canadian Best Practices Portal and Data Cubes), the evaluation was able to find some examples of uptake.

Demonstration of Economy and Efficiency

Efficiency has been demonstrated through recent changes, including the Multi-sectoral Partnerships Approach's focus on performance. Further, economy will continue to be achieved with this approach - the $10.8 million in grants and contributions allocated to the nine projects under review for this evaluation, has allowed the Public Health Agency of Canada to leverage $9.3 million from the private sector. Although this approach is relatively new, it has already expanded the reach and potentially the impact of funded projects.

In terms of the governance structure, recent changes have allowed the Centre for Chronic Disease Prevention to benefit from more streamlined decision-making and priority-setting mechanisms and processes. As a result of addressing the Auditor General's recommendations on Promoting Diabetes Prevention and Control, the newly developed financial coding system is enhancing the ability of the Centre to identify planned and actual spending according to various priorities. Further, the evaluation has found that the Centre is in the process of successfully establishing a performance measurement culture.

Additional efficiencies could be gained in integrating activities further within the Centre and in collaborating further with other areas of the HPCDP Branch, where appropriate.

RECOMMENDATIONS

Recommendation 1:

Review the Canadian Best Practices Portal with a focus on assessing the content of the current best practices

Issues were raised on the content of the best practices available on the Canadian Best Practices Portal (i.e., through key informant interviews, web analytics, and performance measurement survey). The evaluation recommends that the Centre for Chronic Disease Prevention conducts an external review of the content of current best practices to guide revisions that would re-instill trust in a role and a platform that has great potential and is sought-after. Changes should then be made to address the external review recommendations.

Recommendation 2:

Address communication issues

Communication is at the centre of a few issues raised in the evaluation. In particular, the Centre for Chronic Disease Prevention should:

• In collaboration with other government departments (e.g., Health Canada, Heritage Canada), communicate federal public health roles and responsibilities in chronic disease prevention to external stakeholders;
• Develop and communicate a dissemination plan for surveillance products;
• Enhance the promotion of the chronic disease prevention strategy, products and platforms to target audiences; and
• Clarify and communicate how the Multi-sectoral Partnerships Approach's design aligns with a population health approach.

The Centre's Digital Strategy is a step in the right direction to address some communication issues, but it is still relatively new. Once the policy refresh exercise is completed, the Centre must make sure that priorities are communicated to stakeholders.

Recommendation 3:

Continue to integrate chronic disease prevention activities within the Centre for Chronic Disease Prevention as well as formalize mechanisms to ensure this integration

Integration should be carried out within the Centre for Chronic Disease Prevention. The evaluation found limited examples of successful, well-integrated activities within the Centre but also with other areas of the Branch, as appropriate. The Centre recognizes that integration could be improved and has launched, in the Spring of 2014, an Emerging Issues Forum, led by its Surveillance and Epidemiology Division, which should assist in increased integration.

Management Response and Action Plan

1.0 Evaluation Purpose

The purpose of the evaluation was to assess the relevance and performance of the Public Health Agency of Canada healthy living and chronic disease prevention activities (herein referred to as chronic disease prevention) for the period of April 2009 to August 2014.

The evaluation was undertaken in fulfillment of the Financial Administration Act (FAA) and the Treasury Board of Canada's Policy on Evaluation (2009). The evaluation was designed to assist senior management in program planning and decision making. This was a scheduled evaluation as per the Public Health Agency of Canada/Health Canada approved Five-Year Evaluation Plan 2013-2014 to 2017-2018.

2.0 Program Description

2.1 Program Context

The Public Health Agency of Canada's chronic disease prevention activities are led by the Centre for Chronic Disease Prevention within the Health Promotion and Chronic Disease Prevention (HPCDP) Branch.

The foundation of the Agency's activities in the area of healthy living and chronic disease prevention come from the Integrated Strategy on Healthy Living and Chronic Disease (ISHLCD). The ISHLCD was announced in the 2005 federal budget, which committed $300 million over five years and $66.7 million per year in ongoing funding to the Public Health Agency of Canada. This strategy aimed to ensure that Canada had an integrated approach to addressing major chronic diseases by focusing on risk factors, as well as complementary disease-specific work.

Shortly after the ISHLCD was implemented, the Federal Government along with Provincial and Territorial Ministers of Health and/or Health Promotion/Healthy Living (except Québec) endorsed two initiatives that focused on the prevention of disease, disability and injury, and the promotion of health: the Declaration on Prevention and Promotion (September 2010), and Curbing Childhood Obesity: A Federal, Provincial and Territorial Framework for Action to Promote Healthy Weights (November 2011).

Concurrently, Canada signed the United Nations Political Declaration on the prevention and control of non-communicable diseases (September 2011), which emphasized that prevention is the foundation to combat chronic disease.

All of these commitments, along with concurrent investments (i.e., the Canadian Strategy for Cancer Control in 2006, the Lung Health Program 2009-2012, the Automated External Defibrillators initiative in 2013), introduced a new approach to addressing chronic diseases at the national level: focus on prevention, and more specifically upstream prevention through a common risk factor approach.

While these changes were occurring, spending was examined across the federal government. The recent budgetary reallocations impacted the Public Health Agency of Canada's chronic disease prevention activities. For instance, regional offices underwent a transformation where activities were focused on supporting the work of the Agency's Centres rather than management and delivery of programs. Science activities conducted within the Centre for Chronic Disease were merged with health equity activities within one Directorate where they now support the HPCDP Branch.

Another key context piece is the recent Office of the Auditor General (OAG) report on Promoting Diabetes Prevention and Control, which was published in the Spring of 2013. The Auditor General's report, although examining the Canadian Diabetes Strategy, found areas for improvement that were applicable to many of the Centre for Chronic Disease Prevention's activities. The Centre undertook major changes to address the recommendations, including a revised financial expenditure tracking system and a more robust planning and performance measurement system.

Over the last two years, numerous strategic plans have been developed within the Agency as well as within the Centre: from surveillance transformation to knowledge mobilization. All of these plans have also contributed to influencing current chronic disease prevention activities.

At the time of the evaluation, the Centre for Chronic Disease Prevention was examining the alignment between its activities and its policies and programs to clarify changes in programming and direction that have taken place in the past few years.

2.2 Program Profile

The Public Health Agency of Canada, through the Centre for Chronic Disease Prevention, conducts a range of activities to prevent chronic diseases^{Footnote i} .

Surveillance and Epidemiology

The Public Health Agency of Canada conducts surveillance on chronic diseases and conditions among adults including diabetes, hypertension, mental illnesses, respiratory conditions (i.e., COPD and asthma), musculoskeletal conditions (i.e., osteoporosis and arthritis), as well as stroke and cancer. The Agency also monitors upstream factors which are actionable through prevention programming such as healthy weights (obesity), physical activity, sedentary behaviours, sleep, active transportation, built environments, and positive mental health. The Public Health Agency of Canada analyzes data and develops knowledge products where multiple data sources are used for these analyses including surveys (e.g., Statistics Canada's Canadian Health Measures Survey, Canadian Community Health Surveys), the Canadian Cancer Registry, and administrative data (e.g., the Canadian Chronic Disease Surveillance System). In addition, surveillance data gaps are addressed through targeted surveillance initiatives which include collaboration with researchers (e.g., geospatial mapping to identify built environments, walkability scores).

Knowledge Products and Platforms

Under this activity area, the Public Health Agency of Canada develops and/or manages a variety of knowledge products and platforms, including the Chronic Disease Infobase (i.e. Chronic Disease and Injury Indicator Framework^{Footnote ii}  and Data Cubes), the Canadian Best Practices Portal, and the journal, Chronic Diseases and Injuries in Canada.

Interventions

Interventions that focus on common risk factors, such as unhealthy diet, physical inactivity and tobacco use, are funded by the Agency under the Multi-sectoral Partnerships Approach to Promote Healthy Living and Prevent Chronic Disease (herein referred to as the Multi-sectoral Partnerships Approach). To expand the reach of activities, this grants and contributions program promotes the development of multi-sectoral partnerships between non-governmental organizations and private sector stakeholders. Funding is also provided to support the installation of automated external defibrillators in recreational hockey arenas.

From 2008 until 2012, the Agency provided funding to community-based programs focused on addressing chronic diseases (Cancer Community-based Program, Canadian Breast Cancer Initiative, Canadian Diabetes Strategy) and healthy living (Healthy Living Fund). It also funded initiatives, through the Cardiovascular Disease Program, that aimed to address the recommendations from the Canadian Heart Health Strategy and Action Plan.^{Footnote iii} 

Policy Leadership

Nationally, the Public Health Agency of Canada supports the Public Health Network's Healthy People and Communities Steering Committee, and provides scientific support to the Canadian Task Force on Preventive Health Care (CTFPHC) in the development and revision of screening guidelines. Internationally, the Agency hosts the World Health Organization's (WHO) Collaborating Centre on Noncommunicable Disease (NCD) Policy, in addition to contributing to international committees/fora.

To carry out its activities, the Centre for Chronic Disease Prevention works in close collaboration with a variety of stakeholders. Some of the key players include:

• Federal government departments, such as Canadian Institutes of Health Research, Health Canada, Heritage Canada (Sport Canada), and Statistics Canada.
• Provinces and Territories, through the Public Health Network's Steering Committee on Healthy People and Communities but also bilaterally through various activity areas, including surveillance.
• Non-governmental organizations that focus on healthy living and chronic disease prevention, such as the Canadian Cancer Society and the Canadian Diabetes Association.
• Private sector partners that are engaged, or have the potential to be engaged, in multi-sectoral action to promote healthy living and address chronic diseases.
• The international community through the Pan American Health Organization, United Nations, and WHO.

Consult Appendix 4 for further details on key partners.

2.3 Program Logic Model and Narrative

According to the Public Health Agency of Canada's 2013-2014 Performance Measurement Framework, the expected results of the Chronic Disease and Injury Prevention Program are:

• Chronic disease prevention priorities for Canada are identified and advanced; and
• Chronic disease prevention practice, programs and policies for Canadians are informed by evidence.

The activity areas, immediate, intermediate, and long-term outcomes were outlined in the Performance Measurement Strategy for ISHLCD. Three main areas are depicted: partnership building/multi-sectoral collaboration, knowledge development and exchange, and surveillance and analysis.

The partnership building activity area is expected to result in an immediate outcome of multi-sectoral partnerships being initiated. These partnerships then lead to the development of common risk factor approaches to prevention initiatives. In the long-term, it is expected that this will result in chronic disease prevention priorities for Canada being identified and advanced.

Knowledge development and exchange, and surveillance and analysis activity areas are both expected to result to increased access to knowledge products at the immediate outcome level. At the intermediate level, those activity areas lead to use of knowledge products and information, and then to the long-term outcome of chronic disease prevention practice, programs and policies for Canadians being informed by evidence.

The connection between these activity areas and the expected outcomes is depicted in the logic model (see Appendix 2). The evaluation assessed the degree to which the defined outputs and outcomes were being achieved over the evaluation timeframe.

2.4 Program Alignment and Resources

The activities were part of the Public Health Agency of Canada's 2013-2014 Program Alignment Architecture: program 1.2 Health Promotion and Disease Prevention, sub-program 1.2.3 Chronic (non-communicable) Disease and Injury Prevention.

The financial data for the years 2009-2010 through 2013-2014 are presented below (Table 1). Overall, the sub-program had a budget of $234 million over five years^{Footnote iv} .

In 2013-2014, the Centre for Chronic Disease Prevention's staff consisted of a total of 175 full-time employees . The employees worked within five Divisions: Surveillance and Epidemiology Division (86), Partnerships and Strategies Division (30), Interventions and Best Practices Division (23), Prevention Guidelines Division (7), Executive Office (e.g., finance, planning, performance measurement, policy) (20), and the Director General's Office (9). Staff complement is composed of 4 directors, 83 policy analysts, 35 epidemiologists, 30 program analysts, 10 administrative positions, 5 coders, 5 medical positions, and 3 researchers.

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

The evaluation covered the period from April 2009 to August 2014 and included activities funded through the ISHLCD, the Canadian Strategy for Cancer Control, the Canadian Breast Cancer Initiative, the Lung Health Program, and the Automated External Defibrillators initiative. It focused on the HPCDP Branch chronic disease prevention activities carried out by the Centre for Chronic Disease Prevention and it excluded the chronic disease science activities conducted by the Social Determinants and Science Integration Directorate (SDSID), the ISHLCD investments in mental health, the Joint Consortium for School Health, investments in neurological diseases, the Federal Tobacco Control Strategy, and food security activities.

The evaluation issues were aligned with the Treasury Board of Canada's Policy on Evaluation (2009) and considered the five core issues under the two themes of relevance and performance, as shown in Appendix 5. Corresponding to each of the core issues, specific questions were developed based on program considerations and these guided the evaluation process.

A developmental approach was adopted given that activities have changed considerably in the past two years, and to inform a policy refresh exercise that was being conducted at the time of the evaluation. This approach was methodologically flexible and provided real time feedback. This flexible approach allowed the evaluation to assess the performance of grants and contributions programs most of which sunsetted, or were sunsetting, separately from this broader evaluation.

The Treasury Board's Policy on Evaluation (2009) also guided the choice of evaluation design and data collection methods to ensure that the evaluation would meet the objectives and requirements of the policy. A non-experimental design was used based on the evaluation matrix, which outlined the evaluation strategy for these activities.

Data collection started in March of 2014 and ended at the end of August 2014. Data for the evaluation was collected using various methods, which were: document and file review, financial data review, internal and external key informant interviews, international analysis of other countries' chronic disease prevention activities, performance data review, stakeholder mapping exercise, and a stakeholder web survey. A review of the content of the Canadian Best Practices Portal was also conducted. More specific detail on the data collection and analysis methods are provided in Appendix 5. Data were analyzed by triangulating information gathered from the different methods listed above. The use of multiple lines of evidence and triangulation were intended to increase the reliability and credibility of the evaluation findings and conclusions.

3.2 Limitations and Mitigation Strategies

Most evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. The following table outlines the limitations encountered while implementing the selected methods for this evaluation. Also noted are the mitigation strategies put in place to ensure that the evaluation findings can be used with confidence to guide program planning and decision making.

4.0 Findings

This section provides a summary of the findings organized under two broad headings:

• Relevance: the need, priorities, and federal public health role in preventing chronic diseases.
• Performance: the effectiveness, efficiency and economy of the Agency's activities in this area.

4.1 Relevance: Issue #1 - Continued Need for the Program

Chronic diseases are the leading cause of death in Canada and globally. Government and stakeholder action in addressing chronic disease is still needed.

The WHO defines noncommunicable diseases, or chronic diseases, as illnesses that are of long duration and generally slow progression. Each year, chronic diseases cause more deaths around the world than all other causes combined. In 2008, from a total of 57 million deaths that occurred worldwide, more than 36 million (63%) were attributable to chronic diseases, consisting mainly of cardiovascular diseases, cancer, diabetes, and chronic lung diseases^{Footnote 1} . The picture in Canada and most developed countries is comparable. In 2011, cancer was the leading cause of death, claiming 72,476 Canadian lives or 30% of all deaths. The same year, heart diseases caused 20% of all deaths, followed by stroke with 6% and chronic lower respiratory diseases with approximately 5% of all deaths^{Footnote 2} .

Prevalence and Incidence

Within the last decade, the prevalence of many chronic diseases has increased significantly. For example, between 1999 and 2009, the prevalence of diagnosed diabetes among Canadians increased by 70%. Approximately 2.4 million Canadians (6.8% of the total population) are currently living with diagnosed diabetes; however this is likely an underestimate as approximately 20% of cases remain undiagnosedEndnote 3. Continuous progress in the prevention and control of cardiovascular diseases has decreased mortality rates since the 1960s; however, research estimates that the aging of the population, combined with increased rates of obesity and diabetes, may increase prevalence and mortality attributed to cardiovascular diseases in the future^{Footnote 4} .

The incidence of chronic diseases is indicating a slight decrease in the more recent years. More than 200,000 Canadians were diagnosed for the first time with diabetes in 2008-2009. The overall trend in diabetes incidence indicated a slight increase since 1998-1999 but overall appears to be decreasing since 2006-2007^{Footnote 5} . In 2010, about 170,000 new cases of cancer were diagnosed in Canada. Although the trend analysis in overall cancer incidence showed an increase of 11% compared to a decade earlier, after taking into account the aging of the population, the analysis indicates that it has decreased by 3.4%^{Footnote 6} . Of the newly diagnosed, more than half are lung, colorectal, prostate and breast cancers^{Footnote 7} .

Vulnerable Populations

Social determinants such as education, occupation, income, gender and race influence risk factors and rates of chronic disease^{Footnote 8} . Vulnerable groups such as people of low socio-economic status, Aboriginals, and specific ethnic minorities are disproportionally impacted by chronic disease. For instance, people experiencing material hardship are less likely to adopt physical leisure activity or have healthy eating habits because they are more focused on coping with day-to-day life^{Footnote 9} . They are also more likely to respond to psychosocial stress by adopting unhealthy coping habits such as carbohydrate-rich diets, smoking or consuming alcohol^{Footnote 10} .

Aboriginal people experience higher rates of heart disease, diabetes, cancer and asthma than the rest of the population^{Footnote 11} . Further, the prevalence of chronic disease in the younger Aboriginal population is significant: obesity and obesity-related diseases, particularly type 2 diabetes, have increased in the younger First Nations people living in the North over the past two decades. The main changes contributing to the high incidence of obesity are linked to sedentary lifestyle, and low levels of physical activity, as well as a diet increasingly influenced by western-based food habits^{Footnote 12} .

Changes in the Environment

Public health research findings
It was found that many chronic diseases can be prevented by decreasing four main behavioural risk factors: physical inactivity, poor nutrition, smoking and alcohol consumption. More than 50% of the Canadian population aged 20 and over is spending more than 14 hours per week engaging in sedentary activities and only half is reporting being physically active during leisure time^{Footnote 13} . Merely 7% of children and youth attain the recommended daily level of activity^{Footnote 14} . As well, less than half of the population aged 12 and over reports consuming the recommended amount of fruit and vegetables to ensure a healthy diet. Although effective interventions have reduced smoking rates among Canadians aged 12 and over from 26% in 2001 to 20% in 2011, smoking remains a serious risk factor^{Footnote 15} Endnote 15.

Behavioural risk factors can lead to intermediate conditions such as obesity, high blood pressure, raised blood glucose and raised lipids, which are important determinants for many chronic diseases. Obesity increases the risk of many chronic diseases such as type 2 diabetes, hypertension, cardiovascular disease, and some forms of cancer. Since 1981, measured obesity roughly doubled in most age groups and became more severe, with a greater increase in the heaviest weight classes. Data from 2009-2011 showed that 11.7% of children aged 5 to 17 years and 26.2% of adults aged 18 years and over were obese^{Footnote 16} .

Over the past few years, research increasingly demonstrated the linkages between physical and social environment and chronic disease. The effects of rapid unplanned urbanization and globalization have led to increasing sedentary lives and unhealthy diets^{Footnote 17} . The Canadian physical and social environments have seen changes, such as reduced "walkability" of neighbourhoods, disparities in accessibility and proximity of grocery stores, and the sedentary nature of play and work that can negatively impact human health^{Footnote 18 Footnote 19} .

Changes in the population
The proportion of overweight and obese individuals is dramatically on the rise around the world, particularly in urban settings^{Footnote 20} . Trends since the 1980s demonstrate an important increase in the prevalence of those overweight and obese in both sexes and almost all age groups^{Footnote 21} . Currently, childhood obesity is one of the most serious public health challenges given that obese children are more likely to become obese adults with increased risk for chronic disease.

Another change in the context of chronic disease is the increased aging of the population. Over the past 30 years, the population aged 65 years and over has been increasing. This trend is expected to continue in the next few decades and, by 2050, the elderly are projected to comprise 27% of the Canadian population^{Footnote 22} . Canadians are living longer and are therefore more likely to develop chronic diseases and related conditions. In 2009, 89% of Canadian seniors were living with at least one chronic condition^{Footnote 23} .

Finally, research shows that more and more people are living with multiple chronic diseases. The 2009 Canadian Community Health Survey revealed that more than 13% of the Canadian population aged 20 and over have multiple (two or more) concurrent chronic conditions, such as heart disease, stroke, cancer, asthma, chronic obstructive pulmonary disease, diabetes, arthritis, Alzheimer's or other dementia^{Footnote 24} . As the number of chronic conditions in an individual rises, there is an increased risk of premature death, reduced quality of life and increased costs to the healthcare system.

New player
In addition to new research findings, the creation of a strong player on the national scene in 2007 - the Canadian Partnership Against Cancer - has also contributed to changing the landscape by leading key cancer prevention and control activities. Under the Canadian Strategy for Cancer Control, developed in 2006, the federal government brought together partners from across the country under the umbrella of the Canadian Partnership Against Cancer. The Partnership promotes the integration of solutions across the continuum of cancer control, leverages funds, and facilitates knowledge creation and exchange.

Alignment between the Changes in the Environment and Current Activities

In order to address these changes in the environment, the Agency, Branch and the Centre for Chronic Disease Prevention have developed a number of plans and strategies. In 2013, the Centre developed the Preventing Chronic Disease Strategic Plan (2013-2016), which sets out the Centre's role and strategic priorities in addressing chronic disease. These plans, supported by key informant interview data, have indicated that there is a strong alignment between the changes in the environment and current activities. For instance, the Centre now focuses its work in addressing common risk factors. Also, given the creation of the Canadian Partnership Against Cancer, the Centre has limited its activities in cancer knowledge development and exchange.

4.2 Relevance: Issue #2 - Alignment with Government Priorities

The prevention of chronic disease and related conditions, such as obesity, is a priority for the Government of Canada as well as the Public Health Agency of Canada.

Government of Canada

The Pan-Canadian Healthy Living Strategy was first endorsed by Federal, Provincial and Territorial Ministers of Health in 2005, recognizing the importance of promoting healthy living behaviours through coordinated and sustained action^{Footnote 25} . Shortly thereafter, the Public Health Agency of Canada developed the Integrated Strategy on Healthy Living and Chronic Diseases (ISHLCD) to confirm the Agency's integrated approach to addressing chronic disease and promoting healthy living. From there, the Government of Canada has continued to demonstrate that the prevention of chronic disease and related conditions is a priority through signing a number of international agreements.

The Government of Canada reinforced that the prevention of chronic diseases and related conditions is a priority through collaborating with its provincial and territorial counterparts to develop national agreements. In 2010, the Pan-Canadian Healthy Living Strategy was strengthened, building on common risk factors and conditions, and pointing to new areas of opportunity, including preventing obesity^{Footnote 26} . To this end, two initiatives were endorsed between 2010 and 2011:

• Creating a Healthier Canada: Making Prevention a Priority - A Declaration on Prevention and Promotion from Canada's Ministers of Health and Health Promotion, which was a public statement to work together, and with others, to make the promotion of health and prevention of disease, disability and injury a priority for action^{Footnote 27} ; and
• Curbing Childhood Obesity: A Federal, Provincial and Territorial Framework for Action to Promote Healthy Weights, which focuses on reducing prevalence of childhood obesity in Canada and advancing strategies for Federal, Provincial and Territorial collaboration^{Footnote 28} .

In 2011, the Political Declaration of the High-level Meeting of the General Assembly on the Prevention and Control of NCDs^{Footnote 29}  was endorsed by global leaders, including Canada. This declaration established an international agenda on chronic diseases and led to the Rio Political Declaration on Social Determinants of Health, also adopted in 2011, during the World Conference on the Social Determinants of Health, organized by the WHO. As a signatory, Canada expressed its commitment to the implementation of a social determinants of health approach to reduce health inequities and to achieve other global priorities^{Footnote 30} . These declarations resulted in the Global Action Plan for the Prevention and Control of Noncommunicable Diseases, which was endorsed by the WHO and aims to strengthen national efforts to address the burden of chronic diseases and help attain voluntary global targets. This plan was updated in 2013, but remains largely focused on the same objectives, including common risk factors for chronic diseases.

There are also a number of national, disease-specific strategies in place to help address chronic diseases in Canada. These include, but are not limited to, the Canadian Strategy for Cancer Control, developed between 1999 and 2006 by federal, provincial and territorial partners, as well as a multi-stakeholder group of Canada's leading cancer organizations. Since 2007, it has been implemented by the Canadian Partnership Against Cancer. There is also the Canadian Diabetes Strategy, an element of the ISHLCD, which contributes to diabetes prevention and control through surveillance, evidence-based information and support to community-based programs.

While the Speech from the Throne and the Budget do not specifically highlight chronic disease, they do confirm healthy living as a priority, with specific commitments made to lung health in 2008, physical activity, through PartipACTION in 2012, and through a commitment to work with provinces and territories, private and not-for-profit sectors to encourage young Canadians to be more physically active, in 2013.

Public Health Agency of Canada

Since at least 2009, the Public Health Agency of Canada has identified surveillance, common risk factors, the development of knowledge and research and, more recently, multi-sectoral partnerships to address common risk factors, as key priorities. Obesity (or healthy weights) is also a key priority of the work of the Agency. These priorities are outlined in the Agency's five year strategic plan, Strategic Horizons, as well as the Corporate Risk Profile.

Strategic Horizons 2013-2018 identifies the following priorities related to chronic disease prevention:

• Strengthen collaborative engagement to promote healthy weights and prevent obesity, particularly among children, and
• Increase public health surveillance of noncommunicable diseases, common risk factors, maternal/child health, injuries and infectious diseases.

Further, the Agency indicates that it will take a population health and equity analysis approach in planning the specific focus of its key priorities, and it expects this work to result in in improving population health and well-being and reducing health inequalities.

Since 2012, the Public Health Agency of Canada's Corporate Risk Profile has noted the following key priorities:

• Upstream prevention interventions to refocus activities, including science, research surveillance and knowledge development and exchange, on common risk factors;
• Leveraging multi-sectoral partnerships to increase the impact and reach of programs; and
• Working to prevent increasing rates of obesity through identifying, testing and sharing interventions, social marketing and promoting shared leadership.

The Agency's Reports on Plans and Priorities within the scope of this evaluation (2009-2010 to 2013-2014) further confirm that the Public Health Agency of Canada identifies chronic disease, associated conditions, and common risk and protective factors as priorities. The reports indicate that the Agency is continually working to strengthen chronic disease prevention initiatives. In particular, the Agency is working to address common risk factors and to advance protective factors, such as physical activity. These priorities are aligned with priorities of other countries, as determined by the evaluation's international analysis.

Alignment between Priorities and Current Activities

In order to address these priorities, the Agency, Branch and Centre have developed a number of plans and strategies. In 2013, the Centre for Chronic Disease Prevention developed the Preventing Chronic Disease Strategic Plan (2013-2016), which sets out the Centre's role and strategic priorities in addressing chronic disease in Canada and internationally. More specifically, the Centre is implementing aspects of its strategic plan in parallel with its Knowledge Development and Exchange Plan (2013-2016) and the Digital Strategy (2013-2016), which seek to modernize its knowledge modernization activities.

At the Agency level, the Agency Plan to Advance Health Equity (2013-2016) was developed with a priority to strengthen the evidence base through consistent collection, analysis and reporting on social determinants of health and health inequalities. It contributes to Canada's commitment for the implementation of a social determinants of health approach, as outlined in the Rio Political Declaration on Social Determinants of Health. The Agency also developed a Suveillance Strategic Plan 2013-2016, which sets out, among other priorities, the Agency's surveillance transformation priorities. At the Branch level, the HPCDP Branch developed the HPCDP Science Plan (2013-2016) in order to better align research to Branch policies, programs and priorities.

Branch and centre strategic and operational plans reflect priorities set out by the HPCDP Branch, the Public Health Agency of Canada and the Government of Canada. These plans, supported by key informant interview data, have indicated that there is a strong alignment between priorities and current activities. There may be opportunities to clarify, through the Centre for Chronic Disease Prevention policy refresh exercise, how the Agency's chronic disease prevention activities address health inequalities, especially within the Multi-sectoral Partnerships Approach.

4.3 Relevance: Issue #3 - Alignment with Federal Roles and Responsibilities

Various legislative, policy and program authorities identify that the federal government has a role in and responsibilities to conduct surveillance, research, provide leadership, address health promotion and prevent chronic diseases through funding community-based interventions to prevent chronic diseases, identifying best practices and developing screening guidelines. The role played by the Agency in preventing chronic disease has evolved in response to changes in the current environment.

The Public Health Agency of Canada addresses the federal government's broad role in health promotion and disease prevention and control outlined in foundational legislation. The Canada Health Act describes one of the main objectives of Canadian health care policy as being to "protect, promote and restore the physical and mental well-being of residents of Canada". Within this mandate, the Department of Health Act gives the Minister of Health powers, duties and functions relating to "the promotion and preservation of the physical, mental and social well-being of the people of Canada". Finally, the Public Health Agency of Canada Act specifies that some of the activities the Government of Canada can undertake include health protection and promotion, population health assessment, and health surveillance.

The expert studies that helped identify gaps in the public health system and ultimately helped shape the Public Health Agency of Canada (i.e., Lalonde 1974, Kirby 2002, Naylor 2003, Ottawa Charter 1986) also clearly comment on the federal government's role in health promotion, indicating that the federal government should have a specific role to ensure that health promotion and chronic disease prevention activities are sensitive to health equity considerations.

In addition to these broader roles, the Public Health Agency of Canada has specific roles in chronic disease surveillance, sharing of best practices, leadership, capacity building and public health guidelines.

Surveillance

Legislation provides the primary basis for the Agency to conduct surveillance activities. It is clearly laid out in the Department of Health Act that the Minister has powers, duties and functions in matters of investigation and research into public health, including disease surveillance. The Public Health Agency of Canada Act reinforces this role by giving the Governor in Council the power to make regulations regarding the collection, analysis, interpretation, publication and distribution of information relating to public health.

As confirmed in policy and program authorities, the Public Health Agency of Canada has a role in conducting chronic disease surveillance. Further, the ISHLCD provides the Agency the policy authority to conduct enhanced surveillance activities for chronic disease, and the Canadian Strategy for Cancer Control provides the policy authority to conduct cancer surveillance (apart from the Canadian Cancer Registy).

The importance of the Agency's role in surveillance was highlighted by internal and external key informants as it was the most frequently cited federal role. In particular, key informants reported that the Public Health Agency of Canada had a role to provide a pan-Canadian perspective on the trends, incidence and risk factors of chronic disease. It was expected that activities would include: data collection, streamlining, coordinating and ensuring data quality and comparability. Key informants also reported that this role helps provinces and territories build on an existing system and provides them with the capacity to collect data and distribute the surveillance knowledge appropriately.

Sharing of Best Practices

Many internal and external key informants identified the sharing of best practices as a key federal public health role in the area of chronic disease prevention, second to conducting surveillance. Under the ISHLCD, the Public Health Agency of Canada received the authority to conduct this activity as part of the Observatory of Best Practices.

Leadership

There is a legislative basis for the Public Health Agency of Canada's leadership role in chronic disease. Under the Department of Health Act, the Minister has powers, duties and functions to cooperate with provincial authorities in coordinating efforts. The Agency was established under the Public Health Agency of Canada Act to assist the Minister and also to foster collaboration regarding public health with provincial and territorial governments, other countries, and international organizations. Two policy and program authorities further detail the leadership role of the Agency in regard to chronic disease prevention and control: the ISHLCD and the Canadian Breast Cancer Initiative. The ISHLCD also notes the Agency's leadership role in international activities related to chronic disease prevention by continuing to support the WHO Collaborating Centre on NCD Policy, while the leadership role identified in the Canadian Breast Cancer Initiative includes establishing an external advisory committee and coordinating activities with other departments.

Key informants agree that the Agency should be undertaking these policy leadership activities and that there is a role in bringing groups together, including the provinces and territories, non-governmental organizations and the private sector. Key informants noted that this role should include setting up and facilitating the development of policy frameworks and priorities, both nationally and internationally.

Public Health Guidelines

Several policy and program authorities indicate there is a federal role to support the development of national health guidelines for physical activity (ISHLCD), clinical practice guidelines related to diabetes and hypertension (ISHLCD), and cancer (ISHLCD, Canadian Strategy for Cancer Control). Although policy and program authorities identify a role in developing screening guidelines, internal and external key informants were less likely to mention this as a key role in chronic disease prevention for the federal government. An internal key informant noted that this may not be a well-recognized role given that the Public Health Agency of Canada does not develop guidelines; however, since 2010 it has supported the work of the CTFPHC, which is an arms-length voluntary body that produces and updates screening guidelines.

Research

According to legislative authorities (Department of Health Act), foundational documents (Naylor, Lalonde), and the ISHLCD policy and program authorities, the federal government has a role in the development of new knowledge. Although the Centre for Chronic Disease Prevention previously undertook research through the Chronic Disease Science Office, most activities are now led by SDSID. The Centre focuses on epidemiological research and the dissemination of results through the journal Chronic Diseases and Injuries in Canada. It should be noted that key informants were less likely to mention research as a key public health role in chronic disease prevention in comparison to the roles mentioned above.

Change in Direction

The Public Health Agency of Canada also has a role in funding interventions through grants and contributions. This role, however, has shifted over the years. The ISHLCD authorities identified a role in funding community-based initiatives that focus on the prevention and/or management of specific disease. That role has since evolved to nation-wide funding initiatives that address common risk and protective factors through multi-sectoral partnerships (i.e., public-private). This shift aligns with federal/provincial/territorial and international declarations' priorities. Further, the 2013 Speech from the Throne and Budget outlined the government's commitment to work with the private and not-for-profit sectors to encourage Canadians to be more physically active.

Also, as the Centre for Chronic Disease Prevention and Control, the Centre undertook a role in the area of disease control and management, as outlined in the ISHLCD policy and program authorities, to collaborate on integrated management models for major chronic diseases. However, as a result of a budget reallocation decision, and to better align the federal role with provincial and territorial roles, the Centre is now focusing its activities in the area of chronic disease prevention, which addresses priorities identified in federal/provincial/territorial and international declarations.

Alignment Between Role and Current Activities

The Public Health Agency of Canada's current activities in chronic disease prevention are well aligned with its surveillance, best practices, policy leadership, and research roles. In the area of surveillance, the Agency manages the Chronic Disease Surveillance System and the Chronic Disease Infobase, in addition to developing surveillance knowledge products and collaborating with Statistics Canada in cancer surveillance and health surveys. With regards to the sharing of best practices, the Agency manages the Canadian Best Practices Portal, while it continues to provide policy leadership through various activities, including the CTFPHC and monitoring the Obesity Framework. The Agency's research role is expressed through the journal Chronic Diseases and Injuries in Canada and epidemiological studies.

Activities in the the area of disease management and community-based projects have slightly shifted based on a budgetary decision to focus on the prevention of chronic diseases with a focus on common risk factors. These shifts align with signed-upon federal/provincial/territorial and international priorities.

The Public Health Agency of Canada's role in preventing chronic disease is complementary to other federal government departments, provinces/territories and non-governmental organizations. Although duplication does not appear to exist, the difference between the health promotion roles carried out by the Agency, Health Canada and Heritage Canada (i.e., Sport Canada), is unclear to external stakeholders.

As discussed under Program Context, many stakeholders are involved in addressing chronic diseases. Although this evaluation found no evidence of duplication, the multitude of stakeholders working in chronic disease prevention increases the likelihood of this occurring. (Please refer to Appendix 4 for an overview of the roles of other federal government departments and provinces/territories.)

The Public Health Agency and Other Federal Government Departments

A review of other federal government departments' legislation, strategic plans and websites indicates that the role played by other federal government departments in the area of chronic disease prevention, research and statistics/surveillance, appears to be complementary to that of the Agency. However, a few areas require a clarification of roles and responsibilities for external stakeholders. For example, internal and external key informants reported being concerned regarding a potential for duplication with the reporting and analysis function carried out by Statistics Canada, as the Public Health Agency of Canada also conducts similar activities. For example, both groups analyze health-related data, and produce reports. These groups mitigate any issues by collaborating through an advisory committee and working groups to avoid duplication of efforts.

As well, while the Agency, Health Canada's Office of Nutrition Policy and Promotion (ONPP), and Sport Canada's roles are distinct, external key informants had difficulty distinguishing between their activities in the area of nutrition and physical activity, which results in confusion regarding the leadership role that the Agency should assume. These groups, however, understand each other's roles clearly, and engage each other when needed.

Public Health Agency of Canada and Provinces/Territories

Overall, the role of the provinces and territories appears to be complementary to the role of the Public Health Agency of Canada. A review of strategic plans, key strategies and websites finds that, like the Agency, most provinces and territories are conducting activities to address common risk factors. For example, information on provincial and territorial websites targets healthy eating and physical activity as ways to prevent chronic disease^{Footnote 31 Footnote 32 Footnote 33} .

Although provinces and territories conduct activities to prevent chronic diseases, most of their focus is on specific chronic diseases, disease management and treatment; whereas the Agency has recently focused its activities on disease prevention. Key informants agreed that roles were similar, but complemented each other very well, especially since the Agency's activities have moved upstream to focus on the prevention of chronic diseases, therefore limiting potential duplication of roles in the area of disease management and control. Although provinces/territories and the Public Health Agency of Canada have a role in surveillance, provinces and territories' role is to collect and analyze provincial data, while the Agency's role is to coordinate data collection and analyze data at the national level, which is clearly a role for a federal public health entity. This was noted by internal, as well as provincial and territorial key informants, who also felt that the roles carried out by the Agency and provincial and territorial governments complemented each other very well.

Public Health Agency of Canada and Non-Governmental Organizations

While the Public Health Agency of Canada and stakeholder organizations are active in many of the same activity areas (e.g., national leadership, research, health promotion and disease prevention), the potential for overlap and duplication is mitigated by the niche focus of stakeholder organizations and linkages between the Agency and stakeholders. For example, stakeholders may play a national leadership role in some instances (e.g., chairing a national committee, creating guidelines), however, this role is complementary and acknowledged through a funding relationship or formal collaboration with the Agency.

Likewise, research is an activity where the Public Health Agency of Canada, other federal government departments and stakeholders/non-governmental organizations have a role to play. However, current national priority setting, strategies and peer review processes that are in place mitigate overlap and duplication of research efforts. Additionally, while the federal government is not active in advocacy, non-governmental organizations fulfill this research role by publishing position papers, increasing awareness of disease burden, and striving to present the patient voice.

The federal government brings together national surveillance data, through collaboration with provinces and territories, whereas non-governmental organizations are not undertaking population surveillance to any significant degree. They do, however, use the surveillance data generated by the Agency to cite, publish and present surveillance statistics. For example, the Canadian Cancer Society supports the production of an annual series of publications on prevalence of different cancers. Non-governmental organizations have also used the data that is housed within the Public Health Agency of Canada to create smaller registries targeted to specific segments of the population (e.g., patients) for research purposes.

The only area of potential duplication between the Agency and stakeholders is knowledge translation; however, the audiences for these repeated messages tend to differ. While non-governmental organizations are focused on knowledge products for the general public, the Public Health Agency of Canada produces information typically for public health professionals. For example, data from the Canadian Cancer Registry is used by the Agency and the Canadian Cancer Society to produce an annual technical report on cancer rates in Canada. From there, the Canadian Cancer Society uses the information to develop products targeted to the public, at the community-level. Key informants have reported that although many players communicate prevention messages, there is no harm in repeating these messages as long as they are not contradictory. Having these repeated messages tailored to different audiences is beneficial, as they will be more meaningful for the intended audience.

4.4 Performance: Issue #4 - Achievement of Expected Outcomes (Effectiveness)

4.4.1 To what extent have the immediate outcomes been achieved?

Immediate Outcome #1: Integration

There are limited examples of integration of chronic disease prevention activities within the Centre for Chronic Disease Prevention; however, recent changes aim to facilitate integration.

Integration, or the science to policy continuum of activities (i.e., science and surveillance informing policy and programming), was one of the key features of the ISHLCD. While there has been some effort to integrate chronic disease prevention activities, improvements could be made.

Integration in the Previous Model

Internal key informants did not feel that activities were integrated in the previous model. Internal key informants noted that there were attempts to communicate activities but that organizational integration was not strong. According to internal key informants, mechanisms were in place to ensure coordination between functional and program components, but those structures were more conducive to information-sharing than integrating along the continuum of activities. For example, working groups were established for disease-specific program components, which included representatives from various functional components of the ISHLCD, such as surveillance, knowledge development and exchange, and community-based programming.

The development and implementation of the CANRISK initiative was mentioned by internal key informants as a successful example of integration, where science-based data informed programming and where the private sector became an integral part of public health programming^{Footnote v} . The project was initially developed by SDSID's Science Integration Division (previously housed under the Centre for Chronic Disease Prevention). Scientists refined a tool developed in Finland, and adapted it to the Canadian context. A peer-reviewed article was then published to validate the methodology. Once the tool was approved, it was disseminated through retail pharmacies, and the Partnerships and Strategies Division established a contribution agreement with the Canadian Pharmacists Association for promotion of the tool among pharmacists as well as evaluation of its uptake.

Some internal key informants noted that the development and ongoing monitoring of the Obesity Framework was another example of integration of activities. According to internal key informants, the Centre for Chronic Disease Prevention and SDSID collaborated on developing the Agency's approach to addressing obesity, and then activities were coordinated by the Centre's policy analysts with the support of epidemiologists.

Integration in the New Model

Internal key informants noted that the lack of an information-sharing structure at the staff level has limited opportunities to integrate across the Centre for Chronic Disease Prevention. Key informants felt that integration occurred mostly at an ad-hoc, project-basis. However, the Centre for Chronic Disease Prevention has recently taken measures to facilitate integration.

Since June of 2014, on two occasions, the Surveillance and Epidemiology Division convened the Centre's Extended Senior Management Committee members to discuss selected surveillance findings. The first Quarterly Emerging Issues Forum on included discussion papers on three topic areas, including diabetes in pregnancy. The survey results show that participants appreciated the session (86% were satisfied or very satisfied with the session). Issues identified through survey results were addressed in the second forum. For example, instead of having presentations on three topics, two topics were presented with more discussion time allotted.

The two fora were attended by representatives from all areas of the Centre for Chronic Disease Prevention, while the second forum also included attendees from other areas within the HPCDP Branch (i.e., SDSID and the Centre for Health Promotion). Key informants recognized the importance of these fora to facilitate integration across the Centre and within the HPCDP Branch. The Centre is in the process of determining how it will maximize these opportunities and ensure action items are integrated into planning and decision-making processes.

In terms of projects, most Centre key informants mentioned the sedentary behaviour research project as a perfect example of how surveillance information has influenced programming (i.e., grants and contributions funding). The Partnerships and Strategies Division requested data from the Surveillance and Epidemiology Division to identify an area within the realm of physical activity that could be addressed further in a social marketing campaign. The Surveillance and Epidemiology Division staff conducted research on behalf of the Partnerships and Strategies Division and were able to determine a strong link between sedentary behaviour - regardless of physical activity - and chronic disease. Both groups determined that this would be a suitable area to explore further.

The evidence was discussed further and then, through an Ideas Factory meeting, participants from across the Centre for Chronic Disease Prevention and SDSID contributed to refining the approach. Both groups continue to collaborate during the upcoming implementation phase of this project. In addition to strengthening a project, this helped in solidifying linkages between surveillance and programming.

While there appear to be more efforts to integrate activities across the Centre for Chronic Disease Prevention, to date, the involvement of SDSID's Science Integration Division with Centre activities has been limited. With the exception of the CANRISK initiative and involvement in the Obesity Framework, key informants did not provide other examples of how science has contributed to the Centre's chronic disease prevention activities. The consequence of not integrating activities, as noted by internal key informants, is that staff members continue to work in silos and may not benefit from the knowledge and expertise of other parts of the Centre and the HPCDP Branch.

Immediate Outcome #2: Internal and External Collaboration

As noted earlier in the report, many organizations are involved in the prevention of chronic diseases nationally and internationally. Collaboration is key to ensure that organizations work together in the achievement of common outcomes. Although improvements are needed, especially in terms of communicating priorities, overall, the Public Health Agency of Canada collaborates well and frequently with its partners, mainly on a project-by-project basis.

The following section presents an overview of some of the key activities and achievements that stem from a select number of the Agency's collaborations in the area of chronic disease prevention. Please note that it does not represent an exhaustive review of each collaborative relationship. Rather, it represents highlights of some of the major relationships that the Public Health Agency of Canada has developed and maintained in the past five years.

Although collaboration within the Public Health Agency of Canada exists, further improvements could be made.

Regional Operations

The Public Health Agency of Canada's Regional Operations, since the Agency's Regional Transformation in April of 2012, support the work of Agency Centres through centre agreements. The 2014-2017 Centre Agreement between the Centre for Chronic Disease Prevention and Regional Operations has improved in comparison to the 2013-2014 Centre Agreement. The previous version was not clearly linked to the Centre for Chronic Disease Prevention's priorities, whereas the current Centre Agreement includes this link in addition to identifying the intended use of each deliverable. It also includes a business management model check-list, and activities are further aligned with regional expertise.

Internal key informants are satisfied with support provided by Regional Operations, however, there appear to be opportunities to further align the Centre Agreement with current Centre for Chronic Disease Prevention priorities. Some internal key informants noted that some support to the Multi-sectoral Partnerships Approach in the area of partnership development and brokering, and other Centre priority areas, could be strengthened to capitalize on Regional Operations' staff expertise. However, it should be noted that the Centre is one of many other Centres that Regional Operations must serve since 2012, and therefore not all Agency activities can be served regionally.

Social Determinants and Science Integration Directorate

Although the Centre for Chronic Disease Prevention and SDSID collaborate on many projects, interviewees felt that there is opportunity to augment collaboration and facilitate integration, as appropriate, given the Directorate's expertise in health equity (i.e., the Social Determinants of Health Division) and science (i.e., the Science Integration Division).

The Social Determinants of Health Division collaborates with the Centre on various activities, including determining and assessing new best practices focused on health equity for the Canadian Best Practices Portal. The Division also provides expertise in the analysis of the impact of chronic disease in more vulnerable populations. Key informants noted that involvement with the Multi-sectoral Partnerships Approach has been on an ad hoc basis.

The Science Integration Division provides scientific support to HPCDP's Centres. One of the Division's main activities is to review HPCDP scientific publications. In the past five years, other than a few examples mentioned under the Integration section above, it is unclear to what extent the Science Integration Division has supported the Centre for Chronic Disease Prevention's activities.

Communications and Public Affairs Branch

The Centre for Chronic Disease Prevention and the Communications and Public Affairs Branch (CPAB) collaborate well on funding announcements. However, there are opportunities to improve collaboration between the Centre and E-Communications where further clarification of strategic communications and E-Communications requirements could be strengthened.

The Partnerships and Strategies Division and CPAB collaborate frequently, through regular conference calls/meetings and part-time co-location, to facilitate the announcement of projects funded through the Multi-sectoral Partnerships Approach. This has allowed staff to understand requirements around ministerial announcements, but also the importance of planning these announcements and their potential key messages.

Collaboration could be improved, however, as it relates to the online posting of chronic disease knowledge products. The majority of internal key informants expressed concerns with multiple levels of approvals and the unclear requirements surrounding the online posting of products. The lack of clarity could be attributed to the fact that while the E-communications function in CPAB has specific web publishing requirements, the Centre was managing its own web coding, web publishing, and associated processes for several of its online platforms. Of note, this potential confusion may be alleviated if the Centre continues the process of transferring this function to CPAB as part of the Health Canada and Public Health Agency of Canada Shared Services Efficiency Exercise. According to key informants, the challenges to posting products have hindered the Centre's ability to increase access to, and timeliness of knowledge products. (See section 4.4 for further detail.) Although there is an HPCDP Communications Strategy for 2013-2016, which highlights communication priority areas and confirms some of the issues around publishing processes, there may be opportunities to strengthen collaboration between the Centre and the E-Communications function of CPAB to ensure that all parties are aware of each other's priorities and activities and benefit from each other's expertise.

At the federal government level, the Public Health Agency of Canada collaborates frequently with Health Canada and Statistics Canada.

Health Canada

Although the role differentiation between Health Canada's ONPP and the Centre for Chronic Disease Prevention is unclear to external stakeholders, both groups communicate regularly and collaborate frequently through various initiatives.

ONPP provides public health nutrition expertise to various Public Health Agency of Canada projects, including contributing to the Canadian Best Practices Portal, reviewing letters of intent under the Multi-sectoral Partnerships Approach, developing joint products such as the Eat Well Be Active Educational Toolkit and participating on a variety of working groups to support ongoing Agency activities. ONPP participated in the development of the Agency-led Obesity Framework and it continues to play a key role in supporting the Framework by contributing to various task teams.

ONPP also includes the Public Health Agency of Canada in the development of national nutrition policies. In the past five years, however, that work has mostly focused on prenatal nutrition guidelines and the infant feeding guidelines. Beyond project-level collaboration, ONPP and the Agency communicate frequently through various committees including the Food and Nutrition Surveillance working group, which aims to maintain a credible and coordinated approach to food and nutrition surveillance within the Health Portfolio. Key informants noted that staff members know when to involve each other in their work and both the Public Health Agency of Canada and ONPP are very satisfied with support received from each partner.

Statistics Canada

The Public Health Agency of Canada and Statistics Canada collaborate frequently given that the Agency relies on Statistics Canada data to conduct some of its surveillance activities. Recent mechanisms have been implemented to address challenges in coordination and communication, including a Director General-level committee on health statistics.

Within the Public Health Agency of Canada, the Data Coordination and Access Program determines data acquisition needs; acquires data on behalf of the Agency; manages data agreements; provides data quality control services; and serves as the single point of access between the Agency and Statistics Canada. Key informants noted that this new function has been very effective in monitoring requests, managing agreements and, overall, ensuring that Statistics Canada has one single point of access. Prior to the establishment of the Data Coordination and Access Program, external key informants mentioned having had difficulty in determining the correct individual to liaise with and in receiving timely feedback from the Agency on deliverables.

In addition to having a Director General-level committee on health statistics where priorities for the year are communicated and discussed, every health-related Statistics Canada survey includes representation from the Public Health Agency of Canada at the working level and the senior management level. These two levels of committees/working groups allow the Agency to provide feedback on the content and objectives of questionnaires. As one key informant noted, it is very important for the Agency and Statistics Canada to establish a good collaborative relationship given that only the Agency, Statistics Canada and the Canadian Institute for Health Information have access to survey data for a period of 10 to 12 months. After that period, data becomes "public", which means that any stakeholder is allowed to publish on a particular topic related to the data set.

External key informants noted the importance of the Public Health Agency of Canada's activities in convening provincial and territorial stakeholders, and fostering collaboration, and felt that the Agency could increase activities in this area.

Through the Public Health Network's Healthy People and Communities Steering Committee, the Public Health Agency of Canada has mobilized provinces and territories and other federal government departments on a variety of initiatives, including the development and monitoring of the implementation of the Obesity Framework. The framework was mentioned as a key policy document that continues to influence stakeholders' work in determining their plans and priorities. External and provincial/territorial key informants expressed their appreciation of the Agency's leadership and activities in this area.

Since moving to a common risk factor approach, external key informants have expressed an interest in receiving more guidance on nutrition and physical activity from the Agency, as well as promotion of healthy built environments. Key informants felt that the Agency should play a more active role in setting priorities, including working with stakeholders to address issues surrounding sodium and trans fats in food; however, this may be a role more suitable for Health Canada given its role in public health nutrition. This might indicate, as noted under the section on Role, the need to clarify the differences between the role played by the Public Health Agency of Canada and Health Canada, in the area of healthy living and chronic disease prevention.

The Public Health Agency of Canada's collaboration with provinces and territories on surveillance is strong when it comes to the implementation and monitoring of surveillance systems. In addition to collaborating through ongoing surveillance systems and Statistics Canada surveys/registry, the Agency and provinces and territories collaborated well on supporting the work of the Canadian Fitness and Lifestyle Research Institute through the establishment of the Physical Activity Monitor and CANPLAY. Over the past five years, approximately 22 bulletins were produced based on the surveys' data, which were all accessible on the Canadian Fitness and Lifestyle Research Institute website.

Overall, internal and external key informants noted a challenge in establishing surveillance knowledge product priorities at the national level. The current federal/provincial/territorial decision-making and priority-setting body may not fully assist the Public Health Agency of Canada in receiving guidance and feedback from provinces and territories on surveillance plans. The existing committee is composed of technical staff that, although very knowledgeable about systems issues, is not necessarily aware of their own jurisdiction's information needs nor are they able to make decisions on behalf of their jurisdiction. It becomes difficult for the Agency to prioritize its surveillance activities, and for provinces and territories to understand upcoming surveillance plans.

The Public Health Agency of Canada supports the work of the Canadian Task Force on Preventive Health Care.

The Centre for Chronic Disease Prevention's Prevention Guidelines Division supports the work of the CTFPHC by coordinating activities, conducting research on its behalf and providing funding to support knowledge translation and evaluation activities. The CTFPHC was re-established in 2010 and, since then, it has conducted evidence reviews and developed screening guidelines for six conditions/diseases, and is currently completing five other evidence reviews and guidelines. The CTFPHC has also conducted appraisals of six other guidelines developed by other organizations.

Through collaboration with the Public Health Agency of Canada and other key stakeholders, the CTFPHC has successfully developed and disseminated clinical practice guidelines and has been building a base of partnerships and collaborations. According to an internal evaluation report, physicians' level of awareness and understanding varies from high to moderate among guidelines. Furthermore, although research estimates that the overall cycle of new research and guideline development to clinical practice changes can take from 17 to 25 years^{Footnote 34 Footnote 35} , there is early indication of changes in practice based on the breast cancer and cervical cancer screening guidelines. The internal evaluation's key informants indicated already having adopted the recommendations for both guidelines by reducing screening.

Collaboration with non-governmental organizations and academia takes place at a project-level.

Over the past five years, the Centre for Chronic Disease Prevention has informed, consulted, or collaborated with non-governmental organizations and academia at the project-level. Also, as a member of the University of Waterloo's Propel Centre for Population Health Impact (herein referred to as the Propel Centre), the Centre for Chronic Disease Prevention has been collaborating with over 400 policy, practice and research colleagues across Canada.

Key informants representing non-governmental organizations noted that the Public Health Agency of Canada had been very good at liaising with them on multiple fronts, from liaison on implementing the Obesity Framework to informing them of the new Multi-sectoral Partnerships Approach. These external key informants felt, however, that the Agency was not clear on its priorities, whether speaking of chronic disease prevention writ large, or relating to funding priorities under the new grants and contributions program, despite these priorities being communicated on the Agency's website and in the Centre's Strategic Plan.

Beyond informing or consulting with non-governmental organizations and academia, the Centre for Chronic Disease Prevention also collaborates with stakeholders on specific projects or activities. The following examples have been noted by some key informants:

• The Interventions and Best Practices Division collaborated frequently with stakeholders to add new content to the Canadian Best Practices Portal. Over the years, it has also convened knowledge translation stakeholders in annual conferences on knowledge mobilization on chronic diseases.
• There is ongoing collaboration between the Surveillance and Epidemiology Division and the Canadian Cancer Society for analyzing and publishing the annual cancer statistics reports. Stakeholders are very satisfied with this collaboration. Staff members have formed a strong relationship with the Canadian Cancer Society that allows them to advance data analysis and co-author articles in the journal Chronic Diseases and Injuries in Canada.
• The Surveillance and Epidemiology Division also collaborates with other non-governmental organizations and academia on establishing new/emerging diseases to track within a surveillance system, analyzing and disseminating results. It also collaborates regularly with non-governmental organizations and academia to both provide technical inputs on surveillance indicators, measures and analytic methods as well as in disseminating and/or when reviewing messaging within knowledge products.

The Public Health Agency of Canada collaborates with international partners by leading the WHO Collaborating Centre on Non Communicable Disease Policy and through multiple fora.

WHO Collaborating Centre

Since 2005, the Centre for Chronic Disease Prevention has been contributing to the international knowledge base efforts on chronic disease prevention by leading the WHO Collaborating Centre on Non Communicable Disease (WHO Collaborating Centre). Under the Framework for Cooperation, the WHO Collaborating Centre initially had a policy and research mandate where it led the organization of events, developed technical resources/reports, participated in regional networks, and was a point of contact to share expertise on chronic disease policy.

One of its key achievements has been contributing to the development and implementation of a standardized questionnaire, which has been used by countries to serve as a Non Communicable Disease Country Profile. The WHO Collaborating Centre has also contributed to the international knowledge base by developing and sharing information products on chronic disease prevention and health promotion in addition to supporting various international chronic disease summits/fora. Further, additional assistance has been provided by the WHO Collaborating Centre to Pan American Health Organization partners. Some of this work includes assisting Pan American Health Organization countries in preparing for the 2011 United Nations Summit on chronic disease. This entailed a chronology of international events that surrounded chronic disease prevention as well as a guide that provided an introduction to the functions and processes of a United Nations Summit, all of which were made available in four languages.

More recently, the mandate of the WHO Collaborating Centre has shifted to meet current Public Health Agency of Canada priorities. In addition to assisting the international community, one of the main activity areas to be led by the Centre under its Re-designation Form for 2013-2016 is to conduct research on multi-sectoral action. To do so, the WHO Collaborating Centre has produced case studies, such as the ActNow BC initiative and the Quebec whole-of-government approach to public health. Both of these case studies have been, or are in the process of being, shared with international partners and made available in four languages. Internal key informants felt that this new focus was appropriate and should translate into useful activities and products.

International Fora

The Public Health Agency of Canada has represented Canada at a number of international fora over the past five years. The Agency played an important role in providing input during the negotiation phase of the United Nations High-level Meeting on NCDs. In follow-up to the United Nations General Assembly's Political Declaration on NCDs, the Agency contributed its technical expertise to the WHO's efforts to develop the Global Monitoring Framework and Global Action Plan for the Prevention and Control of NCDs 2013-2020. Some internal and external key informants commented on the Agency's leadership (as Canada's representative) and openness to collaboration during these important international discussions including one non-governmental organization that communicated its appreciation formally.

More recently, the Agency participated in the 2014 United Nations High-level Meeting to undertake a review and assessment of the progress achieved in the prevention and control of NCDs. As Canada's representative, the Agency's Centre for Chronic Disease Prevention was invited to address an international group of stakeholders to describe Canada's progress in multi-sectoral action; demonstrating Canada's commitment to the development and strengthening of relationships with non-governmental stakeholers to advance chronic disease prevention. The Centre also participated in the review process against the 2011 Political Declaration where progress was discussed and recommendations were agreed-upon for the way forward.

Immediate Outcome #3: Multi-sectoral Partnerships

Multi-sectoral partnerships have been developed and have resulted in considerable leveraging from private sector partners.

The Public Health Agency of Canada adopted a new approach in funding its grants and contributions to encourage public-private partnerships through matched funding and to fulfill commitments made by the federal government to prevent chronic disease by focusing on common risk factors. In order to qualify for Agency funding, organizations are required to establish public-private partnerships in the non-governmental or private sectors to secure funding against which the Agency will contribute an equivalent amount. Further, recipients are receiving payment once outputs have been produced (also called 'pay-for-performance'). Internal and external informants noted their appreciation of these innovative aspects of the grants and contributions program.

Since 2012-2013, 11 projects have received funding under this new approach. Nine of these projects fell within the scope of the evaluation^{Footnote vi} . As shown in Table 3 below, at the time of the evaluation, on average, projects were approved to receive $1.2 million from the Public Health Agency of Canada, which leveraged an average of $1 million in matched funding from partners.

The Agency recognizes that perspectives from across multiple sectors can be used to enhance solutions to complex health issues. This aligns with federal/provincial/territorial and international agreements that stipulate that there should be multi-sectoral action to address chronic diseases. The Multi-sectoral Partnerships Approach requires all projects to expand beyond traditional partnerships in the healthcare sector and develop new relationships with partners from other sectors, including the private and charitable sector. Reviewed projects had an average of 2.3 partnerships and received funding from both private (e.g., Shopper's Drug Mart, Reebok) and non-governmental partners (e.g., Dieticians of Canada, Dairy Farmers of Canada).

While the Multi-sectoral Partnerships Approach has received positive feedback from stakeholders, with many indicating that it could/should be the way forward for other grants and contribution programs, a number of challenges were raised. Some key informants and stakeholder survey respondents noted that this new approach has created difficulties for smaller, non-governmental organizations as it is difficult for them to secure funding from private sector partners. To some extent, this has been mitigated by the Centre for Chronic Disease Prevention staff's emerging role as a partnership broker, helping match stakeholders with ideas for a project with potential funders that are available in the private sector or non-governmental organizations.

Many external key informants and some internal informants were under the impression that partnerships were being developed in the absence of guidance from the Agency regarding the scope and type of private sector partner that is most appropriate. The Centre for Chronic Disease Prevention has recently developed a Partnership Guide to assist staff in assessing potential partnerships as part of the Letter of Intent assessment process of the Multi-sectoral Partnerships Approach. The guide is based on a number of government values and ethics policies and includes strategies for effectiveness, such as assessing potential partners and addressing developing challenges as they arise. The guide notes that in some circumstances the Centre for Chronic Disease Prevention staff may also take on a convenor or facilitator role to broker relationships between diverse partners. In addition to that, and more importantly, the guide includes a partnership assessment framework, which is used by Centre for Chronic Disease Prevention staff to assess and brief up on challenges, risks and opportunities related to potential partnerships. This guide has not yet been shared externally but it is expected to address some of the concerns raised by key informants.

Immediate Outcome #4: Access to Knowledge Products and Platforms

Many knowledge products and key platforms have been made accessible over the past five years. There are, however, challenges, including timeliness of surveillance products and low awareness of products and platforms. While target audiences have accessed the Canadian Best Practices Portal, access has declined over the past few years and issues have been raised on the completeness and reliability of its content.

Data Cubes

The Centre for Chronic Disease Prevention develops and maintains a data-sharing platform, the Data Cubes, available on the Chronic Disease Infobase. The Data Cubes contain various types of chronic disease health information such as mortality, morbidity, risk factors and associated demographic variables (age, gender, geography). It allows users to use pre-defined reports or create tables, graphs and maps that can be printed, downloaded and analyzed. The Data Cubes' main target population consists of public health researchers.

As shown in Table 4 below, web analytics from 2011-2012 to 2013-2014 demonstrate sustained traffic to the Data Cubes. The number of sessions, users and pageviews stayed relatively constant while the average session duration remained relatively constant from 1 minute 26 seconds to 1 minute 45 seconds.

Internal key informants indicated that promotional activities such as conference booths, presentations and listserv, generate a temporary increase in site traffic; however, site visits slowly return to normal traffic patterns following those promotional efforts.

Surveillance Publications

As shown in Table 5 below, since 2009, the Public Health Agency of Canada produced, in collaboration with other stakeholders, 15 surveillance reports on diabetes, cardiovascular disease, arthritis, obesity and cancer. It should be noted that since 2011, knowledge translation activities have shifted towards suites of smaller products using multiple approaches (e.g., social media, peer-reviewed publications, fact sheets, infographics) to reach varying audiences.

The reports that have been produced were made accessible through multiple channels, including direct emails, the web, listservs, newsletters and e-bulletins. The reports are intended to provide information to a broad audience, including health care professionals, researchers, policy makers and health planners, the general public, as well as community advocates to assist in policy development, priority-setting, and programming.

The Centre for Chronic Disease Prevention Stakeholder Satisfaction Survey for the Diabetes in Canada Report found that the report was accessed by survey respondents and there was a high percentage of satisfaction among those who accessed it. More specifically, the report was read by 71% of survey respondents and those who read it reported a high level of satisfaction with the content and the quality of the analysis (89%). Respondents felt that the report was timely (83%); however, some respondents suggested that the report and data should be updated regularly. There has also been access to this knowledge product: between December 2011 and October 2014, the report's web page was viewed approximately 75,000 times^{Footnote vii} .

Other surveillance reports have contributed to an increased access to knowledge products, as shown by the web analytics covering the period of time between the date of their publication and October 26, 2014:

• The Report from The Canadian Chronic Disease Surveillance System: Hypertension in Canada 2010 received approximately 19,000 pageviews.
• The Life with Arthritis in Canada: A personal and public health challenge report received over 24,500 pageviews.

In addition to publishing surveillance reports, the Centre produced knowledge products in a variety of other formats. Staff authored 27 fact sheets/bulletins and prepared 40 articles for publication in peer-reviewed journals. According to the 2014 Surveillance Stakeholder Assessment of Priority Knowledge Products Survey, executive summaries, journal articles and fact sheets were identified, by respondents, as preferred formats to receive surveillance information. Survey participants reported using journal articles "often" (60%), followed by executive summaries (60%) and fact sheets (52%). Furthermore, the Centre for Chronic Disease Prevention Knowledge Uptake Survey on Fast Facts from the Canadian Community Health Survey: Osteoporosis Rapid Response indicated that the majority (94%) of respondents read and understood the fact sheet.

The web analytics provided below, covering the period between the date of publication and October 26, 2014, varying degree of access of the following other surveillance knowledge products.

• Published earliest, Arthritis: Fast Facts from the 2009 Survey on Living with Chronic Disease in Canada received over 1,100 pageviews.
• Fast Facts about Diabetes 2011 received over 5,900 pageviews.
• There are over 11,500 pageviews of the Tracking Heart Disease and Stroke in Canada - Stroke Highlights 2011.
• More recently, the Fast Facts report on Chronic Obstructive Pulmonary Disease (COPD) 2011 received over 14,000 pageviews.

The web analytics show that surveillance reports attracted more pageviews than fact sheets, which are shorter in length. It is unclear whether this is linked to promotional efforts or interest of stakeholders.

In general, the evaluation's stakeholder web survey identified that less than half (49%) of survey respondents, who identified themselves as being part of a surveillance working group, agreed that the Public Health Agency of Canada's information is timely. Internal and external key informants also mentioned issues about the timeliness of reporting surveillance information. Although there are memoranda of understanding in place with each province and territory, internal staff recognized that there are challenges with receiving timely data submission by some provinces/territories due to technical capacity issues. This hinders the ability of the Centre for Chronic Disease Prevention to analyze data, develop knowledge products, and share them in a timely way. Key informants also reported delays in posting reports due to the Agency's internal approval process, and a gap in clarity and understanding of the rules around publications overseen by the Communications and Public Affairs Branch.

Although timeliness and awareness challenges remain, the way forward with the Digital Strategy and the Centre's surveillance transformation looks promising. The Strategy recognizes these challenges, and promotes collaboration around report dissemination with stakeholders, such as the Public Health Agency of Canada's collaboration with the Canadian Cancer Society on the annual cancer statistics publications. External stakeholders are not subject to the same posting requirements established by the Government of Canada and, furthermore, they may have stronger reach to target audiences. The Strategy also looks at adopting different formats, enabling innovative platforms to engage stakeholders and citizens in chronic disease prevention by using social media, releasing shorter reports and data (i.e., Open Data).

Canadian Best Practices Portal

The Canadian Best Practices Portal, first launched in 2006, and expanded in 2012, is a resource for health professionals and public health decision makers that provides multiple sources of information, including best practice resources and solutions, to plan programs for promoting health and preventing diseases for populations and communities. The portal consists of six different tabs, including: chronic disease, interventions (formerly best practices), resources, public health topics, policy issues, and relevant events, publications, and advisories. The bulk of the information is contained in the interventions tab, where there are over 250 interventions described.

Following an internal User Satisfaction Survey in 2009, work has been ongoing to address the feedback that was received, including improvements to accessibility and ensuring the Portal's relevance. This included:

• An infrastructure and content review;
• Identification of new interventions;
• A review of the protocol for establishing interventions as best practices;
• Developing a protocol for selecting best practices interventions; and
• Setting priorities to enhance best practice content.

More recently in 2013, work began to ensure a quality review is completed for all new interventions added to the Canadian Best Practices Portal, and targeted plans are being developed to support marketing and promotion of the portal in the Agency's Regional Offices.

The Canadian Best Practices Portal is accessible on mobile devices. Interventions found on the Portal have been disseminated through other mechanisms including fireside chats, hosted by the Population Health Improvement Research Network at the University of Ottawa, social media, journal articles, conference workshops and posters, and other promotional material such as brochures. While the Portal and its content are promoted in a variety of ways, awareness of the portal was low among external key informants, such as local public health and provincial and territorial representatives.

Other evidence was gathered to demonstrate that there is access to the Canadian Best Practices Portal. Overall, between April 3, 2009 and April 3, 2014, there were 234,200 visitors to the Portal, 155,442 of which originated in Canada. Web analytics demonstrate a period of growth in pageviews between 2009-2010 and 2010-2011, followed by a slight decline in subsequent years. Other analytics show a similar trend. For example, session duration remained relatively stable between 2009-2010 and 2010-2011, followed by a decline in the average length of a session and the number of pages viewed per session was also relatively stable between 2009-2010 and 2010-2011, but has since declined. In total, 57% of all visits were under 10 seconds long and close to three-quarters (72%) of all visitors were new visitors, which indicate that once accessed, users leave the Portal and do not usually return. This may speak to concerns related to the promotion of the portal to target audiences, raised by internal and external key informants.

Further, of the key informants who were aware of the portal, many of them raised concerns with the quality and completeness of the best practices. Results of the Centre for Chronic Disease Prevention Stakeholder Satisfaction Survey support this finding: respondents indicated a need for more information on the quality of evidence and recommended that claims of effectiveness be clearly supported by facts. To verify these statements, as part of the evaluation, a review of the best practices posted on the portal was conducted (see Appendix 4 for a description). The review found that all of the 34 best practices sampled were missing information. Almost half of the interventions had at least three or four sections of information missing and 41% (n=14) had broken links to websites. These issues demonstrate that the Centre was not able to address all of the issues that were identified in its 2009 User Satisfaction Survey.

A Centre for Chronic Disease Prevention Stakeholder Satisfaction Survey conducted in June 2014 indicated that people come to the Canadian Best Practices Portal to explore content, find interventions and support research. While survey respondents pointed out that the portal provides information not available elsewhere, they also reported challenges with the portal including accessibility, content availability and quality, and marketing/awareness. Regardless, some of the respondents also reported a high degree of satisfaction with the quality and timeliness of the information and acknowledged that the information was relevant for their work.

For the way forward, the Centre for Chronic Disease Prevention anticipates to use the Portal as its web presence in response to the Government of Canada Web Renewal^{Footnote viii} . In addition to adding further content, such as promising and innovative interventions (i.e., information on Agency-funded projects that have had success in achieving outcomes but are not meeting the criteria to be deemed a best practice), there may be opportunities to ensure that the current content is of good quality.

The Journal Chronic Diseases and Injuries in Canada

The journal Chronic Diseases and Injuries in Canada, first published in 1980, is the other key platform used to produce and make information about chronic diseases (and injuries) available. It is a quarterly journal focusing on current evidence relevant to the control and prevention of chronic diseases and injuries in Canada. It publishes peer-reviewed feature articles from authors representing the public sector or academic institutions in a variety of fields including epidemiology, public and community health, biostatistics, behavioural sciences, health services and economics. The journal is housed in the Centre for Chronic Disease Prevention and is served by editors, peer-reviewers, an Editorial Board, and authors internal and external to the government. Since 2009, 26 issues, including supplementary ones, have been published, with a total of 199 articles. These articles included a mix of research findings, book reviews, fact sheets, conference information and report summaries, among other things.

Evidence indicates that the journal is accessible. In December 2013, there were over 1,200 digital subscribers, including epidemiologists, public health practitioners, and policy-makers, as well as academic institutions and science libraries. Up until November 2014, approximately 300 subscribers also received print copies of the journal.

Evidence shows that subscriptions to Chronic Diseases and Injuries in Canada increased between 2009 and 2011 and have since remained relatively stable. A 2012 Centre for Chronic Disease Prevention Stakeholder Satisfaction Survey (n=192) demonstrated a high level of satisfaction with Chronic Diseases and Injuries in Canada. Of the respondents who recalled reading the journal (n=165), 92% agreed or strongly agreed that it increased their knowledge, 82% were satisfied with the quality of analysis, and 78% of respondents indicated that it met their needs for evidence-based information. While most survey respondents seemed satisfied, some reported that increasing awareness of the journal and using social media to promote content, would be beneficial in promoting the product.

Web analytics demonstrated increased traffic to the journal's website over a four-year period. For November 2010 to November 2011, there were approximately 55,000 sessions and 36,000 different users. These numbers increased each successive year and culminated in approximately 80,000 sessions and 54,000 users between October 2013 and October 2014.

The journal Chronic Diseases and Injuries in Canada is working toward several changes in order to increase accessibility and uptake of knowledge. In particular, the publication schedule will change from quarterly to monthly, and it will be used by the Centre as the primary communication vehicle to feature its peer-reviewed science, research and analysis. The journal will also continue to feature work by external authors who show research collaboration with the Agency, through co-authorship with health portfolio staff, Agency or Health Canada funding, or use of Agency or Health Canada data. These changes may greatly assist in addressing issues of timeliness, especially relating to posting of some surveillance information, given that publishing in a scientific journal is not under the same set of requirements as publishing on the Agency website. Although not within the scope of this evaluation, the recent posting of the Chronic Disease and Injury Indicator Framework is a good example of using the journal to promote findings and upcoming work. The Framework was the subject of a special issue of the journal.

In general, although a lot of knowledge products have been produced, there appear to be challenges in increasing access to products and platforms, and promoting them to target audiences. The evaluation stakeholder web survey respondents and internal and external key informants indicated that the Public Health Agency of Canada could improve its visibility. Some informants mentioned that one of the ways that the Agency could increase its visibility would be to make better use of social media. One of the constraints faced by the Centre for Chronic Disease Prevention is that it currently needs to use a generic Public Health Agency of Canada twitter account, which also communicates many other messages, including travel health notices and infectious diseases updates. This limits the visibility of healthy living and chronic disease prevention messages and it is unclear to what extent the Government of Canada Web Renewal will affect government communication through Twitter. The international analysis carried out for the evaluation found that the United States' National Centre for Chronic Disease Prevention and Health Promotion has its own Twitter account with over 7,900 followers and an average of five tweets per day relating to chronic disease.

4.4.2 To what extent have the intermediate outcomes been achieved?

Intermediate Outcome #1: Common Risk Factors and Behaviour Change

Common risk factor approaches have been adopted in almost all projects. In addition, funded projects are contributing to inform Canadians of the benefits of healthy living. However, there is an opportunity to clarify the alignment of the Multi-Sectoral Partnerships Approach with a population health approach. Early results demonstrate behaviour change in project participants.

Common Risk Factors

The Centre for Chronic Disease Prevention, under the Multi-sectoral Partnerships Approach, requires projects to include a focus on at least one of the following:

• addressing healthy living and healthy weights through a primary prevention initiative;
• addressing common risk factors (i.e., unhealthy diet, physical inactivity, tobacco use).

Eight out of the nine projects reviewed have adopted a common risk factor approach. Most projects addressed both diet and physical activity, one project (Air Miles Program) focused solely on physical activity, and one project had a more generalized approach and supported healthy living by developing and disseminating information regarding a healthy lifestyle and chronic disease risk factors directly to Canadians. The C-CHANGE project, addressing promotion and uptake of an updated screening guideline for heart disease, does not appear to have adopted a common risk factor approach. However, it should be noted that this project was initially started under the previous model as a Cardiovascular Disease Program project, and was one of the first projects implemented under the new Multi-sectoral Partnerships Approach.

Not only are current projects shifting prevention activities towards common risk factors, they are also contributing to informing Canadians, in general, of the benefits of healthy living. The Play Exchange, launched by the Minister of Health in February 2014 during the CBC's coverage of the 2014 Olympic Winter Games, is expected to assist in this dialogue. The Play Exchange is essentially a competition where all Canadians were invited to submit innovative ideas/projects to promote healthy living and prevent chronic disease. This year-long process will culminate with the announcement of a winner in January 2015 on CBC and, from there, the winning project will be implemented as part of the Public Health Agency of Canada's Multi-sectoral Partnerships Approach.

Reduce health inequalities among population groups

As reported under section 4.1, vulnerable groups are disproportionately impacted by chronic disease, and are more likely to adopt behavioural risk factors. Similarly to the Agency's other grants and contributions programs, the Public Health Agency of Canada's previous ISHLCD grants and contributions community-based programs employed a population health approach. The aim of the 'population health approach' is twofold: to improve the health of the entire population, and to reduce health inequalities among population groups. In the case of the Multi-Sectoral Partnerships Approach, the Centre for Chronic Disease Prevention indicates focussing its programming on a 'population as a whole' approach. It is not clear whether the stated 'population as a whole approach' is identical to a population health approach, nor is it clear whether this approach is systematically used for all funded projects.

Currently, four of the nine Multi-Sectoral Partnerships Approach projects examined (44%) targeted at least one higher risk (or vulnerable) segment of the population:

• The Special Olympics project targets disabled children;
• Right to Play targets Aboriginal children;
• The Lawson Prescription project is targeted toward a rural community; and
• The Get BUSY project is targeted toward lower-income families.

In addition to projects targeting some higher risk (or vulnerable) populations in their programming, some organizations have partnered with groups that reach those populations. For example, the Air Miles project partnered with the YMCA, a charitable organization, which offers disadvantaged individuals a fee assistance program to help cover the costs of a gym membership.

The policy refresh exercise should provide an opportunity to clarify and communicate the program design in the area of reducing health inequalities among population groups.

Behaviour Change

As noted by the evaluation's stakeholder web survey respondents and internal key informants, the previous grants and contributions community-based programs did not fund projects for a sufficient period of time for project participants to achieve behaviour change. Under the new funding program, projects are funded for a minimum of two years with a possibility of extension for a maximum of five years. This change has been well received by stakeholders.

In addition to lengthening the funding arrangements from the previous ISHLCD community-based programs, under the Terms and Conditions for Population Health, payment has moved away from "reimbursement of eligible expenditures" to "achievement of pre-determined performance expectations or milestones". The pay-for-performance arrangement ensures that organizations measure and report pre-specified outputs that are related to the expected behavioural changes. These changes are anticipated to result in improved outcome measurement and report.

At the time of the evaluation, two projects were at a stage in which they were able to report on some results:

• The Air Miles project, where pilot YMCAs issued air miles to individuals who signed up for a YMCA membership and for attending a YMCA, realized two positive outcomes: more people were exercising and those who were already exercising were doing so more often.
• The Get BUSY project, where The Boys and Girls Clubs of Canada offered a mentorship program for older youths to lead and instruct younger youths in physical and healthy eating activities and provide parental and family outreach, also realized several positive outcomes, including: children reported spending less time in front of a screen (e.g., television, computer) than before the program, children reported an increased activity level compared to before the program, and children demonstrated an increased knowledge of healthy living.

Impact of Multi-sectoral Partnerships

It is too early to determine the longer-term impact of the Multi-sectoral Partnerships Approach. Projects have the potential to make significant contributions to the health of Canadians, including the Play Exchange, which received significant coverage by being announced during the 2014 Winter Olympics. Once the first cohort of projects has completed their activities, a more in-depth analysis of outcomes could be completed to determine how these partnerships have benefited project participants, but also society at large. For example, this analysis could take into consideration the potential of other health-related multi-sectoral partnerships being developed - outside of the Public Health Agency of Canada's programming - but based on the Agency's leadership in this area.

Intermediate Outcome #2: Uptake of Knowledge Products and Platforms

Although there is a lack of performance information to fully assess uptake of information products and platforms, there is indication that some products and platforms were used by stakeholders to assist in their practice, or to adapt interventions.

Data Cubes

There is currently no information on the uptake of the Data Cubes. Upcoming performance measurement activities include developing a methodology to assess uptake.

Surveillance Publications

There is evidence that surveillance products are being utilized by a wide target audience. Some examples of uptake include are available:

• The Centre for Chronic Disease Prevention Stakeholder Satisfaction Survey for the 2011 Diabetes in Canada Report showed that two-thirds of respondents used or intended to use the information provided in the report in their organization's activities. An example of this use could be for research purposes, where a citation analysis for the same report found that the report was referenced 89 times (two self-citations) in other publications such as journal articles, blogs and web pages.
• The Centre for Chronic Disease Prevention knowledge uptake survey on Fast Facts from the CCHS: Osteoporosis Rapid Response reported that the information in the fact sheet influenced decisions, guidelines, policy, or program development by one-quarter of the respondents (25%).
• The evaluation of the Canadian Cancer Statistics Publication Partnership identified the top three uses for the publications: research (61%), teaching or staff training (38%) and public education (34%). Cancer statistics also provided information and support for situation analyses, communications, fundraising and advocacy.

In general, stakeholders confirm using evidence from surveillance products while recognizing that surveillance is usually one line of evidence that informs program decision-making. For example:

• The results of a 2014 Surveillance Stakeholder Assessment of Priority Knowledge Products Survey found that 46% of respondents representing local, provincial or territorial, national or international levels mentioned using surveillance reports 'often'.
• The evaluation's stakeholder web survey found that three-quarters (76%) of respondents, who represented a variety of sectors (e.g., government, academia, non-governmental organizations, healthcare) that have been involved in Public Health Agency of Canada surveillance activities, are using the Agency's information, and 37% agreed that the information influenced their organization's programs, policies or interventions.
• Provinces and territories involved in chronic disease surveillance activities indicated, in their annual Capacity Reports to the Public Health Agency of Canada, that they used surveillance information to develop reports; plan, develop and manage programs/initiatives; and to inform policy and legislation.

Canadian Best Practices Portal

Stakeholders appear to have used the Portal's best practices to assist in their practice or to adapt interventions. The Centre for Chronic Disease Prevention 2014 Stakeholder Satisfaction Survey indicated that of those users that had accessed the portal (n=109), 66% of respondents used or intended to use the information, 39% of respondents discussed the information from the Portal with others, and 32% of respondents indicated that the information helped inform decisions, programs, policy or practice. For example, one respondent reported that their clinic used the information to create a new Vascular Risk Reduction program and implement the Diabetes Screening Program with Chronic Condition Patients.

Chronic Disease and Injuries in Canada

There is some indication that the journal Chronic Disease and Injuries in Canada is used by stakeholders to assist in developing programs, policy or improving practices. The Centre for Chronic Disease Prevention 2012 Stakeholder Satisfaction Survey indicated that of those who recalled reading the journal (n=165), 73% of respondents had used or intended to use information from the journal for professional purposes. It was reported that information was used for presentations, funding proposals and journal articles, shared with colleagues and as a general resource/reference. Further, the journal's 2013 impact factor^{Footnote ix}  was 1.22, a significant increase from the previous year (0.68). The evaluation's international analysis found that it is slightly below the impact factor of Preventing Chronic Disease (1.956), the journal of the United States' Centers for Disease Control and Prevention.

Additionally, the journal has a number of citations, which demonstrates that information from the various issues is being used by others to further research in this area. In particular, there have been 520 citations for articles published between 2009 and 2014, including six for articles from 2014. As expected, the majority of citations are from 2009, 2010 and 2011^{Footnote x} .

Impact of Knowledge Products and Platforms

The anticipated impact on the Canadian population is that Canadians will benefit from evidence-based decision-making that is consistent throughout the country. The evaluation did not intend to assess this longer-term outcome but, the next evaluation of chronic disease prevention activities may be able to further explore this outcome should performance measurement efforts continue to be implemented.

4.5 Performance: Issue #5 - Demonstration of Economy and Efficiency

In general, the prevention of chronic diseases is more cost-effective than the treatment of these diseases. While efficiencies have been realized through steps the Centre for Chronic Disease Prevention took to improve program delivery and performance measurement, further efficiencies could be gained.

Economic Burden of Chronic Diseases and Related Conditions

Preventing chronic disease is a cost-effective activity. The economic burden of chronic diseases and related conditions is an important factor for understanding the impact of the diseases within the broader public health and healthcare systems, and the societal context.

Through the Economic Burden of Illness in Canada, the Public Health Agency of Canada has developed a methodology to assess direct and indirect costs to society^{Footnote xi} Footnote xi. As outlined in Table 6 below, cardiovascular diseases were estimated to cost society $11.8 billion in 2008, while it was approximately $4 billion for cancer, and $2 billion for diabetes. The direct and indirect costs associated with chronic diseases are high, and the costs of associated conditions, such as obesity, are also substantial. For example, it is estimated that obesity cost the Canadian economy approximately $4.6 billion in 2008, an increase of 19% from $3.9 billion in 2000.

Source: Public Health Agency of Canada. (2008). The Economic Burden of Illness in Canada.

The economic impact of chronic diseases suggests that prevention could be a cost-effective area of intervention^{Footnote 36} . One study found that a lifestyle intervention focused on addressing common risk factors was more cost-effective than a pharmaceutical intervention when it comes to preventing diabetes^{Footnote 37} . Another study, which looked into the cost-effectiveness of seven public health interventions focused on reducing chronic disease risks by increasing levels of physical activity, found that physical activity contributed to reducing the incidence of disease and improving health outcomes cost-effectively in comparison to no intervention^{Footnote 38} .

In 2012, during a budgetary reallocation exercise, the focus of the Public Health Agency of Canada's chronic disease-related activities shifted to prevention - and even more so on prevention through a common risk factor approach - and therefore eliminating activities that focus on the control of chronic diseases (e.g., producing knowledge products on disease management). This shift aligns with research findings on the cost-effectiveness of preventing chronic diseases and, as highlighted under the section on Role, has been welcomed by chronic disease stakeholders. It also supports federal/provincial/territorial and international priorities and commitments (see Section 4.2 for more detail).

Funding

There were slight variances between year-end budgets and actual spending during the period evaluated. As illustrated in Table 7 below, the most significant variance took place in 2012-2013 during the Agency's budget reallocation exercise. Some of the planned activities that did not take place that year included the implementation of some surveillance memoranda of understanding, a social marketing campaign, and reductions in publishing and meeting expenses.

*Data verified by the Office of the Chief Financial Officer

Since 2013-2014, the Centre has made considerable improvements in the monitoring and reporting of its financial expenditures, which includes a structure that links direct and indirect costs to funding authorities and alignment between planned activities and financial codes. Subsequent evaluation of the Centre's activities should therefore allow for a more fulsome analysis of planned spending and expenditures.

In terms of alignment between budget and priorities, the evaluation found that the majority of the Centre for Chronic Disease Prevention's budget is allocated to activities that are aligned with HPCDP Branch or Centre priorities. Since 2013-2014, resources are now tracked and linked to the priority area addressed - whether responding to Agency Operational Planning priorities, or HPCDP Branch priorities. For 2013-2014, 73% of funds were directed toward key deliverables in the Centre's operational plan, while 18.4% were directed toward other deliverables (e.g., Surveillance Grants and Contributions, guidelines, lessons learned from Grants and Contributions projects). The remaining (8.6%) was directed toward management offices.

In terms of economy, the Multi-sectoral Partnerships Approach's $10.8 million investment in the nine projects under review for this evaluation allowed the Public Health Agency of Canada to leverage $9.3 million from the private sector. Although this approach is relatively new, it has already expanded the reach and potentially the impact of these funded projects. For example, instead of offering their Get BUSY programs in ten locations, the Boys and Girls Clubs of Canada are able to reach 22 locations across Canada.

Efficiencies

In addition to focusing activities on the prevention of chronic diseases, other recent changes made to the Centre for Chronic Disease Prevention's activities have translated into greater efficiencies. For example:

• As mentioned under the section above, to address the OAG's recommendations on financial tracking, the Centre made considerable changes in the monitoring and reporting of its financial expenditures. The Centre for Chronic Disease Prevention's practice is to continuously monitor expenditures and make adjustments as required. Given these recent changes, it is expected that future evaluations will be able to conduct a more fulsome assessment of economy.
• The ISHLCD had a very complex structure with many levels of governance. Recent changes have allowed the Centre for Chronic Disease Prevention's Senior Management Committee to become a decision-making, priority-setting body where decisions are made at the Centre-level. In addition, as opposed to having a multitude of committees or working groups guiding the work of the Centre, there are two committees that report to the Senior Management Committee: the Digital Strategy Committee and the Talent Management Committee. This has allowed the Centre to streamline its priority-setting and decision-making bodies and processes.
• The Multi-sectoral Partnerships Approach has allowed the Public Health Agency of Canada to gain efficiencies, which are expected to translate into improved results. Recipients are now receiving payment once outputs have been completed. The Centre is currently exploring how it can move to a model where funded recipients receive payments upon achievement of their anticipated outcomes. Regardless of the final model, whether payment is linked to completion of outputs or achievement of outcomes, this ensures that projects are on track and meet established targets.
• The approach to preventing chronic diseases has shifted upstream and now focuses on common risk and protective factors. This may lead to broader impact and greater efficiency of the investment. As determined by evidence (see Continued Need section), and as further noted by key informants, the shift to a common risk factor approach is the more efficient way to tackle chronic diseases given that, for instance, healthy eating reduces an individual's chances of developing diabetes, some forms of cancer, and conditions linked to cardiovascular diseases, such as hypertension^{Footnote 39} .

Areas of Potential Inefficiency

Despite the efficiencies gained, there are areas that could be improved.

• At the time of the conduct of the evaluation, the latest version of the Centre for Chronic Disease Prevention's Data Quality Report was based on data collected from 2009-2010 from the National Diabetes Surveillance System. At that time, there were issues relating to the timeliness and quality of the data received from provinces and territories. Recommendations were presented on enhancing data quality tools to ensure that provinces and territories can better verify data prior to submitting their data sets to the Public Health Agency of Canada. The latest data quality report, which was in the process of being approved at the time of the evaluation, did not find data quality to be an issue for the Canadian Chronic Disease Surveillance System as a result of new measures implemented by the Centre. Timeliness, however, remained an issue.
• There appear to be opportunities in continuing to integrate activities between the Centre for Chronic Disease Prevention's Divisions and, as appropriate, within the HPCDP Branch. To date, although some attempts have been made to integrate activities, more integration could take place. For example, scientists working in SDSID could be further engaged in the planning and development of products and initiatives.

Performance Measurement

The Centre for Chronic Disease Prevention has made several changes in order to implement a performance measurement system to support management decision-making and accountability, as recommended in the 2013 OAG Report. The system is comprised of two components:

• Performance Highlights Posters: following each performance measurement data collection, including the Centre's satisfaction survey and knowledge uptake surveys, a Performance Highlights Poster is developed. Each poster contains background on the product/activity measured, and a description of the methodology, findings, lessons learned or insights from data analysis. In total, seven posters have been developed and shared within the Centre for Chronic Disease Prevention on products and platforms that fall within the scope of this evaluation.
• Directory of Performance Information: all of the Centre for Chronic Disease Prevention's performance data holdings are listed in this directory, as well as upcoming performance measurement data collection activities. The Directory is updated annually to align with priority activities identified during the Planning, Reporting, Monitoring and Financial Management Cycle and is then discussed and approved biannually by the Centre's Senior Management Committee.

In addition to conducting performance measurement for some of its products and platforms, the Centre for Chronic Disease Prevention monitors the performance of its grants and contributions projects. The Multi-sectoral Partnerships Approach funded projects are required to have an evaluation plan in place which shows how evidence will be collected to meet three out of four expected outcomes, all of which are based on the Partnerships' Logic Model. Furthermore, projects must measure a corresponding set of indicators at the short (demographic data on target groups), medium (increased knowledge) and long-term (behavior change - by common risk factor element) levels. Funded projects are required to develop a baseline report within twelve months of starting activities, and annual outcome measures reports to show progress on their achievement of expected outcomes.

There is indication that the Centre for Chronic Disease Prevention is starting to utilize performance measurement data to support decision-making. As an example, the design and delivery of the journal Chronic Diseases and Injuries in Canada was improved based on some of the findings from a stakeholder satisfaction survey. Given the relatively recent changes made to performance measurement in the Centre, key informants anticipate using performance measurement data more and more in the future to make programmatic decisions.

There are a few challenges that affect the Centre for Chronic Disease Prevention's performance measurement activities, including incomplete stakeholder lists. To conduct its measurement activities properly (i.e., survey the right people), the Centre's Program Performance Section relies on stakeholder lists. It appears that those lists may be incomplete or outdated, which then could impede the Centre's ability to receive useful feedback and an accurate assessment of activities/products.

Given that the Centre for Chronic Disease Prevention is currently undergoing an exercise to align its policy and program authorities, it is anticipated that performance measurement activities (including logic models) will be adapted to reflect those upcoming changes.

5.0 Conclusions

5.1 Relevance Conclusions

5.1.1 Continued Need

Chronic diseases remain a public health issue. Prevention is still needed given that the rate at which new cases of some chronic diseases are diagnosed continues to rise (i.e., cancer and diabetes) due to an aging population, while the rate of diagnosis in other diseases, such as hypertension, has stabilized. Further, the number of people living with chronic diseases continues to rise due to improvements in diagnosis and treatment, while more and more people live with multiple chronic diseases.

Common risk factors - physical inactivity, poor nutrition, tobacco use, and alcohol consumption - contribute to chronic disease in both men and women. Obesity also increases the risk of many chronic diseases. Certain factors, including living and working conditions, as well as socio- economic status, are associated with obesity, physical inactivity and indirectly with many chronic diseases.

The Public Health Agency of Canada has been able to adapt to the changing policy environment by shifting its activities in the area of prevention, with a focus on addressing common risk factors.

5.1.2 Alignment with Government Priorities

The federal government and Public Health Agency of Canada continue to identify chronic disease prevention and obesity as priorities. In the past five years, numerous federal/provincial/territorial and international declarations and priority-setting documents have been signed. Further, the prevention of chronic diseases continues to be reflected in a variety of Government of Canada and Agency strategic policy and planning documents, including the federal Speech from the Throne and the Agency's Strategic Horizons.

Current chronic disease prevention activities align with priorities. For instance, the Centre for Chronic Disease Prevention has focused its activities in the prevention of chronic disease through a common risk factor approach. There may be opportunities to clarify, through the Centre for Chronic Disease Prevention policy refresh exercise, how the population health approach will assist in reducing health disparities.

5.1.3 Alignment with Federal Roles and Responsibilities

Various legislative, policy and program authorities identify that the federal government has a role and responsibilities to address chronic diseases by conducting surveillance and research, providing leadership, supporting interventions, identifying best practices, and supporting the development of screening guidelines. The current chronic disease prevention activities conducted by the Public Health Agency of Canada align with the federal public health role and responsibilities in addressing chronic disease.

The federal public health role in preventing chronic diseases does not appear to duplicate the role of other stakeholders. In fact, roles are more inclined to complement each other. Although there is no duplication, there does not appear to be clarity around the role played by the Public Health Agency of Canada and that of other federal government departments, especially Health Canada and Sports Canada. External key informants - representing provinces and territories, non-governmental organizations or at the local public health level - had difficulty in understanding the differences between those federal government departments' roles in chronic disease prevention and also healthy living.

5.2 Performance Conclusions

5.2.1 Achievement of Expected Outcomes (Effectiveness)

In the past five years, there has been limited evidence of the integration of activities within the Centre for Chronic Disease Prevention. However, a few recent initiatives, such as the Surveillance and Epidemiology Division's Emerging Issues Forum, are appropriate mechanisms to encourage staff from the Centre, and even the HPCDP Branch, to learn from each other, and come together to address common issues. In the area of collaboration, the evaluation found that the Centre collaborates frequently with stakeholders, and these activities are well appreciated by stakeholders; however, improvements could be made, especially in terms of clearly communicating priorities.

The Public Health Agency of Canada's new grants and contributions programs on chronic disease prevention and healthy living, the Multi-sectoral Partnerships Approach, is demonstrating that common risk factor approaches are being adopted by funded projects, private sector partners are committed to the projects, and there is early indication that behaviour changes are occurring as a result of funded initiatives. However, there is an opportunity to clarify how the multi-sectoral partnerships' approach aligns with a population health approach.

Many knowledge products have been developed in the past five years, and key platforms continue to be managed. It is anticipated that the Centre's newly implemented Digital Strategy will be able to further increase and facilitate access. The evaluation identified that stakeholders use surveillance knowledge products frequently. Many internal and external informants felt that, in general, products and platforms were not promoted to intended audiences, which may limit their uptake. One platform in particular, the Canadian Best Practices Portal, was either unknown to external key informants or was not trusted as a good source of information. A review of a random sample of best practices posted on the Portal found that all of the 34 best practices had incomplete content and web analytics further showed that once users access the platform, they leave it quickly and do not return.

5.2.2 Demonstration of Economy and Efficiency

In terms of efficiencies, recent changes have allowed the Public Health Agency of Canada to gain efficiencies, including the Multi-sectoral Partnerships Approach's focus on pay-for-performance where recipients are now receiving payment once they demonstrate that outputs have been produced. This ensures that projects are on track and meet established targets. However, the evaluation found that additional efficiencies could be gained in integrating activities further with other areas of the HPCDP Branch, as appropriate.

The Centre for Chronic Disease Prevention experienced challenges in delivering planned activities as a result of budget reallocation decisions. Also, although funds were spent within the Centre, there were some difficulties in systematically coding expenditures based on the complex ISHLCD coding system. The newly developed financial coding system links costs to Treasury Board authorities and is aligned with the Centre's planned activities. These changes have greatly enhanced the ability of the Centre to identify and monitor planned and actual spending according to various HPCDP Branch and Centre priorities.

Continued economy will be achieved with the Multi-sectoral Partnerships Approach - the $10.8 million in grants and contributions allocated to the nine projects under review for this evaluation has allowed the Agency to leverage $9.3 million from the private sector. Although this approach is relatively new, it has already expanded the reach and potentially the impact of funded projects.

Finally, as a result of addressing the Auditor General's recommendations on Promoting Diabetes Prevention and Control, the Centre for Chronic Disease Prevention has made many improvements to its management, planning and reporting processes. Performance measurement activities are now clearly aligned with the Centre's priorities and include a suite of well-researched tools and are an integral part of the planning process.

6.0 Recommendations

The Centre for Chronic Disease Prevention has done considerable work by shifting its activities in the area of prevention and aligning with international and federal, provincial and territorial commitments. In terms of performance, over the past five years, stakeholders have appreciated the surveillance knowledge products produced by the Centre, as well as the collaborative efforts and leadership the Centre has undertaken on a number of fronts, especially in the area of obesity/healthy weights. The Centre has also made many improvements to its management, planning and reporting processes.

Although there have been many achievements, three recommendations are put forward to assist the Public Health Agency of Canada in continuously improving its chronic disease prevention activities.

Recommendation 1:

Review the Canadian Best Practices Portal with a focus on assessing the content of the current best practices

Issues have been raised on the content of the best practices available on the Canadian Best Practices Portal through a variety of ways (i.e., key informant interviews, web analytics, and performance measurement survey). Given that the identification and sharing of best practices have been identified by most internal and external key informants as a key and well-appreciated federal public health role, and that there is currently no other similar platform that exists in Canada, the evaluation recommends that the Centre for Chronic Disease Prevention conducts an external review of the content of current best practices to re-instill trust in a role and a platform that has great potential and is sought-after. Changes should then be made to address the external review recommendations.

Recommendation 2:

Address communication issues

In particular:

• In collaboration with other government departments (e.g., Health Canada, Heritage Canada), communicate federal public health roles and responsibilities in chronic disease prevention to external stakeholders;
• Develop and communicate a dissemination plan for surveillance products;
• Enhance the promotion of the chronic disease prevention strategy, products and platforms to target audiences; and
• Clarify and communicate how the Multi-sectoral Partnerships Approach aligns with a population health approach.

Communication is at the centre of a few issues raised in the evaluation. For one, stakeholders have difficulties understanding the Public Health Agency of Canada's role in comparison to the Health Canada or Heritage Canada's Sport Canada roles when it comes to disease prevention and health promotion. Further to that, stakeholders are unaware of the Agency's priorities and products/platforms, which then has an impact on the level of uptake of those products/platforms. Although surveillance activities and products are clear, or well-known, to stakeholders, there is general lack of awareness of their plans in terms of upcoming product dissemination. The Centre for Chronic Disease Prevention's Digital Strategy is a step in the right direction to addressing some communication issues, but it is still in its early days. The evaluation also found that the Centre should to clarify the alignment of the Multi-Sectoral Partnerships Approach with a population health approach.

Recommendation 3:

Continue to integrate chronic disease prevention activities within the Centre for Chronic Disease Prevention and formalize mechanisms to ensure integration

To offer an efficient suite of programs based on evidence, the Centre for Chronic Disease Prevention needs to integrate its activities better. Overall, the evaluation found very limited examples of successful, well-integrated activities within the Centre but also with other areas of the Branch that could assist in informing program activities or delivering them, based on the Centre for Chronic Disease Prevention's priorities.

The Centre recognizes that integration could be improved and has launched, in the Spring of 2014, an Emerging Issues Forum, led by its Surveillance and Epidemiology Division, where surveillance staff present new trends they see in data, and engage in discussion with other areas of the Centre and also the HPCDP Branch.

Appendix 1 - References

• Aglukkaq, L.(2009). Public Health Agency of Canada, 2009-2010: Report on Plans and Priorities. Retrieved from http://www.tbs-sct.gc.ca/rpp/2009-2010/inst/ahs/ahs-eng.pdf
• Aqlukkaq, L. (2010). Public Health Agency of Canada, 2010-2011: Report on Plans and Priorities. Retrieved from http://www.tbs-sct.gc.ca/rpp/2010-2011/inst/ahs/ahs-eng.pdf
• Aglukkaq, L. (2011). Public Health Agency of Canada, 2011-2012: Report on Plans and Priorities. Retrieved from http://www.tbs-sct.gc.ca/rpp/2011-2012/inst/ahs/ahs-eng.pdf
• Aglukkaq, L. (2012). Public Health Agency of Canada, 2012-2013: Report on Plans and Priorities. Retrieved from http://www.tbs-sct.gc.ca/rpp/2012-2013/inst/ahs/ahs-eng.pdf
• Aglukkaq, L. (2013). Public Health Agency of Canada, 2013-2014: Report on Plans and Priorities. Retrieved from http://www.phac-aspc.gc.ca/rpp/2013-2014/assets/pdf/rpp-2013-2014-eng.pdf
• Auditor General of Canada. (2013). Chapter 5: Promoting Diabetes Prevention and Control. In Report of the Auditor General of Canada. Retrieved from http://www.oag-bvg.gc.ca/internet/docs/parl_oag_201304_05_e.pdf
• Canada. (2008). Speech from the Throne: Protecting Canada's Future. Retrieved from http://www.parl.gc.ca/Parlinfo/documents/thronespeech/40-1-e.html
• Canada. (2013). Speech from the Throne: Seizing Canada's Moment Prosperity and Opportunity in an Uncertain World. Retrieved from http://speech.gc.ca/sites/sft/files/sft-en_2013_c.pdf
• Canada, Department of Finance. (2012). Jobs Growth and Long-term Prosperity: Economic Action Plan 2012. Retrieved from http://www.budget.canada.ca/2012/plan/pdf/Plan2012-eng.pdf
• Canada, Department of Finance. (2013). Jobs, Growth and Long-term Prosperity: Economic Action Plan 2013. Retrieved from http://www.budget.canada.ca/2013/doc/plan/budget2013-eng.pdf
• Canada Health Act, Revised Statutes of Canada (1985, c. C-6). Retrieved from http://laws-lois.justice.gc.ca/PDF/C-6.pdf
• Canadian Strategy for Cancer Control Governing Council. (2006). The Canadian Strategy for Cancer Control: A Cancer Plan for Canada. Retrieved from http://www.partnershipagainstcancer.ca/wp-content/uploads/CSCC_CancerPlan_20061.pdf
• Cancer Care Nova Scotia, Capital Health Cancer Care Program at the QEII Health Sciences Centre. (2011). Cancer Care: A Guide for Patients, Families and Caregivers. Retrieved from http://www.cancercare.ns.ca/site-cc/media/cancercare/CCNS_CH_Guide_05_11(1).pdf
• Department of Health Act, Statutes of Canada (1996, c. 8). Retrieved from http://laws-lois.justice.gc.ca/PDF/H-3.2.pdf
• Elmslie, K. (2012). Against the Growing Burden of Disease. Retrieved from http://www.ccgh-csih.ca/assets/Elmslie.pdf
• Financial Administration Act, Revised Statutes of Canada (1985, c. F-11). Retrieved from http://laws-lois.justice.gc.ca/PDF/F-11.pdf
• Government of Ontario, Healthy Kids Panel. (2013). No Time to Wait: The Healthy Kids Strategy. Retrieved from http://www.health.gov.on.ca/en/common/ministry/publications/reports/healthy_kids/healthy_ kids.pdf
• Government of Saskatchewan. (2004). The Diabetes Provincial Plan. Retrieved from http://www.health.gov.sk.ca/provincial-diabetes-plan
• Haman, F., Fontaine-Bisson, B., Batal, M., Imbeault, P., Blais, J.M., Robidoux, M.A. (2010). Obesity and type 2 diabetes in Northern Canada's remote First Nations communities: the dietary dilemma. International Journal of Obesity, 34 (2), S24-31.
• Herman, W.H., Hoerger, T.J., Brandle, M., Hicks, K., Sorensen, S., Zhang, P., ...Ratner, R.E. (2005). The cost-effectiveness of lifestyle modification or Metformin in preventing Type 2 diabetes in adults with impaired glucose tolerance. Annals of Internal Medicine 142(5), 323-332.
• Kirby, M. (2002). The Health of Canadians - The Federal Role. Retrieved from http://www.parl.gc.ca/Content/SEN/Committee/372/soci/rep/repoct02vol6-e.pdf
• Lalonde, M. (1974). A New Perspective on the Health of Canadians: a Working Document (Catalogue No. H31-1374). Retrieved from http://www.phac-aspc.gc.ca/ph-sp/pdf/perspect-eng.pdf
• Mikkonen, J., & Raphael, D. (2010). Social Determinants of Health: The Canadian Facts. Retrieved from http://www.thecanadianfacts.org/the_canadian_facts.pdf
• Naylor, D. (2003). Learning from SARS: Renewal of Public Health in Canada (Catalogue No. H21-220/2003E). Retrieved from http://www.phac-aspc.gc.ca/publicat/sars-sras/naylor/index-eng.php
• Public Health Agency of Canada. (2005). The Integrated Pan-Canadian Healthy Living Strategy. Retrieved from http://www.phac-aspc.gc.ca/hp-ps/hl-mvs/ipchls-spimmvs/pdf/ipchls-spimmvs-eng.pdf
• Public Health Agency of Canada. (2008). Economic Burden of Illness in Canada, 2005-2008. Retrieved from http://www.phac-aspc.gc.ca/publicat/ebic-femc/2005-2008/assets/pdf/ebic-femc-2005-2008-eng.pdf
• Public Health Agency of Canada. (2009). Investing in Prevention - The Economic Perspective : Key Findings from a Survey of the Recent Evidence. Retrieved from http://www.phac-aspc.gc.ca/ph-sp/preveco-index-eng.php
• Public Health Agency of Canada. (2009). Report on the State of Public Health in Canada 2009. Retrieved from http://www.phac-aspc.gc.ca/cphorsphc-respcacsp/2009/fr-rc/index-eng.php
• Public Health Agency of Canada. (2009). Tracking heart disease and stroke in Canada 2009. Retrieved from http://www.phac-aspc.gc.ca/publicat/2009/cvd-avc/index-eng.php
• Public Health Agency of Canada. (2010). Annual report on the state of public health in Canada 2010. Retrieved from http://www.phac-aspc.gc.ca/cphorsphc-respcacsp/2010/fr-rc/index-eng.php
• Public Health Agency of Canada. (2010). Creating a Healthier Canada: Making Prevention a Priority - A Declaration on Prevention and Promotion from Canada's Ministers of Health and Health Promotion from Canada's Ministers of Health and Health Promotion/Healthy Living. Retrieved from http://www.phac-aspc.gc.ca/hp-ps/hl-mvs/declaration/pdf/dpp-eng.pdf
• Public Health Agency of Canada. (2010). Curbing Childhood Obesity: A Federal, Provincial and Territorial Framework for Action to Promote Healthy Weights. Retrieved from http://www.phac-aspc.gc.ca/hp-ps/hl-mvs/framework-cadre/index-eng.php Public Health Agency of Canada. (2010). Overview of the Pan-Canadian Healthy Living Strategy. Retrieved from http://www.phac-aspc.gc.ca/hp-ps/hl-mvs/ipchls-spimmvs-eng.php
• Public Health Agency of Canada. (2011). Diabetes in Canada: Facts and figures from a public health perspective. Retrieved from http://www.phac-aspc.gc.ca/cd-mc/publications/diabetes-diabete/facts-figures-faits-chiffres-2011/index-eng.php
• Public Health Agency of Canada. (2013). Multi-sectoral Partnerships to Promote Healthy Living and Prevent Chronic Disease. Retrieved from http://www.phac-aspc.gc.ca/fo-fc/mspphl-pppmvs-eng.php#a3
• Public Health Agency of Canada. (n.d.). Canadian Best Practices Portal. Retrieved from http://cbpp-pcpe.phac-aspc.gc.ca/
• Public Health Agency of Canada. (n.d.) Chronic Diseases and Injuries in Canada. Retrieved from http://www.phac-aspc.gc.ca/publicat/cdic-mcbc/index-eng.php
• Public Health Agency of Canada. (n.d.). Chronic Disease Indicator Framework. Retrieved from http://infobase.phac-aspc.gc.ca/cdif/
• Public Health Agency of Canada. (n.d.). Chronic Disease Indicator Framework -CANPLAY 2009-2011. Retreived from http://infobase.phac-aspc.gc.ca/cdif/
• Public Health Agency of Canada. (n.d.). Chronic Disease Indicator Framework - CCHS 2009-2010. Retrieved from http://infobase.phac-aspc.gc.ca/cdif/
• Public Health Agency of Canada. (n.d.). Chronic Disease Infobase. Retrieved from http://infobase.phac-aspc.gc.ca/
• Public Health Agency of Canada Act, Statutes of Canada (2006, c. 5). Retrieved from http://lois-laws.justice.gc.ca/PDF/P-29.5.pdf
• Public Health Agency of Canada, Canadian Institute for Health Information. (2011). Obesity in Canada. Retrieved from http://www.phac-aspc.gc.ca/hp-ps/hl-mvs/oic-oac/index-eng.php
• Public Health Agency of Canada, Statistics Canada, Canadian Cancer Society. (2012). Canadian Cancer Statistics, 2012. Retrieved from https://www.cancer.ca/~/media/cancer.ca/CW/cancer%20information/cancer%20101/Canadian%20cancer%20statistics/Canadian-Cancer-Statistics-2012---English.pdf
• Raphael, D., Daiski, I., Pilkington, B., Bryant, T., Dinca-Panaitescu, M., & Dinca-Panaitescu, S. (2012). A toxic combination of poor social policies and programmes, unfair economic arrangements and bad politics: The experiences of poor Canadians with Type 2 Diabetes. Critical Public Health, 22(2), 127-145.
• Roux, L., Pratt, M., Tengs, T.O., Yore, M.M., Yanagawa, T.L., Van Den Bos, J.,...Buchner, D.M. (2008). Cost Effectiveness of Community-based Physical Activity Interventions. American Journal of Preventative Medicine, 35(6), 578-588.
• Statistics Canada.(2014). Cancer Incidence in Canada, 2010. Retrieved from http://www.statcan.gc.ca/daily-quotidien/140710/dq140710b-eng.htm
• Statistics Canada. (2014). The 10 leading causes of death, 2011. Retrieved from http://www.statcan.gc.ca/pub/82-625-x/2014001/article/11896-eng.htm
• Treasury Board of Canada. (2009). Policy on Evaluation. Retrieved from http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=15024&section=HTML
• United Nations. (2011). Political Declaration of the High-level Meeting of the General Assembly on the Prevention and Control of Non-communicable Diseases. Retrieved from http://www.who.int/nmh/events/un_ncd_summit2011/political_declaration_en.pdf
• World Health Organization. (1986). The Ottawa Charter for Health Promotion. Retrieved from http://www.who.int/healthpromotion/conferences/previous/ottawa/en/
• World Health Organization. (2010). Global status report on noncommunicable disease 2010. Retrieved from http://www.who.int/nmh/publications/ncd_report_full_en.pdf
• World Health Organization. (2011). Rio Political Declaration on Social Determinants of Health. Retrieved from http://www.who.int/sdhconference/declaration/en/
• World Health Organization. (2013). Draft comprehensive global monitoring framework and targets for the prevention and control of noncommunicable diseases. Retrieved from http://apps.who.int/gb/ebwha/pdf_files/WHA66/A66_8-en.pdf?ua=1
• World Health Organization. (2013). Global Action Plan for the Prevention and Control of NCDs 2013-2020. Retrieved from http://apps.who.int/iris/bitstream/10665/94384/1/9789241506236_eng.pdf
• World Health Organization. (2014). Facts about ageing. Retrieved from http://www.who.int/ageing/about/facts/en

Appendix 2 - Logic Model

Enlarge Image - Appendix 2 - Logic Model

Appendix 3 - Summary of Findings

Rating of Findings

Ratings have been provided to indicate the degree to which each evaluation question and issue have been addressed.

Relevance Rating Symbols and Significance:

A summary of Relevance ratings is presented in Table 1 below. A description of the Relevance Ratings Symbols and Significance can be found in the Legend.

Performance Rating Symbols and Significance:

A summary of Performance Ratings is presented in Table 2 below. A description of the Performance Ratings Symbols and Significance can be found in the Legend

Appendix 4 - Role of Stakeholders

Along with the Public Health Agency of Canada, a number of other federal government departments work to address chronic diseases in Canada, specifically:

• Health Canada promotes healthier lifestyles, integrates renewal of the health care system with longer-term plans in the areas of prevention, health promotion and protection, reduces health inequalities, and provides health information. In particular, the Office of Nutrition Policy and Promotion (ONPP) serves as the federal focal point and authoritative source for nutrition and healthy eating policy and promotion in Health Canada. ONPP supports the nutritional health and well-being of Canadians by collaboratively defining, promoting and implementing evidence-based nutrition policies.
• Although not included in the scope of this evaluation, Health Canada also works in the area of Tobacco Control, in leading the implementation of Canada's Federal Tobacco Control Strategy, which, in collaboration with many parties, seeks to reduce, if not eliminate, tobacco use. It also enforces the Tobacco Act.
• Statistics Canada collects, compiles, analyzes, abstracts and publishes statistical information on a variety of topics including population health and welfare, which incorporates chronic diseases. More specifically, Statistics Canada maintains the Canadian Cancer Registry, a database that collects information on cancer incidence from all provincial and territorial cancer registries. The registry provides information that supports health planners and decision-makers at all levels of government to identify risk factors for cancer, monitor and evaluate cancer control programs and conduct research.
  
  Statistics Canada also gathers information related to health status, health care utilization and health determinants through the Canadian Community Health Survey. The data collected is used by federal and provincial departments of health and human resources, social service agencies, and other types of government agencies to monitor, plan, implement and evaluate programs to improve the health of Canadians.
• Sport Canada is part of the Department of Canadian Heritage and seeks to help Canadians participate and excel in sport. This is achieved by enhancing the capacity and coordination of the Canadian sport system and encouraging participation in sport. Sport Canada works with and funds many partners, programs and initiatives that encourage participation in sports and physical activity, in particular, ParticipACTION, Physical and Health Education Canada, and a wide variety of sport-specific organizations.
• Canadian Institute for Health Research (CIHR) creates knowledge and translates it into improved health for Canadians, more effective health services and products, and a strengthened Canadian health care system. It forges an integrated health agenda, encourages inter-disciplinary, integrative health research, collaborates with P/Ts to advance research and engages voluntary organizations.

The CIHR has research institutes on a number of areas related to chronic disease, including, aging, cancer research, circulatory and respiratory health, musculoskeletal health and arthritis, nutrition, metabolism and diabetes, and population and public health.

• Canadian Institute for Health Information (CIHI) leads the development and maintenance of comprehensive and integrated health information that enables sound policy and effective health system management to improve health and health care. They also work with stakeholders to create and maintain a broad range of health databases, measurements and standards, and develop reports and analyses from their own and other data. CIHI is predominately focused on the health care system.
• Provinces and territories also carry out activities to address chronic diseases. Provinces and territories are primarily responsible for the administration of health care systems and for public health in their own jurisdictions. According to provincial and territorial strategic plans, key strategies and websites, provinces and territories engage in a variety of activities that impact chronic disease prevention in their jurisdictions, including:
  • Developing and guiding provincial policy,
  • Providing leadership for health care systems and related programs,
  • Setting direction and priorities for provincial public health policy to ensure timely, effective services,
  • Allocating funding and resources for community agencies and service deliverers, in support of provincial priorities
  • Coordinating with key partners and stakeholders,
  • Conducting surveillance in their own jurisdictions,
  • Producing knowledge products, often using the surveillance data they have gathered, and
  • Developing screening guidelines.

In addition to prevention activities, provinces and territories focus on disease management, including conducting screening and providing treatment for chronic diseases.

Appendix 5 - Evaluation Description

Evaluation Scope

The scope of the evaluation covered the period from April 2009 to August 2014, and included activities funded through:

• Automated External Defibrillators
• Canadian Breast Cancer Initiative
• Canadian Strategy for Cancer Control
• Integrated Strategy on Healthy Living and Chronic Disease (ISHLCD)
• Lung Health Program

The evaluation focused on the Public Health Agency of Canada (PHAC)'s chronic disease prevention activities carried out by the Centre for Chronic Disease Prevention (CCDP). It excluded the chronic disease science activities conducted by the Social Determinant and Science Integration Directorate, the ISHLCD investments in mental health, the Joint Consortium for School Health, investments in neurological diseases, the Federal Tobacco Initiative and food security activities.

Evaluation Issues

The specific evaluation questions used in this evaluation were based on the five core issues prescribed in the Treasury Board of Canada's Policy on Evaluation (2009). These are noted in the table below. Corresponding to each of the core issues, evaluation questions were tailored to the program and guided the evaluation process.

Data Collection and Analysis Methods

Evaluators collected and analyzed data from multiple sources. Sources of information used in this evaluation included:

• Document review - approximately 150 documents pertinent to chronic disease prevention were reviewed for information regarding the relevance (priorities, roles and responsibilities) of the activities.
• File review - approximately 300 files held by the Centre for Chronic Disease Prevention were reviewed to obtain information regarding all aspects of the activities related to chronic disease and in particular the performance (achievement of outcomes, economy and efficiency) of the activities.
• Financial data review - a review of financial data from 2009-2010 to 2013-2014, including budgeted and actual expenditures.
• Key informant interviews - interviews were conducted with 72 stakeholders (Public Health Agency of Canada (n=43); other federal government departments or agencies (n=7); Public Health Network Steering Committee members (n=6); external partners (n=16). Interviews were recorded and transcribed as necessary. Data was analysed with NVIVO.
• International analysis - a review of chronic disease prevention activities conducted by the United States, Australia and the United Kingdom was carried out by scanning the countries' public health websites.
• Literature review - a search for Canadian and international literature from the past five years using search terms of "chronic disease", "chronic illness", "noncommunicable disease" and "noncommunicable illness". After examining documents to ensure relevance, 35 articles were reviewed.
• Performance data review - a review of data on performance of program activities between 2009-2010 and 2013-2014 (stakeholder satisfaction survey, knowledge uptake survey, web analytics, project-level performance and evaluation reports).
• Review of the Canadian Best Practices Portal - 34 interventions were assessed for completeness of data. Four categories were selected and all interventions in those categories were reviewed. Each intervention was scanned to determine if information for each section was included, if that information was complete and whether or not links were active and relevant.
• Stakeholder mapping - a listing of primary organizations and secondary organizations was developed by evaluators and a consultant. Profile and other related information was gathered for each organization to understand key areas of action. An analysis was then conducted to map or summarise these activities in comparison to the roles and responsibilities of the Public Health Agency of Canada with a view to assessing areas of complementarity, potential gaps and potential duplication of investment.
• Stakeholder web survey - an online survey was in the field between May 26, 2014 and June 9, 2014; it was sent to 180 contacts provided by the Surveillance and Analysis Division and the Partnerships and Strategies Division. There were 72 fully completed surveys and 13 partially completed surveys for a response rate of 47%.

Data were analyzed by triangulating information gathered from the different sources and methods listed above. This included: systematic compilation, review and summarization of data to illustrate key findings; statistical analysis of quantitative data from databases; thematic analysis of qualitative data; and comparative analysis of data from disparate sources to validate summary findings.

Appendix 6 - Stakeholder Mapping

List of Organizations Included in Stakeholder Mapping

Primary stakeholder organizations were identified by the Public Health Agency of Canada's Centre for Chronic Disease Prevention. Secondary organizations were identified through an internet search. Secondary organizations were selected based on: the broad disease areas included within the scope of the Agency's chronic disease and the evaluation; activities focused on prevention and/or control; national coverage; and current status of activities.

Primary organizations

• Arthritis Society
• Canadian Cancer Society
• Canadian Diabetes Association
• Canadian Partnership Against Cancer
• Canadian Public Health Association
• Canadian Obesity Network
• COPD Canada
• Chronic Disease Prevention Alliance of Canada
• Heart and Stroke Foundation
• International Union Against Cancer
• Lung Association
• Osteoporosis Canada
• ParticipAction

Secondary organizations

• Active Healthy Kids Canada
• Allergy/Asthma Information Association
• Asthma Society of Canada
• Canadian Active and Safe Routes to School Partnership
• Canadian Breast Cancer Foundation
• Canadian Council for Health and Active Living at Work
• Canadian Public Health Association
• Canadian Sleep Society
• Canadian Stroke Network
• Childhood Obesity Foundation
• Canadian Obesity Network
• Hypertension Canada
• International Union against Cancer
• National Collaborating Centre for Aboriginal Health
• Physical and Health Education Canada
• Pulmonary Hypertension Association Canada

Appendix 7 - Description of Performance Measurement Data

The evaluation relied heavily on a number of surveys to assess the performance of the Centre for Chronic Disease Prevention's (CCDP) activities. The following appendix provides a description of the purpose and methods used for these surveys.

Evaluation Stakeholder Web Survey
The survey was conducted by the Office of Evaluation, as part of the evaluation of the Public Health Agency's Chronic Disease Prevention Activities. The objective of the web survey was to collect information that can be used to assess the relevance and performance of the Agency's activities, along with other lines of evidence (e.g., key informant interviews, document and file review).

A total of 180 survey invitations were sent to surveillance and grants and contributions stakeholders via email addresses provided by CCDP's Partnerships and Strategies Division and Surveillance and Epidemiology Division. The survey was conducted between May 26, 2014 and June 9, 2014. In total, there were 72 fully competed surveys and 13 partially completed surveys, for a response rate of 47%. Respondents represented a variety of sectors, including academic/research/education sector, non-governmental organizations, provincial/territorial governments and healthcare organizations.

Surveys conducted by the Centre for Chronic Disease Prevention's Program Performance Section

Knowledge Uptake Survey on Fast Facts from the 2009 CCHS: Osteoporosis Rapid Response
The intent of the survey was to assess stakeholders' uptake and use of the Fast Facts from the 2009 Canadian Community Health Survey - Osteoporosis Rapid Response.

In February 2013, CCDP administered the web-based survey to 39 known stakeholders. The survey was open for two weeks. The Centre received 19 responses, for a response rate of 49%. The majority of survey respondents work in the academic/research/education sector. Other respondents represented the healthcare sector, provincial/territorial government, federal government and the not-for-profit sector.

Stakeholder Satisfaction Survey for Diabetes in Canada 2011 Report
CCDP administered this survey to determine stakeholders' satisfaction with content, timeliness of information, and intended use of Diabetes in Canada 2011 Report.

The web-based survey was sent to 595 known users of the report and was open between May 14 and May 23, 2012. The response rate was 22% (129 responses). The profile of respondents represented academic/research sector, government and not-for-profit organization.

Stakeholder Satisfaction Survey for the Chronic Diseases and Injuries in Canada Journal
The survey was conducted by CCDP in May 2012 in order to measure stakeholder satisfaction with content, timeliness of information, and intended use of the journal. The survey web-based survey was emailed to 1,026 subscribers to the journal (excluding Public Health Agency of Canada staff) and was open for seven weeks.

There were 192 respondents (response rate of 19%). Of those that responded, 55% worked in the government sector (excluding Public Health Agency of Canada staff), 37% in the academic/research sector and 10% in the not-for-profit sector. Respondents were healthcare professionals (44%), epidemiologists (32%), health/education administrators (17%), policy-makers (6%) and students (1%).

Stakeholder Satisfaction Survey for the Canadian Best Practices Portal
The survey was conducted by CCDP in June 2014 in order to measure stakeholder satisfaction with the Canadian Best Practices Portal content and to assess uptake and use of information.

The survey was administered to 633 key stakeholders who were on the email distribution list of the Portal. It was open for three weeks and covered ten specific topic areas including cancer, cardiovascular diseases, chronic respiratory diseases, diabetes, healthy weights, physical activity, nutrition, mental health and social determinants of health. There were 165 respondents to the survey, which represents a response rate of 26%. Of those that responded, 33% (n=26) worked in the healthcare sector, 23% in the non-governmental sector, and 15% worked in the academic/research sector. Respondents were healthcare providers (29%), program deliverers (22%), knowledge brokers (15%), and academics/researchers (13%).

Other Centre for Chronic Disease Prevention Surveys

Surveillance Stakeholder Assessment of Priority Knowledge Products
An external consultant conducted this assessment on behalf of CCDP to determine surveillance partners' needs for surveillance and intervention evidence (type, content and format), to inform the development of a Centre-approach to developing knowledge products and activities.

CCDP surveillance staff developed a stakeholder list. The online survey was in the field from February 28 to March 12, 2014. The response rate was 24% (143 responses, with 135 of these complete responses out of a possible 597). The survey responders represented a range of institutions, including government (federal, provincial/territorial and local), healthcare sector (hospitals, community health centers), not-for-profit organizations, academic/research institutions, and industry.

Evidence Needs of Public Health Practitioners and Professionals across Canada
The purpose of the survey was to identify evidence needs and knowledge exchange preferences of public health practitioners and professionals across Canada. Survey findings were intended to inform the planning and development of knowledge products and activities.

CCDP administered the survey to a convenience sample made of Canadian Public Health Association members and newsletter (listserv) subscribers. The survey link was sent via email to 5,328 prospective participants. Survey links were open over a 35-day period between April 30, 2014 and June 5, 2014. A total of 428 individuals responded to the questionnaire, for a response rate of 8%.

External Evaluation

Evaluation of the Canadian Cancer Statistics Publication and Partnership
The study, conducted by an external consultant on behalf of the Canadian Cancer Society, included quantitative and qualitative methods. An online stakeholder survey was conducted from November 9 to November 30, 2011. A total of 483 participants completed the online survey. Participants were derived from the mailing list comprised of current subscribers to the publication and stakeholders or collaborators of the Canadian Cancer Society on other projects (n=1,043). Respondents included: media, policy makers (government and cancer agencies), researchers, clinicians, Canadian Cancer Statistics staff, patient organizations, and some members of the general public.

The evaluation also used other methods, such as an online focus group (November 18-22, 2011) with ten participants, and interviews (December 2011) with 29 key informants.

Footnotes

List of Acronyms

CTFPHC

Canadian Task Force on Preventive Health Care

CPAB

Communications and Public Affairs Branch

FAA

Federal Accountability Act

HPCDP

Health Promotion and Chronic Disease Prevention

ISHLCD

Integrated Strategy on Healthy Living and Chronic Diseases

NCD

Noncommunicable Disease

OAG

Office of Auditor General

ONPP

Office of Nutrition Policy and Promotion

SDSID

Social Determinants and Science Integration Directorate

WHO

World Health Organization