ARCHIVED - Promotion of Population Health Grant and Contribution Programs: Summary of Program Evaluations, 2004-2009

1. Context

The mission of the Public Health Agency of Canada (PHAC) is “to promote and protect the health of Canadians through leadership, partnership, innovation andaction in public health.” PHAC’s strategic outcome is “healthier Canadians, reduced health disparities, and a stronger public health capacity.”

The promotion of population health is key to PHAC’s achieving its mission, and strategic outcome. It is based on a comprehensive range of strategies requiring partnerships and collaboration among the different levels of government as well as between government and non-government organizations and communitygroups.

Grants and contributions^{Footnote 1} are key instruments in allowing the government to engage a diversity of skills and resources to further the well-being of Canadians. Grants and contributions are awarded under a set of terms and conditions that are subject to approval by the Treasury Board of Canada (TB) and must be renewed periodically. All major PHAC health promotion programs with a grant and contribution component are incorporated under an umbrella Promotion of Population Health Grants and Contributions (PPHGC) Results-based Management and Accountability Framework (RMAF). The current RMAF was developed in 2003 and approved by TB in 2004, and covers the 16 programs listed in Table 1. The umbrella RMAF is supplemented by individual-program RMAFs that translate the PPHGC-level objectives, outcomes and evaluation questions into individual-program-specific activities, deliverables and evaluation questions.

Table 1. Programs under the Umbrella Promotion of Population Health Terms and Conditions.

1. Aboriginal Head Start in Urban and Northern Communities (AHSUNC)
2. Canada Prenatal Nutrition Program (CPNP)
3. Canadian Breast Cancer Initiative Community-Capacity Building Component (CBCI-CCB)
4. Canadian Diabetes Strategy (CDS)
5. Canadian Health Network (CHN)
6. Centres of Excellence for Children’s Well-Being (CofE)
7. Community Action Program for Children (CAPC)
8. Fetal Alcohol Spectrum Disorder (FASD, formerly Fetal Alcohol Syndrome/Fetal Alcohol Effects)
9. Health Canada/Veterans Affairs Canada - Falls Prevention Initiative (HC/VAC)
10. Federal Initiative to Address HIV/AIDS in Canada (FI) a. Canadian Strategy on HIV/AIDS (CHSA)
11. Hepatitis C Prevention, Support and Research Program (HEP C)
12. Integrated Strategy on Healthy Living and Chronic Disease (ISHLCD)
13. National Collaborating Centres for Public Health (NCCPH)
14. Population Health Fund (PHF)
15. Voluntary Sector Initiative (VSI) -- Sectoral Involvement in Departmental Policy Development

While evaluations have been conducted for the sixteen programs currently under the umbrella RMAF, the RMAF also committed to a review the PPHGC programs as a whole, prior to the next renewal of the promotion of population health grant and contribution terms and conditions by March 31, 2010.

2. Objective

The objective of this Report is to fulfil the commitment in the 2003 umbrella PPHGC RMAF to an evaluation summary to inform the next renewal of the PPHGC Terms and Conditions. The report reviews and summarizes the findings on the overall relevance, impact, and design and delivery of the Promotion of 3 Population Health Grant and Contribution Program. The report is based on the evaluations conducted on the component PPHGC programs from 2004 to 2009.

This report is targeted at those taking part in decisions related to the renewal of the PPHGC terms and conditions and to PHAC program and evaluation managers, as part of the Agency’s continuous learning and improvement journey.

3. Scope and Methodology

This report reviews and summarizes the evidence in the PPHGC component program evaluations conducted from 2004 to 2009 period. However, it is not in itself, an evaluation.

The methodology used was the systematic review and analysis of the individual program evaluation reports and RMAFs provided by PHAC to The Governance Network (TGN). The goal was to extract and categorize key findings, and answer the evaluation questions posed in the PPHGC RMAF. Reviews of other documents and evidence, the independent validation of the evidence in the evaluation reports, and the investigation and generation of (new or existing) information beyond that already in the evaluation reports to more fully answer the evaluation questions, were not included in the engagement. Also, the program evaluations addressed other issues that responded to the needs of their intended audience and objectives. This information will be included in this report only to the extent that they contribute to answering the PPHGC evaluation questions.

4. Promotion of Population Health – an Overview

The promotion of population health approach within PHAC integrates prevention, health promotion, diagnosis, treatment and care.

“Population health” is defined as the health of a population as measured by health status indicators as influenced by the determinants of health. The indicators generally relate to mortality, morbidity, loss of function, and quality of life.

The Ottawa Charter for Health Promotion defines health promotion as “… the process of enabling people to increase control over, and to improve, their health.”^{Footnote 2} The promotion of population health approach focuses on the interrelated conditions and factors that influence the health of populations over the life course, identifies systematic variations in their patterns of occurrence, and applies the resulting knowledge to develop and implement policies and actions to improve the health and well-being of those populations.

A population health approach is focused on the health of an entire population (or sub-population) rather than the health of individuals. Sub-populations may be defined according to geography, age, culture, health risks, and socio-economic characteristics. The empowerment of individuals to make decisions on their own health is also an important element.

In March 1994, the Federal, Provincial and Territorial Ministers of Health endorsed the report Strategies for Population Health: Investing in the Health of Canadians, which lays out a framework to guide the development of policies and strategies to improve population health. This report recognized population health as a key concept for government policy and program development. As a result, PHAC funds initiatives that promote a population health approach. In 2006-07, PHAC’s total actual expenditures amounted to $510.8 million, of which grants and contributions (for programs covered by the three sets of terms and conditions – promotion of population health, public health and blood safety)^{Footnote 3} accounted for $182.2 million^{Footnote 4} or about 36%. Amounts for 2005-06 were $477.2 million in total expenditures, of which $176.6 million (37%) was for grants and contributions. In 2004-05, the amounts were, respectively, $486.7 million^{Footnote 5}, and $221.8 million or 45.6%.

5. Framework: The PPHGC RMAF

Grants and contributions enable PHAC to provide financial support and assistance and to work in partnership with others – including municipal and other levels of government, community-level organizations, institutions and the private sector – to establish a wide variety of population health initiatives to realize the Agency’s strategic outcome of “healthier Canadians, reduced health disparities, and a stronger public health capacity”. The current terms and conditions were renewed in 2004 and adopted by PHAC health promotion programs over the last five years. They provide the operating authority under which grants and contributions made by these programs are administered. The associated management and accountability commitments are reflected in the umbrella PPHGC RMAF, which was part of the 2004 PPHGC Terms and Conditions renewal submission to the Treasury Board of Canada.

The umbrella PPHGC RMAF translated the key characteristics that define a population health approach into a common and standardized set of key activity areas, objectives and results, performance indicators, and evaluation questions that guide performance measurement and evaluation at the individual program level. It is the foundation for the PPHGC component-program RMAFs and also represents a commitment among these programs to the ongoing collection of common performance data to:

• ensure that overall program management and information sharing objectives are met;
• inform reporting on program outcomes;
• contribute to continuous program improvement; and
• guide evaluation-related activities.

RMAF’s have been developed for all programs, except the Falls Prevention Initiative, which was funded by Veterans Affairs Canada (VAC) and jointly delivered by PHAC and VAC through a Memorandum of Understanding.

6. Evaluation Questions

The evaluation strategy in the umbrella PPHGC RMAF identified three key evaluation questions:^{Footnote 6}

• Relevance: To what extent is there a continued need for federal involvement in promotion of population health programs?
• Impact and effect: To what extent have the programs contributed to the achievement of outcomes identified in the PPHGC umbrella logic model?
• Design and delivery: How have programs approached the design and delivery of programs? What has worked well and what has not?

7. Challenges

Reponses to the evaluation questions at the PPHGC level are based on evaluations conducted on the component programs.

As of the date for the final preparation of this report, information was available for fourteen programs. Of those, four were formative evaluations or implementation reviews. Annex 2 summarizes the availability, titles and authorship of the evaluation reports reviewed. The limited data available may have restricted the analysis.

As all evaluations were conducted by the program areas there was no consistent approach across all evaluations, making conclusions on the overall findings more difficult. The evaluations used a variety of quantitative and qualitative techniques such as questionnaire surveys; key stakeholder, informant or participant interviews; survey questionnaires or focus groups; participant database analysis; case studies; and document and literature reviews. Because of the differences among the programs and lack of qualitative or quantitative standards and criteria to assess performance, synthesizing the findings of the PPHGC individual program evaluations was a challenge and concluding on the extent to which the programs individually and together were meeting desired outcomes was not possible.

8. Relevance

The evaluation question on “relevance” in the PPHGC evaluation strategy is:

To what extent is there a continued need for federal involvement in this area?

The associated indicators are:

• the value of the population health activity to the network, organization, and provincial and territorial governments and the relevance of the initiative for different constituencies;
• government and political priorities and stakeholder perceptions of identified need;
• evidence of the nature and magnitude of the problem; and
• evidence in population health literature of the potential impact of the programming.

The evaluations supplied for this review were not expected to directly address the relevance question at the PPHGC level which, to be consistent with the component-program level question, could be restated as: to what extent is there a continued need for federal involvement in the promotion of population health? All the evaluations that addressed the relevance question generally presented the findings so as to explicitly or implicitly advocate for continuing federal involvement in the public health area under consideration.

• Eleven evaluations either directly or indirectly (such as in discussions on “impact and effect”) addressed the question of relevance.
• Four provided evidence based only on stakeholder interviews or surveys that advanced such rationales for continuing federal involvement as the need for national coordination or national strategies, ameliorate regional disparities or as the catalyst for further action by other stakeholders.
• One used the number of programs and services being funded over the last decade and the government’s activities in related areas as the evidence.
• Six others combined opinion-based information with research evidence and epidemiological and surveillance data on the significance of the health issue addressed, such as the extent of the problem in Canada (e.g., the number of 7 individuals affected or at risk, mortality or incidence rates), and information on risk factors, risk groups and the economic burden of illness.
• Two of the formative or implementation evaluations did not address the question of relevance, nor did the AHSUNC summative evaluation, which focused solely on the implementation of the program components and the program’s impact on the participating children, families and communities.

A quick Internet review of activities in the promotion of population health or health promotion would indicate that actions and commitments by national governments operating under different governance models all over the world on the promotion of population health can be taken as explicit acknowledgement of the need for involvement at that level. In Canada, this has been underscored by the creation of the Public Health Agency of Canada in 2004, which not only demonstrates the relevance of the promotion of population health for federal involvement but also that the government wants a higher profile for and greater emphasis on it.^{Footnote 7 Footnote 8}