Evaluation of Health Canada’s First Nations and Inuit Health Branch’s Environmental Public Health Program 2010-11 to 2014-15

Download the alternative format
(PDF format, 456 KB, 59 pages)

Organization: Health Canada

Date published: 2017-02-24

Prepared by

Office of Audit and Evaluation

Health Canada and the Public Health Agency of Canada

September 15, 2016

List of Acronyms

CA: Contribution Agreement
CBWM: Community Based Drinking Water Quality Monitor
CFIA: Canadian Food Inspection Agency
CMHC: Canadian Mortgage and Housing Corporation
DPR: Departmental Performance Report
DWA: Drinking Water Advisory
EC: Environment Canada
EH-ER: Environmental Health and Environmental Research
EHIS: Environmental Health Information System
EHO: Environmental Health Officer
EPH: Environmental Public Health
EPHD: Environmental Public Health Division
EPHP: Environmental Public Health Program
FNECP: First Nations Environmental Contaminants Program
FNFNES: First Nations Food, Nutrition and Environment Study
FNEHIN: First Nations Environmental Health Innovation Network
FNHA: First Nations Health Authority
FNIHB: First Nations and Inuit Health Branch
FNWWAP: First Nations Water and Wastewater Action Plan
Gs&Cs: Grants and Contributions
GCDWQ: Guidelines for Canadian Drinking Water Quality
HQ: Headquarters
INAC: Indigenous and Northern Affairs Canada
O&M: Operations and Maintenance
PAA: Program Alignment Architecture
PHAC: Public Health Agency of Canada
REHM: Regional Environmental Health Managers

Executive Summary
Management Response and Action Plan
1.0 Evaluation Purpose
2.0 Program Description
3.0 Evaluation Description
- 3.1 Evaluation Scope, Approach and Design
- 3.2 Limitations and Mitigation Strategies
4.0 Findings
5.0 Conclusions
- 5.1 Relevance Conclusions
- 5.2 Performance Conclusions
6.0 Recommendations
Appendix 1 – Core Areas of Program Activity Focus
Appendix 2 – Logic Model
Appendix 3 – Summary of Findings
Appendix 4 – Evaluation Matrix

List of Tables

Table 1: Program Expenditures ($)
Table 2: Limitations/Challenges and Mitigation Strategies
Table 3: Most Frequently Cited EPH Risk Areas by Environmental Public Health Officers
Table 4: Most Frequently Cited as "High Priority" Risk Areas
Table 5. Activity areas where EPH risk is mitigated or eliminated in the short term
Table 6: EPHP Expenditures by Region and HQ

Executive Summary

This evaluation provides an update on the relevance and performance of Health Canada's Environmental Public Health Program (EPHP), which was first evaluated in 2011 under Health Canada's Environmental Health and Environmental Research (EH-ER) Cluster Evaluation.

Evaluation Purpose and Scope

The purpose of this evaluation is to assess changes in the performance of Health Canada's EPHP during the past five fiscal years (2010-11 to 2014-15), and to fulfill the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009). The scope of this evaluation included most activities performed by the EPHP, as well as progress made on activities covered under the First Nations Water and Wastewater Action Plan. Not included within the scope of this evaluation were the activities that are now covered under the British Columbia Tripartite Framework Agreement on First Nation Health Governance, which came into full effect in 2014-15, and activities included under recently evaluated horizontal initiatives, namely the Clean Air Agenda and the Action Plan to Protect Human Health from Environmental Contaminants. Furthermore, the evaluation did not assess FNIHB activities related to Inuit and Inuit communities given that those activities were assessed as part of existing external/horizontal evaluation (INAC's Northern Contaminants Program and Environment and Climate Change Canada's Clean Air Agenda).

Program Description

The primary objective of the EPHP is to "identify, address, and/or prevent human health risks to First Nations and Inuit communities associated with exposure to hazards within the natural and built environments." Environmental public health surveillance and risk analysis programming includes community-based and participatory research on trends and impacts of environmental factors, such as chemical contaminants and climate change on the determinants of health (e.g., biophysical, social, cultural, and spiritual).^{Footnote 1} This programming is directed to First Nations communities south of the 60th parallel, and in the case of the Northern Contaminants Program and the Climate Change and Health Adaptation Program, also to Inuit and First Nations north of the 60th parallel.^{Footnote 2}

Within Health Canada, the EPHP is coordinated regionally by Environmental Public Health Services (EPHS) and supported nationally by the Environmental Public Health Division (EPHD). EPH programming is delivered in First Nations communities south of 60º by Environmental Health Officers (EHOs) employed by Health Canada or First Nation communities and/or Tribal Councils in accordance with the National Framework for the Environmental Public Health Program in First Nations Communities South of 60º. Key programming includes environmental public health assessments; training; and, public awareness activities that focus on eight core areas: Drinking Water; Wastewater; Solid Waste Disposal; Food Safety; Housing; Facilities inspections; Environmental Communicable Disease Control; and, Emergency Preparedness and Response.^{Endnote i}

Between April 1, 2010 and March 31, 2015, the EPHP expended approximately $162 million through set contribution funding agreements and direct departmental spending to identify, assess and address human health risks associated with exposure to hazards within the natural and built environments. One key program difference as of 2014 is that activities are now performed in the province of British Columbia by the First Nations Health Authority (FNHA) as a result of the British Columbia Tripartite Framework Agreement on First Nation Health Governance. Another difference as of 2011 is that the former cluster structure of two groups of programming (Environmental Health and Environmental Research) have now been integrated into one program, namely the Environmental Public Health Program at FNIHB.

Conclusions - Relevance

Continued Need

EPH programming is globally recognized as an essential component of public health systems, and is a key contributor to improving the health and well-being of populations in a holistic manner. The EPHP supports a continued and growing need among many First Nations communities to identify and address human health risks associated with exposure to hazards within the natural and built environments. As identified in the previous evaluation, many First Nations communities continue to experience significant EPH risks compared with other Canadian communities. Ongoing challenges result from issues such as: sub-standard housing and living conditions; poor drinking water quality and poorly operated wastewater systems; a lack of certified water plant operators; climate change; local geography characteristics; and, numerous socio-economic inequalities. While the resolution of EPH issues often falls outside the scope of Health Canada's control, there continues to be a strong, demonstrated need for a program like EPHP to influence health promotion and disease prevention outcomes in First Nations communities. As First Nations communities continue to grow and experience more complex environmental public health risks, the need for EPH assessments, training, public awareness, research and surveillance is expected to increase.

Alignment with Government Priorities

The EPHP continues to align with federal priorities and strategic outcomes sought to improve the health status of First Nations and Inuit communities and individuals.

Alignment with Federal Roles and Responsibilities

Since the previous evaluation, Bill S-8 was passed and the Safe Drinking Water for First Nations Act came into force in 2013. The introduction of this Act moves the federal role in EPH risk mitigation beyond advisory to include a possible regulatory role. During this evaluation time period, safe drinking water regulations for First Nations were under development; however, as of 2016, the Act became subject to review by INAC Ministerial direction, and regulatory development was put on hold. Health Canada continues, therefore, to play an advisory role which varies in each region in order to meet specific community needs and priorities.

As different federal departments have a role to play in supporting First Nations communities in addressing EPH risks, it is necessary that the contribution of all federal departments is coordinated and aligned to maximize effectiveness and achieve common strategic outcomes. Currently, the Health Canada role in drinking water remains advisory, and the activities delivered under the EPHP align with this advisory role. Health Canada also does not currently have the mandate to provide funding to communities to address or remediate risks, as this is the purview of INAC. While there is no duplication, there are opportunities for greater coordination and integration at the federal level.

Conclusions - Performance

Achievement of Expected Outcomes (Effectiveness)

Building on the achievements identified in the previous evaluation, over the past five years the EPHP has demonstrated clear progress in achieving its immediate outcomes. For example, findings indicate that data collection is more timely and findings are disseminated at various levels either through recommendations or other types of reporting; there is an increased use of evidence to guide EPH activities with relevant research being conducted in communities to address EPH priorities; and an awareness of EPH risks among First Nations communities has increased overall, which has further contributed to an increased use of, and demand for, EPHP services in communities.

Similar to the previous evaluation findings, while progress on immediate outcomes is evident, the extent to which intermediate outcomes have been achieved is less clear. There are three general steps required to mitigate EPH risks that include activities that identify, then assess, and then address issues (if required from the assessment).

Findings from this evaluation indicate that the EPHP has continued to demonstrate capacity to work effectively with First Nations communities to identify and to assess EPH risks, and that such activities can result in recommendations for First Nations communities and other stakeholders with respect to addressing environmental risks. In addition, there has been continued progress over the past five years with First Nations communities developing their own capacity to identify and assess EPH risks. There has been less progress in communities undertaking of activities required to address EPH risks, which can be complex, and require considerable resources from other funding sources (i.e., outside of the mandate of the EPHP). While there are some areas where this third step is occurring (e.g., short-term remedial actions such as drinking water advisories and food safety precautions), overall there remain challenges for communities to address the more complex EPH risks (e.g., long-term actions involving housing and systems for drinking and wastewater).

The contingencies and factors involved in addressing EPH risks are numerous, and include the consideration of community prioritization and the availability of resources, all of which are external to the EPHP's mandate. This indicates that if the EPHP is expected to contribute to communities addressing their more complex EPH risks, then the program will need to continue coordinating and collaborating with the multitude of other stakeholders involved that assist First Nations communities in determining their priorities and obtaining resources to address EPH risks.

The evaluation was unable to directly assess the longer term outcome of improved health and well-being of First Nations. There is consensus among program representatives that the EPHP is likely contributing to improving health status by taking specific actions towards certain environmental public health risks.

Demonstration of Economy and Efficiency

The EPHP demonstrates some aspects of economy and efficiency in its delivery. Given the manner in which budgets are developed and resources allocated, it is challenging to determine the extent to which planning and budgets align with actual expenditures in each of the program's regions and activity areas. This also contributes to challenges with calculating some measures of efficiency using the financial data available for the program such as ratios of cost per output.

A key component of the EPHP successes can be attributed to the work of Environmental Health Officers (EHOs). While there were numerous areas of efficiency demonstrated for EHOs, the sustainability of their efforts may be impacted by an increasing workload in the area, and significantly greater demands being placed on the EPHP due to: an increased awareness of EPH risks in many communities; an increase in funding for community infrastructure projects; and, the ongoing growth and development of First Nations communities. The impact of workload pressures among EHOs is somewhat varied by region due to differences in capacity and priorities between First Nations communities.

Regarding matters pertaining to the program's performance measurement, the EPHP has generally improved during the past five years. Areas for improvement identified in the previous evaluation show that attempts have been made to advance reporting from strictly counting outputs to also including outcomes. In addition, there was work on the development of some of the data collection systems. Despite these improvements, there remain some challenges with respect to the comprehensiveness of data (e.g., different data collection tools and coverage parameters among regions) and the utility of data collected (e.g., focus on counting inspections but not systematically capturing the content of recommendations).

Recommendations

Recommendation 1

Explore options to enhance collaboration with stakeholders to support First Nations in addressing and mitigating complex environmental health risks (e.g., those that may require longer-term and more costly interventions). While there are many coordination and collaboration activities underway already, it is clear that for the intermediate (and thus longer term) outcomes to be achieved there needs to be continued involvement by EPHP in contributing evidence and assessments to support or promote the mitigation of complex environmental public health risks. In order to better assess the degree to which outcomes are achieved, the impacts of EPH collaboration and coordination on EPH risk mitigation should be documented, with the understanding that whether such risks are actually mitigated is often outside the EPHP's control.

There are numerous other stakeholders involved in various areas of EPH in First Nations communities. These stakeholders include: First Nations communities, various First Nations organizations, other federal government departments and agencies (e.g., INAC, EC, PHAC, CFIA, CMHC), various branches within Health Canada, and provincial governments. INAC and Health Canada require collaboration specifically in a number of areas associated with EPH given their combined contributions to achievement of EPHP objectives. For example, while INAC is the lead on the First Nations Water and Wastewater Action Plan, Health Canada has important supporting roles. Similarly, investigations by EHOs or reviews by public health engineers can result in recommendations for infrastructure improvements that are being funded through INAC programming. EPHP is only one of the players in this complex matrix; however, the EPHP is well-positioned to continue to provide evidence and information required to promote the mitigation of complex EPH risks, given their expertise and strong working relationships and connections to the community. EPHP's coordination and collaboration activities should continue at various levels including community, sub-region, region and national levels.

Recommendation 2

Review the current EHO allocation and workload within and across regions to determine sustainability of service to communities considering various forecasted additional demands and complexities. Building on the findings from the workload analyses currently underway, it will be important for the EPHP to clearly understand the challenges involved resulting from the current EHO allocation and workloads at both regional and national levels. This review would need to take into account the anticipated additional demands and complexities for many First Nation communities as a result of various changing contexts (e.g., increased infrastructure funding, climate change, economic development, population growth).

Recommendation 3

Produce an assessment of the impact of the recommendations made by EHOs to Chiefs and Councils in the area of food, water and housing. Given that the recommendations that EHOs develop are an essential output to achieving some of the desirable outcomes for the program, they are potentially a rich source of information and data for various uses at multiple levels. This would likely enhance the utility of the performance and monitoring data being collected by the program.

Management Response and Action Plan (or Management Response)

Evaluation of the First Nations and Inuit Health Branch's Environmental Public Health Program 2010-2011 to 2015-2016
Recommendations	Response	Action Plan	Deliverables	Expected Completion Date	Accountability	Resources
Recommendation as stated in the evaluation report	Identify whether program management agrees, agrees with conditions, or disagrees with the recommendation, and why	Identify what action(s) program management will take to address the recommendation	Identify key deliverables	Identify timeline for implementation of each deliverable	Identify Senior Management and Executive (DG and ADM level) accountable for the implementation of each deliverable	Describe the human and/or financial resources required to complete recommendation, including the source of resources (additional vs. existing budget)
Review the current EHO allocation and workload within and across regions to determine sustainability of service to communities, considering various forecasted additional demands and complexities.	Program management agrees with the recommendation. Efforts to improve the availability of, and access to, high quality data to promote better decision-making and planning is a priority of FNIHB. Within this context, FNIHB has developed an EHO Workload Assessment Tool to assess sustainability and determine future requirements for EHOs.	Validate EHO Workload Assessment Tool and prepare analysis of EHO workload.	Presentation to senior management on EHO Workload Analysis	September 2017	Tom Wong Executive Director, Office of Population and Public Health	No additional resources required.
Produce an assessment of the impact of the recommendations made by EHOs to Chiefs and Councils in the area of food, water and housing.	Program management agrees with the recommendation. A representative sample of recommendations in the areas of food, housing and drinking water will be reviewed to assess their impacts and determine if required follow up actions are taken. If necessary, based on the review of the implementation of recommendations, policies and procedures will be revised in order to improve implementation of mitigation measures to address EPH risks.	Complete analysis and develop an approach to assess the impacts of a representative sample of recommendations in the areas of food, housing and water, to determine if required follow-up actions are taken, over a period of 6 months. Update existing policies and procedures to improve implementation of recommendations by EHOs in order to address EPH risks.	Presentation to senior management on approach.	March 2017	Tom Wong Executive Director, Office of Population and Public Health	No additional resources required.
			Initiate data collection in selected Regions or communities	June 2017
			Presentation to senior management on results	February 2018
			Updated policies and procedures	April 2018
Explore options to enhance collaboration with stakeholders to support First Nations in addressing and mitigating complex environmental health risks (e.g., those that may require longer-term and more costly interventions).	Program management agrees with the recommendation. EPHP (National Office and Regions) has a long history of collaboration with partners to work towards the mitigation of complex environmental public health risks (e.g., those requiring funding investments and/or longer-term interventions). In the context of INAC's new infrastructure funding for water and wastewater, and their renewed mandate to address Drinking Water Advisories (DWA) on-reserve, a key EPHP priority is to collaborate with INAC to promote the prioritization of investments that address high risk systems from the health perspective. Public health advice and analysis in this area is already being shared at the working level and ADM level.	Establish and document systems and processes to share public health information and advice with INAC regarding strategic water infrastructure investments and drinking water advisories, at the National and Regional levels.	INAC-HC Joint Work plan on strategic collaborative activities related to new infrastructure investment planning and strategies to address DWAs.	March 2017	Tom Wong Executive Director, Office of Population and Public Health	No additional resources required.

1.0 Evaluation Purpose

The purpose of this evaluation was to assess the relevance and performance (effectiveness, efficiency, and economy) of Health Canada's Environmental Public Health Program (EPHP) for the period of 2010–11 to 2014–15. The evaluation was conducted in accordance with the Treasury Board Secretariat of Canada's 2009 Policy on Evaluation^{Endnote ii} and the Financial Administration Act. The evaluation is part of the Five-Year Evaluation Plan of Health Canada and the Public Health Agency of Canada.

An evaluation of the EPHP (known previously as the Environmental Health and Environmental Research Cluster) was first conducted in 2011, and it focussed on many of the same issues covering the fiscal period 2005-06 to 2009-10.^{Footnote 3} This evaluation presents an update on the progress of outcomes and changes made to the EPHP since the previous evaluation, and highlights program achievements, lessons learned, and delivery challenges.

2.0 Program Description

2.1 Program Context and Narrative

Conditions in the environment, both natural and human-built, can affect a person's ability to achieve and maintain good health. A healthy environment includes safe water and food supplies, properly designed, constructed and maintained housing and community facilities, as well as suitable treatment and disposal of wastewater and solid waste. To maintain a healthy environment, it is also necessary to plan for and respond to emergencies and work to prevent and control environmental communicable diseases.

Improving the health of Indigenous people is a shared responsibility among federal, provincial and territorial governments, as well as First Nations and Inuit communities and individuals. The EPHP must, therefore, continue to work collaboratively with a variety of organizations, such as:

First Nations communities and organizations at the local, regional and national levels;
Other areas of Health Canada and the First Nations and Inuit Health Branch;
Other federal departments, including: Indigenous and Northern Affairs Canada (INAC); Environment and Climate Change Canada (EC); Public Health Agency of Canada (PHAC); Canadian Food Inspection Agency (CFIA); and, the Canada Mortgage and Housing Corporation (CMHC); and,
Provincial and territorial governments.

The long term expected outcome for the EPHP is improved health and well-being of First Nations individuals and communities. A logic model depicting how the program intends to achieve this outcome was updated in 2015 (see Appendix 2). This evaluation assessed the degree to which the defined outputs and outcomes in the logic model were achieved over the evaluation timeframe. The updated EPHP logic model in 2015 is very similar to the one used in the 2011 evaluation of the EH-ER Cluster.

2.2 Program Profile

The primary objective of the EPHP is to "identify, address, and/or prevent human health risks to First Nations and Inuit communities associated with exposure to hazards within the natural and built environments." Within Health Canada, the EPHP is coordinated regionally by Environmental Public Health Services (EPHS) and supported nationally by the Environmental Public Health Division (EPHD). All program activities are provided in agreement with and by request of First Nations Authorities. In First Nations communities south of 60º (the main focus of this evaluation), EPH programming is delivered by Environmental Health Officers (EHOs) employed by Health Canada or First Nation communities and/or Tribal Councils in accordance with the National Framework for the Environmental Public Health Program in First Nations Communities South of 60º. All EHOs must have a Certificate in Public Health Inspection (Canada), which is recognized by health organizations in Canada as evidence of satisfactory training and competency. They may also participate in Continuing Professional Competencies in order to maintain their skills, knowledge and expertise.

Key programming under public health assessments continues to include activities that focus on eight core areas (Appendix 1): Drinking Water; Wastewater; Solid Waste Disposal; Food Safety; Housing; Facilities Inspections; Environmental Communicable Disease Control; and, Emergency Preparedness and Response. The role of the EPHP is to assist communities by providing training and education around EPH risks according to community priorities, developing recommendations for addressing EPH risk based on investigations, and reviewing infrastructure plans from a public health engineering perspective. Key activities include public health inspections and assessments, public education and training, and providing advice and guidance.

Environmental public health surveillance and risk analysis programming includes community-based and participatory research on trends and impacts of environmental factors, such as chemical contaminants and climate change on the determinants of health (e.g., biophysical, social, cultural, and spiritual).^{Footnote 4} This programming is directed to First Nations communities south of the 60th parallel, and in the case of the Northern Contaminants Program and the Climate Change and Health Adaptation Program, also to Inuit and First Nations north of the 60th parallel.^{Footnote 5}

Between April 1, 2010 and March 31, 2015, the EPHP expended approximately $162 million through set contribution funding agreements and direct departmental spending to identify, assess and address human health risks associated with exposure to hazards within the natural and built environments. One key program difference as of 2014 is that Health Canada's EPHP activities are now performed in British Columbia by the First Nations Health Authority, as a result of the British Columbia Tripartite Framework Agreement on First Nation Health Governance. Another difference as of 2011 is that the former structure of two groups of programming (Environmental Health and Environmental Research) have now been integrated into one program, namely the Environmental Public Health Program at FNIHB. In addition, research and surveillance activities have focussed primarily on environmental contaminants and laboratory activities (see Appendix 2, EPHP Program Logic Model).

2.3 Program Alignment and Resources

The EPHP is listed as sub sub program number 3.1.2.2 under Health Canada's Program Alignment Architecture, and it contributes to the following strategic outcome: First Nations and Inuit communities and individuals receive health services and benefits that are responsive to their needs so as to improve their health status.

The Program's financial data for the years 2010-11 through 2014-15 is presented below (Table 2). Overall, the Program had expenditures of approximately $162 million over five years.

Table 1: Program Expenditures^{Table 1 - Footnote a} ($)^{Table 1 - Footnote *}
Year	Gs & Cs^{Table 1 - Footnote b}	O&M^{Table 1 - Footnote c}	Salary^{Table 1 - Footnote d}	Total^{Table 1 - Footnote e}
2010-2011	15,700,304	7,367,832	10,367,299	33,435,435
2011-2012	15,007,787	6,554,327	11,071,277	32,633,391
2012-2013	15,006,828	5,742,165	10,192,278	30,941,271
2013-2014	14,787,800	6,029,588	10,873,888	31,691,276
2014-2015	14,554,559	5,601,485	13,009,365	33,165,409
Total	75,057,278	31,295,397	55,514,107	161,866,782
Table 1 - Footnote * Data Source: (Financial data provided by BSFO-FNIHB) Return to Table 1 - Footnote * referrer Table 1 - Footnote a excludes British Columbia across all five years; includes funding from the Clean Air Agenda and the Action Plan to Protect Human Health from Environmental Contaminants Return to Table 1 - Footnote a referrer Table 1 - Footnote b "contributions (326)" from BSFO data Return to Table 1 - Footnote b referrer Table 1 - Footnote c "other operating (310)" from BSFO data Return to Table 1 - Footnote c referrer Table 1 - Footnote d "salaries & wages / student program (301-302)" and "uncontrollable salaries (307)" Return to Table 1 - Footnote d referrer Table 1 - Footnote e excludes "minor capital (312)", "revenue (315)", and "non-insured (311)" Return to Table 1 - Footnote e referrer

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

The scope of the evaluation covered the period from April 1, 2010 to March 31, 2015 and included the environmental public health activities conducted by the Environmental Public Health Division (EPHD) and FNIHB Regions, including progress made on activities covered under the First Nations Water and Wastewater Action Plan (evaluation completed in November 2013).^{Footnote 6} Given that other evaluations^{Footnote 7} have assessed the EPHP's activities specific to Inuit communities, this evaluation focussed only on First Nations communities. The evaluation excluded activities that are now covered under the British Columbia Tripartite Framework Agreement on First Nation Health Governance, which came into full effect in 2014-15. Also, the evaluation did not include FNIHB activities covered under the following horizontal initiatives:

Clean Air Agenda, Adaptation Theme (assessed concurrently as part of an Environment Canada horizontal evaluation),^{Footnote 8} and
Action Plan to Protect Human Health from Environmental Contaminants (evaluation completed in February 2014).^{Footnote 9}

The evaluation aligns with the Treasury Board of Canada's Policy on Evaluation (2009) and considered the five core issues under the two themes of relevance and performance, as shown in Appendix 4. Corresponding to each of the core issues, specific questions were developed based on program considerations and these guided the evaluation process.

An outcome-based evaluation approach was used to assess progress made towards achievement of the expected outcomes, to identify any unintended consequences, and to document lessons learned. Given the EPHP was previously evaluated in 2011,^{Footnote 10} particular emphasis was placed throughout the present evaluation on determining what changes and progress had been made since the previous evaluation.

Data for the evaluation were collected using various methods, including: a review of literature, program documents, and administrative data; key informant interviews with both internal respondents (n=15), and respondents external to Health Canada (n=26);^{Footnote 11} a national survey of EHOs (n=69); and, a comparative analysis with other environmental public health services. More specific details on the data collection and analysis methods performed are contained in Appendix 4. In addition, data were analyzed by triangulating information gathered from the different methods listed above. The use of multiple lines of evidence and triangulation was intended to increase the reliability and validity of the evaluation findings and conclusions.

3.2 Limitations and Mitigation Strategies

All evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. The following table outlines the limitations encountered during this evaluation. Also noted are the mitigation strategies put in place, so that the evaluation findings can be used with confidence to guide program planning and decision making.

Table 2: Limitations/Challenges and Mitigation Strategies
Limitation/Challenge	Impact	Mitigation Strategy
Baseline data on the health status of First Nations individuals living on-reserve is not linked directly to the activities or time period (2010-2015) covered by the evaluation.	Evaluating the program's expected outcome to improve the health status of First Nations individuals and communities is challenging to accurately assess during this time period.	This report uses proxy indicators, as well as data on the health status of First Nations individuals and communities for the period just prior to the evaluation.
While the participation of key informants from First Nations communities was sought, encouraged, and voluntary, the evaluation team was unable to interview the number of First Nations community representatives initially planned.	The perspectives of First Nations community representatives may not be fully represented I n the evaluation's findings.	This report notes when findings may not include the perspective of groups underrepresented in key informant findings.
Over the evaluation timeframe, the program's context changed following the introduction of new federal priorities.	The evaluation took place during a change of government, where priorities shifted, such as: addressing the calls to action from the Truth and Reconciliation Commission of Canada; a review of the Safe Drinking Water for First Nations Act; and, increased resources to address health disparities for Indigenous people and other Canadians.	Relevance of the program was assessed for two time periods: one period with the evaluation scope, and current priorities to assist in the development of forward-looking recommendations.
Detailed performance data for the EPHP were not available for all years covered by the evaluation.	The availability of program performance data is limited.	Gaps in documented performance data were filled with information from other lines of evidence (e.g., survey, interviews) where possible.

4.0 Findings

4.1 Relevance: Issue #1 – Continued Need for the Program

There is a continued need to identify and address human health risks to First Nations communities associated with exposure to hazards within the natural and built environments. This evaluation found that many of the current and projected environmental health risks facing First Nations communities are very similar to those that were outlined during the previous evaluation.

Similar to the previous evaluation, all lines of evidence (literature and document reviews, interviews and survey results) indicate that there remains a clear and consistent need to identify, assess, and address environmental health risks in First Nations communities. As identified in the previous evaluation, many First Nations communities experience significant EPH risks compared with other Canadian communities. The link between housing quality and health has been established and is well documented in the literature/document review. Indoor air quality, safety and overcrowding can have a direct impact on health. Over the last decades, First Nations communities have been faced with health problems as a result of the quality of their housing stock. For example, the First Nations Regional Health Survey (RHS), using data collected in 2008-2010, found that for many First Nations living on reserve, housing and living conditions are substandard with approximately one-quarter of First Nations adults living in over-crowded housing (23.4%), representing a substantial increase since the previous RHS in 2002 (17.2%). In comparison in the general Canadian population, 7% of adults live in over-crowded housing. High household membership or over-crowding is linked with various negative health outcomes such as mould exposure, and transmission of infectious disease.^{Footnote 12}

A survey conducted in 2013 with First Nations respondents found that the majority of First Nations people on reserve feel that environmental issues affect their/their family's health "a great deal", and that this proportion has grown since 2010 (52 per cent, up from 42 per cent).^{Footnote 13} Among the eight core areas of EPH programming, the highest priority areas of concern have been directed to risks associated with housing and water infrastructure issues. As noted in the 2013 survey, the environmental issues that pose the greatest health concern to First Nations people on-reserve are water pollution, air pollution, and mould, each identified by between one in five and one in three. These were also the top three issues identified in 2010. More recently, the Standing Senate Committee on Aboriginal Peoples released a report on its study of on-reserve housing and infrastructure. Among its findings, the Committee heard testimony and visited First Nations communities with "boil-water advisories that had been in place for more than a decade, over-flowing sewage lagoons, roofs covered by tarps because there is no money to repair them, and small overcrowded bungalows where between 16 and 18 people sleep at one time"^{Footnote 14} which raises other possible EPH risks like the potential spread of communicable diseases, mould, and indoor air quality.

As illustrated in Table 3, findings from the current survey of EHOs were relatively similar to those from the previous evaluation's findings. The most frequently cited risk areas across both cohorts were housing, drinking water, and wastewater related risks, and none of the groups interviewed or surveyed identified large-scale changes or differences in areas of risk that had occurred over the past five years.

Table 3: Most Frequently Cited EPH Risk Areas by Environmental Public Health Officers
EPH Area	2011 (n=93)	2016 (n=65)
Housing conditions	92%	99%
Drinking water	91%	95%
Wastewater	80%	72%
Food safety	53%	66%
Solid waste	47%	52%
Communicable disease control	43%	39%
Community emergencies	20%	29%
Note: Facilities inspection, while a core program area, addresses various risk areas. Source: EHO Survey 2011; EHO Survey, 2016

As illustrated in Table 4, among those identifying the above risks in 2016, drinking water and housing risks were most frequently rated as "high priority".

Table 4: Most Frequently Cited as "High Priority" Risk Areas
Core EPH issues identified as Risk Areas	Overall % of Survey Respondents Identifying Area as a High Priority (n=65)
Drinking Water Risks	88%
Housing Risks	80%
Wastewater risks	38%
Food safety risks	35%
Communicable disease control risks	25%
Solid waste risks	14%
Community emergency risks	12%
Note: Multiple responses allowed, therefore, will not add to 100%

With respect to drinking water and wastewater, responses from the EHO survey identified that challenges continued to result from poor or old infrastructure; improper infrastructure maintenance; lack of training and availability of certified operators; systems working at or beyond capacity; climate change; local geography characteristics; and numerous socio-economic issues.

4.2 Relevance: Issue #2 – Alignment with Government Priorities

Similar to the previous evaluation, the EPHP activities continue to align with federal priorities to address environmental public health risks in First Nations communities. The EPHP is aligned with Health Canada's Strategic Outcome 3: First Nations and Inuit communities and individuals receive health services and benefits that are responsive to their needs so as to improve their health status.

The previous evaluation found that the EH-ER Cluster activities aligned with the federal and Health Canada priorities related to environmental public health. Since April 2010, various Government of Canada commitments have supported improving the identification and mitigation of environmental health risks, particularly in the areas of drinking water and wastewater. For example, the 2011 and 2013 Speeches from the Throne outlined the Government's continued work with Indigenous peoples. Budget 2011 outlined funds to support water, housing and First Nations and Inuit health with Budget 2012 through 2015 mentioning support for various aspects of First Nations and Inuit health. More recently, Budget 2016 has included additional commitments specifically in the areas of drinking water, wastewater, solid waste management and housing in First Nations communities.

4.3 Relevance: Issue #3 – Alignment with Federal Roles and Responsibilities

The EPHP role is advisory in nature and has not changed significantly since the previous evaluation. However, there may be potential in the future to move towards a more regulatory role in the area of drinking water.

Health Canada (FNIHB) provides advice, recommendations and guidance to First Nations communities on how to reduce or mitigate potential environmental public health risks in the natural and built environments. In 2012, the Federal Government introduced Bill S-8, the Safe Drinking Water for First Nations Act, with the Act receiving Royal Assent on June 19, 2013 and coming into force on November 1, 2013. The Act enables the Federal Government to develop enforceable federal regulations for access to safe, clean, and reliable drinking water; the effective treatment of wastewater; and the protection of sources of drinking water on First Nations lands. At the time of the finalization of this evaluation report, the Act was under review by INAC Ministerial direction, and the future of the regulatory development project was unknown.

Currently, in the absence of a statutory regime or legislative framework, the activities delivered under the EPHP remain advisory. Health Canada also does not currently have the mandate to provide funding to communities to address or remediate risks, as this is the purview of INAC.

With respect to the other core areas of EPH mitigation, the findings from the current evaluation are very similar to those from the previous evaluation. Health Canada plays an advisory role, and the delivery of this role varies according to region. As a result, activities under the EPHP are provided at the request and/or with the agreement of First Nations authorities. EPHP personnel operate with the understanding that Health Canada does not have any EPH authorities to enforce existing public health standards or regulations (federal, provincial, otherwise) on-reserve.

As outlined in the previous evaluation, there are numerous other stakeholders involved in various areas of EPH in First Nations communities. These stakeholders have not changed substantially over time and include: First Nations communities, various First Nations organizations, other federal government departments and agencies (e.g., INAC, EC, PHAC, CFIA, CMHC), various branches within Health Canada, and provincial governments. INAC and Health Canada require collaboration specifically in a number of areas associated with EPH given their combined contributions to achievement of EPHP objectives. For example, while INAC is the lead on the First Nations Water and Wastewater Action Plan, Health Canada has important supporting roles. Similarly, investigations by EHOs or reviews by public health engineers can result in recommendations for infrastructure improvements that are being funded through INAC programming. For the management of drinking water quality, First Nations are responsible for the construction, operation and management of the systems; INAC sets the standards for these systems and provides funding for the construction, operation and maintenance of the systems; and HC provides support to protect public health, including drinking water quality monitoring and funding of Community-Based Water Monitors. For example, if the HC's Environmental Health Officer's(EHO) review and interpretation of drinking water quality results indicate that drinking water is not safe, the EHO immediately communicates recommendation(s) (such as a "boil water" advisory) to the Chief and Council for their action.

The provision and management of housing on reserve lands is the responsibility of First Nations, with support from the Government of Canada. In addition to government funding from INAC, First Nations are encouraged to identify funding from other sources for their housing needs, including shelter charges and loans. According to documentation, and confirmed through key informant interviews and the EHO survey, despite the large number of stakeholders involved, roles and responsibilities are clearly outlined through various agreements and arrangements depending on the stakeholders involved, with limited overlap or duplication.^{Endnote iii}

Echoing the findings from the previous evaluation, the one area in need of greater attention and highlighted by many of the key informants from First Nations communities and organizations is the continued gap between known risks that have been identified and assessed by Health Canada, and then those risks actually being addressed by the contributions of others. For example, the link between Health Canada's identification and assessment work and the contribution of others like INAC's programming and funding that would be needed to address the more complex risks identified and assessed by the EPHP (e.g., housing and infrastructure changes). This challenging gap was also identified in many of the interviews with Health Canada representatives, and was outlined in open comments by some respondents to the EHO Survey.

4.4 Performance: Issue #4 – Achievement of Expected Outcomes (Effectiveness)

The immediate outcome of "improved environmental public health risk identification" was defined according to four sub-outcomes of:

Timely data collection, analysis, dissemination and reporting;
Evidence based decision-making;
First Nations awareness of environmental public health risks; and
Immediate risk mitigation activities.

Timely data collection, analysis, dissemination and reporting

There are multiple layers of data collection, analysis, dissemination and reporting within the EPHP. At the community level, and similar to the previous evaluation, this process continues to be working well, as EHO expertise and knowledge connects with community priorities to guide the collection, analysis, dissemination and reporting undertaken within a specific community. Data collected at the community level is analyzed and reported through the form of recommendations to Chief and Council. Findings are disseminated through various training and community awareness activities. At the program level, there has been some improvement in data collection systems since the previous evaluation, and continued work in the analysis, dissemination and reporting of the program results, particularly with respect to drinking water. The current evaluation found gaps and challenges that relate to needs for greater data comprehensiveness at a national level, and the utility of data in being able to report on outcomes and use in risk management.

Data collection, analysis, dissemination and reporting occur at various levels under the EPHP program. EHOs work directly with communities through investigations and various other monitoring activities to collect data on aspects of a specific community (e.g., facility, housing, water system) that they then analyze and report back to the community through recommendations to Chief and Council. Further dissemination of results is continued through work with community health representatives, and potentially through related training and awareness activities for community representatives.

Another level of data collection, analysis, dissemination and reporting occurs at the program level, with some aspects of the EHO and CBWM activities getting captured within regional and national data systems. Analyses of these data then feed into various regional and national performance reports (e.g., FNWWAP national reports; EPHP Performance Report).

Data systems

The key data systems in use by the Program to identify and assess risk are the WaterTrax and Environmental Health Information System (EHIS). For various reasons, some regions (e.g., Alberta, Quebec) use alternative data systems, but much of the data from these systems can be rolled up at a national level. The previous evaluation found that there were challenges with the data systems in place at that time, but that considerable progress had been made over the five-year period. EHIS was implemented in 2005, and drinking water data were being compiled in various regional data systems.

For the current evaluation, overall, many Health Canada key informants perceived that the timeliness and quality of data systems for the EPHP had improved during the period under study. Informants indicated that improvements were largely the result of increased attention, training and diligence of regions and EHOs in populating the data systems. According to the FNWWAP Drinking Water Performance Indicators National Report, in 2012-13 all regions had access to a drinking water database. Maintenance of regional databases is necessary, so that drinking water quality results can be communicated in a timely manner, allowing for informed and timely decisions such as recommending the issuance of a drinking water advisory (DWA).

When Health Canada key informants identified weaknesses with the data / data systems, these generally related to the comprehensiveness of the data for national reporting purposes, and the utility of the data for decision-making.

Key informants' concerns about data comprehensiveness included the fact that EPHP does not have access to data from First Nations communities that have entered into a Health Funding Agreement, and therefore cannot report on or conduct analysis of risks facing all First Nations communities at the regional or national levels. Additionally, existing systems are not currently designed to allow for the capture of all relevant EPH activities or risks related to any given community, facility, system or home. As a result, risk analyses and resulting decision-making may at times be based on partial information. Finally, there are multiple systems in place and these do not communicate with each other. EPH data is currently housed in several different databases depending on the program activity and the region, making it difficult to roll-up and analyze regional and national data, and identify risks and trends. For example, the drinking water monitoring data and drinking water advisory data are housed in different databases depending on the Region. Quebec uses Water-Eau, Alberta uses ELPHIS, and the four other Regions use WaterTrax. For data related to public health inspections, investigations, and other activities undertaken, Alberta uses Hedgehog, while the rest of the regions use EHIS. There is no communication or interoperability between systems; data is not standardized and must therefore be rolled up manually for national reporting; and there is little capacity for integrated, complex or comprehensive analyses of environmental health risks and trends across systems/program areas/activities, which would be useful at all levels of decision-making, from First Nations to senior management.

Key informant concerns about data utility tended to focus on the perceived limitations of the data systems in capturing outcomes, and in providing strategic information useful for risk management and decision-making. The data are not yet being used optimally to monitor environmental public health risks and outcomes, or to support risk management. At the community and regional levels, data pertaining to drinking water quality monitoring results is housed separately from data pertaining to all other EPH program areas (inspections, investigations, assessments, surveillance related to facilities and housing). These databases do not speak to each other and do not incorporate geographic information systems data in a way that would facilitate cross-referencing or integrated analysis of risk trends and results across the different program areas. Further, these systems do not integrate data from other systems/departments (e.g., those used in other FNIHB Regions, or those used by INAC in relation to housing or water and wastewater infrastructure projects and reporting. Some additional specific issues mentioned by a few respondents included the inability of the inspections data systems to facilitate risk assessment or prioritization (e.g., provide risk rankings and follow-up schedules on the basis of inspection results).

Staff

Having an adequate number of EHOs is considered to be a possible contributing factor to more timely data collection, analysis, dissemination and reporting at both the community and program level, which should lead to improved environmental public health risk identification. The previous evaluation identified a challenge with the demands for EHOs and their workload, to the extent that there were issues with having to work reactively rather than proactively in some communities. While the current evaluation identified continued challenges related to EHO workload, it is unclear whether the actual number of EHOs has increased from the previous evaluation.

Inspections, Monitoring and Investigations

Water sampling activity is an indicator of data collection, analysis, dissemination and reporting which should lead to improved environmental public health risk identification. While the document review did not obtain data for all years covered by the evaluation, from the information received it appears that the amount of water sampling is increasing overall in First Nations communities, with compliance rates for recommended frequency of sampling increasing as well. According to the 2015 evaluation of the CBWM Program, "there has been a 22% total increase in the number of bacteriological samples collected for all distribution systems from FY2004-05 (76,029 samples) to FY2011-12 (92,785 samples)." ^{Footnote 15} The same evaluation found that while the recommended sampling frequency set out in the Guidelines for Canadian Drinking Water Quality (GCDWQ) was generally not met at a national level over the period from 2008-09 to 2011-12, compliance increased annually from 42% in 2008-09; 45% in 2009-10; 58% in 2010-11; to 60% in 2011-12. However, in 2014-15, the compliance rate declined to 42%. This same trend was found in the available National FNWWAP Reports with the number of water samples analyzed increasing by approximately 6% overall (from 134,659 in 2011-12 to 142,853 in 2012-13). Although compliance rates increased over the 2008-09 to 2011-12 period, there was a sharp decline in 2014-15, highlighting that further improvement is required to meet the recommended monitoring frequencies for bacteriological parameters of the Guidelines for Canadian Drinking Water Quality. HC's EHOs work directly with First Nations communities to increase monitoring compliance for bacteriological parameters in distribution systems with 5 or more connections by identifying targeted communities; addressing non-compliance (eg.,identify suitable back-up CBWMs) and; in the absence of a CBWM, identifying/training a replacement while assisting in monitoring to the extent possible.

The delivery of public health inspections is a core component of EPH services in First Nations communities. Overall, there are similar proportions of routine inspections and requested inspections; however, this proportion differs between communities and across facility types and can vary considerably from year to year, and from region to region.^{Footnote 16} According to the 2014-15 Performance Indicators Report, there were on average approximately 7,000 inspections each year for the period covered by the evaluation. The variability is demonstrated with a low of 5,960 inspections in 2012-13 and a high of 9,175 inspections in 2011-12. The number of inspections is slightly lower, but comparable to the previous evaluation, which identified the number of annual inspections ranged from 8,089 (2009-2010) to 8,877 (2007-08).

While the proportion changes slightly from year to year, the number of requested inspections is usually slightly higher than routine inspections. For example, in 2013-14, the proportion of inspections that were requested was 54% compared with the 46% that were categorized as routine inspections. The majority of requested inspections are in the areas of housing, sewage, and drinking water. In contrast, the routine inspections generally occur for food facilities and community care facilities (childcare, schools and other care facilities). According to a recent report analysing EHO workloads, in 2013-14, the need for some regions to focus on emergency activities (e.g., flooding) resulted in EHOs only being able to meet approximately one-third of their targeted inspection regime (i.e., routine inspections as mandated through National Framework).

EHOs work with First Nations Authorities and other public health agencies to address suspected or confirmed cases or outbreaks of foodborne, waterborne and vectorborne diseases. According to EPHP Performance Reports, the vast majority of the activities related to communicable disease investigations are related to investigating vectorborne illness resulting from rabies risks and animal bites from dogs. The number of these investigations appears to have more than doubled from 135 in 2010-11 to 301 in 2014-15. Activities around Lyme Disease investigation and awareness increased in the last four years while the West Nile Virus work continued, resulting in a large increase in the total number of investigations for vectorborne diseases.

Zoonotic surveillance activities are primarily located in three regions (Atlantic, Ontario and Manitoba) and focus on mosquito, bird and tick surveillance. Surveillance activities seem to have decreased more recently from 378 activities in 2013-14 to only 86 activities in 2014-15^{Footnote 17}. Overall, the workload of EHOs in these areas has increased. ^{Footnote 18} An initial assessment of the gap in the current number of EHOs reveals a shortfall, thus leaving no surge capacity in most communities to address unexpected emergencies or events.

Evidence-based decision-making

The previous evaluation found that the EH-ER cluster was contributing to the evidence and research used to make decisions on EPH-related activities. Similarly, the current evaluation found that the EPHP activities undertaken were largely evidence-based. There was evidence of best practices being applied in the work with First Nations communities, research being conducted and disseminated, and an ongoing effort to update various guidance documents.

Overall, key informants across all respondent groups were of the view that EPH program activities were evidence-based, and that this had largely remained stable over time. Health Canada representatives most often pointed to the fact that program activities were aligned with evidence-based national guidelines (e.g., GCDWQ) and provincial EPH standards and practices (e.g., drinking water quality standards, themselves largely based on the GCDWQ). As well, some Health Canada respondents noted that EHOs and other areas of the program such as the research components rely on their internal networks, and also have access to Branch and external experts with expertise. A few Health Canada key informants noted that all EHOs are certified and part of a professional association that expects they are up to-date on current evidence-based practices.

Key informants from First Nations communities and organizations indicated no concerns with the evidence base of the Program's risk assessment and mitigation (e.g., the EHOs are held in high esteem in communities, reports are viewed as careful, rigorous and trustworthy). According to one respondent, First Nations communities hiring their own EHO through a Health Funding agreement are well-supported by the Health Canada regional office with regular communication on standards and case reviews.

According to the EHO survey, the majority of EHOs (55%) reported that their use of best practices from other jurisdictions, organizations and departments had increased over the past five years. Overall, 38% indicated that this has increased "somewhat" while an additional 17% of EHOs indicated that it had increased "significantly". A large proportion reported their use of best practices had remained the same (40%), while approximately 5% reported this having decreased.

In addition, the program generates evidence through a number of research initiatives. The main research initiatives that produced findings that were further disseminated to various communities, researchers, and internal HC audiences included:

First Nations Biomonitoring Initiative (ongoing),
Northern Contaminants Program (ongoing),
Drinking Water Quality Program (2 years),
First Nations Environmental Health Innovation Network (3 years),
First Nations Food, Nutrition and Environment Study (FNFNES) (ongoing), and
First Nations Environmental Contaminants Program (FNECP) (ongoing).

According to Health Canada key informants, data from these studies provide communities with an evidence base to understand risks of exposure to environmental contaminants within their community. According to Health Canada representatives, the research initiatives are largely community-driven (e.g., the FNECP) or will engage communities in methodology design and knowledge transfer.

From the interviews with First Nations communities, the small number of community representatives who had participated in the FNECP were pleased with the Program and confirmed that the program had helped to understand the nature of contaminants in their community and mitigation strategies (e.g., levels of mercury in local fish, uranium in ground and surface water). Many of the EPHP representatives identified improvements in the community-based research program, including guidelines that have been implemented in the call for proposals for research projects. Similarly, they noted that both national and community-driven research initiatives are customizable to the concerns of the community.

Interviews with First Nations organizations noted a few areas for improvement, such as ensuring that the notification of the AFN of planned research with First Nations communities occurs prior to implementation. For all major research work, FNIHB respected the AFN-FNIHB Consultation Protocol (including FNFNES, Biomonitoring and FNECP). However, there were times in the past where AFN was informed of one-off research projects after the fact, yet this was not a common practice.

The document review found various examples of policies, guidelines, frameworks and procedures that have been developed, updated or adapted by the EPHP during the period covered by the evaluation. Examples included:

Drinking Water Program Manual;^{Footnote 19}
Housing Inspection Resource Manual for EHOs;^{Footnote 20}
Field Reference Manual: Non-Microbiological Drinking Water Quality Parameters;^{Footnote 21}
Guidance on ensuring the safe delivery of drinking water to individual homes and facilities where piped water from a central system is not available;^{Footnote 22} and,
A set of detailed and complete set of instructions in straightforward language for the installation, use, maintenance and decommissioning of cisterns for holding drinking water from a certified safe source.^{Footnote 23}

The target audiences for these products vary including FNIHB regions, EHOs, CBWMs, First Nations communities and other stakeholders.

First Nations awareness of environmental public health risks

Over the past five years there appears to be an increased awareness of environmental public health risks among First Nations community members, along with increased awareness of EPH services in their community. This greater awareness has led to increased access and use of the EPH services in communities. Given the workloads and travel restrictions in some regions, this increased access and use may differ according to area and region. Awareness of the EPHP was lower among the regional First Nations organizations.

The previous evaluation demonstrated that considerable effort and emphasis was placed on the development of public education and awareness materials under the EH-ER cluster activities. The current evaluation found similar levels of effort for the subsequent five-year period.

The evaluation found numerous examples of public education and awareness materials that had been developed by the EPHP during the past five years. From the document review, it was noted that the materials tended to range in topics (drinking water, traditional food safety, infection control, mould) and were delivered in various formats (fact sheets, videos, websites, newsletters, posters, presentations, media releases) to various groups of community members (parents, youth, children, council members, facilities employees). According to the 2014-15 Performance Indicators Report, there were a total of 257 public education materials developed nationally in areas of drinking water, wastewater, housing food safety, and environmental communicable disease control.

A survey conducted in 2013 with First Nations respondents found that more than half of First Nations people living on reserves (54%) say, unprompted, that they recall having seen, heard, or read something about the impacts of the environment (outdoors and indoors) on their health in the previous 12 months. When asked, unprompted, about the topic of this information, the largest proportions of First Nations people mention drinking water safety (20 per cent), mould (18 per cent), and oil and gas pollution (16 per cent). Alternatively, a decrease in the number of First Nations on reserve unable to identify any environmental issue was also observed (16% of respondents during the 2013 survey compared to 31% in 2010).^{Footnote 24}

From the interviews, Health Canada regional representatives and First Nations communities representatives named a variety of health promotion and public education strategies that were used in the communities to raise awareness: radio, door hangers, pamphlets/publications, public health meetings, booths at health fairs and community events, door-to-door interactions, training (food safety) and annual reports. Some community representatives noted that in-person strategies are most effective in terms of outreach and awareness-raising, and tapping into community gatherings is another common practice that is perceived to be effective and culturally acceptable/well-received. A few First Nations community representatives also indicated that awareness of EPH risks in some communities is increasing in part due to knowledge of external factors, such as impacts of oil and gas activity on drinking water quality.

The evaluation captured the perspective of EHOs who have the opportunity to observe possible changes in levels of awareness among First Nations individual community members. As illustrated in Figure 1, approximately two-thirds of respondents to the EHO survey (64%) reported that there had been a positive shift in the community members' awareness and knowledge of EPH risks in the past five years. Most EHOs (83%) reported that the EPHP had made at least some contribution to community members' levels of awareness of EPH risks. This rate is similar to that found in the previous evaluation where in 2011, 54% of EHOs noted a positive shift, and 87% indicated that the Program had made a contribution towards this shift.

Figure 1: Community members' knowledge/awareness of EPH risks and EPHP contribution

**Figure 1A: Knowledge/Awareness of EPH Risks**

Source: EHO Survey, 2016, (n=63).

Many key informants from across the respondent groups tended to agree that awareness of EPH risks has increased over the last five years among First Nations community members through the active dissemination of information, education and site visits. Most Health Canada regional representatives noted awareness has increased in part due to the information that is collected by the EHOs (through inspections, water sampling, and investigations) and reported back to the community leadership, along with risk mitigation strategies. Many Health Canada representatives noted that community awareness is also increased by means of the EHO working collaboratively with different groups, including Health Directors, Housing Director, nursing staff and a variety of community members, and not just Chief and Council.

Most First Nations community representatives interviewed indicated that their communities had good access to the program.This was confirmed with the findings from the EHO survey (Figure 2) with the majority of EHOs (57%) reporting that availability of EPH services for communities had improved during the previous five years. This is similar to the evaluation in 2011 in which 64% of EHOs indicated that availability of services had increased.

Most community representatives reported being satisfied with access to the EPHP services through their EHO, and indicated that the communications and responsiveness provided by EHOs are very good. In some cases, the EHO was perceived as taking a supporting role in the community (e.g., providing evidence to support applications for infrastructure funding to INAC). Their relationship-building efforts and site visits were viewed by many community representatives to be very important to the visibility and credibility of the program and their position with the community. Many community representatives indicated that through their role in capacity building and training, EHOs are perceived as valuable resources, professionals and trainers. It was noted in the EHO survey that the majority of respondents (53%) reported that communities access to skills development and training related to EPH had increased over the past five years (Figure 2). This is a similar rate to the 47% of EHOs reporting this in 2011.

Consistent with the previous evaluation, in general, the EPHP and in particular the EHOs are quite visible with many communities as reported by HC regions and confirmed by many communities. The interviews with First Nations regional organizations suggest that there is less awareness of the EPHP and the role of the EHO among some of the regional organizations interviewed. Interviews with some First Nations communities and organizations noted the need for greater promotion of the program within communities and regions, and a greater sharing of community experiences in addressing EPH risks. Venues indicated for this included regular meetings of EPHP representatives with Health Directors in specific areas or at a regional level.

Figure 2: Community awareness, availability and use of EPH services

**Figure 2A: Community awareness of EPH services (n=63)**

**Figure 2B: Availability EPH services (n=63)**

**Figure 2C: Use of EPH services by communities(n=63)**

**Figure 2D: Access to EPH skills dev't and training (n=55)**

Source: EHO Survey, 2016

Immediate risk mitigation activities

Similar to the previous evaluation, many EPH risks are being mitigated in the immediate or short term as a direct result of EHO's risk assessment activities, advice or recommendations. This appears to be happening more frequently now during drinking water monitoring activities, food handler training sessions, and community education sessions than during the previous evaluation time period.

Both in 2011 and 2016, EHOs reported frequently noticing EPH risks that are being mitigated or eliminated immediately due to their ongoing involvement with community members (Table 5). This is happening most often as a result of drinking water monitoring activities and food handler training sessions, but also occurring as a result of public health inspections, communicable disease investigations, and community education sessions. The extent to which this was reported by EHOs to have occurred was generally more frequently in 2016 during drinking water monitoring activities, food handler training sessions and community education sessions when compared with 2011 rates.

Table 5. Activity areas where EPH risk is mitigated or eliminated in the short term
Activity	Frequently and Very Frequently 2011 (n=93)	Frequently and Very Frequently 2016 (n=49 to 65)
Drinking water monitoring activities (n=65)	66%	78%
Food handler training session (n=61)	59%	75%
Public Health Inspections (n=64)	36%	36%
Communicable disease investigation (n=49)	47%	45%
Facilities plan review (n=52)	54%	52%
Community education session (n=59)	29%	46%
Emergency planning or response activities (n=50)	25%	30%
Source: 2011 EHO Survey; 2016 EHO Survey – Activity column includes sample population from 2016 in brackets

A survey conducted in 2013 with First Nations respondents found that in response to a specific awareness campaign conducted by EPHP, more than one-third of First Nations (36%) indicated that they did something based on the campaign materials they saw. The largest proportions of First Nations took action to protect against harmful chemicals (19%) and to prevent mould in the home (19%), and talked to others about environmental health risks (17%).^{Footnote 25}

Achievement of Intermediate Outcomes

Similar to the previous evaluation, there continues to be progress in increasing First Nations communities capacity to identify and assess EPH risks and to address certain risks through short-term remedial actions when addressing areas such as drinking water advisories and food safety issues. Consistent with the previous evaluation, progress is slower with respect to addressing more complex recommendations. Considerable effort is being made in the area of training and education around EPH risks.

The previous evaluation found that while First Nations community capacity to identify and assess EPH risks has increased, the community capacity to mitigate or address more complex risks had not increased to the same extent. The findings from the current evaluation are consistent with this previous finding. As illustrated in Figure 3, EHOs indicated that while there had been some change in capacity among communities to independently identify and assess EPH risks over the past five years, there was more limited evidence of change in communities capacity to either address and mitigate EPH risks, or manage and administer EPH programming under the guidance of EHOs. This pattern is very consistent with that found with the 2011 survey of EHOs.

Figure 3: Community capacity to identify, assess, address and mitigate EPH risks

**Figure 3A: Community capacity to independently identify and assess EPH risks (n=57)**

**Figure 3B: Community capacity to address and mitigate EPH risks (n=61)**

**Figure 3C: Community capacity to manage and administer EPH programming under the guidance of EHOs (n=57)**

Source: EHO Survey, 2016

According to some Health Canada regional representatives and some representatives from First Nations communities and organizations, short-term remedial actions and drinking water advisories lead consistently to timely reactions by communities, so that immediate behaviour changes can be made by community members. Similarly, issues with food safety are quickly taken into account. A few community representatives reported that they have had no food-related outbreaks in many years, suggesting that the food safety training is effective and changing behaviour.

Similar to what was highlighted in the previous evaluation, many of the representatives from First Nations communities and organizations and Health Canada regional representatives noted that recommendations that are more complex and require costly interventions are more difficult for the communities to respond to as these may require additional funding from INAC or investments by the community that they cannot afford due to other priorities. Some First Nations communities indicated that the human resources available to address and manage all EPH risks within their communities are inadequate, and that they would benefit from having trained resources on site to conduct monitoring and follow-up in various risk areas.

It was noted by some Health Canada representatives that an area for improvement with respect to the program is to enhance the risk-based approach to programming. Routine inspection and monitoring frequencies are important, but in some cases may not be required at current levels for certain facilities or systems that are extremely low risk. Conversely, high risk facilities and systems might require increased inspection or monitoring frequencies and it is important to help communities prioritize recommendations pertaining to such systems and facilities. As previously discussed, data systems might assist in this respect by facilitating risk assessment or prioritization (e.g., provide risk rankings and follow-up schedules on the basis of inspection results). An additional area for consideration outlined in interviews with EPHP representatives was to begin to track both the content and outcome of recommendations (e.g., monitor which recommendations were implemented, and which still require attention or follow-up).

Achievement of Longer-Term Outcomes

Most key informants across all respondent groups indicated that in their opinion the EPHP contributes to the improved health and well-being of First Nations' individuals and communities. The evidence offered by key informants to support this view often centered on the Program's activities, and to the extent that risks are being mitigated, immediate outcomes logically linking to the improved health and well-being outcome. Examples of activities cited included sampling of drinking water to verify quality and, as required, recommendation for drinking water advisories; and avoiding outbreaks of food-borne illness through food safety training and regular inspection of food facilities.

Following the stated program logic, improved health and well-being likely is occurring, particularly where intermediate outcomes are being achieved; however, the evaluation was not able to accurately ascertain this link due to limited data on First Nations health outcomes during this period.

4.5 Performance: Issue #5 – Demonstration of Economy and Efficiency

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results. The financial information available consists primarily of the expenditures by the EPHP as outlined in Table 6.

Table 6: EPHP Expenditures^{Table 6 - Footnote a} by Region and HQ^{Table 6 - Footnote *}
2010-2011 to 2014-2015 ($)
Region	Gs & Cs^{Table 6 - Footnote b}	O&M^{Table 6 - Footnote c}	Salary^{Table 6 - Footnote d}	Total^{Table 6 - Footnote e}	%
Atlantic	3,259,065	2,169,913	4,330,263	9,759,241	6%
Atlantic	33%	22%	44%	100%	-
Quebec	4,148,970	1,618,192	4,505,928	10,273,090	6%
Quebec	40%	16%	44%	100%	-
Ontario	8,517,078	10,100,411	11,216,452	29,833,941	18%
Ontario	29%	34%	38%	100%	-
Manitoba	9,438,823	3,835,898	5,573,052	18,847,773	12%
Manitoba	50%	20%	30%	100%	-
Saskatchewan	21,505,886	2,400,488	3,798,492	27,704,866	17%
Saskatchewan	78%	9%	14%	100%	-
Alberta	7,122,564	6,021,272	10,060,002	23,203,839	14%
Alberta	31%	26%	43%	100%	-
Northern	5,854,801	55,691	209,741	6,120,234	4%
Northern	96%	1%	3%	100%	-
Headquarters^{Table 6 - Footnote f}	15,210,091	5,093,531	15,820,177	36,123,799	22%
Headquarters^{Table 6 - Footnote f}	42%	14%	44%	100%	-
Total	75,057,278	31,295,397	55,514,107	161,866,782	100%
%	46%	19%	34%	161,866,782	100%
Table 6 - Footnote * Data Source: (Financial data provided by BSFO-FNIHB) Return to Table 6 - Footnote * referrer Table 6 - Footnote a includes funding from Clean Air Agenda and Action Plan to Protect Human Health from Environmental Contaminants. Return to Table 6 - Footnote a referrer Table 6 - Footnote b "contributions (326)" from BSFO data Return to Table 6 - Footnote b referrer Table 6 - Footnote c "other operating (310)" from BSFO data Return to Table 6 - Footnote c referrer Table 6 - Footnote d "salaries & wages / student program (301-302)" and "uncontrollable salaries (307)" Return to Table 6 - Footnote d referrer Table 6 - Footnote e excludes "minor capital (312)", "revenue (315)", and "non-insured (311)" Return to Table 6 - Footnote e referrer Table 6 - Footnote f Most of the G&C funds for HQs support regional First Nations organizations and communities, as well as universities' work as they relate to the Climate Change Health Adaptation Program and various other contaminates programs, plus approximately $1million annually for the support of national organizations' (e.g. AFN & ITK) work in environmental health. Return to Table 6 - Footnote f referrer

Departmental Performance Reports (DPRs) provide a source of budgetary, expenditure and variance information based on the PAA. In the case of the EPHP, information is available for the two fiscal years (2013-14, 2014-15) for which the program has not been included in broader reported activities. In the 2014-15 DPR, the program spent approximately $24.5 million less than originally anticipated ($59.6 million) stating that the variance between actual^{Endnote iv} and planned spending was mainly due to the reallocation of funding to sub-sub program Clinical and Client Care for continuity of access of nursing services in remote and isolated First Nations communities, which was partly offset by additional funding received through Supplementary Estimates to support the delivery of the First Nations Water and Wastewater Action Plan. However, it should be noted that, while there was a budgetary variance, more human resources (150 FTEs) were attributed to the program than originally planned (132 FTEs), mainly due to a realignment of resources from plans in order to meet program needs.

Similarly in 2013-14, the program spent just under $6.6 million less than anticipated ($82.4 million), noting that the variance that year was primarily due to reductions associated with the realignment of resources for the implementation of the B.C. Tripartite initiative and a reallocation of funding within this strategic outcome to Clinical and Client Care to address needs and priorities. Similar to the 2014-15 fiscal year, more human resources were dedicated to these activities than originally planned.

EPHP expenditures are captured by an overall funding source. For example, an EHO's salary may be funded with FNWWAP resources (water and wastewater) whereas her annual activities may actually cover other various areas of programming (e.g., communicable disease control, solid waste, etc.) depending on community priorities. This approach to budgeting and assignment of expenditures limits the extent to which the evaluation can quantitatively assess the economy and efficiency of the EPHP from a financial data perspective. The evaluation has collected more qualitative information on stakeholders' perceptions of economy and efficiency, along with some potential areas for improvement which are presented in this section.

Economy

The EPHP has continuously worked with First Nations communities and organizations to offer the most useful, relevant services, policies and knowledge transfer in the various areas of environmental public health. The evaluation found that First Nations communities and organizations are satisfied with the services from the EPHP, indicating that the program is expending resources on areas that are addressing key First Nations priorities and that are relevant. This is a good indicator of the economy of the program. An additional area of economy noted by the evaluation is the strong network of EHOs that can provide surge capacity, share expertise and resources.

Many representatives from First Nations communities and organizations and representatives from EPHP noted that the relationship between the EHOs and community is an important aspect of the economy of the program delivery. The EHO investment in community work plans and adjusting the services from the EPHP to meet the needs at a community level provides some assurance that the services being offered are tailored according to community priorities and needs. This planning and flexibility in programming promotes the deployment of resources based on the priorities of the community and the EPH concerns of the local area.

While the flexibility of the program and services offered was highlighted as a positive aspect in many interviews, there were a few EPHP representatives that indicated there may be a need to determine whether activities do actually align with the program's core mandate and eliminate areas of potential duplication. The two main areas cited as examples included clarification of funding of drinking water sampling at the water plant level in some communities (e.g., role of EHO, CBWM, or plant operator), and the extensive involvement in food safety training to determine whether it is verging on job skills training more than addressing environmental public health risks.

The strong network of EHOs in regions was identified as supporting economic program delivery for the EPHP. The network allows for quick sharing of expertise and resources across EHOs and communities, along with a "surge capacity" to cover emergencies and need for quick response from EHOs. This need to quickly respond to emergency situations such as flooding in First Nations communities is expected to continue to grow with anticipated impacts of climate change.

Efficiency

Already considered a "lean" program, there have been a number of efficiencies developed over the past five years. Changes in approaches to budgeting and funding at the regional level make it challenging to identify national level efficiencies for the EPHP. The workload of some EHOs may be unsustainable in the longer term, potentially impacting on the effectiveness of the EPHP. There is a strong likelihood that there will be significantly greater demands placed on the EPHP given the increased infrastructure changes, ongoing growth and development of First Nations communities as they address EPH risks in various areas. Given the variability in delivery of the EPHP across regions, and differences in capacity and priorities between First Nations communities, these challenges in sustainability and its impact on effectiveness may be more marked in some areas compared with others.

As illustrated in Figure 4, the majority of EHOs surveyed indicated that the efficiency of their work methods had increased over the past five years (62%). As well, the majority also indicated that the quality and level of service they can provide to meet community needs had increased during the same period (56%). These are similar levels of increased efficiency noted by EHOs in 2011 (67%), and increased quality and level of service (63%).

Figure 4: Efficiency and quality of service improvements in past five years

**Figure 4A: Efficiency of EHO's work methods**

**Figure 4B: Quality & level of service to communities**

Source: EHO Survey, 2016 (n=63).

In terms of efficiency, many program representatives, particularly in the regions, characterized EPHP as a lean program with a few believing that the program is actually underfunded. Many regional representatives noted that the workload and activities of EHOs are highly monitored for efficiency. Some regional representatives reported that they are experiencing significant workload pressures. Many regional representatives noted that workload issues are likely to get increasingly challenging as a key cost driver for the program is the number and profile of communities and their priorities. It was noted among regional representatives and other federal government departments interviewed that many First Nations communities are growing and there are consistently more complicated commercial and community infrastructure being constructed. Budget 2016 also committed to substantial increased investments for First Nations community infrastructure projects. In addition, according to interviews with program representatives and First Nations' community representatives, there is increasing emphasis on EPH risks within First Nations communities and additional funding being allocated to address some of these issues. All of these circumstances combined with potential continued work in the development of drinking water and wastewater regulations point to future increased demands that are likely to be placed on the EPHP.

A draft report from the Workload Assessment Working Group found that there are significant shortages of EHOs when workloads were assessed against requirements in meeting the minimum standards set out in the National Framework for the Environmental Public Health Program in First Nations Communities South of 60. The report noted that this shortage could be impacted by various factors, such as time limited FNWWAP contributions (currently funding approximately 40% of EHOs); possible expansion of EHO duties due to environmental changes; the development of a regulatory framework for water and wastewater; and, other human resources factors that are common across the federal government (e.g., anticipated retirements).

Interviews with HQ representatives noted some concerns with respect to their oversight role in understanding and reporting on EPHP efficiencies at a national level given the changes in budgeting and allocation approach (i.e., FNIHB's approach to zero-based budgeting using regional management operational plans) combined with the shift of some administrative responsibilities to the regions. Some regional representatives indicated that the changes in budgeting approach and shift of administrative responsibilities could have created inefficiencies for the EPHP at the regional level as they are required to perform more administrative aspects of program delivery.

Also potentially impacting efficiency is the finding from the key informant interviews with EPHP representatives regarding a perceived lack of clarity of roles and responsibilities between HQ and some regions (as well as provincial health authorities). According to some HQ and regional representatives, there is no clear definition or direction at this point with respect to leadership and responsibilities. Regional representatives varied considerably in their characterization of the relationship between HQ and regions as ranging from "clear and functional" on the one hand to "strained" on the other. It was noted in interviews that regions and HQ have different reporting needs that can lead to different perspectives on program priorities (e.g., changes to the data systems to meet the national reporting needs of HQ versus the more immediate delivery objectives of regions). For example, having all Regions on one database/information system would, from HQ's perspective, simplify and improve efficiency and accuracy of reporting on national performance and public health indicators. However, some Regions prefer different databases because these better suit the needs and practices in place in their Regions. Many HQ and regional key informants noted that until recently (March 2016), there have been no opportunities to meet in-person, leading to an erosion of peer relationships given less direct, ongoing contact between regions and HQ.

Performance Measurement

There have been improvements in data collection and quality control over the past five years; however there remain challenges with the comprehensiveness and utility of data collected through the various administrative and monitoring systems.

EPHP has prepared an annual performance report on various national-level indicators for four of the five years within the evaluation timeframe. In 2012-13, FNIHB finalized the EPH Indicator Reporting Strategy with the goal of increasing the evidence base on EPH conditions and potential risks in First Nations communities on reserve. This strategy was implemented for the 2012-13 and 2013-14 annual reports. From a review of documents, the annual reporting for 2014-15 appears to have reverted back to be more closely aligned with specific outputs/activities according to the logic model developed for the EPHP (similar to 2011-12).

Reports from key informants, which were confirmed with the review of performance data, indicate that there have been some improvements in data collection and quality control over the past five years with respect to EHO activities in communities. There remain challenges with comprehensiveness and utility of data collected through the various administrative and monitoring systems (outside of drinking water monitoring), with reporting from these systems having been variable over the period covered by the evaluation.

Many program representatives interviewed said that there should be increased efforts in improving performance data. The evaluation found that areas cited that could be improved included:

addressing data gaps with contributors (e.g., INAC, First Nations communities that have entered into a Health Funding agreement);
placing more emphasis on results in addition to outputs that would support national reporting on outcomes (particularly related to recommendations);
structure data systems so that they allow for data mining capabilities and potentially linking to other data sources such as GIS – this would make the data more useful in answering questions, planning and allocating resources; and
ensuring that there is capacity within data systems for a regional/community lens to support a more proactive and risk-based service delivery approach.

5.0 Conclusions

5.1 Relevance Conclusions

5.1.1 Continued Need

The EPHP is designed to address a continued and growing need among many First Nations communities to identify, address and prevent human health risks associated with exposure to hazards within the natural and built environments. Environmental public health is an essential component of public health systems, and is a key contributor to improving health and well-being of populations. As was identified in the previous evaluation, many First Nations communities continue to experience significantly challenging conditions with respect to environmental health risks compared with many other Canadian communities. These ongoing challenges result from issues such as sub-standard housing and living conditions, poor quality or maximum capacity water and wastewater systems, lack of certified water plant operators, climate and geography, and numerous socio-economic inequalities.

5.1.2 Alignment with Government Priorities

As was found in the previous evaluation, the EPHP is designed to align with federal priorities and strategic outcomes sought for First Nations communities. Furthermore, the EPHP is aligned with Health Canada's strategic outcome of First Nations and Inuit communities and individuals receiving health services and benefits that are responsive to their needs so as to improve their health status.

5.1.3 Alignment with Federal Roles and Responsibilities

Since the previous evaluation, the Government has passed the Safe Drinking Water for First Nations Act (2013). The introduction of this Act moves the federal role in EPH risk mitigation beyond advisory to include a possible regulatory role. During this evaluation time period, safe drinking water regulations for First Nations were under development; however, as of 2016, the Act became subject to review by INAC Ministerial direction, and regulatory development was put on hold. Health Canada continues, therefore, to play an advisory role which varies in each region in order to meet specific community needs and priorities.

5.2 Performance Conclusions

5.2.1 Achievement of Expected Outcomes (Effectiveness)

The EPHP has clearly demonstrated progress in the achievement of immediate outcomes over the past five years. The findings indicate that there have been further improvements in timely data collection, the results from which are then disseminated at various levels either through recommendations or other types of reporting; an increased use of evidence to guide EPH activities with relevant research being conducted in communities to address EPH priorities; and an overall increased awareness of EPH risks among First Nations communities. This awareness has contributed to an increased use of and demand for EPHP services in communities.

Similar to the findings from the previous evaluation, although progress on immediate EPHP outcomes is quite clear, the extent to which intermediate outcomes have been achieved is less clear. The steps required to mitigate an EPH risk are generally to initially identify, then assess, and then address (if required from the assessment). The evaluation findings indicate that there has been continued progress over the past five years with respect to First Nations communities capacity to undertake the first two steps (identify and assess) EPH risks, but less progress with undertaking activities required to address the EPH risks. While there are some areas where this third step is occurring (e.g., drinking water advisories, food safety issues), overall there remain challenges for communities to address the more complex EPH risks.

The role of the EPHP is to assist communities by providing training and education around EPH risks according to community priorities, developing recommendations for addressing EPH risk based on investigations, and reviewing infrastructure plans from a public health engineering perspective. Given this role, EPHP has the capacity to undertake the first two steps to identify and to assess EPH risks that often result in recommendations for a First Nations community. There is neither funding directly attached to the EPHP nor a mandate to actually undertake the third step of addressing an EPH risk, based on the EPHP recommendation.

The contingencies and factors involved in actually achieving the third step in the outcome of addressing EPH risks are numerous including community priorities and resource availability, all of which are external to the EPHP. This indicates that if the EPHP is expected to contribute to communities addressing their more complex EPH risks, the program will continue to have to coordinate and collaborate with the multitude of other stakeholders involved that assist First Nations communities in determining their priorities and obtaining resources to address EPH risks.

While the program logic would indicate that EPHP's progress in the achievement of immediate outcomes combined with the partial progress in achievement of intermediate outcomes should be contributing to the longer term outcome of improved health and well-being of First Nations, the evaluation was unable to actually assess this link accurately. From a theoretical perspective, this contribution would be occurring to the extent that EPH risks are actually being addressed in First Nations communities.

5.2.2 Demonstration of Economy and Efficiency

The program's strong, positive connections with First Nations communities and organizations, particularly at the community level, assist in ensuring the relevancy of the EPHP's services and advice. The EPHP is expending resources on EPH activities that are contributing to outcomes important to First Nations communities. There were demonstrated efficiencies in the past five years for the program, many of which were highlighted in the areas of service delivery at the regional levels.

A key component of the success of the EPHP is the EHO. While there were numerous areas of efficiency outlined for EHOs, there is a concern that the sustainability of their effectiveness may be at risk given the increasing workload in many of the regions, and the anticipated significantly greater demands being place on the EPHP due to increased awareness of EPH risks in many communities, increased funding for community infrastructure projects, and the ongoing growth and development of First Nations communities as they address EPH risks in various areas. The impact of workload pressures among EHOs is somewhat varied by region due to differences in capacity and priorities between First Nations communities.

Highlighted as an area for improvement in the previous evaluation, the performance measurement for the EPHP has improved during the past five years. Attempts were made at moving the reporting from strictly counting of outputs more into the realm of outcomes. In addition, there was work on the development of some of the data collection systems. Despite these improvements, there remain some challenges with the comprehensiveness and utility of data collected.