Evaluation of the Clean Air Agenda’s International Actions Theme

Final Draft Report

Audit and Evaluation Branch March 2016

PDF (515 KB)

• Executive Summary
• 1.0 Introduction
• 2.0 Background or Context
• • 2.1 Program Profile
  • 2.2 Governance and Management
  • 2.3 Resource Allocation
  • 2.4 Expected Outcomes
• 3.0 Evaluation Design
• • 3.1 Purpose and Scope
  • 3.2 Evaluation Approach and Methodology
  • 3.3 Limitations
• 4.0 Findings
• • 4.1 Relevance
  • 4.2 Performance - Efficiency and Economy
  • 4.3 Performance - Effectiveness
• 5.0 Conclusions
• 6.0 Recommendations and Management Response
• Annex 1: Summary of Findings
• Annex 2: Governance Structures for the CAA International Actions Theme
• Annex 3: Program Logic Model

Report Clearance Steps

Acronyms used in the report

ADM

Assistant Deputy Minister

CAA

Clean Air Agenda

CCAC

Climate and Clean Air Coalition

CCI

Climate Change International

CCS

Carbon Capture and Storage

CED

Clean Energy Dialogue

CEM

Clean Energy Ministerial

CESD

Commissioner of the Environment and Sustainable Development

COP

Conference of the Parties

CTCN

Climate Technology Centre and Network

CTI

Clean Technology Initiative

DFATD

Department of Foreign Affairs Trade and Development

DG

Director General

DM

Deputy Minister

DOE

Department of Energy (U.S.)

DPR

Departmental Performance Report

ECCC

Environment and Climate Change Canada

FSDS

Federal Sustainable Development Strategy

G&Cs

Grants and Contributions

GHG

Greenhouse Gas

GMI

Global Methane Initiative

IAI

Inter-American Institute for Global Change Research

IPCC

Intergovernmental Panel on Climate Change

LULUCF

Land Use, Land Use Change and Forestry

MEF

Major Economies Forum on Energy and Climate

NFCMARS

National Forest Carbon Monitoring, Accounting and Reporting System

NRCan

Natural Resources Canada

PMF

Performance Measurement Framework

R&D

Research and Development

REDD

Reducing Emissions from Deforestation and forest Degradation

RPP

Report on Plans and Performance

SFT

Speech from the Throne

SLCPs

Short-Lived Climate Pollutants

UNFCCC

United Nations Framework Convention on Climate Change

USEPA

United States Environmental Protection Agency

Acknowledgments

The Evaluation Project Team would like to thank the individuals who contributed to this project, particularly members of the Horizontal Evaluation Steering Committee, as well as all interviewees who provided insights and comments crucial to this evaluation.

The Evaluation Project Team was led by Robert Tkaczyk, under the direction of the Environment and Climate Change Canada Evaluation Director, William Blois and included Urszula Adamik and Kevin Marple. The evaluation was conducted by Goss Gilroy Inc., with input from the Evaluation Project Team, Audit and Evaluation Branch of Environment and Climate Change Canada, and the Horizontal Evaluation Steering Committee, composed of program and evaluation representatives from Environment and Climate Change Canada and Natural Resources Canada.

Executive Summary

Context

This report presents the results of the evaluation of the Clean Air Agenda (CAA) International Actions Theme, conducted by Environment and Climate Change Canada’s Audit and Evaluation Branch between October 2014 and March 2015.

The purpose of the evaluation was to assess the relevance and performance of the CAA International Actions Theme and to support its funding renewal. It was also conducted to meet the requirements of the Financial Administration Act and the Treasury Board Policy on Evaluation.

The CAA International Actions Theme’s overall objective is to participate in international partnerships and negotiations to achieve the Theme’s two expected final outcomes: fair, effective and comprehensive international action to address climate change, and innovation in clean energy resulting in global and domestic economic and environmental benefits. The CAA International Actions Theme supports the Government of Canada’s broader initiative known as the Clean Air Agenda, and is delivered jointly by Environment and Climate Change Canada (which is the lead department) and Natural Resources Canada. The CAA represents a part of the government’s broader efforts to address the challenges of climate change and air pollution, with a view to building a clean and healthy environment.

The CAA International Actions Theme ensures that international obligations are met through three key areas of activity, which were the focus of the evaluation:

1. International Climate Change Obligations - the payment of contributions to the United Nations Framework Convention on Climate Change, the Intergovernmental Panel on Climate Change, and the Inter-American Institute for Global Change Research;
2. International Climate Change Participation/Negotiations - engagement in strategic international climate change consultations and negotiations, as well as related technical and policy analysis and advice provided in the context of clean energy, clean energy technologies and forest carbon issues. This includes the management of Canada’s fast-start financing as part of the Copenhagen Accord, but not the projects funded through this financing; and
3. Continued Engagement and Alignment with the United States / the Clean Energy Dialogue - activities that foster enhanced engagement and alignment between Canada and the United States on clean energy technology and innovation.

The total budget for the CAA International Actions Theme was $52.8 million from 2011--2012 to 2015-2016, divided between Environment and Climate Change Canada ($28.8 million) and Natural Resources Canada ($24.0 million). The Theme was originally resourced for $62.8 million over five years; however, $10 million associated with the Department of Foreign Affairs Trade and Development was eliminated as a result of decisions taken in response to Budget 2012.

The evaluation covered the period from fiscal year 2011-2012 to the first quarter of fiscal year 2014-2015 (i.e., April 1, 2011, to June 30, 2014). Given that the current round of CAA International Actions Theme funding addresses the five-year period ending March 31, 2016, consideration was given to the extent to which the program is on track to meet its 2015-2016 deliverables.

Findings and Conclusions

Relevance

The evaluation concluded that the international actions completed under the program were required in order to find comprehensive solutions to climate change as a complex global issue and to meet to discuss Canada’s international interests. The program is aligned with overall federal government priorities, as expressed in Speeches from the Throne and Budgets, as well as Departmental Strategic Outcomes and international commitments. The program is consistent with federal roles and responsibilities for international environmental agreements. Shared jurisdiction with provinces and territories means that they must be involved in implementation of such agreements and have a shared interest in the Theme’s outcomes.

Efficiency and Economy

The program design is appropriate, with separately delivered key areas of activity that contribute to similar outcomes (i.e., the Clean Energy Dialogue is delivered separately). There are a variety of governance mechanisms for various components of the program, with clear collaboration mechanisms between Environment and Climate Change Canada and Natural Resources Canada. Overall, governance is clear and appropriate, although provinces and territories expressed a desire for deeper engagement with the federal government. However, the overall strategy for the CAA International Actions Theme that guides it toward fair, effective and comprehensive international actions and innovation in clean energy is not apparent to all stakeholders, and there are further opportunities to promote Canadian achievements in international fora.

The Theme’s activities are generally thought to be efficient; in particular, the Environment and Climate Change Canada-led assessed contributions have relatively low administrative costs, and additional activities were undertaken to respond to changes in government priorities and additional demands.

Performance information related to the CAA International Actions Theme is collected regularly and reported publicly. Although performance targets are not identified, it would be difficult to define targets given the nature of the expected shared outcomes.

Achievement of Intended Outcomes

Overall, progress has been made toward achieving the CAA International Actions Theme’s expected outcomes. The Theme has fulfilled its financial obligations, promoted the Government of Canada’s climate change objectives in international fora, met a majority of the Government of Canada’s objectives (and the objectives of its key allies) for negotiations and agreements, worked with international partners to advance clean technologies through a number of initiatives such as the Clean Energy Ministerial and Clean Energy Dialogue, and engaged in bilateral and multilateral initiatives on climate change, short-lived climate pollutants, forest carbon, climate financing and clean energy that advance Canada’s overall climate change interests.

With respect to the Theme’s final outcomes, there is evidence to suggest that the Government of Canada’s objectives are being achieved in key United Nations Framework Convention on Climate Change meetings. Also, Canadian investments in clean energy have increased over the evaluation timeframe. However, it is difficult to ascertain the extent to which the Government of Canada’s (or any other country’s) interventions were influential in securing these objectives.

The CAA International Actions Theme was influenced by a variety of external factors that have impacted the program’s success and direction, including changes in the positions of other countries (particularly the United States, with whom Canada’s position has been historically aligned due to the high level of integration of the North American economy), and changes in the economic climate that can impact clean energy and technology investments.

Recommendations

The following recommendations are directed to Environment and Climate Change Canada (Assistant Deputy Minister (ADM), International Affairs Branch) and Natural Resources Canada (ADM, Canadian Forest Service; ADM, Energy Sector; and ADM, Science and Policy Integration) as the departments jointly responsible for the program.

Recommendation 1:

Improve opportunities for provinces and territories to support the departments’ Clean Air International activities.

Despite continued engagement through such mechanisms as the broad federal/provincial/territorial working group on international climate change, as well as issue-specific mechanisms such as the National Forest Sinks Committee, many provincial and territorial representatives feel that engagement is insufficiently frequent or comprehensive, and they are unclear how their inputs to international negotiations are used. Consequently, sub-national and regional fora have been used by non-federal entities to engage internationally and influence climate change negotiations separately from the federal government. Given the role that provinces and territories play in climate change and clean energy, their engagement can help to articulate region-specific issues and opportunities to more fully inform international positions. Existing mechanisms for engagement should be reviewed and strengthened and/or better communicated (as appropriate) so that provinces and territories have improved opportunities to support Clean Air International activities.

Recommendation 2:

Develop and communicate to stakeholders an overarching strategy that defines the mechanisms and processes by which the objectives of the CAA International Actions Theme are expected to be realized.

Despite the existence of a program logic model and various Government of Canada websites which list actions being taken internationally and domestically, there is no central communications tool for the Theme to communicate its strategy and objectives to partners and stakeholders. Provinces and territories, in particular, would find such information useful to help them understand what is to be achieved through the CAA International Actions Theme, how it will affect them, and how they can best contribute.

Recommendation 3:

Identify ways to showcase the achievements and expertise of Canada’s public and private sectors in order to better support international actions and demonstrate that Canada is taking action and achieving results.

The evaluation identified that there were opportunities to better publicize domestic achievements to the international community in such areas as clean technology, climate change adaptation areas and Canada’s Forest Carbon Budget Model. The design of the CAA, with separate international and domestic Themes, means that it is sometimes challenging to share information across CAA program areas in order to profile domestic achievements in international fora.

The responsible Assistant Deputy Ministers from Environment and Climate Change Canada and Natural Resources Canada agree with all three recommendations and have developed joint management responses that appropriately address each of the recommendations. The full management response can be found in Section 6 of the report.

1.0 Introduction

This report presents the results of the horizontal Evaluation of the Clean Air Agenda (CAA)’s International Actions Theme, which covers the timeframe from April 1, 2011, to June 30, 2014. The evaluation was conducted by Environment and Climate Change Canada (ECCC)’s Evaluation Division, Audit and Evaluation Branch, in fiscal years 2014-2015 and 2015-2016, with input from Natural Resources Canada (NRCan)’s Strategic Evaluation Division. The evaluation was identified in the 2014 Departmental Risk-Based Audit and Evaluation Plan. The evaluation was conducted in order to meet the coverage requirements of the Financial Administration Act and the Treasury Board of Canada Secretariat Policy on Evaluation, which require that an evaluation of all ongoing grants and contributions (G&Cs) and direct program spending be conducted at least once every five years

2.0 Background or Context

2.1 Program Profile

The International Actions Theme is part of the Government of Canada’s broader initiative known as the CAA. The CAA addresses climate change and air pollution at the domestic, continental and international levels and involves 11 federal departments and agencies which provide programs organized under five themes:

1. clean air regulatory agenda (CARA);
2. clean energy;
3. clean transportation;
4. international actions; and
5. adaptation.

The overall objective of the CAA International Actions Theme, approved in 2007 and renewed in 2011, is to participate in international partnerships and negotiations to achieve the Theme’s two expected final outcomes: fair, effective and comprehensive international action to address climate change, and global and domestic economic and environmental benefits resulting from innovation in clean energy.

ECCC is the lead department for the CAA International Actions Theme, in partnership with and including the activities of NRCan. It includes three key areas of activity which are detailed below.

1. International Climate Change Obligations

This area focuses on actions related to compliance with existing treaties, including payment of assessed contributions to the United Nations Climate Change Secretariat and contributions to other international climate change initiatives through G&Cs. As part of this program element, Canada provides annual assessed contributions to international organizations that play a key role in enhancing the analysis and assessment of options related to the development of a future climate change agreement under the United Nations Framework Convention on Climate Change (UNFCCC), as well as to the UNFCCC itself, the Intergovernmental Panel on Climate Change (IPCC) and the Inter-American Institute for Global Change Research (IAI) to support their work in providing policy-relevant science information on climate change. The evaluation covered all assessed contributions under this key area of activity.

2. International Climate Change Participation/Negotiations

This area focuses on Canada’s international activities related to climate change. Such activities involve engagement in strategic international climate change consultations, as well as discussions and negotiations across a number of multilateral fora on behalf of the Canadian Government. Such efforts aim to advance Canada’s national interests in the negotiations under the UNFCCC and in international processes that complement the UNFCCC negotiations.

Efforts related to the UNFCCC revolve in large measure around negotiations under the 2011 Durban Platform for Enhanced Action toward a new, global post-2020 framework agreement. The purpose of the framework agreement would be to ensure the widest possible cooperation by all countries and their participation in an effective and appropriate international response for the reduction of global greenhouse gas emissions. Key negotiating issues relate to post-2020 mitigation commitments, adaptation and means of implementation. These negotiations on a framework agreement are expected to be concluded at the Conference of the Parties (COP) 21 (France, 2015), with the new agreement to come into force in 2020. As part of these negotiations, there will be increased pressure on Canada and other developed nations to increase their support for developing countries’ efforts, including through the Green Climate Fund. Ongoing negotiations are likely to be needed after 2015 to elaborate aspects of the framework agreement.

This area involves ECCC and NRCan technical and policy analysis and advice provided in the context of climate change, clean energy, clean energy technologies, and forest carbon issues to support Canada’s approach to international climate change. This expertise is essential for developing an appropriate balanced Canadian position (i.e., reflecting domestic environmental, energy and natural resource interests) in the international climate change arena. ECCC is the overall lead for climate change negotiations and NRCan is responsible for representing Canada at international negotiations and meetings related to clean energy technologies (development, deployment, diffusion and transfer) and land use, land-use change and forestry (LULUCF) and reducing emissions from deforestation and forest degradation (REDD+) and for providing related policy support in these areas. For example, NRCan leads on technology negotiations under the UNFCCC process, and leads Canada’s engagement in the Clean Energy Ministerial (CEM) where Canada contributes to international collaboration on clean energy technologies.

In addition to the UNFCCC, ECCC and NRCan participate in a number of other fora, including the following:

• The Climate and Clean Air Coalition (CCAC) was established in February 2012, CCAC partners are working to address climate change, air quality and health issues by targeting Short-Lived Climate Pollutants (SLCPs) such as black carbon, methane and hydrofluorocarbons, which are responsible for about 30% of global warming. Canada is a founding partner.
• The Major Economies Forum on Energy and Climate (MEF) was launched on March 28, 2009. It is intended to facilitate dialogue among major developed and developing economies, help generate the political leadership necessary to achieve progress at the annual UN climate negotiations (i.e., COPs), and advance the exploration of concrete initiatives and joint ventures that increase the supply of clean energy while cutting GHG emissions.
• The Clean Energy Ministerial (CEM)^Footnote1 was launched by the U.S. in 2010. It brings together 23 major economies to work on collaborative initiatives aimed at advancing the development and deployment of clean energy technologies.
• The Global Methane Initiative (GMI) is a voluntary initiative that serves as an international framework to promote cost-effective recovery of methane and use it as a clean energy source.
• The REDD+ Partnership ended in 2014 and NRCan represented Canada in this forum which served as an interim platform for partner countries to scale up actions and finance to reduce emissions from deforestation and forest degradation in developing countries.

ECCC is the lead department on climate finance and oversaw the delivery of Canada’s fast-start financing^Footnote2. The evaluation did not cover the G&Cs funded through the Fast-Start Financing Initiative itself, as these will be covered as part of another evaluation. It did, however, examine ECCC’s management and oversight of the fast-start financing process

3. Continued Engagement and Alignment with the U.S. / Clean Energy Dialogue

The Clean Energy Dialogue (CED) fosters enhanced engagement and alignment between Canada and the U.S. on clean energy technology and innovation. The explicit goal of the project is to “enhance collaboration on the development of clean energy technologies to reduce greenhouse gas emissions and address climate change.”

The CED is managed separately from the international climate change participation/negotiations and obligations but contributes to related outcomes. As such, it is included within the CAA International Actions Theme for administrative, coordination and reporting purposes.

To advance the CED objectives, the U.S. and Canada established three bilateral government Working Groups to identify key opportunities for collaboration in each of the following priority areas:

1. developing and deploying clean energy technologies (with a focus on carbon capture and storage (CCS));
2. building a more efficient electricity grid based on clean and renewable electricity; and
3. clean energy research and development and energy efficiency.

A CED Action Plan II, which builds on the achievements of CED Action Plan I, was released in June 2012. Action Plan II originally set a two-year framework for the CED, but work continues under this plan, as reported in the CED’s Third Report to Leaders. In this action plan, a greater emphasis is placed on energy efficiency to take advantage of an expanded array of opportunities.

The CED Secretariat is led by ECCC and the U.S. Department of Energy (DOE), and working groups are composed of government officials from the DOE, NRCan, and ECCC. Canada’s Minister of Environment and the U.S. Secretary of Energy co-lead the CED. Initiatives typically involve NRCan, ECCC and DOE and include the participation, as required, of other government departments, representatives of the private sector, provincial, state and territorial governments, and academia.

2.2 Governance and Management

ECCC’s Climate Change International Directorate (CCI) delivers the CAA International Actions Theme in partnership with other ECCC and NRCan organizations. The Assistant Deputy Minister (ADM) International Affairs Branch is the program lead.

The following groups contribute to this Theme on behalf of ECCC:

• CCI is responsible for leading the implementation of the CAA International Actions Theme under the CAA, and delivering Canada’s climate change negotiations and participation in international for a.
• Americas Directorate (International Affairs Branch) is responsible for the Canadian CED Secretariat.
• Science & Technology Strategies Directorate (S&T Branch) is responsible for overseeing annual contributions made to international bodies and is co-lead for the CED’s Clean Energy Research & Development (R&D) and Energy Efficiency Working Group.

The following groups contribute to this Theme on behalf of NRCan:

• Energy Policy Branch (Energy Sector) is responsible for the implementation of the International Climate Change Negotiation/Participation program (energy component) and Canada’s participation in the Clean Energy Ministerial (CEM).
• Office of Energy Efficiency (Energy Sector) is co-lead for the CED’s Clean Energy R&D and Energy Efficiency Working Group.
• Electricity Resources Branch (Energy Sector) is responsible for the CED Electricity Working Group.
• External Relations (Science and Policy Integration) is responsible for leading the CED NRCan Coordination Office.
• Policy and Planning Branch (Innovation and Energy Technology Sector) is responsible for the CED Carbon Capture and Storage working group and co-lead for the Clean Energy R&D and Energy Efficiency Working Group.
• Policy, Economics and Industry Branch (Canadian Forest Service) is responsible for the Forest Carbon Policy subcomponent related to LULUCF and REDD+.
• Pacific Forestry Centre (Canadian Forest Service) is responsible for the Forest Carbon Monitoring subcomponent through the National Forest Carbon Monitoring, Accounting and Reporting System (NFCMARS).

The Director General (DG) Committee on International Actions is the overarching coordinating body for the Theme and convenes on a quarterly basis. The committee brings together DGs with responsibility for leading the implementation of different elements of the CAA International Actions Theme (see Annex 2 for a list of all implicated DGs) in order to provide a forum for information sharing and coordination.

ECCC has the overall lead for the CED, with the ADM of International Affairs Branch managing the Canadian Secretariat responsible for coordinating CED activities for the Government of Canada. NRCan’s ADM Science and Policy Integration has a supporting role and leads the coordination of the CED in NRCan (see Annex 2). The CED includes three working groups, overseen by ECCC’s ADM Science and Technology, NRCan’s ADM Innovation and Energy Technology, and NRCan’s ADM Energy Sector, with support from implicated DGs from NRCan and ECCC.

There are also a variety of partners and stakeholders involved in various aspects of the CAA International Actions Theme. The Federal, Provincial, Territorial Working Group on International Climate Change, established in September 2010, consults on Canadian policies related to international climate change negotiations and other complementary fora. A range of other groups and committees also exist; the key ones are described in more detail in Section 2.4.

2.3 Resource Allocation

The CAA International Actions Theme was allocated resources of $52.8 million over the five-year period from 2011-2012 to 2015-2016. Although the Theme was originally intended to receive $62.8 million over the same period, resources associated with the Department of Foreign Affairs Trade and Development (DFATD, then Department of Foreign Affairs and International Trade) were eliminated following Budget 2012 decisions (approximately $9.5 million over the five-year period). ECCC absorbed costs associated with administering G&Cs that were originally intended to be funded by DFATD, although DFATD continues to provide the actual funding for the G&Cs themselves. Table 1 provides an overview of the original total budget; Table 2 details the allocations for the Theme for ECCC and NRCan by key area of activity for the first three years of the program, which were the focus of the evaluation.

Table 1 : CAA International Actions Theme Allocations (2011-2012 to 2015-2016) ECCC

Source: CAA International Actions Theme original program documents.

Table 2: CAA International Actions Theme Allocations by Key Area of Activity (2011-2012 to 2013-2014)

Source: CAA International Actions Theme original program documents.

2.4 Expected Outcomes

The expected outcomes for the CAA International Actions Theme are presented in the logic model^Footnote4contained in Annex 3. The CAA International Actions Theme final expected outcomes are

• fair, effective and comprehensive international action to address climate change;^Footnote5 and
• innovation in clean energy resulting in global and domestic economic and environmental benefits.

3.0 Evaluation Design

3.1 Purpose and Scope

This evaluation was conducted to support funding renewal for the CAA International Actions Theme and to meet the coverage requirements of the Financial Administration Act and the Treasury Board Policy on Evaluation.

The evaluation focused on horizontal activities in ECCC and NRCan related to the CAA International Actions Theme and covered the activity streams detailed in Section 2.1. The theme also covered the related G&Cs to meet the requirements of the Financial Administration Act.

The evaluation covered the period from April 1, 2011, to June 30, 2014. Given that the current round of CAA International Actions Theme funding addresses the five-year period ending March 31, 2016, consideration was also given to the extent to which the program is on track to meet its 2015-2016 deliverables. The evaluation covered all assessed contributions that are part of the “International Climate Change Obligations” key area of activity. The evaluation did not examine G&Cs funded through the Fast-Start Financing Initiative, as these will be examined as part of another evaluation.

3.2 Evaluation Approach and Methodology

The following data collection methodologies were developed to address the evaluation issues and questions. Evidence gathered was then used to develop overall findings and conclusions^Footnote6.

Document review: An in-depth review of all relevant documents provided by ECCC and NRCan was conducted. Where gaps were identified or additional information was needed to supplement the evidence base, further requests for documents were made to the implicated departments. Documents were reviewed to identify the mandated requirements, goals and objectives, governance structure, authorities, activities, outputs and outcomes for each key area of activity. This data collection method contributed to addressing all evaluation questions.

Administrative data review: An in-depth review of departmental performance and financial data was also completed in order to determine the degree to which outcomes have been achieved and to assess the Theme’s economy and efficiency.

Key informant interviews: A total of 47 interviews were conducted with key informants identified by the departments involved in the CAA International Actions Theme. This included 6 preliminary interviews with internal stakeholders, followed by interviews with 6 internal program senior managers, 13 internal program managers and staff, 11 representatives of provincial/territorial governments, 5 industry and non-government organization stakeholders, and 6 international stakeholders^Footnote7.

Interviews were either conducted in person or by phone, according to the respondent’s preference and proximity to the National Capital Region. Interviews provided information on the relevance and performance of the program’s key areas of activity (i.e., effectiveness, efficiency and economy). All relevant stakeholder perspectives were considered to provide a balanced blend of views on program performance, with approximately 60% of interviewees external to the program and roughly 47% external to the implicated departments.

3.3 Limitations

The confidentiality of the UNFCCC negotiations, together wtih the direction provided by Cabinet to the Canadian delegation, somewhat constrained the evaluators’ ability to investigate the extent to which success in securing Canada’s stated interests in the Convention can be attributed directly or indirectly to efforts by the Government, as evaluators were not able to compare actual decisions made during international negotiations with Canada’s intended position prior to negotiations. As a result, the evaluation was limited in providing an independent perspective on the Theme’s performance, particularly how its negotiation activities influence the international negotiation process within and outside of the UNFCCC. External key informant interviews were used to fill in this data gap.

The diverse nature of the activities, units and staff involved in the CAA International Actions Theme also meant that most key informants were only able to speak to one piece of the larger picture. As such, it was challenging to concisely communicate the findings for such a diverse program, when stakeholders each only possessed a limited perspective on program functioning. This challenge was met by meticulously limiting key informant feedback to only the narrow set of program activities with which they were familiar, and disparate and varied views from informants were noted in the findings in order to reflect the various inputs received about different program areas.

4.0 Findings

This section presents the findings of this evaluation by evaluation issue (relevance and performance) and by the related evaluation questions.

For each evaluation question, a rating is provided based on a judgment of the evaluation findings. The rating statements and their significance are outlined below in Table 3. A summary of ratings for the evaluation issues and questions is presented in Annex 1.

4.1 Relevance

4.1.1 Continued Need for Program

The evaluation found that international action is required in order to find comprehensive solutions to climate change as a complex global issue and to meet Canada’s international interests. Without the CAA International Actions Theme, significant gaps would emerge--most notably, a diminished influence on international climate change negotiations.

• There is a clear societal and environmental need to address climate change.
  • Climate change is an issue of significant concern that is already causing multiple impacts on human, economic and ecological systems. Recent reports from the IPCC^Footnote8, including a 2014 Synthesis Report and an Impacts, Adaptation and Vulnerability Report, underscore both the long-term risks of inaction on the issue and the importance of international action to address it.
  • Public opinion research^Footnote9 in Canada indicates that Canadians are concerned about climate change and believe that international and domestic action is required by governments to address it.
  • According to the Government of Canada’s website on Canada’s action on climate change, climate change is a long-term ongoing issue that requires coordinated action by nations around the world. The Government of Canada recognizes that to combat climate change, it must participate in global efforts and that financial and technical support must be provided to support international initiatives and programs.^Footnote10.
  • Greenhouse gas emissions from the energy sectors in Canada and the United States are the dominant source of domestic emissions, and represented a large majority of Canada’s (81%) and the United States’ (84%) total domestic emissions in 2012. Advancing the transition to low-carbon economies and combatting climate change requires scientific and technological innovation to reduce the “carbon content” of the energy sector^Footnote11.
• While provinces and territories are active on climate change and energy issues, the document review and interviews suggest that no other existing initiatives duplicate the objectives of the program.
• Gaps would exist in addressing the societal and environmental needs in the absence of the CAA International Actions Theme. Most notably, Canada’s ability to influence international climate change negotiations would be highly compromised in the absence of the program (including participation/negotiations, as well as mandatory payments. Program documents also suggest that, without initiatives like the CED, an important opportunity to accelerate collective progress toward a clean energy future would be missed^Footnote12.

4.1.2 Alignment with Federal Government Priorities

The program is aligned with overall federal government priorities, as expressed in Speeches from the Throne and Budgets. The Theme contributes directly to both ECCC’s and NRCan’s strategic outcomes and addresses priorities and commitments related to engaging with international partners by participating in global climate change negotiations, meeting Canada's international obligations, and working with the U.S. to advance clean energy priorities through the CED.

• The evaluation finds that the program is aligned with federal government priorities and Canada’s international obligations and interests.
  • Climate change is referenced in Budget 2011, as well as the 2013 Speech from the Throne (SFT), although the SFT reference relates more to domestic climate change issues than international negotiations.
  • The program is also aligned with the priorities of the Federal Sustainable Development Strategy (FSDS), which articulates Canada’s federal sustainable development initiatives for a period of three years (2013-2016), as required by the Federal Sustainable Development Act. The FSDS states that the Government of Canada will “engage with international partners by participating in global climate change negotiations and implementing Canada's commitments; work with the U.S. to advance clean energy priorities through the Clean Energy Dialogue; and address emissions of short-lived climate pollutants including through fora such as the Climate and Clean Air Coalition and the Arctic Council.”^Footnote13.
  • Canada is a signatory to the UNFCCC, with international commitments stemming from the 2009 Copenhagen Accord and the 2010 Cancun Agreement. Under the 2011 Durban Platform for Enhanced Action, countries are negotiating the establishment of a new, global, post-2020 agreement, which will include commitments by all major emitters to mitigate their greenhouse gas emissions. As a Party to the UNFCCC, the Government of Canada views this process as the most effective and inclusive way to achieve measurable and verifiable progress on climate change. For example, through this process, the Government of Canada will continue to support the Durban Platform and the establishment of a fair and comprehensive global climate change regime by 2015 that will effectively address global climate change and serve Canadian interests^Footnote14.
• There is extensive evidence that the Theme is designed to contribute directly to the specific objectives, priorities and strategic outcomes of both ECCC and NRCan to promote environmental protection and economic development. For example:
  • ECCC and NRCan Reports on Plans and Priorities (RPPs) and Departmental Performance Reports (DPRs) between 2011-2012 and 2015-2016 clearly demonstrate the Theme’s linkages to each department’s Strategic Outcomes and organizational priorities. In the case of ECCC, this includes development and implementation of bilateral and international agreements to address global greenhouse gas emissions, and coordination of Canada’s policy, negotiating positions and participation in relevant international fora of global significance. Activities within this program include implementation of the CED as a mechanism to support bilateral collaboration on clean energy technologies. In the case of NRCan, this includes its participation in international climate change negotiations and its work on forest carbon policy and monitoring and a clean energy technology (including the CED).
  • The Theme also contributes to departmental delivery of commitments made under the FSDS. ECCC and NRCan contribute to Theme I - Addressing Climate Change and Air Quality. Implementation strategies for international work are found under three sub-headings: Leading by Example, Enabling Capacity, and Advancing Knowledge and Communications.

4.1.3 Alignment with Federal Roles and Responsibilities

The program is consistent with federal roles and responsibilities for international environmental agreements. Provinces and territories have a shared interest in the outcomes of this work and can contribute to the implementation of such agreements given their shared jurisdiction over many areas of program activity.

• The CAA International Actions Theme mandate is aligned with federal government jurisdiction and, overall, there is evidence that involvement in international climate change activities is an appropriate role for the federal government.
  • The Government of Canada is responsible for providing leadership on international issues and has the authority to negotiate, sign and ratify international treaties^Footnote15.
  • Jurisdiction over environmental issues is shared in Canada, with the federal government having jurisdiction over environmental issues that cross international and provincial boundaries^Footnote16. Thus, there is a need to ensure coordination among the different levels of government in view of the fact that international agreements involve areas of implementation that fall under provincial/territorial jurisdiction, such as natural resources, energy and transportation.
  • The program is also consistent with the mandates and missions of ECCC and NRCan. The Canadian Environmental Protection Act (1999) indicates that “the Government of Canada must be able to fulfil its international obligations in respect of the environment »^Footnote17.
  • • Within the federal government, ECCC’s climate change responsibilities include reporting on current GHG emissions, estimating future emissions, leading international negotiations, developing and implementing bilateral agreements to address global greenhouse gas emissions (including the CED), and leading policy coordination for the provision of financial support to other countries, notably through the fast-start financing initiative^Footnote18. Additionally, ECCC has the lead on behalf of the federal government for the coordination of action on climate change with provincial/territorial officials.
    • NRCan supports ECCC in international negotiations on matters related to NRCan’s expertise (e.g., technology and forestry). NRCan is responsible for estimating emissions from Canada’s forests, provides leadership in forest carbon negotiations under the UNFCCC and undertakes forest carbon monitoring to fulfil mandatory UN reporting requirements. NRCan is also responsible for technology negotiations under the UNFCCC process^Footnote19.

4.2 Performance - Efficiency and Economy

4.2.1    Program Design

The program design is appropriate, with separately delivered key areas of activity that contribute to similar outcomes (e.g., the Clean Energy Dialogue). However, the overarching strategy that defines the mechanisms and processes by which the objectives of the CAA International Actions Theme are to be realized is not apparent to provincial, territorial and industry stakeholders, and there are further opportunities to showcase or profile Canadian achievements in international fora.

• Documentary evidence and key informant interviews indicate that the program design is appropriate and there is a logical link between program planned activities, outputs, and intended outcomes.
  • A logic model and performance measurement framework is in place to show how activities and outputs relate to advancing Canada’s international climate change and clean energy interests.
  • The current design of the Theme is being delivered as planned with a clear mandate to respond to international commitments and requirements of bilateral and multilateral engagements. Internal informants noted that it had reasonable objectives and the right mix of program activities, but also underscored that the program has demonstrated responsiveness to new conditions as they emerge, for example, by using existing resources to address the Clean Energy Ministerial (CEM), which emerged as a new priority in 2010, and the CCAC, which emerged as a priority in 2012.
• There are further opportunities to showcase or profile Canadian achievements in international fora.
  • Both internal and external key informants report that opportunities to showcase Canadian expertise in areas such as clean technology, climate change adaptation and bio-energy have been missed, as the program was not designed to sufficiently promote information sharing and communications among other related programmatic elements. It is felt that closer collaboration among the different CAA themes, federal departments (e.g., Industry Canada, Transport Canada), provinces/territories and industries involved with energy and clean technology would help profile these achievements internationally and promote what Canada is doing to address climate change and has to offer.
  • Some Canadian achievements, such as with the CCAC and CEM, have successfully been profiled and used in international work. For example, Canadian expertise in CCS and NRCan’s Forest Carbon Budget Model have both been shared internationally and are being used in many countries. Despite these successes, internal and external key informants believe there are more opportunities to profile and link international negotiations with Canadian technologies and expertise to further Canadian interests.
• External key informants suggested that the Theme could benefit from enhanced communication of an overarching strategy outlining how the Theme will achieve its overall objectives, as well as deeper inter-jurisdictional engagement. It is felt that this would better inform international negotiations and initiatives by understanding all the current plans, activities and capacities across Canada, and so ensure that the appropriate parties are onside to implement the results of international actions.
  • While the expected results for the Theme and descriptions of program key areas of activity for the Theme are available online, external interviewees questioned the Theme’s strategy of engaging internationally on the issue of climate change and expressed uncertainty regarding the Government of Canada’s planned GHG emission reduction targets and resulting strategies that will be put in place to achieve those targets. As well, provinces/territories would welcome more opportunities to contribute to developing these strategies and understanding how they will be expected to contribute to their implementation^Footnote20.
• The evaluation found evidence of federal/provincial engagement mechanisms on various issues related to activities undertaken as part of the Theme. For example, the National Forest Sinks Committee has been sharing information and providing joint analysis on technical issues for over ten years. This committee was highlighted by both internal and external key informants as an effective mechanism for FPT engagement. However, this Sinks Committee is limited to only a portion of the relevant work under this theme. While a broader federal/provincial/territorial (FPT) working group on International Climate Change exists ^Footnote21, des intervenants provinciaux et territoriaux qui ont indiqué vouloir un engagement plus provincial/territorial stakeholders who noted that they wanted more meaningful engagement with the federal government did not refer to this group as part of evaluation interviews.
• Provinces/territories desire more engagement on climate change with other jurisdictions. Provinces and territories are taking action outside of a nationally coordinated framework, and some view with skepticism federal requests for input given previous experiences in which they feel their contributions were not well used.
  • Many provincial informants commented that engagement is insufficiently formal, frequent, comprehensive or transparent overall, and they are unclear how their inputs to international negotiations are used. As such, provinces and territories have increasingly formed their own structures (with varying degrees of formality) to communicate amongst themselves and coordinate action domestically, separate from the federal government.
  • Sub-national and regional fora have become mechanisms to take inter-jurisdictional action. These include the Conference of New England Governors and Eastern Canadian Premiers (NEG-ECP)^Footnote22 , the Pacific Coast Collaborative^Footnote23, and the Western Climate Initiative^Footnote24. These initiatives involve only provincial and state representatives and they work to develop regional yet international (Canada-U.S.) strategies to address climate change. As well, the non-profit organization called The Climate Group^Footnote25 is a global network of sub-national government climate leaders that collaborate at an international level to demonstrate impact, share expertise and influence the global climate dialogue. It includes state, provincial/territorial and municipal governments and corporations. These groups are regarded as mechanisms that regional governments can use to understand what is happening internationally, engage in actions that may cross borders (e.g., emissions trading) and influence climate change negotiations.

4.2.2    Program Governance

For the most part, governance of the Theme is clear and appropriate. There are a variety of governance mechanisms for various components of the Theme, with clear collaboration mechanisms in particular between ECCC and NRCan.

• Clear governance is in place for the management and disbursement of funds to support targeted bilateral and multilateral cooperation on climate change, and for Canadian participation in the relevant UNFCCC and other multilateral bodies to influence international climate change and energy outcomes (e.g., CCAC, CTI, CED, etc.).
  • Key internal informants feel the management and governance of the Theme is appropriate. Furthermore, the accountabilities and decision-making processes are thought to be clear and well understood.
  • Clear mechanisms are in place to ensure that ECCC and NRCan coordinate and collaborate effectively on specific issues (e.g., to meet reporting requirements under the UNFCCC).
    • A Director-General (DG)-level committee, created in June 2014, is chaired by the DG of CCI (ECCC) and attended by six other DGs (three each from ECCC and NRCan). The committee, which meets quarterly, brings together DGs with responsibility for leading the implementation of different elements of the CAA International Actions Theme, in order to provide a forum for information sharing and coordination.
    • The document review provided evidence of a variety of governance/ management mechanisms that ECCC and NRCan participate in to achieve outcomes internationally (e.g., MOUs between the departments)^Footnote26. UA clearly articulated governance structure for the CED is in effect that includes both ECCC and NRCan involvement, as well as involvement by U.S. agencies (see Annex 2).
  • While most internal informants were satisfied with overall governance of the Theme and linkages between ECCC and NRCan, a few noted challenges with the interdepartmental nature of the Theme, highlighting information silos and missed opportunities to partner with other departments. However, a number of respondents felt that these relationships (i.e., between ECCC and NRCan) had improved in recent years and that more open communication was now taking place. With the large and diverse set of teams working under the CAA International Actions Theme, these mixed findings would simply suggest that in some areas collaboration and communication is stronger than in others. These respondents suggested more regular senior-level discussions about the Theme and more joint briefings to senior management on areas in which both departments have responsibilities and expertise. The new DG committee may address these issues, as this is part of its role.

4.2.3 Program Efficiency and Economy

The Theme’s activities under ECCC are deemed efficient given the level of expenditures, the additional activities undertaken (e.g., DFATD’s roles and CCAC) without commensurate funding, and the relatively low administrative costs to administer the assessed contributions.

• Table 4 and 5 below indicate the expenditures by ECCC and NRCan, respectively, under the Theme by program key area of activity and in comparison to the original budget for the first three years. The analysis follows.

Table 4: CAA International Actions Theme Budget and Expenditures for ECCC for Key Areas of Activity

Source: ⁽¹⁾ ECCC CAA International Actions Theme program documents. These are revised budgets ⁽¹⁾ and ⁽²⁾. The table excludes PWGSC accommodation allocation and employee benefits.

Table 5: CAA International Actions Theme Budget and Expenditures for Natural Resources Canada for Key Areas of Activity

Source: NRCan CAA International Actions Theme program documents. The table excludes PWGSC accommodation allocation and employee benefits.

• Overall, evidence indicates that the program was delivered below budget (see Tables 4 and 5).
  • For ECCC, a comparison of actual expenditures against revised budgets indicates that the program underspent by 3.4% to 9.6%. Delays in hiring and cancelled international events, mainly in the CED area, explain most of the funding lapses.
  • For NRCan, a comparison of actual expenditures against budgets indicates that the program underspent by 3.5% to 10%. These funds were either transferred to other programs to address pressures within the department, carried forward, and/or lapsed.
• There would appear to have been an increase in efficiency in program delivery over the period examined by the evaluation, as the production of outputs (e.g., number of meetings held, number of initiatives supported) has increased substantially over this time, despite no increase in funding reference level. Performance reporting indicates increasing levels of outputs from 2011-2012 to 2013-2014, and in some cases, by over 50%. This meant that more official submissions were presented and more meetings were attended each year to further Canada’s positions at international events (e.g., the number of UNFCCC meetings attended increased from 11 in 2011- 2012 to 23 in 2013-2014). Furthermore, the program also undertook new activities associated with the CCAC, in order to adapt to the change in government priorities.
• The discontinued role of the DFATD has impacted program delivery and forced efficiencies, primarily in ECCC. It was noted that ECCC has taken on additional roles in international negotiations previously played by DFATD without commensurate resources. For example, DFATD used to be Canada’s point of contact for processing all payments related to Canada’s assessed contributions to the UNFCCC, and was engaged in multilateral discussions on climate change in various international fora, including the UNFCCC and its subsidiary bodies.
• The program was affected to some extent by government-wide and departmental restrictions on travel, as well as reduced travel budgets, which internal interviewees indicated resulted in fewer representatives from Canada attending international fora. The evaluation did not find any evidence that this had a negative impact on the achievement of expected outcomes. Internal interviewees also mentioned that program staff now spend more time on administrative tasks related to approval of travel requests, which has resulted in travel being booked later than expected, thereby increasing the price paid for such items as airline tickets. Cost savings from sending fewer representatives may therefore have been offset somewhat by additional administrative burden.
• Another indicator used for the analysis of efficiency is the costs of administering G&Cs to support international obligations. The cost of administering assessed contributions represented less than 4% of the overall costs of the obligations, which is low compared to a range of charitable organizations^Footnote27. It should be noted that assessed contributions^Footnote28 require significantly less effort to prepare than non-assessed G&Cs. Although no direct program comparison exists within ECCC, the administrative ratios for the Atlantic Ecosystem Initiatives (which at the time of the evaluation had a closed competitive process to administer non-assessed contributions) was approximately 7.8%^Footnote29. Other ECCC G&C programs, which have an open competitive process to disburse non-assessed contribution funding, such as the Lake Winnipeg Basin Stewardship Fund and the Lake Simcoe Clean-up Fund, have a ratio of 0.22 and 0.15, respectively^Footnote30.
• A few internal interviewees mentioned that financial resources for the CAA International Actions Theme are provided with requirements to review and renew funds every five years. Climate change has, however, been an area of focus for the Government of Canada for many years (signatory to the UNFCCC since 1994), and it is expected to continue for many years (post 2020); in addition, the timing of major milestones in the UNFCCC process is not aligned with Canada’s five-year funding cycle. As a result, uncertainties in long-term resource allocation affect the program’s ability to maintain experience and capacities, and administrative review and renewal efforts can coincide with high program demand timeframes when the technical expertise and support is needed to deliver program outputs (e.g., around the time of a Conference of the Parties).
• When asked if there were opportunities to lower program costs, most respondents had difficulty identifying areas for further cost savings. There is limited ability to consider alternative delivery models due to the program mandate to participate in formal international negotiations and discussions, and to meet international, multilateral and bilateral commitments.

4.2.4 Performance Measurement

Performance information is being regularly collected and publicly reported. Although performance targets were not identified, it would be difficult to define targets given the shared nature of the expected outcomes.

• Activities, outputs and outcomes are well articulated in the logic model presented in Annex 3. A CAA International Actions Theme performance measurement framework (PMF) is in place (May 2014) and performance information has been collected and reported by program representatives in line with this framework for the evaluation. The vast majority of the data has been provided for the fiscal years 2011-2012, 2012-2013 and 2013-2014 to provide trend information. Where gaps existed, explanations are provided. This information indicates progress toward the expected outcomes; however, performance targets are not included to help determine if achievements are in line with expectations, given the shared nature of the expected outcomes.
• Appropriate performance information at the direct outcome level is regularly collected and publicly reported in various documents including National Communications on Climate Change, CED Reports to Leaders, FSDS Progress Report, and DPRs, as well as on the Climate Finance website. Canada is also meeting its reporting requirements to the UNFCCC in accordance with guidelines. This reported information relates to progress made in international negotiations, participation in international fora, the extent to which Canadian objectives were reflected in the outcomes of relevant international meetings, the types of international projects funded, and progress made under the CED working groups.
• While the PMF is up to date, a few improvements could be made to performance measurement in the future:
  • For the intermediate outcome, “International climate change agreements, statements, decisions and declarations are consistent with Canada’s objectives,” the proposed methodology to indicate performance (i.e., analyze and rate the results of negotiations) was not deemed feasible by the program and thus needs to be revisited. As well, this outcome is not clearly differentiated from the direct outcome that supports it, “Advancement of Canada’s climate change objectives in international engagement inside and outside the UNFCCC.” Effort should be made by program managers to improve the intermediate outcome and associated indicator to better measure Canada’s influence on key international decisions and agreements^Footnote31.
  • For the intermediate outcome, “Sustained engagement with other countries, international organizations and the private sector on climate change and/or clean energy,” the indicator is at the “output” level (number of meetings) and does not adequately speak to the achievement of this outcome. Effort should be made by program managers to measure not only engagement but also resources leveraged through initiatives such as the CED (e.g., contributions by others to clean energy via the CED).
• Little evidence was provided during the evaluation to indicate how performance data are being used in decision making to illustrate the value of the performance information; however, this information is used for public and accountability reporting and has been and will be used to support program renewal.

4.3 Performance - Effectiveness

Overall, progress has been made toward the achievement of all expected outcomes.

(i) Direct outcome 1: Acceptable

Fulfilment of obligations that enable Canadian participation in international negotiations and contribute to the advancement of Canada’s climate change strategy.

Canada fulfilled its obligations by paying all pledged contributions and meeting reporting requirements over the evaluation period.

• Financial obligations that were met over the evaluation period included (1) contributions to the core budget of the UNFCCC, (2) contributions to the Intergovernmental Panel on Climate Change (IPCC), and (3) contributions to the Inter-American Institute for Global Change Research (IAI). In fulfilling these obligations, the CAA International Actions Theme supported and gained participation in venues that Canada has committed to and that are valuable for promoting Canadian interests.
• As well as financial obligations, Canada also complied with its forest-related reporting obligations by annually producing estimates of the GHG emissions and removal balance of Canada’s managed forest, as well as by developing and continually improving Canada’s National Forest Carbon Monitoring Accounting and Reporting System.
• Canada also collects and reports on performance as per UNFCCC reporting requirements, which include annual National Inventory reports, National Communications produced every four years, and most recently, Biennial Reports illustrating emissions projections to the year 2030 by sector and type of gas. ECCC and NRCan collaborate in the production of these reports which are made available on the UNFCCC website.

(ii) Direct outcome 2: Acceptable

Advancement of Canada’s climate change objectives in international engagement inside and outside the UNFCCC.

Canada is promoting its climate change interests through active participation in UNFCCC and other related international processes.

• There is clear evidence of ongoing, sustained participation in UNFCCC processes, including the Ad Hoc Working Group on the Durban Platform for Enhanced Action (ADP) and subsidiary bodies. Canada has actively developed submissions, statements, and analyses related to key issues within the negotiations (such as LULUCF, technology and financing), both directly and in partnership with key allies. Key informants felt that, through the UNFCCC, the Theme is making progress in ensuring that a comprehensive climate change agreement will provide a level playing field for countries, in line with Canada’s interests.
• The document review and interviews also provide clear evidence of both concerted bilateral efforts with targeted countries (e.g., China, Mexico) and significant engagement and participation in multilateral fora that complement UNFCCC negotiations, including leadership in efforts to address short-lived climate pollutants (SLCPs) through the CCAC, clean energy (e.g., CEM) and forestry issues, all of which advance Canada’s climate change interests. Both ECCC and NRCan conduct ongoing analysis and research to support these initiatives.
• The advancement of Canada’s climate change objectives is measured as the percentage of Canada’s stated objectives met or mostly met through both international negotiations and agreements, as follows:

For ECCC:

• In 2012-2013^Footnote32, amongst the six streams of UNFCCC negotiations, on average 81% of Canada’s objectives were met and 100% of agreements (i.e., the Doha Climate Change Gateway) were fully consistent with Canada’s negotiating mandate.
• In 2013-2014, 71% of Canada’s objectives were met through negotiations that are working toward a global climate change agreement.

For NRCan:

• In 2011-2012, all objectives relating to NRCan’s advancement of clean energy technology were met, specifically at the 2011 Durban climate conference (e.g., supporting the operationalization and proper functioning of the UNFCCC technology mechanism and ensuring that no decisions are made that could negatively affect intellectual property rights). As well, the Durban agreement on LULUCF reflected roughly 90% of Canada’s objectives, including provisions to remove the impacts of natural disturbance from the accounting of national GHG emissions and the ability to more accurately account for carbon stored in harvested wood products.
• In 2012-2013, all objectives were met relating to NRCan’s advancement of clean energy technology. Parties agreed to the final steps needed to make the Climate Technology Centre and Network (CTCN) operational. NRCan also represented Canada in the CEM, advancing Canada as a leader in clean energy and working collaboratively with major economies to advance objectives on clean energy and technology. For LULUCF, 100% of UNFCCC decisions were in line with Canadian objectives and progress was made on financing and institutionalization to support REDD+, 90% of which were in line with Canada’s objectives.
• In 2013-2014, all objectives were met relating to NRCan’s advancement of clean energy technology. At the Warsaw climate conference, the Technology Mechanism was fully operationalized and the CEM leveraged partnerships to take advantage of growth in clean technology globally. Also, a pivotal agreement on the treatment of forests in developing countries was reached (the Warsaw Framework for REDD+) which was in line with Canada’s stated interests.
• Internal and external interviewees noted that in-house expertise in key areas (e.g., LULUCF and forest carbon accounting) has helped the successful promotion of Canada’s interests at the international level, and internal interviewees noted that, where Canadian objectives were not met, it was influenced by external factors such as lack of time or the fact that some countries were unwilling to reach decisions. Provincial/territorial and external stakeholders suggested there may also be a reduced ability for Canada to influence international outcomes resulting from views that Canada would not meet emissions targets set under the UNFCCC. As an example, some external informants indicated that Canada is being excluded from influential informal fora outside the UNFCCC (e.g., C2ES).^Footnote33. Nevertheless, external stakeholders were very complementary toward the high quality of federal staff working in the Theme, who were said to be knowledgeable, helpful and professional.

(iii) Direct outcome 3: Acceptable

Implementation of collaborative actions to advance clean energy technology and reduce GHG emissions.

Canada has worked with international partners to advance clean technologies through a number of initiatives such as the CEM and CED. These have increased clean energy collaboration between Canada and the U.S.

• The vast majority of actions planned for the CED have been implemented. A review of performance information shows that as of June 30, 2014, 73% of planned CED Action Plan II projects (27/37 projects) had been delivered. This evidence suggests the CED has made progress in implementing this direct outcome by fostering greater collaboration between researchers and policy makers on clean energy science and technology through CED projects (e.g., in the areas of carbon capture and storage, renewable energy and energy efficiency).
• Through the projects, the CED has engaged in technical and policy issues with stakeholders from different levels of government, industry, academia, and international organizations. The CED has also produced successful joint research projects (e.g., the North American Carbon Storage Atlas) and facilitated harmonized standards (e.g., Canada’s adoption of the U.S. ENERGY STAR® standard).
• Outside the CED, performance information shows that planned collaborative actions have taken place to advance clean energy technology and reduce GHG emissions (90% of planned actions were implemented in 2011-2012, 75% in 2012-2013, 83% in 2013-2014). This included progress in technology negotiations at the UNFCCC and participation in the CEM process. For example, NRCan’s leadership for the Government of Canada on the CEM has facilitated participation in 5 of the CEM’s 13 collaborative initiatives aimed at advancing a number of technologies related to energy efficiency and use^Footnote34.

(iv) Intermediate outcome 1: Acceptable

International climate change agreements, statements, decisions and declarations are consistent with Canada’s objectives.

While negotiations are ongoing, there is evidence that the Government of Canada’s climate change objectives are being reflected in key UNFCCC agreements and decisions.

• To date, the majority of Canada’s objectives (and the objectives of its key allies) are being reflected in international negotiations and agreements. For example, Canada’s main objective at the Conference of the Parties (COP) 19 in 2013 was to make progress toward the establishment of a new global climate change agreement that is applicable to all Parties. This objective was successfully met, with a clear process and timeline for all Parties to develop post-2020 commitments.
• Canada’s successes in influencing international climate change initiatives include CCAC/SLCPs, Climate Finance, and technology and forestry issues such as REDD+ and LULUCF. More specifically, Canada was instrumental in helping design and operationalize the CTCN under the UNFCCC through ongoing negotiations As well, NRCan’s Forest Carbon Budget Model has been made widely available for use internationally with training and support to provide international leadership in line with Canada’s main priorities.
• Success in getting Canada’s objectives reflected in international agreements or decisions is influenced by a variety of external factors, such as changes in the positions of other countries and in particular the U.S. with whom Canada’s position has been historically aligned due to the high level of integration of the North American economy. Changes in the economic climate also can impact energy and technology investments.

(v) Intermediate outcome 2: Acceptable

Sustained engagement with other countries, international organizations and the private sector on climate change and/or clean energy.

Canada is actively and increasingly engaged on climate change and, particularly, clean energy initiatives internationally.

• There is evidence that Canada is engaged in a comprehensive set of bilateral and multilateral initiatives on climate change, SLCPs, forestry, carbon capture and storage, climate financing and clean energy that aim to advance Canada’s overall climate change objectives and lever opportunities. Canada participates actively in the leadership of many of these initiatives, including the UNFCCC, CED and the CCAC.
  • Canada has been increasingly engaged with other countries, international organizations and the clean energy private sector, as assessed through the number of relevant in-person meetings and teleconferences, which increased from 19 in 2011-2012 to 21 in 2013-2014.
• Through the program, partnerships have been established and linkages created with colleagues in other countries (e.g., Norway), between sectors (non-profits and the private sector) and between provinces/territories.
• The private sector is engaged through its participation in the CEM, as well as engagement with the Climate Technology Initiative’s (CTI) Private Finance Advisory Network. However, some stakeholders feel that there could be increased engagement with the private sector on international climate change action.
• The Government of Canada fully delivered on its $1.2 billion commitment during the fast-start financing period, and the program delivered the management of ECCC’s portion of the total funding ($53.8 million) to ensure sustained engagement with other countries.
• The withdrawal of DFATD impacted Canada’s ability to maximize its international relationships under the Theme. This is because DFATD is involved in a much wider range of international fora and so is better positioned to promote Canada’s interests and share knowledge on other countries’ positions to enhance work under the Theme. Some key informants felt that the withdrawal of DFATD impacted Canada’s ability to maximize ECCC and NRCan’s relationships in bilateral and multilateral fora, as it is no longer able to share knowledge on other countries’ positions and does not play a policy role on international climate change and clean energy issues.

(vi) Final outcome 1: Acceptable

International action to address climate change that is fair, effective and comprehensive.

Overall, the Theme is making progress toward engaging all countries to promote global emission reductions, which is how the government defines fair, effective and comprehensive international action. Performance information related to “fairness and effectiveness” is not expected until 2015-2016.

• Canada has supported decisions by the Parties to work under the Durban Platform to develop a universal legal agreement on climate change by 2015. While negotiations are still ongoing, there is evidence to suggest that Canada’s climate change objectives (such as comprehensiveness) are being achieved in key UNFCCC meetings. For example, decisions at COP 17, 18 and 19, supported by Canada, established and advanced the Durban Platform for Enhanced Action that works toward a legal instrument applicable to all Parties. However:
  • Since an internationally legally binding agreement under the UNFCCC has yet to be reached, it is too early to draw conclusions about the extent to which this agreement will be fair, effective and comprehensive. Performance information is expected to be reported in 2015-2016.
  • It should be noted that the next UNFCCC agreement should not be defined as the final goal of international action to address climate change. Further bilateral and multilateral work, subsequent to the UNFCCC agreement being reached, is expected to contribute to international actions to address climate change (e.g., CEM, CTI, CCAC, etc.).
• That said, there is strong divergence of opinion regarding the extent to which this intended long-term outcome has been achieved. While internal stakeholders were more positive in expecting this outcome to be achieved, external respondent groups were more critical over such issues as a perceived lack of FPT engagement, and uncertainties over whether the Copenhagen target will be met, all of which are thought to diminish Canada’s influence at the negotiating tables.
• Nevertheless, external interviews indicate that federal negotiators have made strong contributions and have provided leadership in key areas. Their inputs were noted to be constructive, and staff members were praised by external stakeholders for their strong technical expertise and the quality of their work.

(vii) Final outcome 2: Acceptable

Innovation in clean energy resulting in global and domestic economic and environmental benefits.

While there is a great deal of evidence to suggest the program is pursuing activities that contribute to clean energy results, there is no direct evidence yet of the economic and environmental benefits of these investments.

• There is evidence^Footnote35 that the CED has deepened bilateral cooperation with the United States on clean energy, which is intended to contribute to enhanced productivity and global competitiveness of Canadian businesses, as well as improve dissemination of energy-efficient technologies in Canada.
• Actions taken within fora such as the CEM, to encourage and facilitate transition to a global clean energy economy, and through the CTCN, to enhance technology cooperation and transfer, contribute to achieving this final outcome.
• To assess overall trends in clean energy investments, performance information from the Bloomberg New Energy Finance database. Globally, investments ($US) have decreased slightly since 2011 ($319 billion in 2011, down to $270 billion in 2013, and back up to $311 billion in 2014)^Footnote36. However, Canadian investments in clean energy have increased ($7.1 billion in 2011, $6.6 billion in 2012, $7.5 billion in 2013, and $9.4 billion in 2014). Canada’s percentage of global clean energy investment has also increased from 2.23% in 2011 to 3.02% in 2014. Examples of successful innovations in clean energy include Canada’s work in CCS technology, such as for SaskPower’s coal-fired units.
  • Industry key informants indicated, however, that industry has yet to make many clean energy investments to address climate change due to uncertainty regarding which technologies would be considered acceptable to address future national requirements.

5.0 Conclusions

There is a continued need for international action to find fair, effective and comprehensive solutions to the global challenges of climate change, and Canada has committed to working internationally to find solutions that meet Canada’s needs and maintain its interests. The CAA International Actions Theme is aligned with this need and, without it, Canada’s ability to influence international climate change negotiations would be highly compromised. Initiatives like the CED and CEM provide the primary conduit for Canada to accelerate collective progress toward a clean energy future.

The Theme contributes to promoting Canada’s climate change priorities regarding environmental protection and economic prosperity. While the program is consistent with federal roles and responsibilities for international environmental agreements, provinces and territories have a shared interest in the outcomes of this work. According to evaluation evidence, the CAA International Actions Theme is aligned with international commitments made by the federal government and the strategic outcomes of ECCC and NRCan.

The design of the CAA International Actions Theme, while appropriate for achieving its intended outcomes, is limited in its ability to identify Canadian successes and expertise and then leverage these internationally to promote Canadian achievements in international fora. Furthermore, provinces and territories desire deeper engagement with the federal government, would like to have their input solicited earlier and would like feedback on how their input was used when preparing federal positions. For the most part, the governance of the Theme is appropriate for the internal program components. A performance measurement framework exists for the Theme and performance information is being collected and is publicly available. Partners and stakeholders also reported that they were not aware of the departments’ overarching strategy for delivering on the Theme’s mandate.

The evaluation found that the Theme is being efficiently delivered, given the level of expenditures, the additional activities undertaken (i.e., DFATD’s roles, CCAC) without commensurate funding, and the relatively low administrative costs to administer the assessed contributions.

Evidence suggests that progress is being made in achieving the expected outcomes of the CAA International Actions Theme. Financial obligations have been met, ongoing participation in climate change negotiations and international fora has advanced Canada’s interests, and ongoing collaboration with the U.S. has furthered Canada’s clean energy interests. There have been noted successes in leadership/expertise related to the work on short-lived climate pollutants, clean energy and forests. However, the CAA International Actions Theme is impacted by a number of external factors that can influence long-term success.

6.0 Recommendations and Management Response

The following recommendations are provided to Environment and Climate Change Canada and Natural Resources Canada as the departments responsible for the CAA International Actions Theme of the CAA

Recommendation 1: Improve opportunities for provinces and territories to support the departments’ Clean Air International activities.
Despite continued engagement through such mechanisms as the broad federal/provincial/territorial working group on international climate change, as well as issue-specific mechanisms such as the National Forest Sinks Committee, many provincial and territorial representatives feel that engagement is insufficiently frequent or comprehensive, and they are unclear how their inputs to international negotiations are used. Consequently, sub-national and regional fora have been used by non-federal entities to engage internationally and influence climate change negotiations separately from the federal government. Given the role that provinces and territories play in climate change and clean energy, their engagement can help to articulate region specific issues and opportunities to more fully inform international positions. Existing mechanisms for engagement should be reviewed and strengthened and/or better communicated (as appropriate) so that provinces and territories have improved opportunity to support Clean Air International activities.

Management Response to Recommendation 1

Recommendation 1: Improve opportunities for provinces and territories to support the departments’ Clean Air International activities.
The Assistant Deputy Minister from Environment and Climate Change Canada (ADM, International Affairs Branch) agrees with the recommendation.

Management Action

Provinces and territories are an integral partner in implementing actions to meet international climate change commitments made by Canada to reduce greenhouse gas emissions. Since 2010, Environment and Climate Change Canada has had formal engagement with provinces and territories and has operated based on the Terms of Reference (TORs) for engagement provided in Annex 1. For over a decade, NRCan / Canadian Forest Service has engaged frequently with provincial and territorial governments through the informal National Forest Sinks Committee on technical and policy issues related to land sector negotiations, forest-related greenhouse gas inventory estimation and accounting, and climate change mitigation analysis.

At the Canadian Council of Ministers of the Environment (CCME) meeting on June 22- 23, 2015, Ministers agreed to establish a Climate Change Committee. This will formalize decisions between federal, provincial and territorial governments and a commitment has been made to enhance current engagement.

Environment and Climate Change Canada (International Affairs Branch) will continue to lead general discussions related to international actions, but other active Departments such as NRCan will be engaged when their respective areas of expertise (forests, technology) are to be discussed, and mechanisms to engage with provincial and territorial counterparts on energy issues will be explored. Technical level meetings (ad hoc processes) led by Environment and Climate Change Canada will continue to be managed separately by responsible Directorates within Environment and Climate Change Canada.

The Clean Energy Dialogue (CED) has engaged over 20 provincial energy and environment agencies in its work, including partnering on projects, sharing information, and participating in workshops and conferences. The CED will continue this engagement moving forward, should funding be available.

Recommendation 2: Develop and communicate to stakeholders an overarching strategy that defines the mechanisms and processes by which the objectives of the CAA International Actions Theme are expected to be realized.
Despite the existence of a program logic model and various Government of Canada websites which list actions being taken internationally and domestically, there is no central communications tool for the Theme to communicate its strategy and objectives to partners and stakeholders. Provinces and territories, in particular, would find such information useful in understanding what is to be achieved through the CAA International Actions Theme, how this will affect them, and how they can best contribute.

Management Response to Recommendation 2

Recommendation 2: Develop and communicate to stakeholders an overarching strategy that defines the mechanisms and processes by which the objectives of the CAA International Actions Theme are expected to be realized.
The Assistant Deputy Minister from Environment and Climate Change Canada (ADM, International Affairs Branch) agrees with the recommendation.

Management Action

Environment and Climate Change Canada commits to developing an improved communication approach that will provide an overarching view of the program, including the elements that make up the program, the goals and objectives, and the expected outcomes where appropriate. This will be communicated to stakeholders, building on and to be delivered through existing mechanisms and channels. The information will be posted on ECCC’s website.

The CED strategic work plan will be integrated into the new communication approach.

Recommendation 3: Identify ways to showcase the expertise and achievements of Canada’s public and private sectors in order to better support international actions and demonstrate that Canada is taking action and achieving results.
The evaluation identified that there were opportunities to better publicize domestic achievements to the international community in such areas as clean technology, climate change adaptation areas and Canada’s Forest Carbon Budget Model. The design of the CAA, with separate international and domestic Themes, means that it is sometimes challenging to share information across CAA program areas in order to profile domestic achievements in international fora.

Management Response to Recommendation 3

Recommendation 3: Identify ways to showcase Canada’s public-sector and private-sector achievements and expertise in order to better support international actions and demonstrate that Canada is taking action and achieving results.
The Assistant Deputy Ministers from Environment and Climate Change Canada (ADM, International Affairs Branch) and Natural Resources Canada (ADM, Energy Sector) agree with the recommendation and have developed a joint management response.

Management Action

Canada has significant expertise in both the public and private sectors, and has been active in showcasing these internationally as part of the program. Canada offers considerable expertise in areas such as Canada’s Forest Carbon Budget Model and clean energy technologies. Canada seeks to identify international opportunities for Canadian firms through a number of international initiatives, such as the Climate Technology Centre and Network (CTCN) and the Clean Energy Ministerial (CEM).

In addition, and to further showcase Canadian achievements and expertise under this program, Environment and Climate Change Canada will continue to work with NRCan and the Department of Foreign Affairs, Trade and Development (DFATD) to reinforce partnerships with other departments including Transport Canada and stakeholders, expand the network by using other related international fora and missions abroad, broaden the knowledge of Canadian achievements and expertise, and increase awareness of opportunities for those public and private sectors that are active internationally.

Annex 1: Summary of Findings^Footnote37

Annex 2: Governance Structures for the CAA International Actions Theme

Annex 3: Program Logic Model