Policy on Results: What is Evaluation?

On this page

• Purpose
• Who
• How
• Evaluation and Performance Measurement

Purpose

Evaluations typically consider one or more issues including whether what is being evaluated is relevant and responsive to a demonstrable need; the results it is producing and whether it is doing so effectively; its efficiency; and how it can be improved. Evaluations can target programs, initiatives, policies, priorities, projects, or other subjects, as needed. They:

• Help program managers and executives improve the achievement of results in their program or department
• Inform Memoranda to Cabinet, Treasury Board Submissions, and expenditure management generally
• Support communication to Parliament and Canadians of the results departments are achieving
• Facilitate innovation and learning within organizations

Who

Large Departments must:

• Have an evaluation function, led by a Head of Evaluation and overseen by a Performance Measurement and Evaluation Committee (PMEC) chaired by the Deputy Head
• Have a five-year evaluation plan based on the principle that all spending should be evaluated periodically and then target evaluations strategically to focus on risks, needs and priorities, providing a rationale for what will not be evaluated, where applicable^{Footnote 1}

Small Departments:

• Must have a Head of Evaluation and a PMEC
• Need not have a formal evaluation plan, but must annually conduct an evaluation planning exercise to identify the department’s five-year evaluation needs by considering information needs to support management decision making, other planned departmental oversight activities, and joint evaluations with other departments

Evaluation functions must be internal to the department but outside of program areas. The credibility and utility of evaluations is derived from respectful collaborative relationships in which Heads of Evaluation and evaluators demonstrate neutrality in behavior, process and decisions, and actively apply evaluation standards, and individual values and ethics. Evaluation reports are approved by the Deputy Head. Centrally-led evaluations can be conducted as necessary.

How

Evaluations typically use social science methods such as document reviews, interviews, surveys, case studies, focus groups, statistics and randomized control trials. They can look at a wide variety of sources to gather information, ranging from program beneficiaries to social media to predictive analytics based on big data. They can be conducted before a program is implemented, to inform its design; during a program, to help it adjust course; or after a program is complete or has been running for some time, to assess its impact.

Common types of evaluations include:

• Impact evaluations, which focus on assessing the results of the program and understanding whether the program caused the results
• Economic evaluations, which consider whether the program was cost-effective, or whether the costs justified the benefits
• Innovation evaluations, which assess pilots where departments are experimenting with new approaches and want to determine efficacy and the feasibility of taking pilots to scale
• Delivery/Process/Formative evaluations, which consider how to improve the implementation or delivery of the program
• Theory of Change evaluations, which explore and test the underlying assumptions of why and how a program works
• Design/Needs evaluations, which aim to develop or assess program design, or redesign an existing program

Evaluation and Performance Measurement

Performance measurement and evaluation are part of the same continuum and help support each other. Heads of both functions report to the Performance Measurement and Evaluation Committee, and both functions help inform program managers, senior management, central agencies, Parliament and Canadians.