Shared Services Canada Five-Year Departmental Evaluation Plan 2021-22 to 2025-26
Table of contents
- Deputy Head Confirmation Note for All Departments
- Context and Background
- Evaluation Planning
- Evaluation Coverage Over Next Five Years
- Appendix 1 – Five-Year Evaluation Schedule
- Appendix 2 – SSC Programs and Planned Spending
- Appendix 3 – Examples of COVID-19 Response Issues and Questions
- Appendix 4 – Examples of Program Evaluation Questions
Deputy Head Confirmation Note for All Departments
I approve the Departmental Evaluation Plan of Shared Services Canada for the fiscal years 2021-22 to 2025-26, which I submit to the Treasury Board of Canada Secretariat as required by the Policy on Results.
I confirm that this five-year rolling Departmental Evaluation Plan:
- Plans for evaluation of all ongoing programs of grants and contributions with five-year average actual expenditures of $5 million or greater per year at least once every five years, in fulfillment of the requirements of subsection 42.1 of the Financial Administration Act,
- Meets the requirements of the Mandatory Procedures for Evaluation, and
- Supports the requirements of the expenditure management system including, as applicable, Memoranda to Cabinet, Treasury Board submissions, and resource alignment reviews
I will ensure that this plan is updated annually, and I will provide information about its implementation to the Treasury Board of Canada Secretariat, as required.
Paul Glover
President
Shared Services Canada
May 2021
Context and Background
Departmental Evaluation Plan
Purpose
This document presents Shared Services Canada’s (SSC) Departmental Evaluation Plan. The Plan is an information and management tool that reflects the results of the Department’s annual evaluation planning exercise. The Plan identifies evaluations for the next five years and the resourcing of the evaluation function.
The scope and content of this plan are consistent with the requirements of the Treasury Board of Canada (TBS) Policy on Results.
Government of Canada Context
Evaluation is a priority of the Government as demonstrated in Mandate Letter Commitments. Specifically, the President of the Treasury Board is charged with instilling a culture of evaluation, measurement and evidence-based decisions across departments for program and policy design and delivery.
Requirements of the Policy on Results
What is evaluation?
The systematic and neutral collection and analysis of evidence to judge merit, worth or value.
Evaluation looks at the relevance, effectiveness, and efficiency of programs listed in the Program Inventory.
The Policy on Results requires that deputy heads annually approve a five-year evaluation plan and provide it to the Treasury Board of Canada Secretariat (Section 4.3.15). The Plan should include:
- A detailed schedule of planned evaluations.
- The extent of evaluation coverage of the Program Inventory, by amount of organizational spending and by program of the Program Inventory.
- The extent of organizational spending and programs that will not be evaluated in the planning period, and justification (i.e. What has been excluded from the five-year schedule and why).
- Advice to the deputy heads on the adequacy of resources for evaluation.
Governance
The Performance Measurement and Evaluation Committee
The Policy on Results calls for deputy heads to establish and chair a Performance Measurement and Evaluation Committee (PMEC) of senior officials. At SSC, PMEC is conducted as part of the Executive Oversight Board (EOB), chaired by the President.
PMEC aims to strengthen the planning, coordination and use of performance measurement and evaluation within SSC, as well as to ensure appropriate resourcing of these functions.
The Policy on Results requires PMEC to:
- Review and advise the deputy head on the establishment, maintenance and implementation of the Departmental Results Framework (DRF), Program Inventory (PI), and Performance Information Profiles (PIPs)
- Review and advise the deputy head on availability, quality, utility and use of performance information including evaluation
- Review and advise the deputy head on departmental evaluation planning, resources and activities, including recommending approval of the departmental evaluation plan
- Review evaluation reports and summaries, including management responses and action plans, and recommend approval to the deputy head
- Review and advise the deputy head on the neutral assessment of the evaluation function
- Monitor follow-up on evaluation recommendations and action plans and advise the deputy head of issues
The Evaluation Function
Evaluation at Shared Services Canada
The Office of Audit and Evaluation (OAE) is a key provider of reasonable assurance and advice on departmental operations to the President SSC and the Comptroller General of Canada.
Reporting directly to the President SSC, the Chief Audit and Evaluation Executive (CAEE) makes recommendations on the relevance and performance of departmental programs, policies, and initiatives through the OAE evaluation function. The CAEE is also the designated departmental Head of Evaluation and represents the departmental link to the Treasury Board of Canada Secretariat for evaluation.
Evaluation Resources
Evaluation Capacity
The focus in fiscal year 2021-22 will be to grow and train the team. The initial capacity for 2021-22 will be sufficient to deliver one final evaluation report. The function will deliver 3 to 4 evaluation reports annually at full capacity.
SSC will continue to grow an evaluation capability to meet evolving needs. The OAE evaluation function will be resourced for 16 full-time equivalent (FTE) positions and an overall annual budget of $1.74M by the end of the 2021-22 fiscal year.
| Salary | O&M | Total |
|---|---|---|
| $ 1.59 M | $ 0.15 M | $ 1.74 M |
| EX-01 | AS-01 | EC-07 | EC-06 | EC-05 | EC-04 | total | |
|---|---|---|---|---|---|---|---|
| Establishment | 1 | 1 | 2 | 4 | 4 | 4 | 16 |
| Filled | 1 | 1 | 1 | 2 | 0 | 2 | 7 |
Evaluation Planning
Key Planning Considerations
The selection and sequencing of the evaluations was driven by:
- The need to address all policy requirements
- SSC’s impact on the Government of Canada COVID-19 response
- The need to review significant spending that has never been evaluated
- The maturity of Programs and their performance measures for results
- The maturity of service design and renewal
- Customer Satisfaction Feedback Initiative (CSFI) scores
- Prioritization of evaluations to inform decision-making on funding renewals
- Prioritization of evaluations that focus on externally-facing outcomes supporting partner departments and clients
- Programs with key initiatives supporting SSC 3.0 strategic priorities
- The allocation for evaluations of Internal Service programs
The Evaluation Planning Process
The Departmental Evaluation Plan is based on analysis and consultations conducted during the Winter of the 2020-21 fiscal year. This included:
- Document review: The review of various sources of information to identify any mandatory evaluation requirements and assess coverage of the Program Inventory. Key documents included the last approved 5-Year Plan, Performance Information Profiles, memoranda to Cabinet, Treasury Board submissions and more.
- Consultations with Branches: Thirty-five separate consultations, involving 127 participants, were held with Assistant Deputy Ministers, Directors General and other key stakeholders in order to review the timing and scope of proposed evaluations, and to confirm their usefulness and value.
- Coordination with Internal Audit. The plan is coordinated with the Risk-Based Audit Plan (RBAP) for Internal Audit engagements to reduce the burden placed on Branches from oversight activities.
- Consultation with the Treasury Board of Canada Secretariat. In accordance with the Policy on Results, the Head of Evaluation also consulted with TBS.
SSC Program Inventory and Components
Cloud
- Cloud services
Data Centre Information Technology Operations
- Enterprise operations
- Linux/ Unix
- Windows and virtualization
- Integrated high performance computing
- Mainframe and bulk print management
- Platform services
- Storage, backup and file services
- Facilities management
Telecommunications
- Conferencing services
- Contact centre infrastructure services
- Toll-free voice
- Mobile devices and fixed-line phones
Security
- Security management and governance
- Secret infrastructure
- Infrastructure security
Enterprise Services Design and Delivery
- Architecture
- Science program
- Service management
- Partner engagement
Networks
- Local area network (LAN)
- Wide area network (WAN)
- Satellite
- Internet
Workplace Technologies
- Email services
- Digital communications and collaboration
- Workplace technology software provisioning
- Workplace technology hardware provisioning
- Workplace technology services
Evaluation Coverage Over Next Five Years
Planned Evaluation Coverage
The Policy on Results introduced the requirement for a Program Inventory, which contains the entire list of programs in the Department. The Program Inventory helps demonstrate how the Department delivers on its mandate, supports the authorization of expenditures through Cabinet, and supports performance reporting.
The Program Inventory represents the entire evaluation portfolio. The Policy on Results requires all programs and planned spending in the Program Inventory (except for internal services) to be evaluated periodically.
The Evaluation team plans to conduct 14 evaluations over the next five years, representing all seven programs in the SSC Program Inventory. This represents 100% of policy coverage requirements.
Internal services are essential to the department’s success as key enablers of SSC 3.0, and two are included for evaluation.
Advisory Services
The evaluation team will also provide advice and assistance for the Police Infrastructure Operations Review and the Service Review in the 2021-22 fiscal year.
Programs Excluded from the Planned Coverage Calculation
Policy Compliance
100%
At the program level, there are no excluded operational programs.
Some SSC Programs are significant in size and will require more than one evaluation for complete spending coverage. For example, Data Centre Information Technology Operations has $588.6M in Planned Spending which would be $2.9B over five years. However the programs contain distinct and standalone components that are outlined in the Performance Information Profiles and captured in financial cost centre data. As a result, the Plan reflects a calibrated approach based on value for decision-making. Some of the planned spending associated with specific components within programs may be evaluated in future years.
Appendix 1 – Five-Year Evaluation Schedule
| Evaluation Name | Strategy2 to EOB (PMEC) | President Approval | Branch | Link to Program Inventory |
|---|---|---|---|---|
| SSC’s COVID Response | May 21 | Apr 22 | All | All |
| Mobile Devices and Fixed Line | Nov 21 | Jul 22 | NSDS | Telecommunications |
| Enterprise Data Centres | Apr 22 | Jan 23 | DCS | Data Centre Information Technology Operations |
| Evaluation Name | Strategy to EOB (PMEC) | President Approval | Branch | Link to Program Inventory |
|---|---|---|---|---|
| Contact Centre Infrastructure Services | Jul 22 | Apr 23 | NSDS | Telecommunications |
| Hosting Platforms and Services | Sep 22 | May 23 | DCS | Data Centre Information Technology Operations |
| Digital Communications and Collaboration | Jan 23 | Oct 23 | NSDS | Workplace Technologies |
Note 1: As required by TBS, evaluations are listed by the year in which they are scheduled for presentation to EOB (PMEC) and the President for approval. The Evaluation Strategy confirms the areas of focus and the evaluation questions with EOB to ensure the evaluation report will inform senior management decision-making.
Note 2: The strategy will present the planned scope and questions for each evaluation for discussion and endorsement by PMEC before the start of data collection.
| Evaluation Name | Strategy to EOB (PMEC) | President Approval | Branch | Link to Program Inventory |
|---|---|---|---|---|
| Cloud Services | Jul 23 | Apr 24 | CTO | Cloud |
| Service Management | Sep 23 | May 24 | OM | Enterprise Services Design and Delivery |
| Evaluation of an Internal Service: Funding Model | Oct 23 | Jun 24 | CFO | Internal Services |
| Networks | Apr 24 | Jan 25 | NSDS | Networks |
| Evaluation Name | Strategy to EOB (PMEC) | President Approval | Branch | Link to Program Inventory |
|---|---|---|---|---|
| Workplace Technology Software Provisioning | Sep 24 | May 25 | NSDS | Workplace Technologies |
| Infrastructure Security | Oct 24 | Jun 25 | NSDS | Security |
| Strategic Direction | Jan 25 | Oct 25 | CTO | Enterprise Services Design and Delivery |
| Evaluation of an Internal Service: (TBD) | May 25 | Feb 26 | TBD | Internal Services |
Appendix 2 – SSC Programs and Planned Spending
- Data Centre Information Technology Operations: $588,595,556
- Telecommunications: $248,205,689
- Internal Services: $242,101,984
- Networks: $331,504,663
- Enterprise Services Design and Delivery: $212,635,650
- Security: $149,215,353
- Workplace Technologies: $129,610,426
- Cloud: $6,185,713
Source: 2021-22 SSC Departmental Plan
Appendix 3 – Examples of COVID-19 Response Issues and Questions
Core Issues and Draft Questions for the Evaluation of the SSC COVID-19 Response:
- Relevance is the extent to which a program addresses and is responsive to a demonstrable need.
- How responsive was SSC to the changing COVID-19 context?
- What were the factors that supported or hindered responsiveness?
- Effectiveness examines the impacts of a program and the
extent to which it is achieving its
expected outcomes.
- To what extent was SSC effective in responding to the pandemic?
- Efficiency is the assessment of resource utilization in
relation to the production of outputs
and outcomes.
- Were the actions taken in the COVID-19 response suitable?
- What are the opportunities to improve efficiency?
Appendix 4 – Examples of Program Evaluation Questions
SSC evaluations will incorporate recent international trends in evaluation and best practices. They can also examine alignment with government priorities and Enterprise 3.0:
- Relevance is the extent to which a program addresses and is
responsive to a demonstrable need.
For SSC, the focus will be on the extent to which the objectives and design
of a program respond to the needs of
partner departments and continues to do so as circumstances change.
- To what extent has the program responded to changing client needs and priorities?
- Is the program aligned with emerging technology trends and best practices?
- Alignment is a component of relevance and it assesses the
linkages between objectives and
government priorities.
- To what extent has the program streamlined and moved towards an enterprise approach?
- Is there an effective engagement process in place to guide SSC in the prioritization of work with clients departments and relevant partners?
- Effectiveness examines the impacts of a program and the
extent to which it is achieving its
expected outcomes. The focus is on immediate and intermediate outcomes, i.e.
the “so what”?
- What is the level of satisfaction of partner departments and other clients regarding the program?
- Has the program achieved the desired outcomes (for example, reducing risk, achieving outcomes for partner departments and CIO clients, and other benefits from modernization)?
- Efficiency is the assessment of resource utilization in
relation to the production of outputs
and outcomes.
- What are the opportunities to improve efficiency?
- What are the innovative practices implemented by SSC to modernize programs?
Some engagements may include support from Internal Audit for lines of evidence that focus on risk:
- Is there a good alignment and understanding of the enterprise risks in the SSC program?
- What is the progress achieved by SSC in reducing risks associated with the program?
Page details
- Date modified: