Internal Audit – Major Project Governance

Final Report

Audit, Evaluation, and Risk Branch

January 2023

Executive summary

The Canada Revenue Agency (CRA) defines major project investments as projects with planned spending in excess of $1 million in any one fiscal year, regardless of the source of funding, and/or projects that have significant strategic, business, or operational risk to the CRA. The CRA's major project investments in fiscal year 2022 to 2023, including programmes, funds, and feasibility studies, accounted for approximately 5.7% ($306.2 million) of the CRA's budget. ^{Footnote 1}

Major project investments are typically identified in the CRA's Strategic Investment Plan, which is a 10-year plan of the CRA's portfolio of major investments and is approved by the Planning and Priorities Committee annually. New projects approved in the Strategic Investment Plan are subject to the gated Major Project Investment Oversight Process to ensure consistent application of principles for the successful delivery of projects.

The objective of the audit was to provide the commissioner, CRA management, and the Board of Management with assurance that the governance over major project investments is effective and supports the timely delivery of projects.

The audit concluded that the governance over major project investments is generally effective and supports the timely delivery of projects. The audit found that the investment planning process was aligned with strategic priorities and considered project interdependencies and sequencing via business and IT architecture assessments. The current governance structure provided adequate support and oversight of major project investments and an appropriate level of risk tolerance was determined for effective project oversight. However, the audit provides recommendations to address issues identified during the audit such as strengthening corporate committee functions, continuing to align the Major Project Investment Oversight Process with Agile and hybrid approaches, and formally integrating the Enterprise Business Architecture function into the CRA portfolio management process.

Summary of recommendations

• The Finance and Administration Branch, in consultation with the Digital Transformation Program Branch, should ensure that Enterprise Business Architecture is formally integrated into the CRA portfolio management process.
• The Corporate Secretariat, in collaboration with the Finance and Administration Branch, should:
  • update procedures and guidelines to ensure effective communication of the mandates and roles of the various corporate committees involved in the oversight of major project investments
  • update the document posting timelines identified in the Corporate Committee Governance Manual to ensure there is adequate time for committee members to review project information in order to provide a value-added oversight function and ensure adherence to the timelines
• The Finance and Administration Branch should ensure that project reports include information about stakeholder consultations and information on stakeholder roles and responsibilities.
• The Finance and Administration Branch should continue to explore opportunities to better align the Major Project Investment Oversight Process with Agile and hybrid approaches to project management.

Management response

The Finance and Administration Branch and the Corporate Secretariat agree with the recommendations in this report and have developed related action plans. The Audit, Evaluation, and Risk Branch has determined that they appear reasonable to address the recommendations.

1. Introduction

The Canada Revenue Agency (CRA) has an annual operating budget of more than $5 billion. The CRA's major project investments in fiscal year 2022 to 2023, including programmes, funds, and feasibility studies, accounted for approximately 5.7% ($306.2 million) of the CRA's budget and the total value of the investment portfolio was $1.154 billion as of July 2022. Project investments address the Government of Canada's priorities and the CRA's strategic priorities as identified annually in the Corporate Business Plan.

The major project investment process, referred to as the CRA portfolio management process, starts with the Strategic Investment Plan process. Every year, the Resource Management Directorate of the Finance and Administration Branch invites branches and regions to submit proposals for investment projects that support the achievement of the CRA's strategic objectives for the upcoming 10 years. The Resource Management Directorate reviews the proposals and recommends investment portfolio options to the Planning and Priorities Committee for approval. The resulting Strategic Investment Plan identifies major project investments over a rolling 10-year period to ensure that investment decisions stay aligned with changing Agency and Government of Canada priorities.

Approved project proposals are subject to the Major Project Investment Oversight Process. This process includes gates or decision points, requiring project sponsors to submit templated reports to corporate committees at the end of the following project stages:

• Business Problem / Opportunity Definition
• Options Analysis
• Detailed Planning
• Execution
• Close-Out and Benefits Realization

The purpose of a gate is to assess a project's readiness to proceed to the next stage. Under certain circumstances, project briefs or amalgamated reports can also be provided.

Within the CRA, there are different governing bodies responsible for the oversight of major project investments. These oversight bodies include the Board of Management, the Planning and Priorities Committee, the Transformation Oversight Committee, and the Resource Management Committee. The responsibilities of these committees are defined in Appendix A and the Major Project Investment Oversight Process is illustrated in Appendix B.

The Strategic Investment Planning and Oversight Division of the Resource Management Directorate supports the governance of major project investments by providing secretariat services to the Resource Management Committee. The division also provides guidance to project teams in the preparation of project reports and ensures that information reported to the oversight bodies is sufficient and reliable for decision-making.

The Corporate Secretariat provides secretariat services to the Planning and Priorities Committee, the Transformation Oversight Committee, and the Board of Management in support of the committees' oversight of major project investments.

2. Focus of the audit

This internal audit was included in the Board of Management approved Risk-Based Assurance and Advisory Plan 2021-2022, which was approved on March 31, 2021. The Assignment Planning Memorandum was approved by the Commissioner on January 26, 2022.

2.1. Importance

This audit is important because of the key role played by governance in ensuring that major projects are well-managed, deliver the expected benefits, and demonstrate good value for money.

The Government of Canada and the CRA have emphasized the importance of strategic planning and the management of investments, as well as the increased focus on transforming the organization in terms of technology and service delivery. Given this, an examination of the governance structure over the selection of major project investments and their governance is important to ensure that major investment projects contributing to the achievement of the CRA's strategic objectives are being approved and implemented as planned.

2.2. Objective

The audit objective was to provide the Commissioner, CRA management, and the Board of Management with assurance that the governance over major project investments is effective and supports the timely delivery of projects.

2.3. Scope

The audit examined the governance of active projects (including programmes, funds, and studies) within the new Major Project Investment Oversight Process^{Footnote 2} from February 5, 2020, to November 30, 2021.^{Footnote 3} It also assessed how projects were selected for entry into the Major Project Investment Oversight Process during the 2021 and 2022 Strategic Investment Plan processes.

The audit focused on corporate governance and included the Director General Steering Committee for the Strategic Investment Plan Process.

2.4. Audit criteria and methodology

The audit criteria and methodology can be found in Appendix C.

The examination phase of the audit took place from December 2021 to June 2022.

The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing, as supported by the results of the quality assurance and improvement program.

3. Findings, recommendations, and action plans

The Finance and Administration Branch and the Corporate Secretariat agree with the recommendations in this report and have developed related action plans. The Audit, Evaluation, and Risk Branch has determined that they appear reasonable to address the recommendations.

3.1. Portfolio planning

3.1.1 Corporate committees consider the interdependencies and sequencing of major project investments across the CRA as well as the alignment with CRA priorities when making investment decisions; however, Enterprise Business Architecture is not formally integrated into the CRA portfolio management process.

The CRA conducted an annual Strategic Investment Plan process to select and prioritize major project investments over a 10-year horizon. As part of the Strategic Investment Plan process, the Resource Management Directorate of the Finance and Administration Branch assessed and ranked proposals and projects in the current portfolio that were in the early stages using an initial ranking methodology, which was updated yearly to align with the changing priorities outlined in the Corporate Business Plan.

In reviewing documentation for the 2021 and 2022 Strategic Investment Plan processes, the internal audit team found that the initial ranking methodology included criteria to assess project interdependencies, resource capacities, and business and IT architecture. The Resource Management Directorate scored both proposals and projects in the early stages of the Major Project Investment Oversight Process based on feedback received from corporate stakeholders. After scoring, the investment options were submitted to the Planning and Priorities Committee for approval.

The internal audit team also found that in prior years, business architecture capacity was limited and existed only in a few branches. The situation improved for the 2022 Strategic Investment Plan process with the creation of a centralized business architecture function in the Digital Transformation Program Branch, which was included as one of the corporate stakeholders for the 2022 process. The Digital Transformation Program Branch was also in the early stages of drafting an Agency Business Capability Model with a goal to have a more scientific approach to strategic investments. However, Enterprise Business Architecture was not fully integrated in the selection and sequencing of proposals as this function was in its early development stage; therefore, it did not have the capacity to conduct an in-depth analysis for all proposals. Enterprise Business Architecture is a blueprint of the enterprise that provides a common understanding of the organization and is used to align strategic objectives and tactical demands. By not formally integrating Enterprise Business Architecture in the planning process, there is a risk that the resulting portfolio of projects might not be optimized to support the CRA's strategic direction.

Recommendation 1

The Finance and Administration Branch, in consultation with the Digital Transformation Program Branch, should ensure that Enterprise Business Architecture is formally integrated into the CRA portfolio management process.

Action Plan 1

The Finance and Administration Branch agrees with the recommendation and will continue to work with the Digital Transformation Program Branch to support the creation and implementation of the Enterprise Business Architecture function for the Agency.

The Enterprise Business Architecture and Technology Exploration Section will consult with branch leaders to validate the content, identify gaps, and seek endorsement of the Agency Business Capability Model by June 2023.

The Resource Management Directorate in the Finance and Administration Branch will work with Digital Transformation Program Branch to use the Business Capability Model as an additional lens for categorizing 2023 Strategic Investment Plan proposals and for providing another horizontal view of the investment portfolio by September 2023.

3.2. Portfolio delivery

3.2.1 The governance structure supports the timely delivery of major project investments, but roles and responsibilities of committees involved are not always clear to the committee members.

Major project investments undertaken by the CRA are identified in the CRA's Strategic Investment Plan. New projects selected via the Strategic Investment Plan process are invited to enter the CRA's Major Project Investment Oversight Process to enable informed decision-making.

During the audit period, the Planning and Priorities Committee, the Transformation Oversight Committee, and the Resource Management Committee were involved in the governance of major project investments. Each committee had the same membership (except for the chair of the committee), but had different mandates and responsibilities.

The internal audit team found that the corporate committees established risk tolerance levels for projects in the portfolio to support effective oversight. The corporate committees monitored projects in terms of scope, schedule, costs, benefits, and risks through a dashboard of light indicators presented at bi-weekly Resource Management Committee meetings and quarterly at Transformation Oversight Committee meetings. These dashboard light indicators expressed the corporate committees' risk tolerance regarding the management of projects. In addition, in 2021, the Resource Management Directorate introduced a new risk and complexity assessment model to require larger and/or riskier projects to be overseen by the Transformation Oversight Committee and the Board of Management.

The internal audit team observed that there were challenges with the scheduling of Transformation Oversight Committee meetings, resulting in project approval delays. To mitigate this situation, project teams sought approval from the Corporate Management Committee, which did not have a mandate for the oversight of major project investments.

Interviews with senior management also revealed that there was confusion about the mandates and roles of the committees involved in the oversight of major project investments, although the mandate of the Resource Management Committee was clearer than that of the Transformation Oversight Committee and the Planning and Priorities Committee.

The audit recommendation in this area of examination is presented below in Recommendation 2.

3.2.2 Sufficient and relevant information is generally provided to corporate committee members to support effective oversight and decision-making. However, project information is not always provided to committee members early enough to allow them to prepare for the corporate committee meetings. In addition, there are opportunities to improve the level of information related to stakeholder consultations.

As part of the Major Project Investment Oversight Process, project teams were required to obtain approval from the oversight committees at the end of each gate in order to proceed to the next gate. The project information presented to the committees commonly included project reports, portfolio dashboards, briefing notes, and outlines.

The internal audit team found that project reports were detailed and contained the relevant information to provide an adequate overview of the project. The Strategic Investment Planning and Oversight Division of the Resource Management Directorate, as the Resource Management Committee secretariat, provided guidance and adequately ensured the accuracy and quality of information. However, information provided on stakeholder consultation in the project reports was limited and the responsibilities of stakeholders were not clearly identified. Detailed information on stakeholder involvement is important to ensure optimal decision making and to mitigate the risk of any unintended impact.

Corporate committee members must review project information in advance of the corporate committee meetings. The CRA Corporate Governance Manual specifies that the full package of project information must be provided three business days prior to the meeting. The internal audit team observed that the full package of project information was not always provided on time, which could result in corporate committee members not having sufficient time to prepare for the meetings, impacting the oversight and quality of the challenge function over major project investments.

Recommendation 2

The Corporate Secretariat, in collaboration with the Finance and Administration Branch, should:

• update procedures and guidelines to ensure effective communication of the mandates and roles of the various corporate committees involved in the oversight of major project investments
• update the document posting timelines identified in the Corporate Committee Governance Manual to ensure there is adequate time for committee members to review project information in order to provide a value-added oversight function and ensure adherence to the timelines

Action Plan 2

The Corporate Secretariat and the Finance and Administration Branch agree with the recommendation.

The Corporate Secretariat will collaborate with the Finance and Administration Branch to provide on-demand information sessions to branches and regions in order to increase the understanding of the roles and responsibilities of corporate committees, including their involvement in the oversight of major projects. These sessions will also highlight the importance of adherence to document production deadlines in support of effective oversight. They will be offered on an ongoing basis and proactive offers of service will be sent quarterly.

A review of the existing Terms of Reference of all corporate committees will be conducted to ensure that their role in the oversight of major project governance is clear, where applicable. Further, existing document posting timelines will be examined to identify possible areas of efficiency and potential for improving the time made available to members for review.

The Corporate Secretariat will also review its internal procedures to add proactive and timely communication with agenda item sponsors. When a Request for Agenda Item is received, the Secretariat will respond with information regarding document production deadlines. The procedures will also be expanded to include instructions for monitoring the timeliness of document production to ensure consistency.

The review of the Terms of Reference and the review and modification of the internal procedures will be completed by March 31, 2023.

The Resource Management Committee Secretariat is scheduled to present the Resource Management Committee Secretariat Updates to the Resource Management Committee, which will include the findings of this audit. This presentation will also provide an opportunity to remind members of the timelines to submit documents. The target completion date for this action plan is December 31, 2022.

Recommendation 3

The Finance and Administration Branch should ensure that project reports include information about stakeholder consultations, such as information on stakeholder roles and responsibilities.

Action Plan 3

The Finance and Administration Branch agrees with the recommendation.

The Finance and Administration Branch will share the Resource Management Committee forward agenda with corporate stakeholders and the Major Project Investment Director General Steering Committee on a regular basis to ensure that they are aware of what projects are presenting. Project managers will be reminded of the requirement to consult with internal stakeholders, other affected branches and regions, and corporate support areas throughout the project lifecycle and each time a report is presented to the Resource Management Committee or other corporate committee, per section 5 of the Protocol for the CRA's Major Project Investment Oversight. Corporate support areas who are not consulted will be asked to raise this issue with the Resource Management Committee secretariat as well as with their assistant commissioner for awareness and action.

The Finance and Administration Branch recognizes that corporate committee members want assurance that stakeholders have been consulted at key stages of the Major Project Investment Oversight Process. Recent updates to the Major Project Investment Oversight Process templates have reaffirmed the importance of consulting corporate stakeholders throughout the project lifecycle. The list of stakeholders provided is not fully exhaustive and will differ from project to project.

The Resource Management Committee Secretariat will continue to invite analysts from the Enterprise Risk Management Division in the Audit, Evaluation, and Risk Branch to participate in peer reviews of early drafts of reports that will be presented to the Resource Management Committee or the Transformation Oversight Committee. This will continue to ensure that risk governance on major projects considers the impact to operations, program delivery, and employees, in addition to project risks.

This action plan will be completed by January 2023.

3.2.3 Expected project benefits are clearly defined, quantified, and tracked; and changes to the benefits were communicated to the corporate committees. After the implementation of a project, realized benefits are reported against the initial benefits identified and any gaps are investigated and explained.

The Resource Management Directorate made enhancements to its benefit management process for major project investments during 2018 to 2019. As part of the new process, the Resource Management Directorate introduced Business Discovery Workshops to help project teams identify and quantify initial benefits at the early stages of a project and then assisted the project teams with benefit maps and registers to track, refine, and finalize benefits throughout the project lifecycle. This process ensured the alignment of project outcomes with the CRA's strategic priorities.

The internal audit team found that the expected project benefits were clearly defined and quantified at the end of the Detailed Planning stage and were monitored continuously throughout the project lifecycle. Project teams also communicated changes to the anticipated benefits during the Execution stage with corporate committees for acknowledgement or approval.

The internal audit team also found that in the Benefits Realization reports, the realized benefits of the projects were reported against the initial expected benefits identified in the Detailed Planning report. Project teams investigated any gaps and implemented measures, as applicable.

3.2.4 The current Major Project Investment Oversight Process is aligned with the waterfall approach, but opportunities exist to better align the Major Project Investment Oversight Process with projects following Agile and hybrid approaches.

The CRA's Major Project Investment Oversight Process is mature and aligned with the traditional waterfall methodology^{Footnote 4} . The Major Project Investment Oversight Process follows a funding and costing model that relies on a structured gating process requiring project teams to identify their project scope, costs, and schedule in the planning stage. However, the internal audit team noted that more than 80% of projects in the CRA's major project investment portfolio were following an Agile or hybrid (i.e., a mix of waterfall and Agile) approach.

Project teams using Agile methodology develop deliverables iteratively or incrementally throughout the lifecycle of a project; therefore, the scope, costs, and schedule are less certain at the early stages. In addition, project teams often perform Agile planning and execution activities during the same project stage. The internal audit team noted that this blend made it challenging for project teams to prepare project reports using existing waterfall-based tools and processes. The Resource Management Directorate modified the process to allow concurrent reporting of projects during the Detailed Planning and Execution stages, but the internal audit team found that this still resulted in duplication of information and difficulties in reporting to corporate committees.

The internal audit team conducted a benchmarking exercise^{Footnote 5} with other tax organizations to identify best practices in major project governance. The internal audit team observed that these organizations experienced similar issues with Agile projects, in particular with aligning these projects with the waterfall governance model. Through benchmarking and additional research, the internal audit team was not able to identify a generally accepted framework for governing projects in the context of public administration. The internal audit team recognizes the challenges of aligning the oversight gates of the waterfall approach with the system development approach of Agile projects. The traditional gated waterfall approach to oversight will likely continue to be required in some form, regardless of the project management approach followed, as it supports accountability and provides senior management with line of sight to ensure that projects continue to be good investments for the CRA. Given that the federal government's digital objective is to become "agile, open and user-focused"^{Footnote 6} and that the majority of projects in the current portfolio follow an Agile approach, the CRA should continue to explore how to best align the Major Project Investment Oversight Process with Agile.

Recommendation 4

The Finance and Administration Branch should continue to explore opportunities to better align the Major Project Investment Oversight Process with Agile and hybrid approaches to project management.

Action Plan 4

The Finance and Administration Branch agrees to continue exploring opportunities to support alternative delivery approaches, including Agile. The Finance and Administration Branch has increased its flexibility in recent years to be more nimble in supporting digital investments, creating programmes (Application Sustainability Programme, Security Sustainability Programme, and the Digital Acceleration Transformation Fund), and allowing projects to operate in multiple stages at once.

The Finance and Administration Branch will work with the Agency's Agile Centre of Expertise to develop best practices for major project investments using Agile and other alternative delivery approaches. This will include guidance on establishing effective project governance structures and artefacts. In addition, the Agile Centre of Expertise has developed a new course centred around "Starting an Agile Project aligned with the Project Gate Assessments Gating process". The Finance and Administration Branch will participate in the course pilot on November 14 and 15, 2022, and continue to work with the Information Technology Branch to strengthen the connection between the Project Gate Assessments Framework, the Major Project Investment Oversight Process, and Agile sprint planning and review.

The Finance and Administration Branch will also continue to work with other government departments through the Investment Planning and Project Management Community of Practice to share best practices and lessons learned. The CRA has presented to this community the lessons learned on a recently completed major project investment. The Project Management Centre of Expertise presented its Community of Practice to this committee in September 2022, and has been approached by other members to collaborate and share best practices.

In addition, the Resource Management Committee Secretariat is scheduled to present the Resource Management Committee Secretariat Updates, which will include the findings of this audit, to a Major Project Investment Director General Steering Committee and the Resource Management Committee. This presentation will provide an opportunity to inform members of the ongoing efforts, described above, being made to ensure the Major Project Investment Oversight Process remains flexible to accommodate projects using different approaches. The target completion date for this action plan is December 31, 2022.

4. Conclusion

The audit concluded that the governance over major project investments is generally effective and supports the timely delivery of projects. The investment planning process was aligned with CRA and Government of Canada priorities and considered project interdependencies and sequencing via business and Information Technology architecture assessments. The current governance structure provided adequate support and oversight of major project investments, and an appropriate level of risk tolerance was determined for effective project oversight.

However, recommendations were provided to address issues identified during the audit, such as improving governance by strengthening corporate committee functions, continuing to align the Major Project Investment Oversight Process with Agile and hybrid approaches, and formally integrating the Enterprise Business Architecture function into the CRA Portfolio Management process.

5. Acknowledgement

In closing, we would like to acknowledge and thank the Finance and Administration Branch and the Corporate Secretariat for the time dedicated and the information provided during the course of this engagement.

6. Appendices

Appendix A: Corporate committee responsibilities in relation to major project investments

Methodology

The methodology for examination included the following:

• Document and file review: The internal audit team reviewed and analyzed meeting minutes, records of decisions, plans, data, reports, policies, and procedures, including CRA, Treasury Board, and industry standards and guidance.
• Interviews: The internal audit team interviewed select management and staff at Headquarters.
• Data analysis: The internal audit team reviewed and analyzed CAS Project System data.

The audit approach provided management the opportunity to review and comment on the findings before they were finalized. The internal audit team held bi-weekly meetings with the Finance and Administration Branch throughout the audit to support effective exchange of information, provide progress updates, and discuss any issues.