Follow-up Review on Project Management and Oversight

From: Employment and Social Development Canada

Official title: Follow-up Review on Project Management and Oversight.

On this page

• 1. Background
  • 1.1 Context
  • 1.2 Review objectives
  • 1.3 Scope
  • 1.4 Methodology
• 2. Review findings
  • 2.1 Clear criteria and definitions for both “projects” and “initiatives” across the Department have been drafted
  • 2.2 Boundaries of project program management framework have been expanded beyond the portfolio of investment projects to capture and govern other key initiatives within the Department
  • 2.3 Project management processes to allow for a scalable approach have been drafted
  • 2.4 Progress has been achieved with respect to improving the baseline knowledge of project management
  • 2.5 A comprehensive training and skills framework to build upon the training content and skills matrix has been established
  • 2.6 Controls to prohibit unskilled or untrained project resources from assuming responsibility or accountability for project management functions have been established
  • 2.7 Role and mandate of change management leader organizational readiness has not been officially supported by management
  • 2.8 Governance
  • 2.9 Benefits management framework and quality assurance
• 3. Conclusion
• 4. Statement of assurance
• Appendix A: Review criteria assessment
• Appendix B: Progress made vis-à-vis the actions identified in the approved management action plan
• Appendix C: Summary of testing results

Alternate formats

Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.

Acronyms used in this report

CMLOR

Change Management Leader Organizational Readiness

DAC

Departmental Audit Committee

DGPOC

Director General Project Oversight Committee

EARB

Enterprise Architecture Review Board

EPMO

Enterprise Project Management Office

ESDC

Employment and Social Development Canada

IA

Internal Audit

MAP

Management Action Plan

MPIB

Major Projects and Investments Board

PDO

Project Delivery Office

PMA

Project Management Advisor

PMCD

Project Management Competency Development

PMIS

Project Management Information Solutions

PMO

Project Management Office

PPMF

Project Program Management Framework

PPPM

Policy Program and Project Management

QA

Quality Advisor

TISMB

Transformation and Integrated Service Management Branch

1. Background

1.1 Context

Project management is the systematic planning, organizing, monitoring and controlling of allocated resources to accomplish identified project objectives and outcomes. It differs from operations as it is non-repetitive, time limited and renders a specific product or service. Effective project management is key to providing value for money and demonstrating strong stewardship in project delivery.

In recent years, project management was identified and acknowledged as a high-risk area within Employment and Social Development Canada (ESDC)’s corporate risk profile, and branches recognized project management as one of their top learning needs.

In 2014 to 2015, ESDC adopted the Treasury Board Project Management Framework to improve its project management maturity and capacity, including the development of its project management related processes; formalization of accountabilities, roles and responsibilities for key project stakeholders; and the establishment of the Enterprise Project Management Office (EPMO) mandate.

Internal Audit (IA) of ESDC conducted a review of Project Management and Oversight within the Department in 2017. The report was tabled to the Departmental Audit Committee (DAC) in November 2017. At the time of presentation to the DAC, senior management requested that IA conducts a follow-up review in 12 months, to assess the progress made towards the implementation of the approved Management Action Plan (MAP).

1.2 Review objectives

The objectives of the follow-up review were to:

• assess progress made with respect to the planned actions identified in the MAP  approved by the Deputy Minister for the 2017 Review of Project Management and Oversight
• assess the status of project management governance since the initial review that was conducted in 2017, and
• assess the Benefits Management Framework and the quality assurance

1.3 Scope

The scope of this follow-up review included a horizontal assessment of project management processes, practices and an assessment of individual projects. The scope also included a review of a sample of both completed and in-flight projects. In addition, we reviewed to what extent the approved MAP was implemented.

1.4 Methodology

This follow-up review was conducted using a number of methodologies and techniques which have included (but are not limited to):

• interviews with management and staff
• documentation review, and
• project file sampling

2. Review findings

2.1 Clear criteria and definitions for both “projects” and “initiatives” across the Department have been drafted

Through interviews and documentation reviews, IA has noted the following areas of strength with respect to the development and implementation of clear criteria and definitions for both projects and initiatives:

• progress was made specifically with the definitions, formalization and understanding of projects and initiatives. A presentation was delivered to the Director General Project Oversight Committee (DGPOC), February 26, 2019, on project versus initiative definition, criteria and project classification. This activity is a critical dependency to many other activities within EPMO (in other words, development of benefit realization and establishing a competency framework). At the conclusion of our fieldwork, we were informed that the governance level review of the definition, criteria and project classification was expected to be conducted at the Major Projects and Investment Board (MPIB) on April 15, 2019
• as evidenced by the ESDC Strategic Framework, a clear correlation between corporate projects and investment projects exists, which enhances strategic alignment
• a governance journey map exists. It maps out eight governance committees responsible for decision-making over different types of projects

Work has begun on the criteria for initiatives and projects, the outcome will provide a clear definition or criteria for both projects and initiatives to ensure proper rigour and oversight.

IA encourages EPMO to continue with the revision and update of scheduled MAP activities.

2.2 Boundaries of Project Program Management Framework have been expanded beyond the portfolio of investment projects to capture and govern other key initiatives within the Department

IA observed that progress has been made to ensure key initiatives are included in the EPMO’s inventory of supported projects and initiatives. Specifically, departmental initiatives (tied to the strategic priorities) have been mapped to investment projects.

IA noted the following areas for improvement with respect to the expansion of the PPMF beyond the portfolio of investment projects:

• the Benefits Directive has been developed. Multiple internal consultation activities between governance committees and project managers have taken place. It was noted that reaching an agreement between governance parties has been a challenge and has resulted in approval delays for the directive. As a result, the Benefits Directive has not yet been approved or deployed as originally planned for June 2017. The directive was approved on March 8, 2019

2.3 Project management processes to allow for a scalable approach have been drafted

Through interviews and documentation reviews, IA has noted the following areas of strength with respect to the design and implementation of scalable processes for project management:

• a draft document defining scalability and classification of projects exists. The document includes definition criteria based on risk levels, complexity and duration. The definition of scalability is found under Policy Program and Project Management (PPPM). The PPPM was presented to DGPOC on February 26, 2019. It is anticipated that the PPPM will be presented to MPIB  on April 15, 2019
• projects are proactively supported by Project Management Advisors (PMAs) and reviewed and evaluated by project Quality Advisors (QAs)
• PMAs are responsible for providing expertise and support function including assessing documentation completeness within projects, compliance against project management gating, and project risks and complexity. PMAs have also been responsible for facilitating benefit realization workshops with project managers. They are responsible for providing project management advice to the project managers throughout the end-to-end lifecycle of the projects. Interviewees indicated that PMAs are responsible for communicating material findings to QAs
• QAs have been responsible for interacting with the PMAs to provide feedback on the evaluation of project artefacts, without any direct communication with project managers. QAs have been responsible for making recommendations to MPIB whether project artefacts should be further enhanced and/or updated. Please see Appendix C: Summary of testing results for mandatory project documentation in Project Management Information Solutions (PMIS)
• within the PMIS, standard templates exist

However, IA also noted the following areas for improvement with respect to the design and implementation of scalable project management processes:

• timeliness of feedback provided by the QAs specifying areas for improvement over project management artefacts does not always permit project managers to have the time to address comments prior to MPIB governance gating reviews. It was noted by interviewees that opportunities exist to improve collaboration and communication amongst the project managers, PMAs and QAs collectively, such as having earlier and more frequent touch-points prior to governance presentations for DGPOC and MPIB
• interviewees indicated that project teams do not clearly understand the roles and responsibilities or value in their approach (no direct interaction with the project managers) of the QAs

IA encourages EPMO to continue with the revision and update of scheduled MAP activities and to further expand the current quality assurance to a project quality management system.

2.4 Progress has been achieved with respect to improving the baseline knowledge of project management

Through interviews and documentation reviews, IA has noted the following areas of strength with respect to the project management knowledge across management:

• IA observed that progress has been made in elevating the baseline of knowledge across management with respect to the value and core principles of project management. A training roadmap exists. Mandatory awareness training was implemented and is required to be completed by all individuals holding executive level positions
• a revised curriculum of training courses is currently being developed by the EPMO. These courses are being tailored to specifically align with project management role competency profiles per the PMCD Framework

However, IA also noted the following area for improvement with respect to project management knowledge across management:

• IA was unable to correlate between the current training requirements to courses that project managers have previously completed. A close correlation would demonstrate that project managers have satisfactorily met the current training requirements

IA encourages EPMO to continue with the revision and update of scheduled MAP activities

2.5 A comprehensive training and skills framework to build upon the training content and skills matrix has been established

Through interviews and documentation reviews, IA has noted the following areas of strength with respect to developing a comprehensive project management training and skills framework:

• progress has been made in which a PMCD Framework was completed and approved on March 22, 2019. The PMCD Framework encompasses mandatory project management curriculums and approaches that the EPMO will offer
• courses are being progressively reviewed and released with an overall plan for completion by fall 2020 and rollout of the courses is anticipated for April 2019

2.6 Controls to prohibit unskilled or untrained project resources from assuming responsibility or accountability for project management functions have been established

Through interviews and documentation reviews, IA has noted the following areas of strength with respect to management developing and implementing controls to prohibit unskilled or untrained project resources from assuming responsibility or accountability for project management:

• the PMCD Framework was approved in March 2019 and controls are anticipated to be put into place to prohibit unskilled or untrained project resources from assuming responsibility or accountability for project management functions
• the controls entail a certification program that was developed to ensure that all project management resources have baseline knowledge of project management and ESDC project management processes. The certification program will be a prerequisite for other project management training courses
• completion of the certificate program will be a prerequisite for access to PMIS
• IA was informed during interviews that, under the updated PMCD Framework, ESDC will conduct a pilot project requiring that project managers for major projects be assessed in 2019 to 2020 to determine if skillsets and competencies are adequate for their assigned projects. Following the outcomes of the pilot project, a course of action will be determined with respect to competencies and project resourcing controls

2.7 Role and mandate of change management leader organizational readiness has not been officially supported by management

Through interviews and documentation reviews, IA noted the following areas of strength with respect to change management best practices:

• the EPMO processes that have been implemented include change management activities to support the human elements of projects. In support of these requirements Transformation and Integrated Service Management Branch (TISMB) is developing and implementing a centre of expertise in this specific area to support both project and operational resources with expertise and best practices
• within ESDC, Assistant Deputy Ministers have received project management training, which encompassed the learning of change management principles
• significant strategic progress has been made by TISMB to prioritize enterprise wide project focus towards organizational change management. A change management directorate has been created under TISMB
• the Change Management Leader Organizational Readiness (CMLOR) team has been established to support the understanding, language and impacts of change within the organization
• the CMLOR Team has been tasked with establishing change management standards, methodologies, and approaches for ESDC
• the CMLOR Team is responsible for improving training and skills to better support enablers within the organization with respect to change
• the CMLOR team is collaborating with EPMO as project management processes have already identified the requirements for change management activities
• transformation hubs exist to bring together stakeholders to discuss change management strategies
• CMLOR within TISMB is in the process of setting up a change management service offering (supports and knowledge) as a center of expertise beyond TISMB, it is currently underresourced to offer this service across all ESDC projects

However, IA also noted the following areas for improvement with respect to change management:

• although processes and documentation requirements exist within PMIS for change management documentation, during interviews project managers identified lack of knowledge, skills and experience with change management
• project managers interviewed indicated there is a lack of change management expertise awareness. Some project managers have noted that they have not been engaged by the CMLOR team

2.8 Governance

Progress has been made to ensure key initiatives are included in the EPMO’s inventory of supported projects and initiatives. Specifically, departmental initiatives (tied to the strategic priorities) have been mapped to investment projects.

IA anticipated that a single governance structure would manage and oversee the processes in place for project management, including change management. However, there are currently multiple management groups within ESDC that provide guidance and oversight on project management. For example, TISMB is responsible for overseeing transformative projects through a Project Delivery Office (PDO), while EPMO has responsibility for overseeing the entire ESDC project portfolio. As a result, projects may be faced with numerous levels of similar oversight and guidance. Interviewees also indicated that transformation is not consistently understood by different management groups (branch level and committees).

ESDC branches have the ability to create independent Project Management Offices (PMOs) or PDO s in addition to the existing EPMO. During interviews, project managers expressed concerns with the need for multiple reviews that resulted in comments relating to project documentation and rework, as both their respective branch PDO and the EPMO would provide feedback. A Responsible, Accountable, Consulted and Informed chart is being developed by EPMO to attempt to clarify accountabilities and responsibilities that currently exist amongst PMOs, PDOs and the EPMO.

IA encourages EPMO to have a single point of expertise for project management to guide both the governance of MPIB and non-MPIB activities. Non-MPIB activities include project management governance items from the Enterprise Architecture Review Board (EARB), TISMB, Shared Services Canada, etc.

2.9 Benefits Management Framework and quality assurance

The Benefits Directive has been developed. Multiple internal consultation activities between governance committees and project managers have taken place. It was noted that reaching an agreement between governance parties has been a challenge and has resulted in approval delays for the Directive. As a result, the Benefits Directive has not yet been approved or deployed as originally planned for June 2017. However, the Directive was approved on March 8, 2019.

Some elements of a project management quality system are in place including a quality assurance function. This is significant progress towards further maturing the EPMO project management processes. IA noted the following areas for improvement with respect to the quality assurance support provided by EPMO:

• project managers indicated during interviews that at the onset of the projects, they were unaware or not advised, by the EPMO project advisors, of mandatory gating presentations required for other non-MPIB governance committees (for example Enterprise Architecture Review Board) or external entities such as Shared Services Canada. In addition, project managers were not always aware of the requirements for mandatory artefacts, associated with non-MPIB committees, as it is not part of project management gating requirements (such as security authorization and assessment) per the PMIS processes. These activities had significant and unforeseen impacts on their project schedules and resource management
• during interviews, project managers expressed that the EPMO senior project advisors provide a valuable support and guidance function, but at times they appear to be challenged by capacity

Beyond a quality assurance function, other elements of a project management quality system that would further improve the quality processes may include items such as continuous improvement and knowledge transfer throughout the project lifecycle.

3. Conclusion

Three out of seven actions included in the MAP related to the 2017 Review on the Project Management and Oversight have been completed. Four out of seven have been partially implemented. Refer to Appendix B which outlines progress made against each action identified in the approved MAP.

With respect to governance, the Department has made progress in broadening project management boundaries beyond investment projects and adopting change management best practices. Further support from management and efforts are required for continuing improvement.

The Benefits Management Directive has been developed and approved in March 2019. A quality assurance function has been implemented and illustrates further maturity to the EPMO processes. The project quality management function with further strengthening would address other attributes such as continuous improvement and prevention over inspection.

4. Statement of assurance

In our professional judgement, sufficient and appropriate review procedures were performed, and evidence gathered to support the accuracy of the conclusions reached and contained in this report. The conclusions are based on observations and analyses at the time of our review. The conclusions are applicable only for the Follow-up Review on the Project Management and Oversight. The evidence was gathered in accordance with the Treasury Board Policy on Internal Audit and the International Standards for the Professional Practice of Internal Auditing.

Appendix A: Review criteria assessment

Review Criteria

It was anticipated that the Department would have further matured the project management governance in the following areas:

• EPMO scope would have broadened beyond investment projects

Rating: Sufficiently controlled; low-risk exposure

Review Criteria

It was anticipated that the Department would have further matured the project management governance in the following areas:

• Change management best practices would have been adopted and utilized to support human elements of projects.

Rating: Controlled, but should be strengthened; medium-risk exposure.

Review Criteria

It was anticipated that the Department would have implemented the Benefits Management Framework and the quality assurance, and these would be operating as intended.

Rating: Controlled, but should be strengthened; medium-risk exposure

Review Criteria

It was anticipated that the Department would have undertaken the actions addressing all recommendations from the Review of Project Management and Oversight and meeting the targeted deadlines for implementation.

Rating: Controlled, but should be strengthened; medium-risk exposure

Appendix B: Progress made vis-à-vis the actions identified in the approved management action plan

Recommendations 

1. Develop and communicate clear criteria and definitions for both “projects” and “initiatives” across the Department.

Follow-up review findings / status: Definitions for both “projects” and “initiatives” have been developed and are in the process of being approved by management (in other words, DGPOC and MPIB).

Office of primary / secondary interest: EPMO/Strategic and Service Policy Branch

Rating: Past original target completion date, this item is still in progress

Recommendations

2. Expand the boundaries of the PPMF beyond the portfolio of investment projects, to capture and govern all projects within ESDC.

Follow-up review findings / status: This item has been addressed. The boundaries of the project and program management practice have been expanded beyond the portfolio of investment projects to capture and govern all projects within ESDC.

Office of primary / secondary interest: EPMO/ Strategic and Service Policy Branch

Rating: This item has now been completed.

Recommendations

3. Modify some of the project management processes to allow for a scalable approach to planning and executing simple or low value projects. Many organizations refer to this scalable methodology as “Project Management Lite”.

Follow-up review findings / status: Progress has been made with this item. A draft definition of scalability was presented to DGPOC in February 2019 and is scheduled for review and approval by MPIB in April 2019.

Office of primary / secondary interest: EPMO

Rating: Past original target completion date, this item is still in progress

Recommendations

4. Develop a baseline of knowledge across management with respect to the value and core principles of project management. This could be achieved through the development and delivery of mandatory awareness training for all executive level positions.

Follow-up review findings / status: A certification program has been developed to ensure that all project management resources have a baseline knowledge by taking a prerequisite course before giving access to PMIS.

Interviewees indicated that current curriculums are being reviewed and revised by EPMO.

Office of primary / secondary interest: EPMO/Human Resources Services Branch

Rating: Past original target completion date, this item is still in progress.

Recommendations

5. Establish a comprehensive training and skills framework that builds upon the training content and skills matrix that has already been developed.

Follow-up review findings / status: A competency framework directive has been developed and approved in March 2019.

In addition to this, EPMO has developed the training curriculum which was completed mid-February 2019. It is anticipated that training curriculum will be deployed on April 1, 2019.

EPMO is in the process of assessing all project managers education and experience to determine training specific requirements.

Office of primary / secondary interest: EPMO/Human Resources Services Branch

Rating: This item has now been completed.

Recommendations

6. Put in place controls to prohibit unskilled or untrained project resources from assuming responsibility or accountability for project management functions.

Follow-up review findings / status: A certification program has been developed to ensure that all project management resources have a baseline knowledge by taking a prerequisite course before being provided access to PMIS.

Office of primary / secondary interest: EPMO/Human Resources Services Branch

Rating: This item has now been completed.

Recommendations

7. Develop change management impact assessments and communications plans for all significant projects within ESDC to ensure support and expediency, as well as to minimize impacts when implementing changes.

Follow-up review findings / status: Substantial progress has been made by TISMB to prioritize enterprise wide project focus towards organizational change management.

Although effort has been made within ESDC to promote change management practices within project management, the role of CMLOR remains undefined and unclear within the organization.

Office of primary / secondary interest: EPMO/TISMB

Rating: Past original target completion date, this item is still in progress.

Appendix C: Summary of testing results

A sample of 10 project files were selected for review and assessment. As described in the Project Governance Framework, several artefacts (mandatory project documentation) are required throughout the project management lifecycle. Through sample testing of project files, the following was noted:

• All 10 projects were required to have a costing artefact
  7 out of 10 had a costing artefact
• Only 8 projects were required to have a business requirement artefact
  7 out of 8 had a business requirement artefact
• Only 8 projects were required to have a project management plan
  7 out of 8 had a project management plan
• Only 7 projects were required to have a scheduling artefact
  7 out of 7 had a scheduling artefact
• Only 7 projects were required to have a risk assessment artefact
  7 out of 7 had a risk assessment artefact
• Only 7 projects were required to have a change management artefact
  4 out of 7 had a change management artefact
• Only 7 projects were required to have decision artefacts
  2 out of 7 had records of decision artefacts