Internal Audit of Passport Integration with the Global Case Management System
Internal Audit & Accountability Branch
February 2016
Table of contents
Executive summary
Objective
The objective of the audit was to provide reasonable assurance that adequate system design and integration controls have been established to ensure the effective, efficient and secure integration of Passport Canada’s Information Systems with Immigration, Refugees and Citizenship Canada's Global Case Management System (GCMS).
Why This Is Important
Formal project management supports the implementation of adequate system design and integration controls, which are necessary for the delivery of major initiatives through systematic and predictable operations. An audit of system design and integration controls of a “system under development”, as this one, provides an early assessment of how long-term projects are faring, and offers the opportunity to address any control weaknesses before the project is complete.
Key Findings
Immigration, Refugees and Citizenship Canada has established several levels of internal reporting and oversight for the integration project but some system controls expected were not in place in the early stages of system development. Business process re-engineering was established as a need, but was not performed to identify improvements and efficiencies early in the project lifecycle. Although business requirements continue to be in development, the integration project was not sufficiently structured and did not include a plan for security requirements. As the integration project continues, further areas for improvement include the need to formalize risk management, project communication, reporting, and systems development validation practices in order to support successful completion and the achievement of project objectives.
Conclusion
System design and integration controls have been developed for the integration of Passport Canada's Information System with the GCMS to support overall management of the integration project. However, further improvement is required to ensure an effective, efficient and secure integration.
Management has accepted the audit findings and developed an action plan to address the audit recommendations.
Statement of Conformance
The conduct of this engagement conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the Quality Assurance and Improvement Review program.
The audit findings and conclusion are based on a comparison of the conditions that existed at the time of the audit against established criteria agreed upon with Immigration, Refugees and Citizenship Canada management prior to the commencement of the audit.
Background
Introduction
On July 2, 2013, Canada’s passport program became integrated into Immigration, Refugees and Citizenship Canada (IRCC). As part of the integration, the authority and overall accountability of the Passport Program was transferred to IRCC, while the responsibility for service delivery has been shared with three departments: Employment and Social Development Canada (ESDC) for domestic service delivery, Foreign Affairs, Trade and Development Canada (DFATD) for overseas domestic service delivery and IRCC as the program lead. A tri-party Memorandum of Understanding was established formalizing the new roles and responsibilities of the three Departments involved and stating that IRCC is the accountable body for the provision of related functional guidance to ESDC and DFATD.
With IRCC now being fully accountable for Canada’s Passport Program, the delivery of passport services is being modernized to accommodate growing changes in service delivery methods and higher passport security and integrity demands. Modernization goals target program improvements including streamlining processes and technologies that enable the issuance of secure Canadian travel documents through authentication of identity and entitlement, facilitating travel, and contributing to international and domestic security.
Under this new accountability model, the Passport Program Modernization Initiative (PPMI) was established with a cornerstone goal of modernizing the passport program business processes and an original cost estimate of $101 million. The PPMI was divided into three main components:
- Activities related to the transfer of the passport program;
- 1.1 Transition (role and responsibility and personnel);
- 1.2 Rationalization of internal services;
- Modernization of the passport program business processes (including new issuance system); and,
- Optimization of the service delivery network.
A Passport Program Modernization Project (PPMP) was established to implement the first and second components of the PPMI. The activities related to the first component of the PPMI have been completed. IRCC is continuing to work on the second component of modernizing the Passport Canada program via business process improvement and re-engineering while also developing a new passport issuance system.
The focus of this audit is on this second component of the PPMI (hereafter referred to as the integration project) which includes the integration and centralization of information from Passport legacy systems into IRCC’s Global Case Management System (GCMS).
Objective and Scope
The objective of the audit was to provide reasonable assurance that adequate system design and integration controls have been established to ensure the effective, efficient and secure integration of Passport Canada’s Information Systems with IRCC’s Global Case Management System (GCMS).
The scope of this audit was to evaluate the system design and integration controls that have been established over the activities related to the integration project of the PPMI. These activities, as per the PPMI’s project charter, include: project management and planning, modernizing the business processes, developing an Information Technology (IT) solution and managing change.
The scope also included assessments of the technical architecture and systems design and, where appropriate, included an examination of automated decision making controls and enhancements. The scope of this engagement also included an analysis of the integration of process improvement and process re-engineering concepts within overall system development, to ensure that the integration project was designed to identify and remove wasteful functions and processes. This audit covered the period from July 2013 to March 31, 2015.
This audit was designed to focus on and evaluate key areas identified during the audit’s planning and risk assessment phase that were identified as being critical to the overall success of the Integration Project. In order to effectively measure each focus area, this audit was designed around and leveraged leading practices and standards from Information Systems Audit and Control Association (ISACA), the Treasury Board of Canada, Communications Security Establishment of Canada, and other industry leading guidance.
This audit was not scoped to evaluate the first or third component of the PPMI, or the PPMI as a whole. Other items out of scope were:
- Financial / Costing arrangements between delivery partners;
- Applied technical testing of application or infrastructure; and
- Supply and network arrangements with Shared Services Canada.
The criteria used were:
- An adequate governance framework exists and adequate governance and project initiation controls and oversight mechanisms have been established.
- An adequate project management framework and approach exists that is commensurate with the size, complexity and Government of Canada (GC) policy requirements.
- Business process re-engineering or improvements of passport issuance processes to identify efficiencies have been conducted.
- Relevant business requirements have been identified and defined.
- Security and integrity enhancements and requirements have been identified and defined. Relevant system and solution technical, architectural, functional and design attributes/requirements have been identified and defined.
- A holistic and secure process-based approach to software and system development and systems integration has been established and leveraged.
Findings and Recommendations
Project Planning and Management
Project planning and management is key to the success of any large integrated IT centric project. Sound and effective planning and staging of key project deliverables and milestones not only ensures that the project is delivered on time and on budget, but also meets the expected outcomes and business needs. Further, effective project management including risk management and communications is a fundamental control that keeps the project on track and stakeholders informed.
It was expected that effective project planning and management controls for the integration project had been established and that the integration project had been effectively scoped to align with PPMI objectives. In addition, it was expected that the Integration Project was supported by adequate project governance and oversight controls and that an effective approach to management, including risk and performance management and effective communications and reporting on key integration project areas.
IRCC leveraged industry expertise to evaluate different integration options.
Prior to PPMP project initiation, IRCC and Passport Canada leveraged industry expertise to evaluate the different integration options of Passport Canada into other federal departments. Various cost savings were examined that may be realized via various alternate service delivery options and IT system deployments. This process explored three major, high level options in IT for the development and implementation of a new and modernized passport IT system that meets the business needs.
This analysis was the primary impetus that defined the PPMP, including key project deliverables, the Business Case, Project Charter, Project Management Plan and detailed scope documentation.
IRCC has established several levels of internal reporting and oversight for the integration project.
Overall, the documented approach to project management is sound and the Department has aligned a dedicated project management team to the integration project with requisite skills and expertise in managing large complex projects.
Within this approach, IRCC established several oversight mechanisms that tracked and reported on key project areas in a specific, objective, measurable, repeatable manner. Further, the integration project has undergone periodical Health Checks following a consistent methodology and included reporting on project scope, costs, and timelines to validate the project plan and progress to date.
The Department also established a suite of plans and approaches on how they intended to conduct project management, monitor progress and manage project risks effectively. These detailed plans were included in key project documentation including the business case, project charter and in other supporting project documentation.
The integration project is not sufficiently structured or scoped.
The PPMI Business Case and the project charter broadly define the scope of the project. However, the integration project’s tasks and activities are grouped into a single project and Work Breakdown Structure for the entire PPMI, as well as a single governance structure, rather than breaking the integration project down into smaller and more manageable and achievable sub-projects. A more detailed Work Breakdown Structure with smaller sub-projects including details of the IT project components would facilitate monitoring of tasks and timelines, and would allow for any necessary course corrections early in the project lifecycle.
In a response to the broadly defined scope, two subordinate and parallel projects with alternate, clearly defined and more focused project scopes and objectives evolved. The two subordinate projects (GCMS Passport Release 1, and Passport e-App - Release 1) resulted as the integration project team recognized the need to apply more discrete and focused project management practices and structure. While this has been a practical, operational response to the integration project challenges, the governance structure is designed to manage only one project and larger milestones rather than smaller discrete sub-projects. With the increased operational need for timely decision-making, the single governance body could slow the approval process and hinder the efficiency of operational delivery of the smaller integration project.
Risk management lacks traceability.
Risk management activities are conducted as part of the integration project and risks are communicated, actively discussed and reviewed through various reporting tools. However, the risks identified in the PPMI Business Case were not consistently addressed in the integration project risk assessment. As a result, IRCC could not trace if an appropriate mitigation response was in place for the integration project based on the stated Business Case risks. The risk identification and assessment process for the integration project also did not consistently describe impact, likelihood or vulnerability factors. In addition, the risk assessment did not describe the impact to the integration project’s objectives as per IRCC guidance on risk management. Although risk responses were recorded, mitigation action plans were not consistently described and there was limited documentation to confirm the implementation of mitigation activities. Without a more rigorous process, the integration project may be exposed to unexpected risks without mitigation strategies in place.
Integration project performance targets are not explicit.
The integration project’s performance measurement strategy detailed how it intends to monitor and track the integration project’s performance against the three high-level, core project objectives and goals. However, the performance measurement strategy did not define any specific or measurable performance targets or Key Performance Indicators beyond the three targeted, high-level core project components.
The Department was in the process of developing an “Outcome Realization and Management Plan” that included “outcome performance indicators,” however, it was not finalized and was being developed late in the project cycle. Without clearly defined targets, IRCC may not be able to focus the integration project efforts toward the stated objectives or be able to demonstrate how it has achieved the intended outcomes.
Integration project communication protocol has not been developed.
The integration project documented a communications management approach which established high-level reporting elements. A broad range of reporting and communications occurred via project summary reports and reporting dashboards. Also, the communications approach described how the integration project will track and report on project updates and changes, and what its performance measurement strategy is for realizing project goals.
However, the communications approach did not establish a formal communication protocol to identify how information would be shared, reported and managed so that the key stakeholders were appropriately engaged throughout the integration project to support decision making and accountability. Thus, the risk is that information is over or under-shared and leads to an inefficient or ineffective process.
Furthermore, although project reporting provided updates on project status, the reporting did not track total project spending against budgeted activities. Tracking actual costs and resource usage is an essential, core aspect of project control. As key information was not regularly disseminated to all key stakeholders, there is an increased risk that stakeholders do not have full information they require to support decision-making. However, this type of information may require significant effort and resource commitment for the Department to implement systems to support the tracking process necessary to collect and record accurate and timely data. Confirmation of the level of investment the Department is willing to undertake would clarify the approach to adequately track project costs against planned activities.
Recommendation 1 (High): IRCC should revise the Work Breakdown Structure to reflect more manageable sub-projects with sufficient level of detail to ensure a common understanding of all project tasks and deliverables to support project management.
Recommendation 2 (Medium): IRCC should strengthen the risk management process for the integration project to ensure that risks are evaluated, mitigated and reported.
Recommendation 3 (Medium): IRCC should complete the Outcome Realization and Management Plan and ensure it has clearly defined Key Performance Indicators to support performance measurement and the demonstration of the realization of intended outcomes.
Recommendation 4 (Medium): IRCC should develop a communication protocol for the integration project to support a more efficient and effective information sharing process that supports decision making and accountability.
Business Requirements and Business Process Improvement
Establishing clear and effective business requirements and conducting business process re-engineering are key to the integration project’s success. Failing to conduct business process re-engineering and establishing well-defined business requirements inhibits the integration project’s ability to meet the efficiency, security and integrity improvement goals and objectives within the PPMI.
It was expected that IRCC identified and defined relevant business requirements, and appropriate security enhancements and requirements for the modernization of passport program business processes. In addition, it was expected that IRCC conducted business process re-engineering on the passport issuance processes to identify efficiencies.
Business requirements continue to be in development.
IRCC clearly outlined the objectives of the PPMI and developed various iterations of pre-project business requirements, some high level business requirements, and the functional, technical and specification requirements of the new passport issuance system and related processes.
While the integration project described some high level business requirements, IRCC did not supplement their capabilities with the appropriate skill sets and experience to effectively identify program level business requirements. As a result, actual passport program business requirements were not identified for improvement, nor were key passport program business process security and integrity control enhancement requirements. While detailed functional and specification requirements were well documented, they did not identify any IT security, privacy or user-centric requirements nor did they overtly map back to project objectives or goals. Without specific project objectives, IRCC will not be able to fully determine if the integration project was a success with any specificity.
Business process re-engineering was established as a need, but was not performed.
The integration project clearly established the need to conduct business process streamlining and re-engineering early in its lifecycle, citing the need to address gaps in passport processes, security, integrity and technologies, and ultimately operate more efficiently. High level project goals and objectives were documented and included: improving the service delivery and accessibility of passports to Canadians; strengthened security and integrity of passport issuance; increased program efficiencies; and strengthened organizational and workforce excellence.
Although there was a clear recognition of the need to conduct business process re-engineering to achieve project goals, the integration project did not establish a defined approach to undertake re-engineering activities, and to further identify business process gaps between current process performance and documented business goals and objectives beyond the high level descriptions in the business case.
IRCC’s project management and system development practices and methodologies were not designed to include business process re-engineering activities and processes and methodologies were not well understood by the organization. The core process re-engineering team did not have any formal or informal training in business process re-engineering. Furthermore, externally contracted business process re-engineering resources were only involved with the project for a limited period.
As a result, the integration project did not identify potential improvements to the process. As a result, the integration project may not be able to fully demonstrate how it will realize some of the expected fiscal savings and demonstrate how it is meeting project improvement goals and objectives.
Recommendation 5 (High): IRCC should identify key Passport Program business requirements, including security and integrity control enhancement requirements; and formalize the approach to conducting Business Process Re-engineering.
IT Security and System Development
IT Security planning was incomplete
As an IT system is being developed or modified, the security profile can be potentially exposed to new and changing risks. It was expected that initial security assurance requirements are identified, approved and updated as needed during the planning phase.
An IT security plan is identified in the project schedule, however it is not complete. There is no documented evidence that demonstrates the plan is being followed, and the initial security assurance requirements required to guide the plan have not been documented. Key security planning deliverables that are missing include: a Security Control Profile based on government-wide (ITSG-33) guidance, a Preliminary Threat and Risk Assessment, a Concept of Operations document and high-level listing of target system security controls.
Through GCMS, IRCC now has external connections with multiple departments, which raises the risk profile of the system. Defining and evaluating IT security requirements throughout an integrated IT project reduces the risk that the integrated solution will experience a breach.
System development practices lack formal validation.
IRCC performs Quality Assurance (QA) reviews within the development of GCMS to validate system outputs. However, the QA reviews are performed on an ad hoc basis and formal quality assurance standards have not been established. As a result, there is a risk that the system may be launched containing functional and security flaws. Sound quality assurance and validation exercises within the system development lifecycle help to ensure that the system is meeting expected functional requirements and does not contain security flaws.
Recommendation 6 (Medium): IRCC should ensure that IT security assurance requirements are completed and that appropriate IT security development validation exercises are undertaken.
Conclusion
System design and integration controls have been developed for the integration of Passport Canada's Information System with the GCMS to support overall management of the integration project. However, further improvement is required to ensure an effective, efficient and secure integration.
Appendix A – Management Action Plan
Recommendation 1 - High Risk
IRCC should revise the Work Breakdown Structure to reflect more manageable sub-projects with sufficient level of detail to ensure a common understanding of all project tasks and deliverables to support project management.
Management Response and Action Plan
Management accepts this recommendation as it relates to the modernization project.
The OPI (Passport Modernization Project Office) will act on the recommendation by:
Action Plan:
The Project plans to secure expenditure authorities to advance the project to the next phase: all stakeholders will participate in a review of the WBS to substantiate costing assumptions and project plans. The Project Delivery Office (PDO) in Operations Performance Management Branch (OPMB) will coordinate this review under the leadership of the Passport Modernization Project Office (PMPO).
- Given the scope of the audit, in its review of opportunities to further breakdown the WBS, PMPO and OPMB-PDO will engage SIMB specifically on the IT elements.
OPI: ADM Operations
Due date: March 31,2016
Recommendation 2 – Medium Risk
IRCC should strengthen the risk management process for the integration project to ensure that risks are evaluated, mitigated and reported.
Management Response and Action Plan
Management accepts this recommendation as it relates to the modernization project.
For overall Project Risk Management at the Project level, OPMB/PDO has established an effective governance structure from the working level through to DM-level, which includes regular meeting at all levels, implicated DG reviews, a Project Management Board (PMB) and a thorough Risk Management process with risk identification, assessment and mitigation processes using the PMBOK v5 Project Management framework. A crosswalk between the business case core assumptions and the risk register has been done to ensure risk management is comprehensive.
Action Plan:
The PMPO will lead (with OPMB-PDO support):
- A review of project governance and supporting processes to assess level of risk management and identify and address any gaps.
OPI: ADM Operations
Due date: February 29, 2016 - A review of the project risk management activities, including validation of risk level and associated mitigation strategies.
OPI: ADM Operations
Due date: March 31, 2016
Recommendation 3 – Medium Risk
IRCC should complete the Outcome Realization and Management Plan and ensure it has clearly defined Key Performance Indicators to support performance measurement and the demonstration of the realization of intended outcomes.
Management Response and Action Plan
Management agrees with this recommendation.
The OPI (Passport Modernization Project Office) will act on the recommendation by:
Action Plan:
- The Outcome Realization and Management Plan has been completed and integrated into the draft of the 2nd business case for the Modernization of the Passport Program/Transformation Plan which was approved by IRCC Senior Management.
OPI: ADM Operations
Due date: Completed – May 2015 - The Business Case is being revised to reflect recent developments in the project. For example, Annex F contains a clear description of each performance indicator, including targets and data sources, mapped to the Outcomes of the Passport Program and the Modernization Initiative.
OPI: ADM Operations
Due date: March 31, 2016
Recommendation 4 – Medium Risk
IRCC should develop a communication protocol for the integration project to support a more efficient and effective information sharing process that supports decision making and accountability.
Management Response and Action Plan
Management accepts this recommendation as it relates to the modernization project.
The Passport Modernization Project Office (PMPO) will be established in IRCC as the central authority under the leadership of an Executive Director directly accountable for the full scope of the project, including planning, implementation, budget control and resourcing, governance, information sharing and reporting. The Executive Director will report to the ADM, Operations. The PMPO will ensure more efficient and effective information sharing and clearer accountabilities and responsibilities for decision making, and that project related information is disseminated in a timely and consistent manner to enable informed decision-making and accountability, to all key stakeholders having a need to know, and that sensitive information.
Action Plan:
- Clarify and document roles and responsibilities of the PMPO and all project stakeholders.
OPI: ADM Operations
Due date: February 29, 2016 - Revise Project Charter and governance framework to ensure consistent communication protocols are in place for the project.
OPI: ADM Operations
Due date: February 29, 2016 - Seek senior management approval for updated project documents
OPI: ADM Operations
Due date: March 15, 2016 - Ensure all project employees and stakeholders are made aware of communication protocols.
OPI: ADM Operations
Due date: March 31, 2016 - Implement and monitor protocols.
OPI: ADM Operations
Due date: March 31, 2016 and ongoing.
Recommendation 5 – High Risk
IRCC should identify key Passport Program business requirements, including security and integrity control enhancement requirements; and formalize the approach to conducting Business Process Re-engineering.
Management Response and Action Plan
Management agrees with the recommendation.
The OPI (Passport Modernization Project Office) will act on the recommendation by:
Action Plan:
- Key Passport Program business requirements including security and integrity controls have been completed since the audit was finalized. Validation with OPIs is anticipated for end of fiscal year 2015-16.
OPI: ADM Operations
Due date: March 31, 2016 - A Strategic Process Business re-engineering strategy was completed in Q2 of fiscal year 2015-16. The main objective of the strategy is to formalize and capture an approach to conducting business process re-engineering through continuous, innovative and adaptable operational improvement.
OPI: ADM Operations
Due date: Completed (October 2015) - Consultation and re-engineering process execution will begin in Q3 of fiscal year 2015-16. Activities include:
- Planning, design and implementation involving critical internal and external stakeholders’ collaborations.
OPI: ADM Operations
Due date: September 2016 - Business process re-engineering to support the passport issuance processes to gain efficiencies, including requirements on relevant business, security and solution and technology requirements.
OPI: ADM Operations
Due date: December 2016 - Identification of gaps between current process performance and documented business goals and objectives.
OPI: ADM Operations
Due date: December 2016 - Identification and documented requirements of improvement of core processes associated with performance gaps.
OPI: ADM Operations
Due date: March 2017
- Planning, design and implementation involving critical internal and external stakeholders’ collaborations.
Recommendation 6 – Medium Risk
IRCC should ensure IT Security assurance requirements are completed and that appropriate IT security development validation exercises are undertaken.
Management Response and Action Plan
Management agrees with the recommendation.
Action Plan:
- IRCC SIMB currently has a Security Assessment and Authorization (SA&A) process in place. The SA&A process was completed for GCMS Release 8.1 (launched on July 31, 2015) and for GCMS Release 9 (launched on November 21, 2015). SA&A letter of acceptance was submitted to the Program Delivery Manager for approval in December 2015.
OPI: ADM Corporate Services
Due date: Completed - SIMB has now implemented the IRCC Risk Management Framework (RMF), consistent with Government of Canada and industry standards (i.e.: Treasury Board’s Enterprise Security Architecture framework, the ISO 27001 ISMS model, and ITSG-33.) The RMF was implemented at IRCC on September 30, 2015 and serves to facilitate IRCC’s formal Security Assessment & Authorization process. The Directive on SA&A is scheduled for approved by December 31, 2015.
OPI: ADM Corporate Services
Due date: February 29, 2016 - SIMB is in the final stages of communicating the RMF to all stakeholders as it applies to all IRCC systems. SIMB will deliver awareness presentations to stakeholders on IT Security Risk Management Framework and training will be offered as required.
OPI: ADM Corporate Services
Due date: March 31, 2016 - SIMB will incorporate a framework for quality assurance reviews during system code development. The current informal peer review of development code will be integrated.
OPI: ADM Corporate Services
Due date: March 31, 2016
Page details
- Date modified: