Language selection

Government of Canada / Gouvernement du Canada

Search

The Treasury Board of Canada Secretariat’s Response to the Office of the Auditor General of Canada’s Report on Phoenix Pay Problems: An Estimate of Costs to Stabilize Phoenix and Operate the Pay System

From Treasury Board of Canada Secretariat

This report is the Treasury Board of Canada Secretariat’s response to the recommendation in Report 1 of the 2017 Fall Reports of the Auditor General of Canada to track and report on the cost of:

  • resolving pay problems
  • implementing a sustainable solution for all departments

On this page

Executive summary

Report 1 of the 2017 Fall Reports of the Auditor General of Canada to the Parliament of Canada recommended that:

The Treasury Board of Canada Secretariat, with the support of Public Services and Procurement Canada, and in partnership with departmentsFootnote 1, should track and report on the cost of:

  • resolving pay problems; and,
  • implementing a sustainable solution for all departments.Footnote 2 Footnote 13

The Comptroller General of Canada, who leads the Office of the Comptroller General within the Treasury Board of Canada Secretariat (TBS), agreed with the recommendation and committed to providing a government-wide cost estimate regarding the Phoenix pay system. This report presents the findings of TBS’s cost estimation exercise and has three main components:

  1. Historical costs related to the implementation of the Pay Centre and the Phoenix pay system from the 2009 to 2010 fiscal year to the end of the 2017 to 2018 fiscal yearFootnote 3 using actual data from departmental financial systems as of December 31, 2017, extrapolated to the end of March 2018
  2. Projected investments required to stabilize the Phoenix system to sustainable performance levels
  3. Annual operating costs of pay administration for Public Services and Procurement Canada (PSPC) and for departments

This cost estimate does not include funding for the future transformation of the federal government’s pay administration (potential replacement of the Phoenix pay system), announced in Budget 2018, because its scope has not been determined, and it is being developed independently of the effort to stabilize Phoenix.

The methodology used to estimate costs was evidence-based to the extent that data were available. Where there were gaps in evidence, accepted techniques to estimate costs were used, including the development of ground rules and assumptions (detailed in Appendix A).

The degree of confidence in the components of the cost estimate varies according to the availability of data:

  • there is high confidence in the estimate of historical costs, as the estimate is supported by high-quality data from departmental financial systems
  • there is lower confidence in the estimate of future costs to stabilize the Phoenix pay system, as these:
    • are mathematically derived estimates based on incomplete data or proxies
    • rely significantly on assumptions for elements that could not be measured
  • there is reasonable confidence in the annual operating costs of pay administration over the stabilization period, as the estimate is derived from a combination of actual data and proxies where there are data gaps

Table 1 summarizes the planned and unplanned costs related to the implementation of the Pay Centre in Miramichi, New Brunswick, and the Phoenix pay system.

Table 1. Planned and unplanned costs, by activity, of implementing the Pay Centre and the Phoenix pay system
(in millions of dollars)
Activity or item Actual costs as of March 31, 2018 Planned costs per year Unplanned costs per year Unplanned, one-time costs
Historical costs
Original project costs (2009 to 2010 fiscal year to the 2017 to 2018 fiscal year)
 309.2 N/A N/A N/A
Planned operating costs (2013 to 2014 fiscal year to the 2017 to 2018 fiscal year)
 380.9 N/A N/A N/A
Unplanned operating costs (2016 to 2017 fiscal year to the 2017 to 2018 fiscal year)
 360.6 N/A N/A N/A
System stabilization costs
Costs per year, unplanned
 N/A N/A 9.8 N/A
One-time costs, unplanned in the stabilization period
 N/A N/A N/A 50.8
Operating costs
Operating costs, per year, planned
 N/A 160.0 N/A N/A
Operating costs, per year, unplanned
 N/A N/A 326.6 N/A
One-time costs, unplanned in the stabilization period
 N/A N/A N/A 73.6

All historical costs are reported as-spent, and system stabilization and operating costs are in constant year, Canadian dollars. Some numbers may not add due to rounding.

Actual spending on the pay solution as of March 31, 2018, comprised the following:

  • project costs for both the Pay Consolidation Project (Pay Centre in Miramichi) and the Pay Modernization Project (Phoenix)
  • planned spending to operate the pay system
  • unplanned spending to enhance capacity to address Phoenix implementation challenges

Estimated system stabilization costs include an unplanned, one-time cost to build a test system and annual costs to develop missing functionality. Once the system is stabilized, it is expected that the government will incur a one-time expense to review pay files in order to restore employee confidence in the system.

Estimated annual operating costs for pay administration across all departments include planned costs of $160.0 million and unplanned costs of $326.6 million. PSPC will incur a further $37.7 million in unplanned, one-time costs to improve workload management tools and implement system upgrades. TBS will also incur unplanned, one-time operating costs of $25 million to provide departments with funding to increase staff to support employees who have pay issues.

Possible changes in annual operating costs after stabilization are not included, as it is unknown how the implementation of system stabilization activities will impact planned and unplanned pay system operations. The planned estimate of $160.0 million to operate the pay system will need to be reassessed once stabilization is reached.

This cost estimate excludes the following:

  • announcements in Budget 2018 to explore alternatives to Phoenix ($16 million)
  • pension administration
  • unreported claims
  • unassessed damages

Best estimates of capacity and existing trend lines indicate that it could take approximately five years to stabilize the Phoenix pay system. Given the dynamic environment in which efforts to address problems with Phoenix are occurring, this estimate is a snapshot in time based on available data and assumptions as at March 2018. For methodological purposes to support a forecasting model, an inventory of technical system change requestsFootnote 4 (CRs) has been selected as a comparator. Given the differences between the Regional Pay System (RPS) and Phoenix, this comparator, while not perfect, is the best available option for this cost estimate. While the cost estimate’s assumptions take into account expected productivity gains and pressure points, PSPC has indicated that a more detailed qualitative assessment of CRs is underway. Therefore, although this cost estimate projects that funding will need to continue for approximately five years based on current assumptions, PSPC expects that its investments to date, its increased processing capacity, its improved data analytics and strategic implementation of fixes will result in shorter timelines to stabilize the queue of transactions and address pay outcomes for staff.

Purpose and scope of the cost estimate

In his 2017 Fall Reports of the Auditor General of Canada to the Parliament of Canada, Report 1: Phoenix Pay Problems, the Auditor General of Canada recommended that TBS track and report on the cost to:

  • resolve pay problems
  • implement a sustainable pay solution for all departments

In response, TBS committed to work in partnership with departments to establish a cost estimate for the Phoenix pay system. This report is provided to the Standing Committee on Public Accounts in fulfillment of TBS’s commitment to provide a cost estimate and a framework to track future costs.

This report addresses the costs of the federal pay system in three parts:

  1. Historical costs: These are the actual costs incurred by PSPC, TBS and other departments from the initiation of the Pay Consolidation Project in 2009 until March 31, 2018 (the end of the 2017 to 2018 fiscal year). These costs include both planned and unplanned costs incurred in response to implementation challenges. The estimate of these costs is based on actual data extracts from financial and expenditure systems as of December 31, 2017, extrapolated to March 31, 2018.
  2. System stabilization and enhancement costs: These are estimated future costs to:
    • stabilize the system
    • add functionality to meet operational needs
    • undertake a pay file review to restore employee and public trust in the system

    The estimate of these costs was developed using historical systems data and assumptions. For methodological purposes to support a forecasting model, an inventory of technical system CRs has been selected as a comparator. Given the differences between the RPS and Phoenix, this comparator, while not perfect, is the best available option for this cost estimate.

  3. Annual operating costs for pay administration: These are annual operating costs based on funding commitments and projections from actual data extracts from departmental financial and expenditure systems that are projected for the next fiscal year.

Costs that could not be estimated or that are outside the scope of this estimate are outlined in the section titled Out of scope. Specifically, the cost of a possible replacement system, as announced in Budget 2018, is excluded from this estimate because the scope of this work is still being developed and is independent of efforts to stabilize Phoenix. A detailed scoping of this cost estimate is provided in Appendix A.

The overall cost estimate is based on data that were available at the time of analysis. Given the dynamic environment of the pay system, this estimate is a snapshot in time based on available data and assumptions as at March 2018. Considerations are provided to explain the limitations for each component of the estimate.

As the federal pay environment continues to evolve and better data become available, the confidence in future cost estimates can be expected to improve.

Costing methodology

This cost estimate was conducted by a team of costing experts under the leadership of the Comptroller General of Canada, using a mixed-method approach in alignment with international best practices. For each component of the estimate, methods were selected to best match the process being costed and the data that were available.

  • Historical costs were estimated using a cost accounting approach. This component of the estimate was largely based on both planned and unplanned costs incurred by PSPC and other departments, as coded in departmental financial systems. There is high confidence in this component because the analysis is based on actual data as of December 31, 2017, extrapolated to March 31, 2018.
  • System stabilization and enhancement costs were estimated using a combination of evidence-based, mathematical approaches. The cost estimate for Phoenix stabilization was modelled statistically using records of the actual number of system fixes and changes made from July 2017 to March 2018. PSPC has indicated that the project is in a very fluid state, which could accelerate results. For example, PSPC is entering a new phase of technical support to deal with systems, and it is adopting a more strategic approach to CRs to maximize their impact and prioritize the best return from each action. The cost estimate model for the pay file review is based on an analogy (a multiplier) of a similar task undertaken by the Government of Canada Pension Centre. There is less confidence in this component because of the number of unknowns that remain and the lack of certainty regarding the tasks required to stabilize Phoenix.
  • Annual operating costs for pay administration during stabilization were estimated using a combination of accounting and statistical methods. Funding to increase capacity is assumed to be spent in the year allocated, and future costs are derived from data collected to estimate historical costs. There is reasonable confidence in this component, as it is based on a combination of funding and recent cost data. The planned estimate of $160.0 million to operate the pay system will need to be reassessed once stabilization is reached.

Unplanned system stabilization and operating costs have been risk-adjusted where possible. The methodology for the development of this cost estimate is detailed in Appendix B.

Historical costs

In this section

This section provides a summary of the actual costs incurred by PSPC, TBS and other departments from the initiation of the Pay Consolidation Project in 2009 through to the implementation of Phoenix in February 2016, until March 31, 2018.

Background

Prior to the implementation of Phoenix, public service pay was administered at the department level by compensation advisors who worked alongside human resource (HR) advisors. This collaboration was required, in part, because the RPS was a stand-alone IT system, with no data linkage between it and the HR system of any department. In contrast, the Phoenix pay system relies on data from departmental HR systems to enable pay actions.

What is commonly referred to as “Phoenix” is part of the Transformation of Pay Administration Initiative and began as two distinct projects: the Pay Consolidation Project and the Pay Modernization Project. Under the Pay Consolidation Project, savings were to be generated by consolidating the compensation services of 46 departments and agencies at a single Pay Centre in Miramichi, New Brunswick. At the same time, through the enhanced automation and functionality of the Pay Modernization Project, compensation would be linked directly to the government’s HR systems to initiate compensation actions and outcomes. Both projects and their outcomes were inextricably linked.

That said, it is not possible to make direct linkages between costs incurred in pay administration under the RPS and compare them with Phoenix. The Phoenix environment is much more expansive than that of the RPS, and Phoenix relies on the integrity of data now entered upstream in HR systems to perform accurately and on time.

Challenges with implementing Phoenix contributed to an unanticipated growth in the volume of cases for both the Pay Centre team and the Pay Operations team that support Phoenix, and emergency funding was issued to support both functions. Since this funding was issued following Phoenix’s implementation, the unplanned costs of the Pay Centre become entwined with the post-implementation costs of Phoenix.

Since 2009, through to March 31, 2018, PSPC has spent $805.4 million on the pay system, of which:

  • $309.2 million was for planned project activities:
    • $118.6 million was for the Pay Consolidation Project
    • $190.7 million was for the Pay Modernization Project
  • $246.2 million was for regular operations of the pay system
  • $249.9 million was for enhanced capacity and emergency measures post-February 2016

As the employer of the public service, TBS invested $7.4 million in response to Phoenix implementation challenges:

  • a Claims Office to reimburse public servants for expenses incurred as a result of pay issues
  • a new Human Resources Management Transformation team to address policy issues

The TBS Claims Office was established in June 2016. As of March 31, 2018, it had reimbursed $292,000 to employees for direct expenses incurred as a result of pay issues. This amount is considered to be less than the total potential value of outstanding claims because:

  • the number of claims received is significantly less than the number of employees who have been underpaid
  • not all the claims that have been received have been processed

Another consequence of issues with the pay system was an increase in workload at the Canada Revenue Agency (CRA). The volume of erroneous and revised T4 slips increased CRA’s volume of tax queries and file reviews. Budget 2018 announced additional funding of $5.5 million over two years, starting with $1.7 million being allocated for the 2017 to 2018 fiscal year.

The continuing challenges with Phoenix’s implementation required individual departments to develop additional capacity to support affected employees. Many departments incurred costs to hire additional compensation advisors and pay overtime to existing staff.

As of March 31, 2018, the unplanned cost to the compensation function in departments was estimated to be:

  • $70.1 million for departments supported by the Pay Centre
  • $31.6 million for those that maintained their own compensation functions

The planned cost of compensation functions of departments not supported by Miramichi was $134.7 million.

Estimate of historical costs, 2009 to 2010 fiscal year to the 2017 to 2018 fiscal year

Table 2 shows an estimate of historical costs of implementing the Phoenix pay system as of March 31, 2018.

Table 2. Estimate of historical costs of implementing the Phoenix pay system, as of March 31, 2018
(in millions of dollars)
Activity or item Original project costs, 2009 to 2010 fiscal year to the 2017 to 2018 fiscal yeartable 2 note * Planned costs, 2016 to 2017 fiscal year and the 2017 to 2018 fiscal yeartable 2 note * Unplanned costs, 2016 to 2017 fiscal year and the 2017 to 2018 fiscal yeartable 2 note *

Table 2 Notes

Table 2 Note *

Columns may not add due to rounding.

Return to table 2 note * referrer

Table 2 Note **

This figure is for the 2013 to 2014 fiscal year to the 2017 to 2018 fiscal year.

Return to table 2 note ** referrer

Pay Consolidation Project (PSPC) 118.6 N/A N/A
Pay Modernization Project (PSPC) 190.7 N/A N/A
Regular operations of the pay system (PSPC) (from the 2013 to 2014 fiscal year) N/A 246.2table 2 note ** N/A
Enhanced capacity and emergency measures (PSPC) N/A N/A 249.9
Departmental compensation costs
Departments supported by the Pay Centre, unplanned costs
 N/A N/A 70.1
Departments not supported by the Pay Centre, planned costs
 N/A 134.7 N/A
Departments not supported by the Pay Centre, unplanned costs
 N/A N/A 31.6
Treasury Board of Canada Secretariat (including the Claims Office and the Human Resources Management Transformation team)
 N/A N/A 7.4
Canada Revenue Agency
 N/A N/A 1.7
Total historical costs 309.2 380.9 360.6

Considerations

Although these historical costs are based on actual expenditures, costs to departments may be understated. Organizations may have responded to the increased workload by reallocating resources rather than incurring unplanned expenses. This reallocation may not have been systematically captured in departmental financial systems, which would not allow for these expenditures to be supported by evidence. TBS will explore the feasibility of creating financial coding or a consistent methodology to enable departments to better capture costs that are directly attributable to Phoenix.

Projected system stabilization and enhancement costs

In this section

Projected system stabilization and enhancement costs comprise a combination of estimated costs to:

  • improve the performance and functionality of Phoenix
  • undertake a pay file review to restore employee confidence in the pay system

Background

The assumption used in the development of this cost estimate is that the pay system will be considered stable when the backlog of work has been reduced and there is a manageable incoming workload for the Pay Centre team and for the Pay Operations team at PSPC. More specifically, it is assumed that stabilization will be achieved when the Phoenix change request backlog does not have more than two months of activity, which is the same as the change request backlog associated with the RPS, as indicated by historical data.Footnote 5 Given the differences between the RPS and Phoenix, this comparator, while not perfect, is the best available option for this cost estimate and is the basis on which projected system stabilization costs have been estimated. The path to stabilization can be assessed using:

  • data from the transaction queue
  • incident reports (system errors)
  • the backlog of change requests (customizations and enhancements to software)

The transaction queue constitutes all changes or corrections that are to be made to the pay files of individual employees and is primarily the responsibility of the Pay Centre. The rate at which the Pay Centre resolves cases depends on:

  • the capacity of the workforce to process individual cases
  • the functionality of Phoenix to enable these tasks

The efficiency of the Pay Centre’s workforce is expected to improve as staff become more experienced and the Pay Centre implements different ways of working. The Pay Operations team works to improve the performance and functionality of Phoenix by addressing incident reports and change requests (CRs).

The Pay Operations team uses incident reports to identify, prioritize and resolve Phoenix system errors (bugs). Analysis of the data shows that the rate of incident reports is declining and that the Pay Operations team has sufficient resources to resolve system errors in a timely manner.

The Pay Operations team uses CRs to prioritize and implement changes and enhancements to the Phoenix system. CRs are prioritized according to type:

  • mandatory/legislative CRs to implement changes such as new tax calculations and the implementation of new collective agreements
  • stabilization/enhancement CRs to add new features or improve the usability of Phoenix

As the name suggests, mandatory/legislative CRs take precedence over stabilization/enhancement CRs. Analysis of the data shows that the Pay Operations team does not have sufficient resources to manage the incoming workload and clear the backlog of CRs.

The unplanned additional resources required for the Pay Operations team to address the backlog of CRs are estimated to be 59 FTEsFootnote 6 per year at an average cost of $9.8 million per year, plus a one-time expense of $1.8 million to build a test system.

As Phoenix approaches a stable state and the backlog is reduced, it will be necessary to undertake some form of review of pay files to restore confidence in the system. A review of this magnitude within the Government of Canada is unprecedented. The most comparable existing process is the file review undertaken by the Pension Centre when an employee retires. Although the two tasks are not identical, the Pension Centre’s file review was the best available proxy for this cost estimate. The rough-order-of-magnitude cost estimate to undertake a complete pay file review is $49.0 million.

Estimate of system stabilization costs from the 2018 to 2019 fiscal year onward

Table 3 shows estimated annual costs of system stabilization.

Table 3. Estimated annual system stabilization costs per year until Phoenix stabilization, based on the 2018 to 2019 fiscal year
(in millions of dollars)
System stabilization activity Unplanned costs per yeartable 3 note * Unplanned, one-time coststable 3 note *

Table 3 Notes

Table 3 Note *

Columns may not add due to rounding.

Return to table 3 note * referrer

Unplanned development capacity (PSPC) 9.8 N/A
Building of test system (PSPC), one-time costs N/A 1.8
Review of pay files (PSPC), one-time costs N/A 49.0
Total system stabilization costs 9.8 50.8

Considerations

Best estimates of capacity and existing trend lines indicate that it could take approximately five years to stabilize the Phoenix pay system. Given the dynamic environment in which efforts to address problems with Phoenix are occurring, this estimate is based on available data and assumptions as at March 31, 2018. For methodological purposes and to support a forecasting model, an inventory of technical system CRs has been selected as a comparator. Given the differences between the RPS and Phoenix, this comparator, while not perfect, is the best available option for this cost estimate.

While the cost estimate’s assumptions take into account expected productivity gains and pressure points, PSPC has indicated that a more detailed qualitative assessment of CRs is underway. Therefore, although this cost estimate projects that funding will need to continue for approximately five years based on current assumptions, PSPC expects that its investments to date, its increased processing capacity, its improved data analytics and strategic implementation of fixes will result in shorter timelines to stabilize the queue of transactions and address pay outcomes for staff.

The rate of resolution of CRs should be monitored to ensure that these improvements are having the expected benefits on the transaction queue.

PSPC has also indicated that it will adopt a managed services model for systems support to assure outcomes at a fixed price, gradually transfer risk, and leverage private sector innovation through incentives.

Under this model, PSPC will leverage the expertise of external resources, which will assume more of the responsibility to deliver specific outcomes at a fixed cost. Implementing the functional managed services model will increase the number of external resources that are expected to strengthen pay system operations. This approach will enable a significant portion of public servants who work in compensation to focus exclusively on enhancing and stabilizing Phoenix.

In June 2018, PSPC was establishing a Pay Advisory Board to provide strategic and operational input and systems-related recommendations on enhancements and performance. The board will include representation from key federal departments, including TBS, and private sector partners, including Oracle and IBM. System-related decisions will be evaluated, and key recommendations will flow from this working group. Linkages to existing governance structures will also be established.

Operating costs for pay administration

In this section

This section summarizes the planned and unplanned operating costs to be incurred by PSPC, TBS, CRA and other departments for the 2018 to 2019 fiscal year.

Background

The planned annual operating costs for pay administration activities are estimated to be $160.0 million per year ($92.6 million for PSPC and $67.3 million for departments not supported by the Pay Centre)Footnote 7. As already described, the challenges with implementing Phoenix and the related resourcing issues of the Pay Centre led PSPC and other departments to change their respective processes for paying employees. Much of the same data collected to undertake the historical analysis were used to estimate these annual operating costs. The planned estimate of $160.0 million to operate the pay system will need to be reassessed once stabilization is reached.

It is assumed that PSPC will maintain increased capacity at the Pay Centre and for the Pay Operations team, at a cost of $258.0 million annually until Phoenix is stabilized ($145.3 million for Pay Centre capacity, $107.5 million for unplanned systems costs, and $5.2 million for the integrated teamFootnote 8). This includes the ongoing pursuit of innovations and service delivery improvements, data analytics and working with the private sector on application-managed services of pay system operations. Unplanned, one-time costs of $37.7 million are expected to be incurred to support activities such as improving workload management tools and making necessary system upgrades to Phoenix.

Similarly, departments will continue to support their employees until the pay system is stabilized or a new pay delivery model is successfully implemented. The unplanned operating cost of delivering this service is projected to remain at current levels of $64.9 million annually ($46.8 million for unplanned costs by departments supported by the Pay Centre and $18.1 million for unplanned costs by departments not supported by the Pay Centre). In addition:

  • TBS will continue to operate its Claims Office until all pay issues have been resolved and a sustainable solution has been implemented
  • TBS’s Human Resources Management Transformation team will be maintained until the end of the 2018 to 2019 fiscal year
  • TBS will coordinate the distribution of funding to departments to increase staff to support employees who have pay issues ($25 million) in the 2018 to 2019 fiscal year

CRA will receive $3.8 million in the 2018 to 2019 fiscal year from Budget 2018 to support its increased workload of tax queries and file reviews. CRA’s ongoing costs have not been estimated, as it is unclear whether this function will be required beyond two years, and these unplanned costs are immaterial to the final estimate.

Estimate of operating costs (planned and unplanned)

Table 4 shows projected planned and unplanned annual operating costs for the 2018 to 2019 fiscal year and unplanned, one-time costs of pay administration.

Table 4. Projected planned and unplanned operating costs for the 2018 to 2019 fiscal year and one-time operating costs to be incurred during stabilization period
(in millions of dollars)
Activity or item Planned operating costs per yeartable 4 note * Unplanned operating costs per yeartable 4 note * Unplanned, one-time coststable 4 note *

Table 4 Notes

Table 4 Note *

Columns may not add due to rounding.

Return to table 4 note * referrer

Regular operations of the pay system (PSPC) 92.6 N/A N/A
Enhanced capacity and emergency measures (PSPC)
Pay Centre capacity
 N/A 145.3 N/A
Systems, one-time costs
 N/A N/A 37.7
Systems, unplanned costs
 N/A 107.5 N/A
Integrated team
 N/A 5.2 N/A
Subtotal for enhanced capacity and emergency measures (PSPC)
 N/A 258.0 37.7
Departmental compensation
Departments supported by the Pay Centre, unplanned costs
 N/A 46.8 N/A
Departments not supported by the Pay Centre, planned costs
 67.3 N/A N/A
Departments not supported by the Pay Centre, unplanned costs
 N/A 18.1 N/A
Funding to departments to increase staff to support employees with pay issues
 N/A N/A 25.0
Subtotal for departmental compensation
 67.3 64.9 25.0
Treasury Board of Canada Secretariat (including the Claims Office) N/A 3.7 N/A
Treasury Board of Canada Secretariat, one-time costs (including Human Resources Management Transformation team) N/A N/A 7.0
Canada Revenue Agency, one-time costs N/A N/A 3.8
Total operating costs per year, based on the 2018 to 2019 fiscal year, and one-time costs to be incurred during stabilization period 160.0 326.6 73.6

Considerations

Unplanned operating costs are assumed to remain at current levels until Phoenix is stabilized. However, these costs are not projected because it is unknown how they will be impacted by stabilization efforts. The ongoing operating costs of the pay system can be better estimated in the future as the impact of system stabilization activities are better understood.

As with historical costs, operating costs to departments may be understated, as opportunity costs are not captured in departmental accounting systems. TBS is exploring the option of creating financial coding or a consistent methodology to enable departments to more completely capture costs that are directly attributable to Phoenix.

Risk analysis of unplanned operating and stabilization costs

When developing a cost estimate for a complex IT project, it is important to understand that uncertainty regarding the variables can significantly impact the results of the cost estimate. The unplanned activities contain some element of uncertainty with respect to the number of additional FTEs required to operate the system and the level of effort required to complete CRs.

The risk associated with this estimate was modelled using statistical methods, and it was found that:

  • the most likely estimate of unplanned operating and system stabilization costs is $337 million
  • it is not likely that unplanned operating and system stabilization costs will exceed $346 million based on the assumptions used for this report

The resulting range is very narrow because most of the unplanned costs are the result of hiring additional people, who have been assumed to be available when required. Additional information is provided in Appendix B.

Out of scope

It is important to draw clear boundaries when defining the scope of a cost estimate. This section summarizes the more noteworthy costs that are excluded from this report:

  1. “Future transformation” funding of $16 million provided in Budget 2018 to explore alternatives to Phoenix is outside the scope of this cost estimate because it is independent of the efforts to stabilize Phoenix.
  2. Pension administration costs have increased because the Pension Centre hired additional staff to address data integrity issues originating from the implementation of Phoenix.
  3. Future claims that will be made to the TBS Claims Office cannot be estimated because there is no means to ascertain their number and the extent to which public servants have incurred eligible expenses because of Phoenix.
  4. Unassessed damages that may be negotiated or awarded by the courts. The value of these damages was not estimated because they could not be quantified, and attempting to do so could be seen to prejudice ongoing negotiations and court proceedings. However, it is acknowledged that there is a possibility of significant additional costs.

Conclusion

The purpose of this report is to communicate TBS’s estimate of costs to operate the pay system and stabilize Phoenix. Table 5 summarizes this estimate.

Table 5. Estimate of costs to operate the pay system and stabilize Phoenix
(in millions of dollars)
Item Actual costs as of March 31, 2018 Planned costs per year Unplanned costs per year Unplanned, one-time costs
Historical costs (details available in Table 2)
Original project costs (2009 to 2010 fiscal year to the 2017 to 2018 fiscal year)
 309.2 N/A N/A N/A
Planned operating costs (2013 to 2014 fiscal year to the 2017 to 2018 fiscal year)
 380.9 N/A N/A N/A
Unplanned operating costs (2016 to 2017 fiscal year to the 2017 to 2018 fiscal year)
 360.6 N/A N/A N/A
System stabilization costs (details available in Table 3)
Costs per year, unplanned
 N/A N/A 9.8 N/A
One-time costs, unplanned, in the stabilization period
 N/A N/A N/A 50.8
Operating costs (details available in Table 4)
Operating costs per year, planned
 N/A 160.0 N/A N/A
Operating costs per year, unplanned
 N/A N/A 326.6 N/A
One-time costs, unplanned in the stabilization period
 N/A N/A N/A 73.6

All historical costs are reported as-spent, and system stabilization and operating costs are in constant year, Canadian dollars. This cost estimate is based on accepted international best practices and is evidence-based.

To date, the unplanned spending on the pay system has been primarily for employees who are directly involved in processing pay transactions. A key assumption of this report is that the volume of work in the transaction queue is heavily correlated with the development work (CRs) that remains. If the backlog of CRs were mitigated and the missing system functionality were implemented, it is expected that the transaction queue will decline to an acceptable size with little or no additional intervention. PSPC has indicated that a more detailed qualitative assessment of CRs is now underway in support of this approach.

It should be noted that it is unlikely for any system to have a CR backlog of zero; there will always be a level of ongoing errors or issues to resolve. System stabilization is therefore estimated to be a CR backlog that does not have more than two months of activity.

Given the dynamic environment in which efforts to address problems with Phoenix is occurring, this estimate is a snapshot in time based on available data and assumptions. Therefore, although this cost estimate projects that funding will need to continue for approximately five years based on current assumptions, PSPC expects that its investments to date, its increased processing capacity, its improved data analytics and strategic implementation of fixes will result in shorter timelines to stabilize the queue of transactions and address pay outcomes for staff.

Appendix A: complete ground rules and assumptions

In this section

Scope

The scope of this cost estimate has been broadly defined as all costs directly attributable to the implementation and stabilization of the Phoenix pay system. Specifically, this estimate comprises five types of costs:

  1. Phoenix pay system costs include the cost of the original project, materials, contracts and employees’ time to implement all fixes and enhancements so that Phoenix functions at an acceptable level of performance. These costs also include any short- and medium-term enhanced capacity required to reduce the rate of incident reports and the backlog of CRs.
  2. Operating costs include planned and unplanned costs:
    • planned costs include costs of materials, contracts and employees’ time required for regular systems operations for the Pay Operations team, the Pay Centre and all government departments
    • unplanned costs are additional operating costs incurred in the short and medium term to:
      • increase capacity of the Pay Centre and all government departments to manage employee compensation files
      • reduce the queue of transactions in Phoenix
  3. The cost to review every employee pay file to restore employee and public confidence, once Phoenix has been stabilized
  4. TBS’s cost to:
    • operate the Claims Office until all pay issues are resolved
    • maintain the Human Resources Management Transformation team until the end of the 2018 to 2019 fiscal year
  5. CRA’s costs include additional resources for CRA to review and correct the tax files of employees who have been affected by Phoenix implementation challenges

Ground rules

Four ground rules for this cost estimate frame the expected outcomes that will impact the estimate:

  1. Functionality and stabilization: Phoenix was originally designed as part of a complete HR-to-pay process that would replace the aged RPS. This cost estimate considers “stabilization” to include all known system fixes and enhancements required to reach a level of functionality and reliability that is at least equal to that of the RPS, using historical benchmarks (for example, CRs) from actual RPS data. The benchmark for functionality and stability of the backlog of CRs is that there are no more than two months of activity, based on analysis of the RPS.
  2. Departmental costs: Challenges with Phoenix’s implementation resulted in additional costs to departments. As much as possible, costs are fully loaded, including elements such as employee benefits and pensions, office accommodation, and shared services costs:
    • Unplanned costs: This cost estimate includes the cost to departments of increasing their compensation function above historical levels, based on expenditures recorded in departmental financial systems.
    • Opportunity costs: The persistence of Phoenix implementation challenges required individual departments to develop additional capacity to support affected employees. Some departments incurred costs to hire additional compensation advisors and pay overtime to existing staff. In addition, many departments postponed work or reassigned staff, but these changes may not have been coded in the compensation function of departmental financial systems. Therefore, the opportunity costs associated with postponed work and uncoded reassignments of staff could not be accounted for in this cost estimate.
    • Productivity costs: Employees have spent time attempting to resolve pay issues while at work. Similarly, some employees have taken increased sick leave as a consequence of the stress caused by financial hardship. However, there are no data to enable a cost estimate for this lost productivity.
  3. Damages and expense claims: Federal public servants across the country have experienced hardship because of problems with their pay, resulting in claims of unfair labour practices, grievances, and other potential complaints to the Federal Public Sector Labour Relations and Employment Board. Efforts are being made to compensate employees for expenses incurred (for example, banking fees and late payment penalties) through TBS’s Claims Office and other means. Because it is not possible to predict the outcome of all negotiations and court proceedings, only negotiated damages and claimed expenses are accounted for in this cost estimate.
  4. HR systems costs include the cost of PSPC to validate data to resolve pay issues caused by inaccurate employee information in departmental HR systems.

Assumptions

In this cost estimate, assumptions are used to fill gaps where data are missing or incomplete. Some of these assumptions may be updated or replaced with actual cost and performance data as work progresses:

  1. Governance: Phoenix stabilization activities will be governed and managed according to best practices (under a single business owner and under a single project lead).
  2. Time needed to stabilize the Phoenix system: Best estimates of capacity and existing trend lines regarding CRs were used to estimate the approximate time needed to stabilize the Phoenix pay system. Given the differences between the RPS and Phoenix, using CRs as a comparator, while not perfect, is the best option for this cost estimate.
  3. Reducing the CR backlog and transaction queue: It is assumed that additional resources will be allocated to address the CR backlog and transaction queue.
  4. Capacity: This cost estimate assumes that it will be possible to recruit staff to increase capacity when and where needed.
  5. Productivity and learning: This cost estimate assumes that new staff will need time to become fully productive in their new positions.
  6. Productivity and work design: PSPC is continuing to make changes to improve the efficiency of the Pay Centre. Because this is an ongoing activity, data to assess the impacts of these changes are not yet available. It is assumed that existing staff at the Pay Centre and in the Pay Operations team have yet to reach maximum productivity.
  7. Funds spent: Additional funds have been provided to PSPC to stabilize Phoenix. It is assumed that all funds have been spent on the activities for which they were intended. It is also assumed that data provided by departments from their financial accounting systems are accurate.
  8. Oracle software support: It is assumed that Oracle software will support the Phoenix pay solution during the time frame considered in this cost estimate.

Appendix B: complete methodology

In this section

This analysis was conducted under the leadership of the Comptroller General of Canada. The analysis team was composed of costing professionals who have backgrounds in computer engineering, industrial engineering, and accounting. The costing models were overseen by a Fellow of the International Function Point Users Group and a number of Certified Cost Estimator/Analyst designation holders.

Historical costs

This section describes how planned and unplanned operating costs to departments were calculated from regular and expected costs.

An evidence-based approach was taken to ensure comparable and consistent results across all departments. Unplanned costs were calculated from data gathered from departmental financial systems. Departments were given a choice of:

  • providing raw data taken directly from their departmental financial system
  • completing a template that captured actual costs per fiscal year and month for the prescribed period of seven years

Due to time constraints, the Phoenix costing team was able to analyze only the compensation function. However, the data received for the accounting operations and HR systems functions are being kept for future analysis. Additional data used in this analysis were each department’s Phoenix go-live date and whether the department’s compensation function was transferred to the Pay Centre in Miramichi.

The Phoenix go-live dates were used to determine the cost of the compensation function in each department for the preceding yearFootnote 9. Unplanned cost components were estimated by comparing the costs incurred in the 12 months prior to implementing Phoenix with the costs incurred in the following months. Unplanned costs incurred in the second year were pro-rated because the data used in this analysis were as of December 31, 2017 (three months before the end of the fiscal year).

Estimating from incomplete data extracts

Once the data were received, it was discovered that some data extracts were not coded in a way that allowed analysts to identify the compensation function. Departments that submitted such data were provided the choice of submitting new data or accepting an estimate based on the average unplanned cost per employee of a comparable department.Footnote 10

System stabilization and enhancement costs

Pay system costs

This methodology estimates whether current allocated development resources are sufficient to clear the backlog of CRsFootnote 11. The cost estimate of stabilization and enhancements to Phoenix was developed by:

  • defining what it means to achieve stabilization
  • estimating the workload to achieve stabilization
  • determining whether stabilization can be achieved with existing resources
  • estimating the cost of any additional resources required to achieve stabilization

This analysis concluded that additional resources are likely required. The resulting cost estimate is for the unplanned costs of additional resources required to reach system stabilization. PSPC indicates that the shift toward application-managed systems, while leveraging the private sector, is forecasted to hasten the realization of improvements to pay system operations.

To initiate the analysis of the stabilization costs, a time frame needed to be established as an initial assumption. Based on a review of literature, five years was selected as a reasonable length to initiate the analysis. This assumption was subsequently tested as part of the costing methodology.

Methodology

Phoenix will be considered “stabilized” when:

  • the backlog of CRs has been mitigated
  • the volume of incoming CRs has reached a steady state

Phoenix went live during the winter and spring of 2016, but the best available data to estimate the current state of Phoenix is to use CRs for the time frame from July 1, 2017, to March 31, 2018. Over this period, the Pay Operations team implemented a total of 103 CRs, or approximately 137 CRs per year.

However, there were 253 outstanding CRs as of March 31, 2018, plus an additional 42 CRs that had been identified by PSPC. One of the new CRs indicated the need to examine individual and mass retroactive actionsFootnote 12. The Phoenix costing team determined that no fewer than 10 additional CRs would be required to automate retroactive functions in a way that would enable the Pay Centre to meaningfully reduce the transaction queue. To account for this effort, 10 more CRs have been added to the list, bringing the total number of CRs in the backlog to 305. Added to this is a projected 263 new CRs per year. In order to resolve the backlog of CRs, it is estimated that 346 CRs need to be implemented each year. The Pay Operations team will require an additional 59 FTEs to achieve this target.

This estimate was validated using a second methodology. A function-point analysis of completed and current CRs was performed in order to determine the work effort required to stabilize Phoenix. This analysis estimated that 62 FTEs would need to be added to PSPC’s functional and Chief Information Officer Branch teams, which is only three FTEs higher than predicted by the primary method.

In conducting this analysis, it was determined that money has not been allocated to develop a test system. An additional seven contractor (technical) FTEs for one year have therefore been added to the estimate. This $1.8 million is a one-year charge only and is not included in unplanned operating costs.

Additional staffing would be acquired through a combination of contracts and staffing. Outside resources are available for between $254,000 and $275,000 per contractor per year, and the fully loaded cost of departmental resources is $105,200 per employee per year. The higher cost of contracted labour reflects the temporary nature of their employment and the expertise required for the work.

Pay file review

A pay file review of this magnitude within government is unprecedented. The most comparable function that exists within government is the file review by the Pension Centre when an employee retires. Although the two tasks are not identical, the Pay Centre’s file review was the best available proxy for this component of the cost estimate.

The cost estimate for the pay file review was developed using an analogy method; in other words, the known level of effort to complete a file review at the Pension Centre was adjusted to reflect the scope of tasks involved in a pay file review for quality assurance. Because this analogy is based on an early scoping of activities required to review a pay file, it was assumed to be no better than a rough-order-of-magnitude quality estimate, and a range of ±50% was applied to the result.

The Pension Centre’s process has two parts:

  1. An analyst at the AS-02 level reviews an employee’s pay information to see whether there is anything unusual and resolves any outstanding issues. This step takes approximately 90 minutes per file and costs about $64.00.
  2. The file is forwarded to an analyst at the AS-03 level who undertakes a quality assurance verification. This step takes approximately 45 minutes per file and costs about $34.00.

The total cost of reviewing a pay file is estimated to be $98.00.

It is difficult to estimate the number of pay files that have been and will be affected by Phoenix when the pay file review begins. In order to restore employee confidence, it is assumed that every pay file will be reviewed. There are approximately 200,000 staff served by the Pay Centre and 100,000 staff who are not; however, staff turnover and seasonality means that there are a greater number of pay files implicated in the review. It is estimated that approximately 500,000 pay files will require review.

Figure 1 shows the calculation of the estimated cost for the pay file review.

Figure 1. Calculation of estimated cost for the pay file review

Pay file review = (cost per file × number of files) ±50%

= ($98.00 × 500,000) ±50%

= $24.5 million to $73.5 million

Operating costs of pay administration for the 2018 to 2019 fiscal year

Projected operating costs for PSPC include:

  • ongoing costs for the regular operation of the pay system
  • unplanned capacity costs
  • unplanned, one-time costs for specific items

TBS’s costs include those for:

  • the Human Resources Management Transformation team
  • the Claims Office team
  • the Phoenix costing team

CRA’s costs were based on the Budget 2018 announcement of $5.5 million over two years, starting in the 2017 to 2018 fiscal year. The methodology used to calculate future expenses to departments was derived from the results of the methodology used to calculate historical costs.

Risk analysis of unplanned operating and stabilization costs, per year

When developing a cost estimate for a complex IT project, it is important to understand that uncertainty regarding the variables can significantly impact the results of the cost estimate. Risk-modelling software (Oracle Crystal Ball) was used to model the key cost drivers and generate a risk range.

This analysis combined the above costs associated with stabilization and enhancements with the other unplanned operating costs described in this report. The primary drivers of this analysis were:

  • the number of FTEs required to operate the pay system
  • contractor costs
  • the total number of employees in government
  • the average cost for providing compensation to employees

Figure 2 illustrates the resulting distribution of the cost estimate. The range was taken at the 50th confidence interval (where possible outcomes are equally likely to be over or under the estimate) and at the 80th confidence interval (a risk-averse estimate where only 20% of scenarios would exceed the estimate). The resulting distribution is very narrow because most of the unplanned costs are the result of hiring of additional people, who have been assumed to be available when required.

Figure 2. Risk analysis of unplanned operating costs, per year (approximating a normal distribution curve)
Risk analysis of unplanned operating costs, per year. Text version below:
Figure 2 - Text version

Figure 2 shows a normal distribution curve, which is bell shaped, for the projected unplanned operating costs per year. The range is from $301 million to $382 million, where the 50th confidence interval cost is $337 million and the 80th confidence interval cost is $346 million.

© Her Majesty the Queen in Right of Canada, represented by the President of the Treasury Board, 2018,
ISBN: 978-0-660-26574-2

Page details

Date modified: