Implementation of Service Transformation

Final Report

Approved February 11, 2010

Table of Contents

• Executive Summary
• 1. Introduction
• 2. Background
• 3. Risks
• 4. Objectives
• 5. Scope
• 6. Statement of assurance
• 7. Methodology
• 8. Observations and recommendations
• Annex A - Management Control Framework over Implementation of Service Transformation
• Annex B - Glossary

Executive Summary

Context

The Staffing and Assessment Services Branch (SASB) has undertaken a comprehensive business transformation to reposition itself as a common service provider for staffing and assessment across the public service, to complement the services available within federal organizations. SASB's intention was to base future services on its clients' most important needs and priorities, including services it traditionally did not offer. Accordingly, in 2006, the Branch adopted a three -year project to put the necessary elements in place.

Rationale

In view of SASB's re-engineering of its operations to be significantly based on cost recovery, management was interested in obtaining assurances with regard to the various controls and mechanisms to enable the Branch to achieve the transformation of its services.

Objective

The audit objective was to determine whether the Service Transformation management control framework, including processes and structures for providing services to other departments and agencies, has been put in place, is operating effectively and addresses risk appropriately.

Conclusion

Since February 2006, SASB has put considerable effort into planning and preparing for the transformation, complete with processes and structures for providing services to other federal organizations. Overall, the transformation project had a clear vision and most of the elements of a sound structure.

SASB has since made significant progress in putting in the tools needed for the transformation such that most elements of a sound management control structure are in place. The organizational structure has been defined and staffing gaps are being addressed with rigorous planning strategies. Included is training to better prepare employees for the new business environment and ensure optimal use of tools and processes. IT systems for business processes are becoming more robust and efficient through upgrades, while providing for more integrated approaches in processing data. Controls have been put in place or are being developed to ensure that operations are running smoothly across the Branch. The feedback and consultation mechanisms SASB has put in place will allow it to collect valuable information on its current offerings and on how to position itself and adapt its products and services to meet the evolving needs of its clients.

To thrive in this new business environment, SASB must continue to enhance integration of processes so that business processes stay efficient and long-run costs are lowered. It must also capitalize on the business intelligence gained through its operations and further develop its risk assessment capacity. This will help management adjust course in this dynamic environment.

Management has given due consideration to the observations from this audit and has provided sound action plans to address each of the audit recommendations.

1. Introduction

In May 2008, the Internal Audit Committee (IAC) of the Public Service Commission (PSC) approved the Audit of the Implementation of Service Transformation as part of the Internal Audit Plan for the fiscal year 2008-2009. The planning phase for this audit started in February 2009. A preliminary survey report was presented at the IAC August meeting. The members recommended the audit be finalized within six months' time by examining the readiness of the Staffing and Assessment Services Branch's (SASB) Service Transformation initiative.

2. Background

In response to the new staffing environment, SASB undertook a comprehensive business transformation to reposition itself as a common service provider for staffing and assessment across the public service, to complement the services available within federal organizations. SASB was to base future services on its clients' most important needs and priorities, including services it traditionally did not offer. Transformation was to be attained within the three-year period ending March 2009. By April 2008, SASB was ready to offer staffing and assessment services based on a new cost-recovery business model in line with central agency requirements. While mandatory services provided to federal organizations are funded by parliamentary appropriations, optional services are based mainly on cost recovery. SASB's operations have been converted from an allocations basis to one of cost recovery, where appropriate.

The transition model used for SASB identified a key lead individual for each of four main building blocks: human resources (HR) and change management; products and services; financial management; and service delivery and marketing. A master plan was designed with key milestones and applicable timelines to be implemented in three phases. This was later reduced to two phases, with phases two and three being combined. A Project Management Office (PMO) was also created to oversee the project as a whole. On April 1, 2008, the PMO, was disbanded and key milestones not yet achieved were absorbed into the daily operations of the organization.

Processes and key activities still coming on-line in April 2008 included such tasks as direct access, a marketing action plan, retention and staffing in SASB, and forecasting. Individuals responsible for the main “pillars” of change started to move on to other areas as the PMO closed.

3. Risks

Prior to the start of the audit, SASB's environment for cost recovery was reviewed for apparent risks and controls. This indicated that SASB faced challenges in the classification and staffing of new positions, as well as in staff training, and in the time to carry out new functions and launch a business based on cost recovery. The SASB funding envelope, even with cost-recovery revenues, does not cover the required organizational structure. SASB needs to be flexible to quickly adjust its organizational design and HR priorities in response to feedback, particularly from clients, if it is to successfully navigate in the environment of the current Public Service Employment Act (PSEA). In order for SASB to meet these challenges, there must be a high degree of communication at all levels for decision-making purposes.

4. Objectives

The objective of this audit is to determine whether the Service Transformation management control framework, including processes and structures for providing services to other departments and agencies, has been put in place, is operating effectively and addresses risk appropriately.

We expect that:

• the organization at its highest level has communicated a strategic need for the organization to change;
• the organization has planned for the transition; and
• the framework for providing assessment services to other departments and agencies has been put in place, is operating effectively and addresses risks appropriately.

The audit criteria for the examinations are listed in Annex A.

5. Scope

The audit covered the:

1. establishment of new or revamped business lines and processes and the mechanisms to enable them; and
2. necessary tools being incorporated into daily operations, their status, and applicable timelines for completion.

The audit did not look into the accuracy and timing of revenue billing, as these have been covered by the Cyclical Audit (Revenue). Nor has the audit examined the efficiency of the billing process, which has been the subject of a consulting contract. However, the audit has taken note of work undertaken as part of the billing process review and its impact on the Branch's operational performance.

6. Statement of assurance

The internal audit was conducted in accordance with PSC standards, based on the Institute of Internal Auditors, standards and the Treasury Board of Canada's policy. The PSC's Internal Audit Directorate (IAD) is working toward full compliance with all applicable standards. We have examined sufficient evidence and collected the information necessary to arrive at the conclusions made. In some cases, the evidence sought was not available, resulting in an observation to this effect.

7. Methodology

The PSC Standard Audit process includes three principal phases: the preliminary survey phase, the detailed examination phase, and the reporting phase. Methodologies included interviews with management and staff; and review and analysis of key processes and documents, including systems-generated reports and financial data.

For this audit, a preliminary risk assessment was used to identify lines of inquiry. Internal Audit developed draft criteria using the document Core Management Controls: A Guide for Internal Auditors (Draft-November 2007) of the Office of the Comptroller General of Canada (OCG) and guidance from the Project Management Institute.

The Director, Internal Audit, reviewed and signed off on all deliverables. Briefings and validations of observations were ongoing during the course of the audit with the Vice-President of SASB and his representatives as part of the audit process. SASB has provided requested documents and access to employees in a timely manner in order to expedite this audit.

The planning phase for this audit started in February 2009. A preliminary survey report (PSR) including a series of recommendations, was presented to the IAC at its August 2009 meeting. In light of the management action plan included in the PSR, the members recommended that IAD proceed with a detailed examination in six months' time and prepare a final report for the following meeting. The criteria for the examination phase were based on key SASB plans relating to establishing a complete management control framework for its new business processes.

8. Observations and recommendations

Need for change

Control Objective - The organization at its highest level has communicated a strategic need for the organization to change.

We expected that senior management had set out clear messaging for the imperative for change.

The review revealed that discussions on the requirement for change had taken place at all levels (Branch Management Committee, Executive Management Committee, and the Internal Audit Committee). The imperative for change was set in process with the adoption of the current PSEA and a review of the suitable mandate for SASB. Senior management, including the President and the VPs, led the discussion with departments and agencies about SASB's changing role.

The audit found that SASB had communicated a vision for the transformation project. Specifically, this was to achieve a government-wide approach to staffing that focuses on attracting and retaining talented individuals to meet the current and future needs of the public service by creating a flexible environment for differing organizational needs; providing common staffing tools and support; and supporting accountability, oversight and the values of fairness, transparency, representativeness and access.

Planning for change

Control Objective - The organization has planned for the transition.

The review revealed that SASB had undertaken an evaluation of the impact of the current PSEA to ensure that all stakeholders, within the PSC and outside, understand the consequences of the Act on them and the work they do.

It was expected that the transformation project would have effective mechanisms for governance and control: a charter, oversight, and clear roles and responsibilities, as well as clear expectations.

We found that the planning phase of the transformation was well conducted utilizing a project work plan , with key milestones, challenges and next steps. No formal charter existed. However, approval from the proper authorities validated the direction adopted for the project.

Operational plans were available for each of the pillars for implementation and risks were identified in advance of the implementation. The first pillar, human resources and change management, involved building and planning HR capacity and developing learning strategies, and then implementing change management and a new organizational structure. The products and services pillar involved introducing and reviewing pilot projects, articulating SASB's role in relation to oversight and creating a catalogue of services and products. The financial management pillar meant developing a costing methodology and a funding strategy and model, as well as a new resource management approach. The last pillar entailed the promotion of services and products, and the development of service standards and a service delivery model. The overall initiative fell under the direction of a director general dedicated full-time to this project.

One of the critical factors missing from the planning phase was the identification of expectations and a formal reporting process for assessing progress. Consequently, successes or impediments to implementation were not readily identifiable. This deficiency was particularly noticeable in light of the relatively loose and fragmented governance structure.

Implementing change

Control Objective - The framework for providing assessment services to other departments and agencies has been put in place, is operating effectively, and addresses risks appropriately.

i) Tools

We expected that the project would have resulted in employees having the tools and resources needed in the new market environment. The audit found that the project was partly successful in delivering to employees the tools and resources initially identified. Many of the key initiatives are implemented, but others were in various stages of approval, development or rollout. This implementation was rendered more complex by the interrelationships between many of the key deliverables and shortages of funding. Some of the key gaps are: an efficient billing system, products and service standards, and a service delivery model. Staff indicated that these areas are being developed as resources become available, with plans for the corresponding tools and training to follow.

In response to the preliminary survey, the Branch confirmed it had instituted organizational charts, and stated that the staffing of key positions was under way. Detailed examination revealed that SASB has been active in addressing these HR structural issues. The establishment of organizational charts and defined roles, as well as a review and update of succession plans, helps clarify the responsibilities of employees as the organization's structure and functions evolve. Though roles and responsibilities have not been formally defined, management is aware of this gap and will allocate the necessary resources to this issue when appropriate. These elements also support the Branch's HR management strategy for both the short term – given the Branch's reliance on a contingency workforce – and the long term – i.e. staffing for strategic positions such as financial analyst and costing analyst for the Integrated Services Directorate (ISD). These two positions are instrumental in the development of solutions to SASB's product costing.

Concurrently, SASB has been carrying out various training initiatives both in the regions and at HQ to prepare employees for the new business environment. SASB's Learning Passport, updated to include, among other things, training requirements related to project management and business acumen, was distributed to all SASB employees in May 2009. In addition, training on IT system updates in the fall of 2009 ensured that users could utilize the enhancements to their fullest and optimize operations.

Management also identified and developed tools to improve business processes. The Sales Order Management system (SOM) simplifies the sales order process and captures more customer information, which could indirectly enhance customer satisfaction and retention. This cost-effective, Excel-based tool, developed in-house, was deployed in August 2009. Usage of SOM by the employees has been slower than expected, affecting both the processing of the sales order and the capture of data. Management is looking at ways to increase use of the system. SASB is also developing the Work Order Management system (WOM), to augment customer satisfaction and retention by making the service process after customer acceptance (and signature) more efficient. The implementation of the WOM was targeted for winter 2009-2010 but could be delayed until the SOM is more widely used.

Another tool, the Revenue Management System (RMS), developed in-house to facilitate the billing of cost recovery operations, has some redundancies and inefficiencies. The Branch agreed to make ad hoc improvements to the current version, RMS 0.5, to smooth billing until a new version could be installed. The introduction of RMS 1.0 is slated for June 2010. This new version should further reduce duplication of effort and bring additional efficiencies through the on-line interface, thus ensuring that bills for services are computed and invoiced in a timely and complete manner.

In connection with modernizing billing, SASB identified some systems that had to be integrated with RMS. For example, the Second Language Evaluation Scheduling System, which is used for booking appointments, required redundant entries to input all of the data into RMS. The coordination of this system into a single-entry process in RMS 0.5 improves efficiency, simplicity and accuracy due to less repeated manual input. The Test Inventory Control System (TICS), another system with similar efficiency problems, will soon be ready for integration as well. The implementation of RMS 1.0, harmonizing these systems, will further simplify the process.

SASB recognized the need to review product pricing and develop a proper costing methodology in line with government requirements. The alignment of prices with costs was addressed in a previous audit (Cyclical Audit - Revenue). This will help ensure the appropriate revenue streams and Branch stability.

Recommendation

1. SASB should complete its planned integration of key business process IT tools to adapt to changing ways of doing business within the timeframes set by management. These tools are as follows:

1. RMS 0.5 upgrades and RMS 1.0 implementation
2. WOM implementation
3. TICS integration

Management Response

RMS 0.5 upgrades, including the report, have been implemented since the summer of 2009. Users have also been trained.

• Responsibility: ISD
• Timeline: Completed

RMS 1.0 is being developed by the Information Technology Services Directorate and the implementation date is June 2010. Finance and Administration Directorate (FAD) is the product owner and project authority. They will be providing project management.

• Responsibility: FAD
• Timeline: June 2010

The WOM is developed and is ready to be rolled out. We need to work with the delivery units to pick up the pace and use the current tools (SOM and Estimator) before we implement the WOM. We are currently targeting implementation and training during the first quarter of 2010-2011.

• Responsibility: ISD
• Timeline: June 2010

The TICS implementation will be delayed until the first quarter of 2010-2011.

• Responsibility: ISD
• Timeline: June 2010

ii) Client Feedback

The audit examined internal and external communications channels. It found that SASB had established a series of tools and events to provide Branch personnel with information on the Transformation Project and to gather information from them. For example, there was a Branch Intranet section devoted to this initiative. Staff could provide feedback and obtain additional information on certain aspects of the transformation project.

For external communication, consultations were held with users and stakeholders as part of the strategic and operational planning of the transformation project. For the external stakeholders group, feedback was collected through both formal methods, e.g. rounds of consultations with external clients, and informal methods, e.g. regular contacts with departments and agencies concerning client relations. Management indicated it had developed a client survey to be sent to clients immediately after services are rendered, thereby giving SASB immediate feedback to improve, refocus and redesign services. The audit noted that further to the client survey conducted by the (PPC) for its products and services, SASB has designed a feedback processto collect and analyze information and comments from clients on its services. As of August 2009, SASB sends bimonthly surveys to its clients for all products and services. A preliminary analysis on findings is shared with the Branch Management Committee (BMC). The first draft of this analysis, which consisted of a compilation of the raw data, as well as an overview of key results, indicated that the two top areas for improvement identified by customers were billing delays and the cost of services. These two issues are being addressed through initiatives discussed later in the audit report.

The new client satisfaction survey may serve as an example of a best practice. The auditors are aware that the number of respondents is limited, making it difficult to draw firm conclusions. Still, the survey provides information on both the clients' profiles and needs as well as on their perception of the organization's performance. This feedback mechanism allows SASB to collect business intelligence to identify changes to the risk environment, better target its market, and gather information on areas where services and products can be improved or adjusted to meet the clients' requirements. SASB's risk environment is evolving rapidly with the imminent changes in demographics, financial pressures on its clients and the need for product development, to name a few factors. The survey also provides SASB with the opportunity to do prospective work to develop new markets, products and services by collecting information on clients' needs beyond current offerings.

SASB's action plan indicated it would draw on information generated by extensive and ongoing client outreach and consultations to be undertaken during the current fiscal year to position PSC products and services to meet actual and forecasted demand. This information on client needs will give SASB the right mix of resources to continue to develop and deliver staffing and assessment solutions to departments and agencies at the level of excellence desired.

The audit found that SASB has complemented the existing advisory bodies with a number of external and internal groups providing feedback, advice and guidance on the staffing and assessment services it provides, e.g. Client Advisory Committee and Services and Products Advisory Group. The newly created External Advisory Committee should have its first meeting early in 2010-2011. The strategic outlook and information garnered from these bodies, composed of representatives of the private and public sectors and academia, are instrumental in the design and improvement of the Branch's offerings in a new business environment. For the most part, the meetings for these advisory committees are scheduled to align with the PSC's planning cycle and SASB's cost-recovery activities.

Since August 2009, SASB has been producing monthly sales data updates to help focus its outreach efforts. The data are extracted from the RMS by FAD. Analysis of these data provides a picture of the current situation and allows management to draw information from trends and patterns to support strategic decision making. It supports other initiatives such as the short-term marketing action plan as well. This plan, to be executed over a six- to nine-month timeframe, is part of a strategy developed in the fall of 2009 to lay the foundation for longer-term marketing initiatives. The action plan notably includes a review of the fee structure, improved awareness of the process and the products, and greater focus on both SASB personnel and the clients, as well as the branding of its products and services and their positioning in a market environment. This project will help both senior management and employees to better understand the brand and position in view of developing a comprehensivecorporate messaging guide for future branch marketing, corporate and sales activities and tools.

iii) Risk assessment

The auditors sought confirmation that management identifies the risks that may preclude achievement of its objectives. Use of risk assessment was noted in various documents and forms. The Project Management Office had an overall risk assessment that was published in the PSC's Operational Plan. Through the preliminary survey, the auditors noted, however, that this risk assessment had changed little over the last several years. Risk management is very important in SASB's new environment. Specifically, PSC's service offerings are often essentially a risk mitigation strategy for departments and agencies facing HR capacity shortages. To that end, PSC risks are amplified as HR shortages emerge elsewhere in the system. Accordingly, robust risk assessment would help management adjust course in this dynamic environment.

In response to the preliminary survey observations, SASB noted that it was committed to reviewing the risk factors and mitigation strategies on a quarterly basis to ensure that the risks identified remain valid and new risks are identified and mitigation strategies put into place where warranted.

Detailed examination indicated that SASB continues to incorporate risks relating to the transformation of services into the PSC's Report on Plans and Priorities (RPP). The BMC members review and update these internal and external risks and mitigation strategies on a quarterly basis, along with the environmental scan.

This high-level review, the analysis of client feedback and input from the various advisory committees are key strategic elements in gathering and analyzing information on SASB's business environment. However, the auditors could not discern a definite process to review and update risks and mitigation strategies on a regular basis. As the public service is set to face challenges on many fronts, SASB needs to quickly develop a process to understand, assess and address its business risks on a regular, if not continuous, basis.

Recommendation

2. SASB management should implement a risk assessment process that identifies significant operational risks and mitigation strategies on a regular basis to assess and address its business risks, and contribute to the quarterly review and update of risks relating to the transformation of services incorporated into the Report on Plans and Priorities (RPP).

Management Response

The SASB Branch Management Committee will be performing quarterly reviews of the risks associated with cost recovery services and other mandatory services. This exercise will determine if the previously identified risks are still relevant, review and update the current mitigation strategies, and identify new potential risks, impacts, probabilities and mitigation strategies. This exercise will contribute to the corporate risk update and the RPP.

• Responsibility: Integrated Services Directorate
• Timeline: Ongoing; however, quarterly reviews of risk are presently done informally. They will be conducted formally starting in March 2010.

iv) Controls

Along with a risk assessment process, we expected SASB to identify its key controls to ensure smooth operations and manage risks. The preliminary survey revealed that SASB had not yet identified controls or assessed operating results. Although many aspects of the new business processes were in place, there were very few controls identified in relation to monitoring or assessing outputs, such as for timeliness. Efforts were being made toward obtaining a stable status before putting monitoring systems into place. The auditors could not locate any standard monitoring or reporting formats being used to assess functioning of its key controls. Discussions with key players disclosed that the need to report on new controls ended once the PMO was dismantled and the project implementation was deemed complete.

Even though the Branch had identified key performance indicators as a component of its initial Service Transformation Business Plan, it recognized that many things have happened since then. Still, SASB agreed to review and assess existing controls used to monitor Branch operational performance for adequacy and adjust these to align with the key risks identified through ongoing risk analysis.

Detailed examination indicated SASB has reviewed and assessed various controls it uses to monitor its operational performance for adequacy. For instance, analyses of data such as revenue against targets or sales volumes are produced monthly and provided to the BMC. Audit review indicated financial controls and monitoring, including challenging of justifications and follow-ups, are performed monthly as part of the monthly variance reporting process. The Branch has also expanded the use of existing tools such as surveys to gather information on client satisfaction, current and future needs, and to collect business intelligence on improvements to its products and services. A review of service standards first developed in 2008-2009 is under way and will be followed by the development of performance indicators, due to be finalized by March 31, 2010. The new service standards and performance indicators should be implemented in April 2010. SASB indicated that a communications strategy will be developed as part of the implementation plan.

One of the major components of the transformation project, the billing process review, has also been addressed in the Cyclical Audit - Revenue. This major endeavour, launched in October 2009, is aimed at streamlining the billing process, while improving the quality of the billing information and ultimately improving invoice issuance and processing time. The auditors noted that this initiative is progressing through the close collaboration of personnel from SASB and FAD.

During the preliminary survey, other than verbal updates on cost recovery and monthly updates to Executive Management Committee (EMC) on SASB's financial performance and operating volumes, the auditors could not locate any standard monitoring or reporting formats on status of the transformation to update senior management or the IAC. Senior management thus did not have a comprehensive appreciation of the status of the transformation project and of its results, fuelling uncertainty as to the degree of risk that the PSC is exposed to. The VP, SASB, at the request of the IAC members, has since been presenting more specific information on the follow-up to its action plans.

Recommendation

3. SASB should complete the development, validation and implementation of its service standards and performance indicators.

Management Response

SASB is finalizing the review of the service standards.

• Responsibility: Integrated Services Directorate (ISD)
• Timeline: February 2010

Service performance indicators are being developed along with an implementation strategy and plan, including a communications plan to clients. A monitoring and reporting strategy will also be developed.

• Responsibility: ISD
• Timeline: April 2010

Conclusion

Since February 2006, SASB has put considerable effort into planning and preparing for the transformation, complete with processes and structures for providing services to other federal organizations. Overall, the transformation project had a clear vision and most of the elements of a sound structure.

SASB has since made significant progress in putting in the tools needed for the transformation such that most elements of a sound management control structure are in place. The organizational structure has been defined and staffing gaps are being addressed with rigorous planning strategies. Included is training to better prepare employees for the new business environment and ensure optimal use of tools and processes. IT systems for business processes are becoming more robust and efficient through upgrades, while providing for more integrated approaches in processing data. Controls have been put in place or are being developed to ensure that operations are running smoothly across the Branch. The feedback and consultation mechanisms SASB has put in place will allow it to collect valuable information on its current offerings and on how to position itself and adapt its products and services to meet the evolving needs of its clients.

To thrive in this new business environment, SASB must continue to enhance integration of processes so that business processes stay efficient and long-run costs are lowered. It must also capitalize on the business intelligence gained through its operations and further develop its risk assessment capacity. This will help management adjust course in this dynamic environment.

Management has given due consideration to the observations from this audit and has provided sound action plans to address each of the audit recommendations.

Annex A - Management Control Framework over Implementation of Service Transformation

Annex B - Glossary