Audit of the ITB Work Order Process

Final Report

Corporate Audit and Evaluation Branch
January 2011


Table of Contents

Executive Summary

Background: The Information Technology Branch (ITB) has an established process to track and manage work requested by internal and external clients, called the ITB Work Order (WO) process. Clients submit work requests to ITB using a work order form, commonly referred to by its form number, RC502. A work order request can range in complexity from a computer purchase to a large multi-year system development project. In Fiscal Year (FY) 2009-2010, 1,613[Footnote 1] work orders were processed by ITB.

There are various sources of funding for work orders submitted to ITB. For work orders funded by Inter-Branch Charging (IBC), cost estimates are prepared by ITB in a summary document, referred to by its form number, RC503, which requires client financial authority signature prior to work commencing. In FY 2009-2010, IBC funded work orders accounted for $23.9M[Footnote 2] in expenditures, and represented 65 percent[Footnote 3] of the number of work orders processed by ITB.

ITB is structured to support the ITB WO process as follows:

The CRBM, WOMC acts as secretariat for the Work Order Review Committee (WORC). The WORC meets weekly and is attended by members from CRBM, DTIM, and Solutions. The WORC reviews all incoming work orders, determines an ITB Office of Primary Interest (OPI) for each work order, and resolves issues related to ITB OPI determination, description of work, and source of funding.

Objective: The objective of the audit was to provide assurance that an effective ITB organizational structure and key controls are in place to ensure that established goals and objectives for the ITB Work Order process are met.

The examination phase of the audit took place from May 2010 to September 2010.

The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing and the Information Systems Audit and Control Association’s (ISACA) Control Objectives for Information and related Technology (COBIT).

Conclusion: ITB has an established organizational structure to support the ITB WO process. However, the following opportunities for improvement were identified:

Action Plans: ITB is in agreement with the recommendations and has developed action plans to address the opportunities for improvement. ITB will develop, publish, and communicate a comprehensive program mandate, which will identify the owner of the ITB WO process, related accountabilities, and formal goals and objectives for the ITB WO process.

ITB will establish an authority to ensure that CPIs and roles and responsibilities that support the ITB WO process are updated, centrally referenced, and communicated to clients and ITB stakeholders. To reduce delays in processing work orders, ITB will replace the current MS Word RC502 template with an electronic form (E502).

ITB will examine the requirement for continued use of the RC503 sub-process and the IBC project funding model.

ITB will implement a process to monitor and report on the overall effectiveness of ITB cost estimating activities, and develop, publish, and communicate formal service standards for the ITB WO process as part of the comprehensive program mandate.

Introduction

The Canada Revenue Agency (CRA) relies on Information Technology (IT) systems to deliver programs and services to internal and external clients. The CRA manages and maintains over 450 national line-of-business applications and over 350 production databases[Footnote 4] as well as the required IT infrastructure[Footnote 5]. As the needs and expectations of clients evolve, it is important to effectively and efficiently build new systems or modify existing systems to meet business requirements.

The Information Technology Branch (ITB) has an established process, called the ITB Work Order (WO) process, to track and manage IT related work requested by internal CRA clients as well as the Canada Border Services Agency (CBSA). Clients submit work requests to ITB using a work order form, commonly referred to by its form number, RC502. A work order request can range in complexity from a computer purchase to a large multi-year system development project.

IT related work that follows the ITB WO process is typically planned, budgeted, tracked and accounted for separately within the CRA Corporate Administrative System (CAS) using Internal Orders (IOs). IOs capture work order details to support the ITB WO process in estimating and tracking financial information. The ITB WO process includes receiving work requests, creating IOs in CAS, estimating ITB salary, capital, and Operating and Maintenance (O&M) costs, approving estimates, allocating budgets, completing the work, and closing the IOs.

The sources of funding for work orders submitted to ITB include ITB base, Inter-Branch Charging (IBC), corporate, Investment Project (IP), and CBSA development. For work orders funded by IBC, CBSA development, and in some cases corporate or IP, a summary document referred to by its form number, RC503, is prepared by ITB, and requires client financial authority signature, prior to work commencing. The RC503 document summarizes cost estimates for the requested work, including development and ongoing maintenance costs.

In Fiscal Year (FY) 2008-2009, IBC funded work orders accounted for $25.9M [Footnote 6] in expenditures, and represented 61 percent [Footnote 7] of the number of work orders processed by ITB. In FY 2009-2010, IBC funded work orders accounted for $23.9M [Footnote 8] in expenditures, and represented 65 percent [Footnote 9] of the number of work orders processed by ITB.

In 2006, ITB conducted the Work Order Process Improvement (WOPI) Initiative, an internal ITB study that examined the ITB WO process. The study involved stakeholders in ITB and Finance and Administration (F&A) Branch. Several issues were identified, and recommendations resulted in improvements to the IBC and the corresponding billing process, but not all WOPI recommendations were implemented as other priorities took precedence.

ITB is structured to support the ITB WO process as follows:

The CRBM, WOMC acts as secretariat for the Work Order Review Committee (WORC). The WORC is comprised of members from CRBM, DTIM, and Solutions, and meets weekly to resolve work order issues. Where required, ITB Office of Primary Interest (OPI) representatives are invited to the weekly meetings. The WORC reviews all incoming work orders, determines ITB OPIs for each work order, and resolves issues related to ITB OPI determination, description of work, and source of funding. In FYs 2008-2009 and 2009-2010, the WORC reviewed 1,559 and 1,613 work orders respectively[Footnote 10].

The ITB WO process enables ITB to fulfil client requests, and supports the coordination of work across ITB.

Focus of the Audit

The objective of the audit was to provide assurance that an effective ITB organizational structure and key controls are in place to ensure that established goals and objectives for the ITB WO process are met. The scope of the audit included ITB, Assessment and Benefits Services Branch (ABSB), Appeals Branch, Ontario Region and Pacific Region.

The audit methodology included review and analysis of CRA Corporate Policy Instruments, roles, responsibilities, data repositories (including CAS), monitoring and reporting processes. Interviews were also conducted with staff and management in the branches and regions referenced above.

Cost estimating methodologies, which are part of the ITB WO process, were excluded from the scope of this audit, as ITB is implementing the ITB Cost Estimating Model project in FY 2010-2011. However, the audit reviewed responsibilities related to monitoring and reporting of cost estimating activities.

The planning phase of the audit took place from October 2009 to April 2010. The examination phase of the audit took place from May 2010 to September 2010.

The audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing and the Information Systems Audit and Control Association’s (ISACA) Control Objectives for Information and related Technology (COBIT).

Findings, Recommendations, and Action Plans

1.0 ITB Organizational Structure

1.1 Process Ownership

COBIT recommends that the ownership of a process be assigned to one group to ensure accountability for process design, effective interaction with other processes, performance measurement, and results.

Ownership and accountability for the ITB WO process, including the RC503 sub-process, have not been documented and communicated to clients and ITB stakeholders. The audit found that the CRBM, WOMC administers the ITB WO process and CRBM, RMS provides financial management services for the ITB WO process, but current documentation does not identify which organization within ITB is the owner of the ITB WO process.

Recommendation

ITB should define and document a process owner, including accountabilities, for the ITB WO process, and inform clients and ITB stakeholders.

Action Plan

CRBM, Branch Planning, Coordination and Reporting (BPCR) Division will develop, publish, and communicate a comprehensive program mandate, which will identify the owner of the ITB WO process and related accountabilities.

Completion date: Q3 2011-2012

1.2 Goals and Objectives

COBIT recommends that organizations should define, document, and communicate goals and objectives for a process.

The audit found that although many of the documents supporting the ITB WO process describe the activities of the process, none state its goals or objectives.

Clear goals and objectives would avoid differing expectations among clients and ITB stakeholders, which can have unpredictable consequences for the ITB WO process and increase the risk that errors will occur that could lessen the quality of process deliverables. There is also an increased risk that decisions to modify the process may create unintended consequences. Ultimately, the process may not work as intended.

Recommendation

ITB should establish formal goals and objectives for the ITB WO process, and inform clients and ITB stakeholders.

Action Plan

CRBM, BPCR will develop, publish, and communicate a comprehensive program mandate, which will include formal goals and objectives for the ITB WO process.

Completion date: Q3 2011-2012

1.3 Policies, Procedures, Roles and Responsibilities

CRA Corporate Policy Instruments (CPIs), include policies, directives, standards, procedures, and guidelines. CPIs that support the ITB WO process should be approved, up-to-date, communicated, and reviewed periodically.

Organizational success is dependent upon roles and responsibilities being clearly defined. Uncertainty can affect both individual and organization performance. Roles and responsibilities of clients and ITB stakeholders of the ITB WO process should be established, up-to-date, and communicated.

The audit found that CPIs that support the ITB WO process exist and are managed independently by ITB stakeholders. However, CPIs that support the ITB WO process are not all updated and communicated to clients and ITB stakeholders.

Roles and responsibilities of various stakeholders of the ITB WO process have been documented. However, not all have been updated or communicated to clients and ITB stakeholders. Roles and responsibilities are maintained independently by ITB stakeholders and are not always published or centrally referenced.

There is no identified ITB authority to ensure that all CPIs, and roles and responsibilities related to the ITB WO process are approved, consistent, up-to-date, and centrally referenced. Without up-to-date CPIs and roles and responsibilities, clients and ITB stakeholders may not fully understand or comply with current process requirements. Furthermore, additional work may be required for the WOMC to address client and ITB stakeholder enquiries, all of which can result in work order processing delays.

Recommendations

ITB should designate an authority to ensure that CPIs and roles and responsibilities that support the ITB WO process are updated, centrally referenced, and communicated to clients and ITB stakeholders.

ITB should establish a schedule to formally review and update the CPIs and roles and responsibilities.

Action Plans

CRBM intends to hold focused discussions with ITB’s Branch Initiatives Steering Committee (BISC) and F&A in order to establish an authority with these responsibilities. Since CRBM is the current service owner of Financial Management for IT Services, it is anticipated that CRBM will become the ITB WO process authority, and will be governed by BISC.

Completion date: Q4 2010-2011

CRBM intends to hold focused discussions with the BISC and F&A in order to establish a cycle for regular reviews and updates of CPIs and roles and responsibilities supporting the ITB WO process.

Completion date: Q4 2010-2011

1.4 Supporting Tools

Supporting tools should be in place to effectively administer the ITB WO process. The Client User Guide for the ITB WO process states that CAS is the only tool for the ITB WO process. The audit found that ITB stakeholders are aware that CAS is the tool used to track work order progress and related costs.

The CRBM, WOMC performs many activities outside of CAS, using other tools to support the ITB WO process. The ITB WO process is manually administered by the CRBM, WOMC and supported by a variety of office automation tools, including Microsoft (MS) Outlook, MS Excel, MS Word, scanning hardware and scanning software. A 285-page document has been developed to describe the activities performed by the CRBM, WOMC to administer the process, including initial screening of work orders, maintaining work orders in CAS, scanning, mailing and tracking RC503s, as well as supporting the WORC.

The current manual manipulation of data contributes to the likelihood of errors, and may strain CRBM, WOMC resources. ITB has been working on an interim solution to automate many of the manual activities (such as the current manual step of initially screening new work orders), as well as to reduce the number of tools, including the MS Word version of the work order form.

Recommendation

ITB should leverage CAS, where feasible, to better support the ITB WO process.

Action Plan

CRBM, BPCR will investigate the opportunity to further streamline the ITB WO process by integrating CAS for Information Technology (CASFIT), Electronic RC502, and other ITB sub-processes within CAS in order to reduce the manual processes and further increase efficiencies.

Completion date: Q3 2011-2012

1.5 Implementing Changes to the ITB WO Process

Changes to a process should be planned, documented, tested, and communicated to stakeholders to ensure they are implemented effectively. The audit found that changes made to the ITB WO process in 2010 were not effectively implemented. The CRBM, WOMC was not involved early enough to effectively plan and prepare for the required changes made to the ITB WO process in 2010. These changes included:

This late involvement resulted in a work order processing backlog between April and August 2010. In addition, a lack of supporting documentation for clients and ITB stakeholders to address the new ITB WO process requirements has caused confusion and further delays in processing work orders.

Recommendation

ITB should seek early engagement with other CRA organizations whose initiatives may impact the ITB WO process, to ensure future changes are implemented and communicated in a timely and effective manner.

Action Plan

CRBM, BPCR has initiated and will continue to develop an Advance Schedule of Change regarding the ITB WO process, which involves ITB and F&A stakeholders. This early engagement will continue to be developed by BPCR in conjunction with their program planning.

Completion date: Q4 2010-2011

2.0 Compliance with Processes and Procedures

2.1 Processing of Work Orders

Work orders submitted to ITB go through various stages of processing and review by the WOMC and the WORC prior to work commencing. To reduce delays in processing work orders, work order forms should be filled out sufficiently by clients prior to submission to ITB, as per established guidelines. ITB guidelines state that description of work should include an explanation of the work, the scope of the work, and the ITB OPI, if known. Sufficiently completed forms would facilitate the WOMC and the WORC’s prompt understanding of work being requested, identification of an ITB OPI, and confirmation of a valid source of funding.

The audit found that description of work, ITB OPI determination, and source of funding issues are causing delays in the processing of work orders. Results of an audit test conducted on work orders reviewed by the WORC indicated that for:

Delays in processing work orders were attributed to:

In April 2008, a briefing note was submitted to ITB Management to address ITB OPI determination issues for work orders not directly aligned with ITB’s application/ infrastructure/service areas, referred to as orphan orders. An escalation process was implemented in 2009, as a short-term solution, to ensure ITB OPIs were assigned to orphan orders in a timely manner. The audit found that this escalation process is working in resolving orphan orders.

Work orders continue to be delayed at the WORC due to insufficient description of work, ITB OPI determination issues that are not raised through the escalation process, and source of funding issues. Delayed work orders negatively impact the efficiency of processing new work orders, as the WOMC and the WORC must expend time and effort to resolve these issues.

Recommendation

ITB should review the causes for delays in processing work orders, and improve existing controls to reduce these delays.

Action Plan

CRBM, WOMC has initiated a project, Order to Cash (ORCA), Phase 1, which is creating an electronic form (E502) to replace the current MS Word RC502 template used in the ITB WO process and increase the quality of information from client requests. This additional automation will remove existing barriers and increase performance. Subsequent phases will be pursued in the future by the CRBM, WOMC.

Completion date: Q4 2010-2011

2.2 ITB Review of Work Order Estimates

Cost estimates produced for work orders should be reviewed to ensure completeness and consistency. ITB, RMS has implemented a process to review cost estimates that meet their identified criteria, prior to communicating estimates to clients. However, there is no evidence that all estimates that meet RMS criteria have been reviewed. Furthermore, a formal monitoring process is not in place to ensure that all cost estimates meeting RMS criteria are reviewed.

A communiqué was published on the ITB, CRBM, RMS Intranet site that includes the criteria for review of completed cost estimates, effective August 2006. The criteria include all work order requests that:

When cost estimates fall under these criteria, ITB OPIs are to provide the estimates to RMS for review. RMS reviews cost estimates to ensure completeness and consistency, determines if standard costs need to be added, completes the CRA costing templates, and clarifies whether estimated O&M costs are for contractor costs. Summary reports of major projects are prepared for senior management. RMS does not challenge the accuracy of estimates, as this is the role of the ITB OPI.

Without a process in place to ensure that cost estimates meeting RMS criteria are being reviewed, there is a risk that estimates submitted to clients are incomplete or inconsistent, which could negatively impact ITB and/or client budgets.

Recommendation

ITB should establish a formal monitoring process to ensure compliance with the RMS requirement to review cost estimates.

Action Plans

CRBM, RMS will strengthen the monitoring process to ensure compliance with the requirement to review cost estimates. CRBM, RMS will:

CRBM, RMS will study from a cost benefit standpoint the possibility of including, in the financial analyst CASFIT roles, the requirement to monitor estimates for the purpose of ensuring that those that meet the criteria are reviewed by the RMS Costing Group.

Completion date: Q2 2011-2012

In addition, RMS will undertake discussion with F&A, RMD and clients about the possibility of establishing Agency-wide inter-branch notification procedures.

Completion date: Q4 2011-2012

2.3 The RC503 Sub-Process

As per ITB documentation, an RC503, a summary document of all cost estimates for a work order, must be generated by the ITB OPI and signed by the client financial authority for work orders that have a funding source of IBC, CBSA development, and in some cases corporate or IP, before work can commence.

Roles and responsibilities regarding RC503 completion, sign-off and commencement of work have been documented; however the audit found that current practices differ from documented procedures. Clients and ITB stakeholders are not all compliant with the ITB requirement to sign, send and return RC503s for IBC funded work orders. Furthermore, clients are not all signing and returning RC503s in timely fashion. The trend indicates that the level of compliance to this ITB requirement is decreasing. Results of an audit test of IBC funded work orders extracted from CAS indicated that for:

In another audit test, 274 RC503s that were signed for a sample of IBC funded work orders submitted in FY 2009-2010 were analyzed. Results indicated that:

The RC503 sub-process is a manual, time consuming process with many steps involving clients and ITB stakeholders, any one of which can delay the process.

In addition, some clients and ITB stakeholders may not see the need to comply with the RC503 sub-process. In these cases, an RC503 may not be generated or signed, but the work request is still completed. ITB could be at financial risk if work starts prior to receiving a signed RC503 and the client decides to cancel their work order. Relationships between ITB and clients could be negatively impacted. Furthermore, compliance to existing procedural requirements may decrease if “exceptions to the rule” are allowed.

Recommendation

ITB should examine the requirement for the RC503 sub-process, improve existing controls to ensure the timely signoff of RC503s, and determine whether it is feasible to automate the RC503 sub-process.

Action Plan

CRBM, BPCR will examine the requirement for continued use of the RC503 sub-process and the IBC project funding model.

Completion date: Q3 2011-2012

3.0 Monitoring and Reporting

3.1 Effectiveness of Estimating Activities

The effectiveness of cost estimating activities should be monitored and reported to senior management.

The requirement for CRBM to report on the overall effectiveness of the cost estimating activities is documented in the ITB CED. However, the directive has gone through numerous revisions since September 2009, and has not yet been approved and communicated.

The audit found that ITB is not monitoring and reporting on the overall effectiveness of the cost estimating activities. However, ITB is drafting procedures to support monitoring and reporting as part of the Cost Estimating Model (CEM) project.

Without a formal process in place, ITB is not able to formally report on the overall effectiveness of cost estimating activities, such as comparing accuracy of estimates to actual expenditures, measuring the consistency of estimates across ITB, and measuring the timeliness of generating estimates.

Recommendation

ITB should implement a process to monitor and report on the overall effectiveness of ITB cost estimating activities.

Action Plan

As outlined in the ITB CED, RMS will implement a process to monitor and report on the overall effectiveness of ITB cost estimating activities. The process will consider projects funded by corporate, IP, and IBC.

The RMS Costing Group will use a checklist to evaluate compliance on a project’s cost estimate. Although the RMS Costing Group will evaluate only a representative sample of cost estimates for projects funded by IBC, it will evaluate all cost estimates for projects funded by corporate and IP sources.

The RMS Costing Group will report the results of their evaluations annually to the Director General (DG) of CRBM.

Communicate the ITB CED.
Completion date: Q4 2010-2011

Begin new process.
Completion date: Q1 2011-2012

Deliver first report to the DG of CRBM.
Completion date: Planned for Q1 2012-2013

3.2 Service Standards

Service standards provide a benchmark by which performance can be evaluated. They assist in measuring and improving current performance, reporting on service availability, and confirming that goals and objectives of the process are being met.

The audit found that no formal service standards for the ITB WO process exist.

Without formally established service standards and performance indicators for the ITB WO process, ITB cannot accurately measure and report on whether or not service standards have been met.

Recommendation

ITB should establish formal service standards for the ITB WO process.

Action Plan

CRBM, BPCR will develop, publish, and communicate a comprehensive program mandate, which will include formal service standards for the ITB WO process.

Completion date: Q3 2011-2012

Conclusion

ITB has an established organizational structure to support the ITB WO process. However, the following opportunities for improvement were identified:

By addressing the findings above, ITB can strengthen the ITB WO process in order to ensure that the process is managed effectively and efficiently, in a consistent, transparent, and timely manner to support the delivery of programs and services that meet client needs. ITB is in agreement with the recommendations and has developed action plans to address the opportunities for improvement.


Footnotes

Page details

Date modified: