Language selection

Government of Canada / Gouvernement du Canada

Search

Public Services and Procurement Canada
Evaluation of the Real Property Services Program

Evaluation of the Real Property Services Program (PDF, 1.4MB)

On this page

Executive summary

i. Federal organizations require office space and other real property assets in order to fulfill their mandates. These assets require capital investment and regular, on-going maintenance as they go through the asset life cycle. Through its Accommodation Management and Real Property Services Program, Public Services and Procurement Canada (PSPC) manages 88% of federal government office space and also provides real property services to numerous other federal organizations. These services are essential to ensuring that PSPC and other government departments have accommodation that meets their needs and enables them to carry out their mandates.

ii. This evaluation examined the relevance and performance of the Real Property Services (RPS) program. The objective of this program is to provide responsive and cost effective real property services to federal organizations and to Parliament. The program's total expenditures in 2016 to 2017 were $3.28 billionFootnote 1 with 2,705 full time equivalents.

iii. The Real Property Services Branch (RPSB) of PSPC is accountable for the program; however, the program is delivered by RPSB, Parliamentary Precinct Branch (PPB) and PSPC regions.

iv. The RPS Program continues to be relevant to the Government of Canada. There is both a legislative and policy requirement for PSPC to provide real property services. The program is aligned with PSPC's strategic outcome of sound stewardship and with the department's strategic priorities of value-for-money, innovative solutions and service excellence. The program is also responding to a continuing need for real property services. Demand for RPS services increased by 23% from 2012 to 2013 to 2016 to 2017 and 86% of clients expect to have an increased demand in at least one service line offered by PSPC's RPSB. As well, the program has played a key role in the delivery of the Accelerated Infrastructure Program (AIP) Initiative, PSPC's initiative in support of the Economic Action Plan. However, the level of need may be affected, in the next few years, as AIP funding comes to an end.

v. The program is, for the most part, achieving its immediate outcomes: to ensure that custodians have access to timely and quality real property services. Clients rate the quality and timeliness of RPS services quite highly. Projects over $1 million are being completed on time and on budget, based on departmental data. As well, projects are not fully complying with PSPC's requirements under the National Project Management System (NPMS). Compliance rates are currently 66%, compared to a target rate of 90%.

vi. The program is efficient. Program costs for management and oversight of its major private sector contractor, which delivers a substantial portion of the program, are at levels similar to the previous contract while providing a more comprehensive management and oversight regime. As well, the fees charged to clients for internal staff fees are appropriate given the underlying cost structure of the program, although the program would benefit from more clarity around corporate service support charges. Finally, revenues per full-time employee, despite decreasing slightly from 2013 to 2014 to 2014 to 2015, have increased 17.4% over the period covered by the evaluation.

vii. The evaluation looked at 2 aspects of program design: clarity of accountabilities and responsibilities between PSPC and clients; and the functionality of the program database. Based on the reviews of these, program design could be improved. Client agreements do not consistently articulate overall accountabilities for projects, creating the potential for misunderstandings on the part of clients. The program database maintained on the departmental SIGMA system lacks adequate documentation, contains a number of fields that appear to overlap with one another and others that are not used in a consistent manner. In addition, financial data contained in this system is not easily reconciled with other sources of program financial information.

Management response

Management has had the opportunity to review the report, and agrees with the conclusions and recommendations found therein. The findings and recommendations will assist RPS in further improving our services to clients and our operations. A Management Action Plan has been developed to address the evaluation recommendations.

Recommendations and management action plan

Recommendation 1

The Assistant Deputy Minister, Real Property Services Branch should ensure that the execution of the annual assessment of conformance to the NPMS is centralized, that the rigour of the process is improved and that the results are used to drive continuous improvement in compliance.

Management response 1

The Real Property Services Branch will enhance the annual conformance review by adding in centralized direct reviews of a sample of regional project files. This process will improve consistency and rigor across the country through the validation of the regional review processes and will ensure that the results are used to drive continuous improvement in compliance. The following actions will be undertaken:

Management action plan 1.1

Report on initial run of centralized reviews.

Management action plan 1.2

Adjust process and have centralized reviews approved by the NPMS governance for inclusion into the National Project Management system Directive on the Monitoring of Real Property Projects' Conformation of the National Project Management System.

Recommendation 2

The Assistant Deputy Minister, Real Property Services Branch should take steps to ensure that RPS Program officials have a clear understanding of the authorities, allocation methodologies and the current rationale for charges allocated to the program through other internal transfers, not the subject of Memoranda of Understanding (MOUs).

Management response 2

Real Property Services Branch (RPSB) will, in collaboration with Finance Branch (FB), ensure that RPS Program officials have a clear understanding of the authorities, allocation methodologies and the rationale for any new charges being allocated to the program through other internal transfers, not the subject of MOUs.

Management action plan 2.1

Program management will assess completeness of its files for other internal transfers (OITs) with respect to authorities, allocation methodologies and the rationale:

Management action plan 2.2

Program management will identify which of the OITs need to be reassessed in terms of continued need for charging, that is, current rationale. (This will include but may not be limited to the 3 OITs that were questioned in the report as to whether there is a current rationale, that is, for Human Resources Branch, Information Technology Investment Fund and Integrity Framework):

Management action plan 2.3

For future OITs, FB will clearly communicate to Program Management the authorities, allocation methodologies and the rationale.

Recommendation 3

The Assistant Deputy Minister, Real Property Services Branch should review and update its client agreement documents to ensure that they all provide clear and consistent statements of the overall accountabilities of PSPC and client departments for programs or projects or work and that signatures are present.

Management response 3

A Directive on Real Property Service Client Agreements is in development to provide clarity on the purpose and roles and responsibilities of the various agreement instruments available to RPS managers who are responsible for putting into place formal agreements with clients. More specifically, for each type of agreement instrument, this directive will describe:

Management action plan 3.1

This directive will ensure a nationally consistent and structured approach to the use and application of the various client agreements across the real property service lines:

Recommendation 4

The Assistant Deputy Minister, Real Property Services Branch should undertake a comprehensive review of the RPS SIGMA database to improve documentation on the database; to ensure the accuracy of project and financial data; to address possible redundancies; and to ensure the consistency and completeness of project information entered on the database.

Management action plan 4.1

Introduction

1. This report presents the results of the evaluation of the Real Property Services Program (RPS). This engagement was included in the Public Services and Procurement Canada 2016 to 2019 Risk-Based Audit and Evaluation Plan.

Profile

Background

2. PSPC, through its Accommodation Management and Real Property Services Program, provides federal departments and agencies with office and common use accommodation. As well, the department administers and maintains various public works, such as buildings, bridges and dams, the Parliamentary Precinct and other heritage assets. In addition, PSPC provides, on an optional basis, real property services to other federal departments and agencies. These services are provided under 3 main service lines: project delivery, property and facility management and advisory services (see Appendix A: Program activities for details).

Authority

3. PSPC is mandated, under the Department of Public Works and Government Services Act, to construct, maintain and repair, and to administer all federal government real property, public works and immovables outside of the Yukon, the Northwest Territories and Nunavut, not under the administration of any other federal organization. The act also empowers PSPC to provide real property management, architectural and engineering services to other federal government departments, boards or agencies on their request.

4. Other legislation and policies providing PSPC with a mandate to provide real property services on a cost recovery basis are:

Roles and responsibilities

5. The RPSB has overall accountability for the RPS Program, while PSPC regions are responsible for delivery.

Resources

6. The RPS Program's expenditures were $3.284 billion in 2016 to 2017. This includes disbursements of $1.36 billion related to expenditures on behalf of other government departments and disbursements of $1.5 billion on behalf of PSPC assets. It also includes total operating expenses of $400 million.Footnote 2 The Real Property Services Revolving Fund is the funding mechanism for the RPS Program. There were 2,705 full-time equivalent employees in 2016 to 2017.

Logic model

7. A logic model is a visual representation that links a program's activities, outputs and outcomes; provides a systematic and visual method of illustrating the program theory; and shows the logic of how a program is expected to achieve its objectives. It also provides the basis for developing the performance measurement and evaluation strategies, including the evaluation matrix.

8. A logic model for the program was developed based on a detailed document review, meetings with program managers and interviews with key stakeholders. It was subsequently validated with program staff. The logic model is provided in Exhibit 1: Logic model.

Exhibit 1: Logic model

Exhibit 1: Logic model - Long description below

Image description

Exhibit 1: This exhibit depicts the logic model of the Evaluation of the Real Property Service Program. The logic model depicts program activities, outputs, immediate outcomes, intermediate outcome, long term outcome, and the overall objective of the program.

Objective

The objective of the Real Property Services Program is to ensure that federal government organizations and the Parliament of Canada receive responsive and cost-effective real property services.

Activities

Real property services include 3 areas of activity:

  • project delivery
  • property and facility management
  • advisory services

The project delivery, property and facility management, and advisory services activities are comprised of more detailed program activities that are included in the logic model.

The project delivery activity is comprised of the following 3 activities:

  • project inception
  • project identification
  • project delivery

The property and facility management activity is comprised of the following 4 activities:

  • daily operational functions
  • mandated and other maintenance
  • building/asset management planning
  • inspections and repairs

The advisory services activity is comprised of the following 5 activities:

  • portfolio management advice
  • accommodation management advice
  • real estate management advice
  • property development advice
  • property management advice
Outputs

The activity, project delivery, leads to the following 4 outputs:

  • investment analysis reports (IARs)
  • statements of requirement
  • project planning and management documents
  • completed real property projects

The activity, property and facility management, leads to the following 3 outputs:

  • building/asset management plans
  • building condition/performance/compliance reviews and reports
  • safe, clean and functional federal real property assets

The activity, advisory services, lead to the following 3 outputs:

  • analysis provided to federal RP custodians
  • options provided to federal RP custodians
  • recommendations provided to federal RP custodians
Immediate outcomes

The outputs for project delivery contribute to the following immediate outcome: Custodians of federal real property have access to timely and quality oversight and management of project delivery in federal real property assets, in compliance with client and other requirements and standards.

The outputs for property and facility management contributes to the following immediate outcome: Custodians of federal real property have access to timely and quality oversight and management of operations and maintenance in federal real property assets, in compliance with client and other requirements and standards.

The outputs for advisory services contributes to the following immediate outcome: Custodians of federal real property have access to timely and quality information and advice in support of informed real property-related decision-making.

Intermediate outcome

The 3 immediate outcomes contribute to the following intermediate outcome: Custodians of federal real property assets benefit from effective and efficient services which, from a whole of government perspective, are cost-neutral as a result of leveraged economies of scale.

Long term outcomes

The intermediate outcome leads to the following long term outcome: Deliver high-quality, central programs and services that ensure sound stewardship on behalf of Canadians and meet the program needs of federal institutions.

Focus of the evaluation

9. The objective of this evaluation was to determine the program's relevance and performance in accordance with the Treasury Board Policy on Results. The evaluation assessed the program for the period from April 2012 to March 2017.

10. The evaluation has focused on the achievement of the program's immediate outcomes of providing timely quality services that meet client and other requirements and standards, in each of the program's 3 service lines: Property and facility management, project delivery and advisory services. The intermediate and long-term outcomes will be evaluated as part of a planned horizontal evaluation that will encompass the Real Property Services Program, the Federal Accommodations Program and the Federal Holdings Program.

Approach and methodology

11. Multiple lines of evidence were used to assess the program. These included:

Document review
Documents included legislative and policy documents, departmental planning and reporting documents and a wide range of program planning, management and operational documents.
Comparative review
National and provincial information and data obtained from an on-line search and from provincial governments were reviewed to contextualize the program and provide comparative data against which the program could be assessed.
Financial analysis
Financial data related to the program's budgets, expenditures and staff resources was examined.
Interviews
26 interviews were conducted: 21 with senior RPS managers, one with Treasury Board and 4 with provincial representatives.
Data analysis
Program data analyzed included longitudinal data on real property for which PSPC is the custodian and on properties managed on behalf of other government departments, and on the number, value and characteristics of program projects over the last 5 years (2012 to 2017).
Surveys
A survey of program clients was conducted. The survey was sent to 118 client representatives and there were 30 valid responses, for a response rate of 22.3%. As well, data from the RPSB 2016 Client Feedback Survey carried out by RPSB in 2016 to 2017 contributed to the evaluation.

12. More information on the approach and methodologies used to conduct this evaluation can be found in Appendix B: About the evaluation.

Findings and conclusions

13. The findings and conclusions below are based on multiple lines of evidence. They are presented by evaluation issue (relevance, performance and program design).

Relevance

1. Is there a legislative or policy requirement for the Real Property Services Program

14. To address this evaluation question, the evaluation team reviewed federal government legislation and policies that require and/or authorize PSPC to provide real property services to federal organizations.

15. The Department of Public Works and Government Services Act mandates PSPC to construct, maintain and repair, and to administer all federal government real property, public works and immovables outside of the Yukon, the Northwest Territories and Nunavut, not under the administration of any other federal organization. The Act also empowers PSPC to provide real property management, architectural and engineering services to other federal government departments, boards or agencies on their request.

16. The Treasury Board Policy on Management of Real Property confirms PSPC's role as the designated custodian of all federal government general-purpose office accommodation in Canada, as well as the obligation to provide office accommodation to departments and to set standards for such accommodation.

17. The Treasury Board Common Services Policy names PSPC as the designated custodian of office facilities assigned to PSPC and provides authority for PSPC in planning, development and acquisition, design, construction and refit as well as property management and special technical services.

18. The Revolving Funds Act states that the authority is with the Minister of PWGS to spend/make expenditures out of the Revolving Fund for the purpose set out in the Department of Public Works and Government Services Act.

19. Based on the review of these documents, there is both a legislative and policy requirement for PSPC to provide real property services, in respect of both properties for which it is custodian and for property under the custodianship of other government organizations.

2. Is the Real Property Services Program aligned with federal priorities and departmental strategic outcomes

20. PSPC's strategic outcome is: "To deliver high-quality, central programs and services that ensure sound stewardship on behalf of Canadians and meet the program needs of federal institutions".

21. In support of this strategic outcome, the department has identified 3 organizational priorities:

22. To evaluate RPS' alignment with the above, the evaluation team conducted interviews with 21 senior managers of the RPSB who are responsible for program delivery, enabling the program (for example client relations, program management), or who are clients of the RPS Program.

23. These senior managers were asked to rate the contribution of the RPS Program to the 3 departmental priorities and to sound stewardship, based on a 3 point scale ranging from 'not at all', through 'moderately' to 'very much'. The results are summarized in Exhibit 2: Real property services senior management perceptions—Real property services' contribution to departmental priorities and strategic outcome, below.

Exhibit 2: Real property services senior management perceptions—Real property services' contribution to departmental priorities and strategic outcome
RPS Services demonstrate not at all moderately very much
value for money 0% 18% 83%
innovative solutions 0% 53% 48%
service excellence 5% 35% 60%
sound stewardship 5% 5% 90%

24. As shown in the above table, of the 19 interviewees who responded to this question, a majority viewed the RPS Program as contributing very much to value for money and service excellence, while just under one-half were of this view with respect to innovative solutions. In regards to PSPC's strategic outcome, 90% of interviewees believed that the RPS Program contributes very much to sound stewardship. Almost all interviewees believe that the RPS Program contributes at least moderately to the 3 priorities and to the departmental strategic outcomeFootnote 3.

25. Based on comments supporting the ratings, the lower score for innovative solutions reflects a number of factors, including the constraining influence of policy requirements, workload and the fact that RPS is slowly divesting itself of a risk averse culture.

26. The RPSB 2016 client feedback survey was distributed to 28 client departments. The survey included questions in the areas of service delivery, process aspects and engagement aspects of RPS services. Respondents provided rated responses on a 5-point scale where, depending on the question, "1" meant "strongly disagree" or "very dissatisfied"; "3" meant "neither agree nor disagree" or "neither satisfied nor dissatisfied"; and "5" meant 'strongly agree" or "very satisfied".

27. Two questions related to the departmental priorities concerning value for money and service excellence were posed. In addressing service excellence, clients were asked to rate their satisfaction with the quality of the service provided. The average score provided by respondents was 3.79 out of a possible 5. With respect to value for money, clients moderately agreed (average score of 3.21 out of 5) with the statement "the proposed real property solution met our needs in terms of time, budget and scope."

28. In the survey of clients of RPS projects conducted by the evaluation team, clients were asked to rate the services provided by RPS against the 3 departmental priorities and the strategic outcome, using the same 3 point scale as was used in the stakeholder interviews.

29. The results, shown in Exhibit 3: Client perceptions—Real property services contribution to departmental priorities and strategic outcome, indicate that all clients who responded to the survey (26) believe that the RPS services demonstrated service excellence to some degree. Similarly, almost all (95%) believe that the program demonstrated value-for-money and 90% believe it demonstrated sound stewardship. However, only 76% indicated that they believed the program demonstrated innovative solutions, compared to 100% of RPSB senior managers. In addition, with the exception of service excellence, far fewer clients than RPS senior managers indicated that the RPS Program demonstrated these attributes "very much". For example, only 14% indicated that their RPS project demonstrated innovative solutions "very much", compared to 48% of RPS senior managers.

Exhibit 3: Client perceptions—Real property services contribution to departmental priorities and strategic outcome
RPS Services demonstrate Not at all Moderately Very much
Value for money 5% 62% 33%
Innovative solutions 24% 62% 14%
Service excellence 0% 38% 62%
Sound stewardship 10% 52% 38%

30. Based on the above lines of research, the RPS Program is aligned with the 3 departmental priorities and with the PSPC strategic outcome. Both RPS senior managers and external clients view the program as contributing to these, although clients believe the program contributes to a lesser extent than do RPS senior managers.

3. Is there a continued need for the Real Property Services Program

31. Continued need is the extent to which a program continues to meet the needs of the client group or groups at which it is aimed. Within the Government of Canada, there are 63 federal organizations with custody of real property assets, including 54 departments and agencies. PSPC is the custodian of the vast majority of office space in the Government of Canada, with 88% of the total square metres of office space. In addition, PSPC and other federal organizations hold and are custodians of special purpose and common purpose real property assets (for example laboratories, training facilities) that support their mandates.

32. The client groups of the program are:

33. To address the continued need for RPS services provided to these clients, the evaluation team interviewed 21 senior managers within RPSB. The interviews were used to gather senior managements' views on whether demand for real property services has increased in the last 5 years and on whether they expected demand to increase in the next 5 years. The team also reviewed RPS data on the real property asset base served by the RPS Program over the last 5 years and on the volume of services provided. Data from the RPSB 2016 client feedback survey and from the survey of clients carried out as part of this evaluation also informed findings with respect to continued need.

Exhibit 4: Senior management's views on demand for services
Interview question Number of respondents Increase Fluctuating Stable Decrease
Has other government department's (OGD) demand for real property services increased or decreased over the last 5 years? 19 17 1 1 0
Do you expect OGD demand for real property services to increase, stay the same or decrease over the next 5 years? 19 17 1 1 0

34. As seen in Exhibit 4: Senior management's views on demand for services, of the 19 interviewees who provided their perspectives on this topic, 17 (89%) expressed their belief that the demand for services had increased in the last 5 years. Of the 2 remaining interviewees, one noted that demand for services has been fluctuating while the remaining interviewee indicated that demand for some services had been stable. Areas of increasing demand in the last 5 years included environmental services (11 out of 19) and workplace solutions (8 out of 19). Other services mentioned by interviewees included capital development, multi-department service hubs (for example science hubs), accommodation, real estate, project management, property and facility management and infrastructure projects.

35. When asked whether they expected demand to increase or decrease over the next 5 years, 17 out of 19 (89%) interviewees who responded to this question predicted that demand will increase. The 3 main service areas predicted to increase are the development of service hubs (45%), environmental services (35%), and workplace fit-ups and activity based workspaces (30%).

36. With regard to why departments are coming to PSPC to meet their needs for real property services, interviewees cited several reasons, including limits on financial delegations of many custodian departments, which make it more difficult for them to directly procure real property services from the private sector; the economies of scale represented by PSPC; the availability of a wide range expertise within PSPC which it would not be economical for most departments to develop internally; and the increasing desire on the part of many custodian departments to focus their resources on their mandated activities, rather than on facilitating functions, such as real property.

37. Program and other data on the volume of real property assets managed by PSPC, for which the RPS provides real property services, also indicates the continued need for the RPS.

38. Between fiscal years 2012 to 2013 and 2016 to 2017, PSPC's share of the federal real property inventory of all space categories (office, special purpose and common purpose), based on floor area, has remained steady, at 27% with a slight increase to 30% in 2014 to 2015, indicating a stable requirement for these services in support of PSPC assets.

Exhibit 5: Public Services and Procurement Canada—Share of federal real property/square metres managed
  2012 to 2013 2013 to 2014 2014 to 2015 2015 to 2016 2016 to 2017
PSPC share of federal real property inventory (%) 27 27 30 27 27
Square metres managed 7,379,066 7,442,580 7,277,624 7,166,131 6,953,870

39. RPS services in support of PSPC assets are provided primarily to the federal accommodation, federal holdings and Parliamentary Precinct programs. In fiscal year 2012 to 2013, the RPS Program provided real property services for 7,379,066 square metres of space for which PSPC is the custodian. Since then, the total square metres of space managed by the RPS Program has remained relatively stable with a slight decrease to 6,953,870 square metres in 2016 to 2017, as shown in Exhibit 5: Public Services and Procurement Canada—Share of federal real property/square metres managed. This decrease, seen mostly for leased and lease-purchased assets, is likely indicative of PSPC's move towards reducing the overall amount of space occupied by public servants through its Workplace 2.0 ProgramFootnote 4.

40. Data on the total number of other government departments (OGDs) who are clients of the RPS Program indicates the number of client departments has decreased slightly during the period covered by the evaluation, as shown in Exhibit 6: Other government department clients. However, the number of regular, ongoing clients who are custodians has remained steady at 22Footnote 5.

Exhibit 6: Other government department clients
  2012 to 2013 2013 to 2014 2014 to 2015 2015 to 2016 2016 to 2017
Number of ongoing OGD custodian clients 22 22 22 22 22
Number of OGD clientstable 1 note 1 110 108 98 99 99

Exhibit 6: Other government department clients: Table 1 Note

Table 1 Note 1

Because the data on custodians and number of OGD clients came from 2 separate sources, the evaluation team was unable to disaggregate the data on total number of clients between custodians and tenants.

Return to table 1 note 1 referrer

41. Data on the dollar value of RPS expenditures largely supports the views of stakeholders regarding the continued need for the RPS Program. Program data for the years 2012 to 2013 to 2016 to 2017 shows that there has been an increase in demand for real property services over the last 5 years. As Exhibit 7: Total expenditures—Public Services and Procurement Canada and other government department clients shows, total project expenditures declined from $2.58 billion in 2012 to 2013 to $2.38 billion in 2014 to 2015. By 2016 to 2017 however, project expenditures had increased to $3.17 billion, an increase of 33% over 2014 to 2015.

42. This pattern recurs in the expenditure data for the 2 main client groups (PSPC and OGDs), although less markedly for PSPC. Total expenditures in support of PSPC clients declined from $1.39 billion in 2012 to 2013 to $1.33 billion in 2014 to 2015 but subsequently increased by 31% to $1.74 billion in 2016 to 2017, compared to 2014 to 2015. The decline in the earlier years likely reflects the impacts of spending reduction initiatives, such as the Deficit Reduction Action Plan, while more recent increases appear, based on program data, to reflect the impacts of funding provided under the Accelerated Infrastructure Program, Phases 2 and 3, the Long-Term Vision and Plan for the Parliamentary Precinct and other special initiative funding.

Exhibit 7: Total expenditures—Public Services and Procurement Canada and other government department clients

Exhibit 7: Total expenditures – PSPC and other government department clients - Long description below

Image description

Exhibit 7: This graph presents real property services expenditures on behalf of Public Services and Procurement Canada and other government department clients, respectively and in total. Data is provided for each of the years 2012 to 2013 to 2016 to 2017.

2012 to 2013

  • Public Services and Procurement Canada: $1,391,540,738.00
  • Other government departments: $1,192,432,272.00
  • Total: $2,583,973,010.00

2013 to 2014

  • Public Services and Procurement Canada: $1,370,479,741.00
  • Other government departments: $1,160,526,677.00
  • Total: $2,531,006,417.00

2014 to 2015

  • Public Services and Procurement Canada: $1,330,148,523.00
  • Other government departments: $1,047,971,583.00
  • Total: $2,378,120,106.00

2015 to 2016

  • Public Services and Procurement Canada: $1,395,920,964.00
  • Other government departments: $1,265,559,303.00
  • Total: $2,661,480,266.00

2016 to 2017

  • Public Services and Procurement Canada: $1,744,564,131.00
  • Other government departments: $1,427,356,646.00
  • Total: $3,171,920,777.00

43. As Exhibit 7: Total expenditures—Public Services and Procurement Canada and other government department clients shows, total expenditures in support of OGD clients declined from $1.19 billion in 2013 to 2014 to $1.05 billion in 2014 to 2015; between 2014 to 2015 and 2016 to 2017, however, they increased by 36% to $1.43 billion. As noted earlier, the RPS Program is an optional service for OGDs in respect of buildings for which the latter are custodians but mandatory for OGDs who are tenants. Unfortunately, program expenditure data for OGDs could not be disaggregated between these 2 OGD clients groups.

44. The program data provides some insight into client demand by RPS' 3 main service lines: property and facilities management, project delivery and advisory services.

45. As Exhibit 8: Property and facility management services expenditures—Public Services and Procurement Canada and other government department clients shows, expenditures under the Property and Facilities Management service line have declined over the last 5 years, from $712 million in total expenditures in 2012 to 2013 to $614 million in 2016 to 2017 (14%). This decline holds true for expenditures in support of OGD clients, which declined by 16% over the same 5 year period. Expenditures in support of PSPC clients, however, declined by 23% between 2012 to 2013 and 2015 to 2016 but increased by 13% between 2015 to 2016 and 2016 to 2017.

46. Based on information provided by program management, the decline in the dollar value of projects carried out in this service line in the period 2014 to 2015 to 2015 to 2016 results, in part, from less operations and maintenance (O&M) funding for PSPC assets, the wind down of the Alternate Forms of Delivery (AFD) contract with SNC Lavalin in 2014 to 2015 and the subsequent transition to the RP-1 contract in 2015 to 2016, which resulted in a good deal of project carry over.

Exhibit 8: Property and facility management services expenditures—Public Services and Procurement Canada and other government department clients

Exhibit 8: Property and facility management services expenditures – PSPC and other government department clients - Long description below

Image description

Exhibit 8: This graph presents program expenditures under the property and facility management services line, incurred annually on behalf of Public Services and Procurement Canada and other government department clients, respectively, and in total. Data is presented for each of the years 2012 to 2013 to 2016 to 2017.

2012 to 2013:

  • Public Services and Procurement Canada: $639,218,413
  • Other government departments: $73,101,190
  • Total: $712,319,602

2013 to 2014

  • Public Services and Procurement Canada: $607,640,263
  • Other government departments: $69,543,560
  • Total: $677,183,823

2014 to 2015

  • Public Services and Procurement Canada: $569,240,746
  • Other government departments: $87,511,103
  • Total: $656,751,849

2015 to 2016

  • Public Services and Procurement Canada: $491,299,054
  • Other government departments: $83,916,657
  • Total: $575,215,711

2016 to 2017

  • Public Services and Procurement Canada: $552,861,421
  • Other government departments: $61,410,972
  • Total: $614,272,393

47. By contrast, as shown in Exhibit 9: Project delivery services expenditures—Public Services and Procurement Canada and other government department clients, project delivery services has experienced an increase in business volume from 2012 to 2013 to 2016 to 2017, from $1.77 billion in total expenditures to $2.13 billion, a 20% increase. Virtually all of this increase is attributable to an increase in expenditures for the PSPC client group, from $671.7 million to $1.03 billion over this time period. Within the project delivery services line, 3 areas experienced large growth: repairs, space optimization and fit-up and major capital projects (>$1M). The sources of a large portion of the funding for these areas include the Accelerated Infrastructure Program (AIP), the Long-Term Vision and Plan for the Parliamentary Precinct, engineering assets funding, the National Investment Strategy, program integrity funding, workplace modernization funding and other special initiative funding.

48. The project delivery service line expenditures in support of OGD clients remained relatively stable over this period, declining from $1.1 billion in 2012 to 2013 to $916.8 million in 2014 to 2015 before recovering to $1.098 billion again in 2016 to 2017. This more stable expenditure pattern likely reflects the fact that OGDs, while they received AIP funding, did not receive funding under some of the other special initiatives, in particular, the Long-Term Vision and Plan for the Parliamentary Precinct and the engineering assets programs that provided large amounts of funding for PSPC assets.

Exhibit 9: Project delivery services expenditures—Public Services and Procurement Canada and other government department clients

Exhibit 9: Project delivery services expenditures – PSPC and other government department clients - Long description below

Image description

Exhibit 9: This graph presents RPS expenditures under the project delivery services line, incurred annually on behalf of Public Services and Procurement Canada and other government department clients, respectively, and in total. Data is presented for each of the years 2012 to 2013 to 2016 to 2017.

2012 to 2013:

  • Public Services and Procurement Canada: $671,689,709.00
  • Other government departments: $1,098,947,703.00
  • Total: $1,770,637,412.00

2013 to 2014

  • Public Services and Procurement Canada: $674,594,276.00
  • Other government departments: $1,061,652,351.00
  • Total: $1,736,246,627.00

2014 to 2015

  • Public Services and Procurement Canada: $711,965,248.00
  • Other government departments: $916,800,373.00
  • Total: $1,628,765,621.00

2015 to 2016

  • Public Services and Procurement Canada: $811,795,319.00
  • Other government departments: $1,075,889,646.00
  • Total: $1,887,684,965.00

2016 to 2017

  • Public Services and Procurement Canada: $1,029,035,868.00
  • Other government departments: $1,098,841,893.00
  • Total: $2,127,877,761.00

49. The RPSB has been focusing on establishing advisory services as a stand-alone service line in recent years, rather than as an adjunct activity in support of other service lines. As shown in Exhibit 10: Advisory services expenditures—Public Services and Procurement Canada and Other government departments clients, the data suggests they have had some success in this respect. Total expenditures for advisory services increased from $64.1 million in 2012 to 2013 to $184 million in 2016 to 2017 (187%). Total expenditures in this service line in support of PSPC programs have increased from $44.5 million in 2012 to 2013 to $101 million in 2016 to 2017, a 127% increase.

50. Total expenditures for advisory services to OGDs have increased steadily since 2012 to 2013, from $19.6 million to $83.6 million in 2016 to 2017, a 327% increase, indicating the program has been successful in establishing this as a more formal separate service line with OGDs. However, given that at least some of this work would, previously, have been carried out as part of property and facility management or project delivery services projects, the increase in expenditures under this service line is likely offset, to some extent, by reductions in expenditures for these types of projects in the other 2 service lines.

Exhibit 10: Advisory services expenditures—Public Services and Procurement Canada and other government departments clients

Exhibit 10: Advisory services expenditures – PSPC and Other government departments clients - Long description below

Image description

Exhibit 10: This graph presents real property services expenditures under the advisory services service line, incurred annually on behalf of Public Services and Procurement Canada and other government department clients, respectively, and in total. Data is presented for each of the years 2012 to 2013 to 2016 to 2017.

2012 to 2013:

  • Public Services and Procurement Canada: $44,476,325
  • Other government departments: $19,589,881
  • Total: $64,066,206

2013 to 2014

  • Public Services and Procurement Canada: $78,084,637
  • Other government departments: $29,322,439
  • Total: $107,407,076

2014 to 2015

  • Public Services and Procurement Canada: $37,758,590
  • Other government departments: $43,660,107
  • Total: $81,418,696

2015 to 2016

  • Public Services and Procurement Canada: $37,098,442
  • Other government departments: $54,301,587
  • Total: $91,400,029

2016 to 2017

  • Public Services and Procurement Canada: $101,117,989
  • Other government departments: $83,645,696
  • Total: $184,763,686

51. The funding programs accounting for the largest dollar value in services in recent years are the Accelerated Infrastructure Program (AIP), Phases 2 and 3Footnote 6, the Long-Term Vision and Plan for the Parliamentary Precinct, and capital and operating projects for OGDs. The combined value of these 3 programs of work accounted for 46.7% of all expenditures for fiscal year 2016 to 2017. The vast majority of the projects funded under these programs were under the project delivery service line.

52. AIP2 and AIP3, in particular, have been a major factor in the total dollar value of work, especially in the last 2 years. Based on the program data, PSPC's expenditures under this program were $304 million in 2015 to 2016 and $652 million in 2016 to 2017. Funding is estimated to be $559 million in 2017 to 2018.

53. AIP2 and AIP3 have not had an equal impact on the 3 service lines provided by the RPS Program. In 2016 to 2017, AIP accounted for only a small portion of expenditures for the advisory service and property and facility management service lines. For project delivery services, however, in 2016 to 2017, AIP accounted for 20% of expenditures.

54. Most of the AIP expenditures—$243 million of the $304 million in 2015 to 2016 and $509 million of the $652 million in 2016 to 2017are for project delivery Services carried out by the RPS Program on behalf of OGDs, indicating that PSPC has played a key role in delivery of the AIP initiative across government.

55. Excluding the value of AIP projects, total RPS expenditures for OGDs would have declined from $1.19 billion in 2012 to 2013 to $.96 billion in 2016 to 2017, a drop of 19%. It is not clear what the impact on RPS revenues will be given that future AIP funding is not guaranteed.

56. To further evaluate whether the RPS Program is delivering high quality services that meet client and other requirements, the evaluation team reviewed pertinent questions from the RPSB 2016 Client Feedback Survey. This survey was distributed to 28 OGD clients representing 83% of RPS' portfolio in terms of space and services usage. When asked about their expectations regarding future demand for the program's services, 86% of clients stated they expected to have an increased demand in at least one service line.

57. With respect to specific services, 57% of survey respondents indicated they expected demand to increase for advisory services while 54% expected increased demand for workplace solutions in the future. Other areas that are expected to experience smaller increases include real estate services (50%), facilities management (39%) and environmental services (25%) (Exhibit 11: Future service demand by clients (Real Property Services Branch 2016 Client Feedback Survey)).

Exhibit 11: Future service demand by clients (Real Property Services Branch 2016 client feedback survey)

Exhibit 11: Future service demand by clients (Real Property Services Branch 2016 Client Feedback Survey) - Long description below

Image description

Exhibit 11: This graph summarizes client expectations of their future demand for specific types of RPS services, based on the Real Property Services Branch 2016 client feedback survey. Data is presented by service type and indicates the percentage of survey respondents indicating that they expected to have a need for each of the specific service types:

  • advisory services: 57%
  • environmental services: 25%
  • facilities management: 39%
  • real estate services: 50%
  • workplace solutions: 54%

58. In addition, 5 of the 28 clients indicated that they expect a need for additional services not currently being offered by RPS. These services include leading the engagement and management of services provided by Shared Services Canada; valuation of client infrastructure portfolios; site management and maintenance contracts for remote areas and northern sites; developing medium and long-term space solutions; and coordination of whole of government initiatives.

59. In the client survey carried out as part of this evaluation, clients were asked whether they expected demand for the program's services to increase, stay the same or decrease in the future. Of the 21 clients who responded to this question, 18 indicated they expected demand to increase moderately or to remain stable.

60. The results from the 4 lines of research support the view that there is a continuing need for RPS services and that this need will extend into the future. However, given the sizeable contribution of time-limited AIP funding on recent increases in program expenditures, program revenues may decline somewhat from current levels.

Conclusions: Relevance

61. In conclusion, the evaluation team found that the RPS Program continues to be relevant. There are legislative and policy requirements for the program, with respect to both PSPC assets and for real property assets under the custodianship of other government departments. The program is aligned with PSPC's strategic outcome and with PSPC priorities that support the achievement of that outcome.

62. Furthermore, there is a continued need for the program to maintain PSPC real property assets and, when requested, assets of other government departments, as indicated by the stable PSPC asset base; the recent increases in the dollar value of services provided to both PSPC and to other custodians; and by the role of the RPS Program in key government initiatives, such as the AIP. However, the level of need may decline somewhat in future years if AIP funding comes to an end.

Performance

Outcome achievement

63. The evaluation examined the degree to which the program achieved its intended immediate outcomes. The intended outcomes of the program are identified below, followed by an assessment of the extent to which they have been achieved.

64. The RPS Program has 3 related immediate outcomes. These are:

65. The indicators against which we evaluated these outcomes apply to all 3 outcomes, for the most part. As well, the data from the same sources was used to evaluate all 3 outcomes. Because this data could not always be disaggregated to the individual outcomes, and in order to minimize repetition we have integrated the discussion of findings on these outcomes.

66. To evaluate achievement of these outcomes, the evaluation team assessed the following indicators:

67. The indicators were assessed using the following sources:

4. Are real property services projects completed on time, on budget and on scope

68. The interviews with senior managers of the RPSB were used to gather management's views as to whether most projects are completed within agreed upon timelines and agreed upon budgets. Of the 19 interviewees that answered the question "Are most projects completed within agreed upon timelines/budgets?", all responded with either "mostly" or "yes" to timelines; however, one interviewee responded "no" in regards to budgets. Where there were delays, the main causes cited were procurement and/or contractual issues as well as unexpected site conditions including information technology readiness. The main causes of budget overruns were client expectations regarding the project scope; worksite related issues such as weather; unexpected building conditions; and the fluctuating cost of materials.

69. The RPSB 2016 client feedback survey partially supports this view. When asked whether they agreed that the real property solution provided by RPS met their needs in terms of time, budget and scope, clients indicated moderate agreement, with an average rating of 3.21 out of 5 on this question.

70. The Major Crown Projects sector (now project management) monitors on time, on budget and on scope for Major Crown Projects (projects over $1 million). Based on the departmental performance reports for the years 2012 to 2013 to 2015 to 2016 and the Departmental Results Report for 2016 to 2017, the percentage of real property projects over $1 million that were on time, on budget and on scope averaged over 95%.

71. While it is not a business requirement, Project Management began monitoring on time, on budget and on scope for all projects, including those under $1 million. In December 2017, they implemented an interactive reporting tool for all projects in the National Capital Region. The tool was launched nationally in April 2018.

72. The client survey conducted as part of this evaluation aimed at determining whether clients' projects were completed on time and on budget and if not, why not. The results of this survey indicate that only 61% of respondents replied that the project was completed according to the agreed upon schedule and 62% replied that it was completed according to the agreed upon budget.

73. Based on PSPC data, the RPS Program has delivered a high percentage of its large (over $1 million) projects on time and on budget. Two-thirds of clients surveyed as part of the evaluation believe their projects were completed on time and on budget. The results indicate that there is potential for improvement in ensuring clients are in agreement with planned project scope, budget and schedule.

5. Are clients satisfied with real property services

74. The results of the RPSB 2016 client feedback survey indicate that 82% of clients are satisfied with their overall client experience with RPS services, while 18% had concerns. These clients were also asked a series of specific questions relating to service delivery, process aspects and the engagement experience. The overall client rating, based on these questions, was 3.89 out of 5, indicating a fairly high degree of satisfaction. These clients rated the quality of RPS services quite highly, at 3.79 out of 5. Clients were also quite satisfied with the amount of time it took to receive the service, which they rated at 3.75 out of 5. However, when asked whether the real property solution provided by RPS met their needs in terms of time, budget and scope, clients rated RPS at 3.21 out of 5, indicating only moderate agreement that this was the case.

75. The evaluation team also addressed client satisfaction in its survey of RPS project clients. The survey results with respect to these questions are summarized in Exhibit 12: Client satisfaction with quality, timeliness and cost of real property services.

76. As shown in Exhibit 12, 84% of clients surveyed indicated that they were somewhat or very satisfied with the quality of work carried out by RPS. Among the 4 clients (16%) who were dissatisfied, the only specific concern cited (by one individual) was focused on how costs were calculated for the overall project, not the quality of work.

Exhibit 12: Client satisfaction with quality, timeliness and cost of real property services
Client satisfaction with: Very dissatisfied (%) Somewhat dissatisfied (%) Somewhat satisfied (%) Very Satisfied (%)
The quality of work carried out by PSPC or its contractors 4 12 44 40
The time PSPC took from your project request to project initiation 0 9 52 39
The time it took to complete this project 0 23 32 45
The cost of this project in relation to the final product that was delivered 5 19 43 33
Communications between the client and PSPC officialstable 2 note 1 0 14 14 67

Exhibit 12: Client satisfaction with quality, timeliness and cost of real property services: Table 2 Note

Table 2 Note 1

One individual indicated they did not know in response to this question; thus the percentages don't add to 100%.

Return to table 2 note 1 referrer

77. Exhibit 12 also shows that with respect to timeliness, the vast majority of clients surveyed were satisfied, overall, with the time it took to initiate their project, with 91% indicating they were somewhat or very satisfied. A smaller percentage, 77%, indicated they were somewhat or very satisfied, overall, with the time it took to complete the project.

78. With respect to costs, a lesser majority, 76%, of respondents indicated that they were satisfied with the cost of their project. When asked if they had any specific concerns with project costs, as shown in Exhibit 12: Client satisfaction with quality, timeliness and cost of real property services, 67% indicated they did not. Among those who said they had concerns, the concern most frequently cited was with how costs are determined and a lack of communication regarding costs and breakdowns of costs on invoices.

79. Clients were also queried regarding their satisfaction with communications between them and PSPC officials during the project. Based on the survey results, 81% of clients were satisfied with project communications, as shown in Exhibit 12: Client satisfaction with quality, timeliness and cost of real property services.

80. Clients of projects that primarily involved the provision of advice were asked whether the advice aided in their decision-making. Seven of the 8 clients who responded to this question indicated that the advice aided their decision-making to a moderate or significant extent. This result suggests that the advisory services business line is meeting client requirements, as was suggested by the large increase in expenditures on this service line.

81. Clients were also asked whether they would use RPS for a similar project in the future. Of the 21 clients who responded to this question, 18 indicated that they would use RPS for a similar project.

82. RPS senior managers interviewed noted that client dissatisfaction sometimes reflects a failure by clients to understand the need to sign off on funding before projects can proceed and the need to come to PSPC very early in the planning process in order to meet their timelines for projects.

83. These results from the client survey carried out as part of the evaluation are consistent with the results with respect to client satisfaction from the other lines of research. Overall while clients are, for the most part, satisfied with RPS services, there are areas of concern, such as the costs and timeliness of projects. Concerns related to cost include lack of clarity on invoices, private sector contractor disbursement costs, number of hours charged for PSPC internal labour, PSPC labour markup and other PSPC fees. The findings from the interviews with RPS senior management indicate that a number of client concerns reflect the need for improved communication, transparency and clarity with clients regarding all costs and the importance of early engagement and approval by clients.

84. In an effort to improve service delivery, RPSB is adopting a client-centric service delivery approach. As part of the transformation, RPSB has created the roles of account executives and service delivery managers. Account executives will be the primary point of contact for clients and service delivery managers will be added to account teams, where necessary, to coordinate and facilitate delivery of all real property services to the client.

6. Do real property services projects conform to the National Project Management System

85. The NPMS is a standardized project management system that PSPC organizations are required to follow when conducting projects. All real property projects over $25,000 in value are required to demonstrate the completion and approval of specific control points and documents within NPMS, although requirements for smaller projects are less demanding. NPMS is important as it supports effective project management.

86. The RPSB annually assesses the conformance of real property projects to NPMS. Annual conformance reports have been completed and published for the last 5 fiscal years, 2012 to 2013 to 2016 to 2017. The objective of the reports is to assess conformance to NPMS within each PSPC region, through a standardized set of criteria related to document and deliverable production and storage, and project approval signatures. These regional self-assessments are combined into an overall national assessment. In addition, the reports identify regional and national issues and recommend corrective actions.

87. In the interviews with RPSB senior managers, 12 out of 18 interviewees who responded to this question (67%) indicated that there is a high degree of overall compliance with NPMS.

88. The evaluation team's review of the 4 NPMS reports however, indicates that overall conformance to the NPMS rates are well below the targeted rate of 90%. Nationally, conformance has declined from 82% to 66% over the course of the 4 years.

89. As shown in Exhibit 13: National Project Management conformance reports—Levels of conformance in regions, the reports also reveal significant year-to-year variation within regions and among regions. For example, in the National Capital Area (NCA), the rate of conformance was 65% in 2012 to 2013, but declined to 41% by 2014 to 2015, before rebounding to a high of 74% in 2015 to 2016. In the Western region, conformance declined from 98% in 2012 to 2013 to 57% in 2015 to 2016. By contrast, in the Ontario region conformance increased from 50% to 67% over the same period.

Exhibit 13: National Project Management conformance reports—Levels of conformance in regions
Regions Report number 1
2012 to 2013		 Report number 2
2013 to 2014		 Report number 3
2014 to 2015		 Report number 4
2015 to 2016		 Draft Report number 5
2016 to 2017
NCA Operations (with major Crown projects (MCP) in report number 4 and number 5) 65% 73% 41% 74% 58%
Quebec 84% 78% 66% 74% 81%
Western 98% 94% 47% 57% 91%
Pacific 72% 71% 90% 75% 60%
Atlantic 79% 100% 84% 53% 84%
Ontario 50% 77% 77% 67% 81%
PBB planning and opérations 100% 50% 100% 100% 82%
Major Crown projects 100% 92% 75% n/a n/a
National 82% 81% 67% 66% 79%

90. The fourth (2015 to 2016) report indicates that the decline in the overall rate of conformance over the years is attributable to improved rigour of the regional self-assessments in some regions and that higher scores in earlier years were likely overstated. This report and the third report (2014 to 2015) also attribute the variations in ratings among regions to inconsistencies in the rigour of the review process. The fourth report concludes that improved rigor and standardization of the self-assessment process across the regions has affected the scores more than an actual change in the quality of the project files and that the more recent results are more accurate estimates of the actual level of conformance.

91. A key factor contributing to the lower than optimal ratings is a recurring problem with documents missing from project files. By 2015 to 2016, the issues with missing documents were described as "becoming somewhat alarming". The report indicates that the lack of adequate storage area to file documents prior to the existence of GCdocs and the slow rate of migration to GCdocs are major issues underlying the levels of conformance.

92. The 4 national compliance reports also highlight several other recurring issues underlying the lack of full conformance, including: lack of a standardized NPMS training program on the requirements of NPMS; staff turnover; lack of staff resources; heavy workloads in the project management community; and a lack of understanding of key self-assessment terms, such as 'major issue'.

93. The recurrence of these issues year-after-year indicates that the current low rates of conformance are not likely to improve until and unless these issues are addressed.

94. In addition to the low rates of conformance, the reports also have identified a number of areas of concern with regard to the methodology utilized to measure conformance. The evaluation team agrees with the reports' concerns with the methodology. Areas of concern include the following:

Selection of projects
Regional staff select the projects for assessment. Samples are not randomly selected, thus, the extent to which they are representative of all projects in the regions cannot be determined.
Regional self-assessment
Regions self-assess, which creates the potential for bias due to differing interpretations of conformance and varying degrees of rigor in the self-assessments.
Staff Training
Variations in the extent to which regional staff are trained in the self-assessment process can contribute to inconsistency in the ratings (for example regions vary in how they define "major issues").

95. In conclusion, NPMS conformance at 66% overall in 2015 to 2016, is considerably below the target of 90% conformance. As well, the reports have identified a number of weaknesses in the methodology used to assess conformance.

Conclusions: Outcome achievement

96. The RPS Program is achieving its immediate outcomes, for the most part. The program is providing clients with high quality real property services that generally meet client requirements, based on client satisfaction data. However, there is potential for improvement in both the tracking of time, budget, and scope for projects under $1 million and in ensuring clients are in agreement with planned project scope, budget and schedule.

97. As well, the program is not fully meeting other requirements and standards, as indicated by the relatively low rates of project compliance with PSPC's NPMS requirements. The evaluation team did not address whether, from a whole of government perspective, the RPS Program provides services that are cost-neutral as a result of leveraged economies of scale. This outcome will be addressed in a future horizontal evaluation of the full range of RPSB programs, including the Federal Accommodation Program and the Federal Holdings Program.

Efficiency

98. Demonstration of efficiency is defined as an assessment of resource utilization in relation to the production of outputs and outcomes. Efficiency refers to the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output.

99. To evaluate whether the RPS Program operates efficiently, the evaluation team looked at 3 areas:

7. Do the costs of oversight and management of the major program delivery contract (Real Property-1) reflect an efficient organization

100. To address this question, we reviewed actual costs for management and oversight of the RP-1 real property services contractFootnote 7 for fiscal years 2015 to 2016 and 2016 to 2017, based on time reporting data provided by the program, compared to estimates of these costs developed by them in 2015, as part of a management and oversight costing exercise. In addition, interviews were conducted with RPSB officials involved in management and oversight of the RP-1 contract to discuss costs and other aspects of the regime.

101. Under the RP-1 contract, Brookfield Global Integrated Services (BGIS) is responsible for the delivery of services and the RPSB is accountable for stewardship and performance. In support of exercising these accountabilities, the RPSB has put into place an extensive contract management and oversight regime. This regime is substantially more extensive, according to program officials, than the management and oversight regime in place during the previous contract (2005 to 2014) and was developed in response to perceived shortcomings in the previous regime. It includes 2 main reporting categories of management and oversight: "contract oversight" and "core support", both of which include a number of sub-categories.

102. The RP-1 contract was selected for the evaluation of management and oversight costs for several reasons:

103. The evaluation team found that, based on a costing exercise carried out in 2015, management and oversight costs for contractor-provided property management and project delivery services were estimated at approximately $33 million, or 6.6% of the contracted business volume for 2015 to 2016. Based on time reporting data provided by the program, actual management and oversight costs were $20.9 million, or 5.2% of business volume nationally in 2015 to 2016 and $23 million, or 3.9% of business volume in 2016 to 2017. These percentages are considerably below the 2015 estimate of management and oversight costs of 6.6% (Exhibit 14: Real Property-1 real property services contract management and oversight costs for property management and project delivery services as a percentage of business volume).

Exhibit 14: RP-1 real property services contract management and oversight costs for property management and project delivery services as a percentage of business volume
  Estimated 2015 to 2016 Actual
2015 to 2016		 Actual
2016 to 2017		 Estimated 2017 to 2018
Oversight costs $33,337.7 $20,924.9 $23,014.6 $29,218.4
Business volume $505,429.4 $405,656 $592,661 $455,547.9
% oversight 6.6% 5.2% 3.9% 6.4%

104. Based on discussions with program officials, the reported management and oversight costs likely underestimate the actual costs, for a number of reasons, including:

105. The regional management and oversight cost data provided by the program for 2015 to 2016 and 2016 to 2017 support the above explanation. The data shows that there were significant regional variations in these costs. For example, in 2015 to 2016 regional costs varied from 4% in the NCA to 8.5% in the Pacific Region. In 2016 to 2017, they varied from 2.3% in the NCA to 7.8% in the Pacific Region (Exhibit 15: Real Property-1 real property services contract regional management and oversight cost data).

Exhibit 15: RP-1 real property services contract regional management and oversight cost data
Regions 2015 to 2016 2016 to 2017
Atlantic 5.7% 5.9%
Quebec 4.8% 5.3%
NCA 4.0% 2.3%
Ontario 5.4% 4.0%
Western 8.4% 6.1%
Pacific 8.5% 7.8%
National total 5.2% 3.9%

106. As shown in Exhibit 14: Real Property-1 real property services contract management and oversight costs for property management and project delivery services as a percentage of business volume, the RPSB currently estimates that management and oversight costs for 2017 to 2018 will be approximately 6.4% of business volume, only slightly below the original 2015 to 2016 cost estimate of 6.6% and considerably above the actual reported data for 2015 to 2016 and 2016 to 2017, shown above. The projected costs (6.4%) for 2017 to 2018 are below industry norms of 8 to 10% for management and oversight costs during contract inception, but higher than industry norms for mature contracts (3 to 4%).

107. Program management expect that, as the contract matures and oversight becomes more efficient, these costs will decline to 4 to 5%—a level slightly above industry norms—in a couple of years. It is not clear yet whether this is a reasonable expectation. The reported management and oversight costs under the previous AFD property management contract were 4.9%; however, this figure is thought to be an underestimation, as management and oversight costs were not consistently tracked and reported for this contract. In addition, the federal government has higher costs than the private sector related to such things accountability, reporting and bilingualism than is typically the case for the private sector, suggesting it may be difficult to approach industry norms.

108. In conclusion, projected costs for management and oversight for 2017 to 2018, at 6 to 7% of business volume are higher than industry norms, however they are likely comparable to the actual, as opposed to the reported, management and oversight costs for the previous AFD major contract. Given this, and given that the current management and oversight regime is more comprehensive in scope and involves more organizational stakeholders, costs in the 6 to 7% range appear to reflect a more efficient organization, compared to the one overseeing the previous real property services contract.

8. Is the mark-up charged to clients on the hourly billable rate for Public Services and Procurement Canada internal staff appropriate

109. To evaluate against this indicator, the evaluation team reviewed RPS Program documents describing the hourly billable rate and the corporate and other costs that are included in the rate. As well, the evaluation team met with officials from across the department to obtain more detailed information on the nature and amount of the costs and the methodologies used to allocate these costs to the revolving fund. In addition, in our survey of RPS project clients, we asked clients whether the mark-up was a concern for them.

110. As a cost recovery program, RPS is required to recover the full costs of its services from clients. Costs of goods or services (disbursements) purchased from the private sector are passed through to clients at cost. The costs of PSPC internal staff time spent on RPS projects are recovered by charging clients a fee which incorporates a mark-up on the direct labour costs of internal staff.

111. To determine the fee charged to clients for internal staff, PSPC first establishes an hourly billable rate (HBR) for each staff member, based on their direct salary costs (salary and employee benefit plans). The department then applies a mark-up of 1.8Footnote 8 the HBR to account for indirect expenses, such as routine administrative tasks, training, occupancy costs, operations management and unrecoverable charges and service support charges. Exhibit 16: Expenses included in internal staff costs: 2016 to 2017 (in $ millions) provides a breakdown of the estimated costs for these in fiscal year 2016 to 2017.

Exhibit 16: Expenses included in internal staff costs: 2016 to 2017 (in $ millions)
Internal staff costs—Components Amount % of total internal staff costs
A. Direct expenses
Direct labour costs (1.0 × HBR) $220.4 55%
B. Indirect expenses
Routine administrative tasks and training (RTT) $26.8 7%
Occupancy costs 15.5 4%
Unrecoverable charges 1.9 0%
Operations' management 46.4 12%
Indirect O&M 12.6 3%
Service support charges
Corporate allocation model (CAM) sub-total 46.1  
Memoranda of understanding sub-total 9.1  
Invoices sub-total 19.2  
Total support services 74.4 19%
Total indirect expenses (0.8 × HBR) $177.6 45%
Total internal staff costs (A + B) $398.0 100%

112. A number of these indirect expenses are straightforward: time for routine administrative tasks and training is expected to be kept at 10% of billable time. Occupancy costs are fixed at 4% of internal staff costs, and are based on guidance provided by the FB. Unrecoverable charges are costs incurred that cannot be recovered from clients. They are highly predictable and represent less than 1% of internal staff costs.

113. Given the above, the evaluation team focused on the appropriateness of the operations management and indirect operations and maintenance (O&M) costs incurred by the program, which, combined (12% + 3%), represent 15% of internal staff costs; and on the service support charges which, at $74 million and 19% of internal staff costs, are the largest cost elements recovered through the mark-up on the hourly billable rate.

114. Operations management costs include the costs of time spent by PSPC staff on activities related to the planning, organizing, supervising and monitoring of billable resources. Similarly, indirect operations and maintenance (O&M) costs are unrelated to a specific project but are incurred in support of the program as a whole. Beginning in 2016 to 2017, these costs have been allocated to the revolving fund based on a well-defined set of business rules designed to ensure that only costs incurred in support of the program are charged to the program. Prior to this, these costs were set at 20% of internal staff costs, based on a costing exercise carried out in 2005. Program management monitors these costs to ensure they do not exceed a target ratio of 1 to 8 (one operations management resource per 8 direct program delivery resources).

115. It is noteworthy that, since this approach has been implemented, the operations management and indirect O&M costs have declined from 20% to 15% of internal staff costs (Exhibit 16: Expenses included in internal staff costs: 2016 to 2017 (in $ millions). It is not clear whether this reflects improved efficiency or whether these costs have been over estimated under the previous costing methodology.

116. Service support charges include internal departmental charges for corporate services. These charges are calculated and allocated through the corporate allocation model (CAM), which is managed by the FB. Service support charges also include charges to the revolving fund for services provided by other branches on the basis of MOUs (for example construction site safety inspections) and other internal transfers from the RPS Program for the costs of other enabling departmental programs or services, such as certain information technology (IT) costs not included in the CAM.

117. Corporate costs recovered from the revolving fund through the CAM include costs incurred by the FB, Human Resources Branch; Chief Information Officer Branch; Integrity Regime Renewal Branch; and the Policy, Planning and Security Branch.Footnote 9 These charges amounted to $46 million and represented 12% of internal staff costs (Exhibit 16: Expenses included in internal staff costs: 2016 to 2017 (in $ millions).

118. The costs for activities included in the CAM are calculated on the basis of the actual costs of the activity or on the percentage of staff time and effort allocated to the activity. They are then allocated to individual program branches based on one of the following methodologies:

119. The processes for calculating and allocating CAM charges appear to be rigorous. CAM charges are supposed to include only the costs of "basic" activities in support of program branches and are to exclude "infrastructure" activities, defined as beneficial to the department as a whole. RPS officials have expressed concern that they are being charged for infrastructure-related activities.

120. Discussions with RPS officials and with officials of the FB indicate that these concerns arise from 2 factors. Firstly, there are differences between RPS and departmental branches in the interpretation of "basic" and "infrastructure" activities. Secondly, because CAM charges are based at the level of activities, they may include some sub-activities that could be defined as infrastructure; however, the reverse is also true, in that some excluded activities may include sub-activities that are basic. Given these offsetting impacts, FB officials are of the view that the ultimate impact on CAM charges is minimal. It was not within the scope of the evaluation to undertake a detailed analysis of individual CAM charges to independently assess these impacts to assess the veracity of this view.

121. With respect to MOUs and other internal transfers, the evaluation team met with financial management advisors (FMAs) and other PSPC officials responsible for MOUs or other invoiced charges to validate the amounts and to obtain additional documentation on charges to the RPS revolving fund and on the allocation methodologies used.

122. There are currently 7 MOUs in place between the RPS Program and other PSPC branches that enable these branches to recover the costs of services not covered under the CAM. These MOU-based charges amounted to approximately $9.1 million in 2016 to 2017 and represented 2% of total internal staff costs, as shown in Exhibit 16: Expenses included in internal staff costs: 2016 to 2017 (in $ millions).

123. For these MOUs, cost estimates are developed annually and included in the MOUs, actual time and effort expended is monitored and the RPS Program is invoiced at year-end for the actual costs. Based on our review of the MOUs, the charges to the RPS, and the interviews with FMAs and program officials, these charges are based on a rigorous charging methodology.

124. In addition to CAM charges there are a number of other internal transfers from the RPS Program to other departmental branches, to recover the costs of programs and services provided to RPSB by these branches. In 2016 to 2017 there were 18 such transfers. These charges amounted to $19.2 million in 2016 to 2017 and represented 5% of total internal staff costs, as shown in Exhibit 16: Expenses included in internal staff costs: 2016 to 2017 (in $ millions). These charges cover a range of departmental and other costs, such as the Desktop Transformation Program and the IT Infrastructure Fund, the Integrity Framework and Fairness Monitoring programs; costs of the Workers' Compensation Board; and legal services provided by the Department of Justice.

125. The evaluation team reviewed 12 of these other internal transfers.Footnote 10 Based on this review, all of these invoice-based charges and their allocation methodologies were authorized by a decision of the Executive Committee and/or the Deputy Minister. Four of these other internal transfers are allocated on the basis of RPS' share of departmental FTEs. In 4 other cases, charges are based on the actual percentage of the charging programs' services that are attributable to the RPS Program (for example percent of staffing transaction). In 2 cases, the charge is based on an estimate of the share of these programs' business volumes generated by the RPS. In the case of legal services, the charge is based on a 3 year rolling average of legal services costs generated by the RPSB. In the case of the transfer to the Human Resources Branch to address its structural deficit, it is believed that it is based on FTEs, but program officials are not sure what the allocation methodology is.

126. While the allocation methodologies appear reasonable, for the most part, in the case of some of these charges the program funding issues that necessitated these transfers do not appear to have been updated in some years. For example, the authority for the transfer to Human Resources Branch (HRB) to address the latter's structural deficit dates back to 2006, while the authority for the transfer to the Chief Information Officer Branch (CIOB) for the Information Technology Infrastructure Fund (ITIF) dates back to 2010 and is based on the distribution of FTEs among departmental branches at that time. Given the amount of time that has passed, the fact that additional transfers to the CIOB were authorized in 2014 for the distributed computer environment and end user devices, the likelihood of changes in the departmental distribution of FTEs since then and that charges for both of these organizations are included under the CAM, it may be timely for the RPS Program to request updated information on the continued need for these charges.

127. In the case of the Integrity Framework program, the business case submitted in support of cost recovery included $400,000 per annum in database upgrades. It seems unlikely that there would be an ongoing need for $400,000 for upgrades and this may also be an area where the RPS could seek clarification.

128. Based on the above, for the most part, the service support charges to the RPS Program appear to be reasonable. While RPS officials have expressed concerns regarding the inclusion of infrastructure costs under CAM, it is not clear what the ultimate impact of the current CAM allocation methodology is on the amounts being charged to RPS. However, there is likely benefit to the program seeking clarification regarding its concerns with officials of FB and branches that charge the RPS through the CAM.

129. In the case of a few of the other internal transfers, there is likely a benefit to the RPS Program obtaining an update on the continued need for these charges. However, resolution of these questions is not likely to materially affect the overall costs to the program for support services. Given that it reflects these costs accurately, the mark-up on the hourly billable rate appears to be appropriate.

130. In addition to reviewing the appropriateness of the mark-up on labour costs, the evaluation team included a question in the client survey, asking clients whether project costs were a concern on their project. Of the 21 respondents who answered the question, 7 respondents indicated they had a concern with costs, with only one citing the mark-up on labour costs as a concern. The main concern revolved around communication with respect to costs, not the costs themselves.

131. In conclusion, while there are a few areas of potential concern with respect to the support service charges that need to be addressed, the mark-up on the hourly billable rate appears to appropriate, given that it accurately reflects the underlying cost structure of the RPS Program.

9. Do revenues per full-time equivalent reflect an efficient organization

132. As part of its review of program efficiency, the evaluation team examined whether the program had realized improvements to the average amount of revenue per full-time equivalentFootnote 11 over the 5 year evaluation period. Revenues from OGDs were obtained from the annual audited statement of operations for the RPS revolving fund, as were internal staff fees for projects carried out on behalf of PSPC clients. The disbursements for projects carried out for PSPC clients were obtained from program data.

133. As Exhibit 17: Revenue per full-time equivalent—Real Property Services Program shows, revenues per employee increased modestly from 2012 to 2013 to 2013 to 2014 and then declined slightly in 2014 to 2015. Since then, revenues per employee have increased, reaching $1.21 million in 2016 to 2017, a 17% increase over 2012 to 2013.

Exhibit 17: Revenue per full-time equivalent—Real Property Services Program
Fiscal year FTEs Total revenuestable 3 note 1 Revenues per FTE
2012 to 2013 2,590 $2.680 billion $1.035 million
2013 to 2014 2,259 $2.628 billion $1.163 million
2014 to 2015 2,263 $2.450 billion $1.083 million
2015 to 2016 2,459 $2.742 billion $1.115 million
2016 to 2017 2,705 $3.286 billion $1.215 million

Exhibit 17: Revenue per full-time equivalent—Real Property Services Program: Table 3 Note

Table 3 Note 1

Total revenues include flow through for disbursements to the private sector for OGDs and PSPC.

Return to table 3 note 1 referrer

134. It is not clear what factors have contributed to this improvement; however, as noted elsewhere in this report, this period has seen the injection of substantial amounts of funding for the Accelerated Infrastructure Program and for the Long-Term Vision and Plan of the Parliamentary Precinct, which may have impacted on revenue per employee. While funding for the latter is expected to be stable for some years, departmental funding projections for the Accelerated Infrastructure Program currently do not extend beyond 2018 to 2019, as noted earlier in the report.

10. What cost savings have been achieved by the program over the last 5 years

135. The evaluation team had intended to assess cost savings achieved by the RPS Program by comparing the costs of real property services provided by PSPC with those of provincial government organizations. It also planned to compare the cost savings achieved under the current delivery regime, which makes more extensive use of private sector contractors, with cost savings achieved through past sub-contracting.

136. To obtain comparative cost information, a review of on-line information on the real property services of the 10 provincial governments was conducted. The evaluation team also contacted 5 provincial government organizations responsible for the provision of real property services to all or part of their respective governments' organizations. Four of the organizations agreed to be interviewed and provided data in response to our request.

137. The interviews with provincial government representatives and the review of provincial government data did not provide a basis for comparison. There were several reasons for this, including:

138. The evaluation team had also intended to compare cost savings achieved from the current RP-1 contract for delivery of real property services, with cost savings achieved by the contract in place from 2005 to 2006 to 2013 to 2014, as reported in a 2008 evaluation report. However, based on discussions with program officials and on a review of program documents and past studies, comparing the costs of real property services under these 2 regimes would most likely be inconclusive for the following reasons:

139. More generally, the costs of real property services cannot be looked at in isolation. They need to be looked at in the context of the age and condition of buildings; the type of building (for example office, special purpose, heritage); available funding for the conduct of real property services; and other factors, such as future plans (for example disposal) for the asset or assets being compared. Given these considerations, it was determined not to proceed with the comparison of cost savings.

Conclusions: Efficiency

140. The program is providing efficient services, based on the indicators used in the evaluation. Program costs for management and oversight of its major private sector real property services contractor, which delivers a substantial portion of the program, are at levels similar to the previous contract while providing a more comprehensive management and oversight regime, indicating a greater degree of efficiency. As well, the fees charged to clients are, for the most, appropriate given the underlying costs structure of the program. Finally, revenues per full-time employee, despite declining slightly for 2013 to 2014 to 2014 to 2015, have increased 17% over the evaluation period.

Conclusions: Performance

141. In conclusion, the program is, for the most part, providing high quality, efficient services that meet client requirements. However, there is a lack of comprehensive information as to whether projects under $1 million are being completed on time, on budget and on scope. In addition, projects carried out under the program are not fully compliant with PSPC's NPMS requirements. As well, there are a few areas of potential concern regarding the allocation of support service charges to the program that require clarification by the program.

Program design

11. Are the respective accountabilities and responsibilities of Public Services and Procurement Canada and client organizations clearly articulated in program documents and are they well understood

142. The evaluation team assessed whether the accountabilities and responsibilities of PSPC and client organizations are clearly articulated in standard program documents and well understood by PSPC officials and client representatives. This was measured through 4 lines of evidence: interviews with RPS senior management, a review of the RPS 2016 client feedback survey, the client survey carried out as part of the evaluation, and a review of a number of client agreement documents.

143. In the interviews with senior managers of the RPSB, interviewees were asked whether the accountabilities of PSPC and clients are made clear in client agreement documents. 17 of them provided their views on this question. Of these, 11 indicated that accountabilities are made clear, with 5 interviewees specifically referencing project charters in this regard. However, a few interviewees indicated that project charters are not always signed or implemented or, more generally, that clarity could be improved around roles and responsibilities, governance, processes or funding. It was noted that recent program changes, confirming Account Executives as the primary point of contact at the national level for demand and escalation, Service Delivery Managers (SDM) as the primary point of client contact for service delivery and the Regional Service Delivery Manager as the primary point of contact at the regional level may help to clarify the interpretation of accountabilities.

144. The evaluation team also reviewed the RPSB 2016 Client Feedback Survey in relation to this issue. The 28 clients surveyed only moderately agreed that they were properly informed of any cost-sharing or risk management (3.25 out a possible score of 5) and that all aspects of the project were explained to them (3.21 out 5). In response to an open-ended question about areas where improvement is needed, 3 clients identified clarity around PSPC internal processes and project team roles and responsibilities.

145. The client survey carried out as part of this evaluation asked clients whether they had a clear understanding of accountabilities on their project and, if so, how they obtained this understanding. Only 13 of the 21 individuals who responded to this question (62%) indicated that they had a clear understanding to a significant or very significant extent. One-third of respondents indicated that they did not have a clear understanding at all or only did to a moderate extent.

146. The evaluation team also reviewed a number of RPSB client agreement documents to determine the extent to which they articulated, in a consistent fashion, the specific accountabilities and responsibilities of PSPC and clients, respectively. Documents reviewed included Deputy Minister-level departmental service agreements (DSAs) covering a range of departmental services; national service agreements (NSAs) covering real property services; MOUs authorizing clients to work directly with PSPC's contractors; service line agreements (SLAs) covering the provision of ongoing services; and project charters, governing specific real property projects.

147. The evaluation team reviewed a random sample of 6 DSAs of the 22 active in 2016 to 2017; 6 of 6 active NSAs, 12 of 49 active MOUs and 4 of 4 active SLAs. The team compared the text in individual clauses in these agreements under key agreement terms, such as implementation, finance, risk management, performance measurement, issues resolution and communication/governance to determine whether they addressed accountabilities and responsibilities in a consistent manner.  Based on this clause-by-clause review, the evaluation team determined the concordance of these agreements with respect to specific accountabilities and responsibilities.

148. The evaluation team also reviewed 17 project charters based on the extent to which the project charters contained clauses that addressed all of the mandatory sections of the agreements. This modified methodology was necessitated as, in project charters the wording within each section is tailored to the specific type of services being provided.

149. The evaluation team found that the overall rate of concordance for all agreements is very high, at 92%. NSAs and MOUs had concordance rates of 100%, while DSAs and project charters had rates of 92% and 93%, respectively. SLAs reviewed had a substantially lower concordance rate, at 59%. This was due primarily to the use of clauses in several sections that were highly customized to the particular program of work (that is fire inspection services versus environmental services). As well, 2 of the agreements included clauses relating to the provision of office support and to coordination with Shared Services Canada (SSC). Given  comments provided in the stakeholder interviews indicating that lack of clarity around responsibility for coordinating work with SSC has been an issue on some projects, this clause, in particular, would appear to be an important one that should be added to all agreements where coordination with SSC is a potential issue.

150. The evaluation team also examined the extent to which PSPC and clients exercised their responsibilities with respect to signing and dating these documents. This aspect of the review also included a review of specific service agreements (SSAs). The results of this examination indicate that, overall, 91% of documents reviewed had required dates and signatures. All project charters, SLAs, SSAs and MOUs reviewed were properly signed and dated; however, only 50% of DSAs and 60% of NSAs were. DSAs and NSAs define the PSPC/client relationship at the departmental and branch level, respectively and provide the overarching framework for the negotiation of agreements for individual programs or projects of work individual projects. Given this, it is important that they are properly signed and dated as the principles and parameters outlined in these documents could be ignored by clients or PSPC officials, if not signed.

151. Based on the above analysis, the specific accountabilities and responsibilities of PSPC and client organizations are articulated in a consistent and comprehensive manner in the various agreements. However, a significant percentage of DSAs and NSAs are not signed and dated.

152. In addition to the above analysis, the evaluation team included a second analysis that focused on overall accountability. The Treasury Board of Canada defines accountability as, "taking responsibility for performance in light of agreed expectations". The main differences between accountability and responsibility are that responsibility can be shared while accountability cannot; and being accountable means not only being responsible but being answerable for your actions. The agreements in our sample were reviewed specifically for statements of overall accountability between PSPC and client departments. Overall accountability is important in the case of, for example, contractor non-performance or project failure.

153. Based on this review, the evaluation team found that overall accountabilities are not consistently articulated in these documents. The DSAs outlined PSPC and client overall accountabilities; however, the NSAs did not refer to overall PSPC accountabilities, although they did articulate several specific client accountabilities.

154. The MOUs indicated that clients had overall accountability for work carried out on their behalf by PSPC's real property services contractor when they authorize work directly with the contractor. While they identified responsibilities for PSPC, they did not identify any accountabilities. This raises the question of whether PSPC or the client would be accountable for contractor non-performance.

155. Only one of the 4 SLAs reviewed identified overall accountabilities of PSPC or the client. Similarly, only one of the 15 project charters reviewed identified overall accountabilities of PSPC or the client department.

156. In conclusion, clients only moderately agree that they have a clear understanding of accountabilities, based on the RPS 2016 Client Feedback Survey and on the survey of clients conducted as part of this evaluation. Further, our review of client agreement documents indicate that, while they outline specific accountabilities and responsibilities in a consistent and comprehensive manner, overall accountabilities for the program or project of work, are not consistently articulated.

12. Does the program database effectively support program management, evaluation and performance monitoring

157. As noted earlier, in the course of the evaluation, the evaluation team obtained a download of program financial and project data from the RPS Program database, which is maintained on the departmental SIGMA system. This program data was utilized to assess program project, expenditure and revenue volume over the 5 year scope of the evaluation, overall and by service line. As well, it was used as the source population for the sample projects used for the client survey and for other analyses, such as on time and on budget analysis.

158. In the course of defining the data requirements for the evaluation and working with the population and sample databases, the evaluation identified a number of issues with the program database that created difficulties for the evaluation and that potentially could impact on the usefulness of this data for project and program management, and for program performance monitoring and reporting. These issues included the following:

159. While these issues did not provide insurmountable obstacles to arriving at evaluation findings, they created difficulties for the evaluation and limited the scope and depth of data analysis. As well, they undermine the usefulness of this data for program management and performance monitoring.

Conclusions: Program design

160. Program design could be improved. Client agreement documents do not consistently articulate overall accountabilities for programs or projects of work, creating the potential for misunderstandings on the part of clients. The program database maintained on the departmental SIGMA system lacks adequate documentation, contains a number of fields that appear to overlap with one another and others that are not completed consistently. In addition, financial data contained in this system is not easily reconciled with other sources of program financial information

General conclusions

161. The RPS Program is providing a wide range of quality, efficient real property services that are responding to a continued need and are meeting client requirements. While improvements are required to ensure services meet departmental standards for project management; to ensure that the RPS has a clear understanding of service support charges being allocated to the program; to clarify the respective accountabilities of PSPC and client departments; and to ensure that the program database effectively supports program management and performance monitoring, these weaknesses have not impacted program efficiency or effectiveness in a major way.

Management response

162. Management has had the opportunity to review the report, and agrees with the conclusions and recommendations found therein. The findings and recommendations will assist RPS in further improving our services to clients and our operations. A management action plan has been developed to address the evaluation recommendations.

Recommendations and management action plan

Recommendation 1

The Assistant Deputy Minister, Real Property Services Branch should ensure that the execution of the annual assessment of conformance to the NPMS is centralized, that the rigour of the process is improved and that the results are used to drive continuous improvement in compliance.

Management response 4

The Real Property Services Branch will enhance the annual conformance review by adding in centralized direct reviews of a sample of regional project files. This process will improve consistency and rigor across the country through the validation of the regional review processes and will ensure that the results are used to drive continuous improvement in compliance. The following actions will be undertaken:

Management action plan 1.1

Report on initial run of centralized reviews

Management action plan 1.2

Adjust process and have centralized reviews approved by the NPMS governance for inclusion into the National Project Management System Directive on the Monitoring of Real Property Projects' Conformance to the National Project Management System.

Recommendation 2

The Assistant Deputy Minister, Real Property Services Branch should take steps to ensure that RPS Program officials have a clear understanding of  the authorities, allocation methodologies and the current rationale for charges allocated to the program through other internal transfers, not the subject of MOUs.

Management response 5

Real Property Services Branch (RPSB) will, in collaboration with FB, ensure that RPS Program officials have a clear understanding of the authorities, allocation methodologies and the rationale for any new charges being allocated to the program through other internal transfers, not the subject of MOUs.

Management action plan 2.1

Program Management will assess completeness of its files for other internal transfers (OITs) with respect to authorities, allocation methodologies and the rationale:

Management action plan 2.2

Program management will identify which of the OITs need to be reassessed in terms of continued need for charging, that is, current rationale. (This will include but may not be limited to the 3 OITs that were questioned in the report as to whether there is a current rationale, that is, for Human Resources Branch, Information Technology Investment Fund and Integrity Framework);

FB will advise program management with respect to the current rationale for OITs brought to its attention and address any OITs that need to be discontinued.

Management action plan 2.3

For future OITs, FB will clearly communicate to program management the authorities, allocation methodologies and the rationale.

Recommendation 3

The Assistant Deputy Minister, Real Property Services Branch should review and update its client agreement documents to ensure that they all provide clear and consistent statements of the overall accountabilities of PSPC and client departments for programs or projects or work and that signatures are present.

Management response 6

A Directive on Real Property Service Client Agreements is in development to provide clarity on the purpose and roles and responsibilities of the various agreement instruments available to RPS managers who are responsible for putting into place formal agreements with clients. More specifically, for each type of agreement instrument, this directive will describe:

Management action plan 3.1

This directive will ensure a nationally consistent and structured approach to the use and application of the various client agreements across the real property service lines (SL):

Recommendation 4

The Assistant Deputy Minister, Real Property Services Branch should undertake a comprehensive review of the RPS SIGMA database to improve documentation on the database; to ensure the accuracy of project and financial data; to address possible redundancies; and to ensure the consistency and completeness of project information entered on the database.

Management Action Plan 4.1

Appendix A: Program activities

The Real Property Services Program provides 3 main types of real property services:

Project delivery: RPS is the federal centre of expertise for real property project management. RPS project delivery services cover all of the phases of a real property project, including project definition, initiation, analysis, planning, design, implementation and closeout. Services for project delivery include design and construction, asset rehabilitation and fit-up space.

Property and facility management: RPS is the federal centre of expertise for property and facilities management. RPS property and facilities management staff support federal real property custodians by providing operating and maintenance expertise for their assets. RPS makes use of the private sector, primarily the real property contractor (Brookfield Global Integrated Solutions), for the delivery of property and facilities management services to federal real property assets. In cases where a federal asset is managed by the PSPC Real Property Contractor under contract to PSPC, RPS employees are responsible for overseeing compliance with the contracts, according to oversight management procedures developed by the Real Property Services Branch. RPS employees approve work authorizations, certify that work has been performed by the real property contractor and validate that services have been rendered in accordance with the terms and conditions of the contracts. Quality monitoring reviews are also used as a method of validating the quality of services provided by the real property contractor. RPS conducts this work in support of the management of federal office facilities and specialized facilities.

In addition to the management, oversight and monitoring role that RPS plays in contracts with the private sector, RPS staff also continue to provide in-house services to federal organizations nationwide. These services range from project delivery services for highly specialized architectural, engineering, project management or technical services to custodians of federal real property across the country to the full suite of property management services to the Parliamentary Precinct buildings, including cleaning and janitorial personnel. While these cases are in the minority in the context of the Program's total volume of work, they are provided directly by RPS Program staff without relying on a private sector contractor.

Advisory services: RPS provides assistance in planning or decision-making for real estate solutions. These services include operational guidance that identifies opportunities for increased efficiencies, including how to optimize office space or enhance building services. It also includes financial advice which aims to help clients plan for future growth and development. As well, RPS experts can carry out complex real estate transactions on behalf of clients. RPS's strategic guidance is provided to align a client's real property strategy with their program objectives. RPS experts develop innovative solutions based on industry best practices to provide best value to clients and their programs.

Appendix B: About the evaluation

Authority

The Deputy Minister for PSPC approved this evaluation, on recommendation by the Audit and Evaluation Committee, as part of the 2016 to 2019 Risk-Based Audit and Evaluation Plan.

Evaluation objectives

The evaluation examined the RPS Program, delivered by the RPSB. This evaluation had 2 objectives:

Approach

The evaluation was conducted in accordance with the Treasury Board Policy and Directive on Results. The evaluation took place between June 2016 and January 2018 and was conducted in 3 phases: planning, examination and reporting. To assess the evaluation issues and questions, the following lines of evidence were used:

Document review: Documents included legislative and policy documents; client agreements; departmental planning and reporting documents (for example annual departmental plans and departmental results reports); and a wide range of program documents such as annual reports and studies, program planning and management documents, and program guidelines and directives.

Comparative review: A national and provincial review was conducted to contextualize the program and provide baseline data against which the program could be assessed. Substantial differences limited the comparability of the programs.

Financial analysis: Financial data related to the program's budgets, expenditures and staff resources was examined to assess the efficiency of the program. In addition to the expenditure and revenue data obtained from a download of program data from the SIGMA database, the evaluation reviewed the annual financial statements of operation for the RPS Revolving Fund; data on the various costs that are included in the determination of the mark-up on the hourly billable rate charged for internal staff fees; and data on the management and oversight costs for the RP-1 and RP-2 real property services contracts.

Interviews: 26 interviews were conducted: 21 with senior RPS managers, one with Treasury Board and 4 with provincial representatives.

Data Analysis: Program data was analyzed to provide information on the of the number and total square metres of real property for which PSPC is the custodian; the percentage of real property assets PSPC assets represent; and the number of buildings managed by RPS on behalf of other government departments. As well, the evaluation obtained and analyzed data on the number, value and characteristics of program projects over the last 5 years. As well, the evaluation obtained and analyzed on time and on budget, data for a sample of projects. The evaluation also obtained and analyzed data on management and oversight costs for contracted services; on the costs included in the mark-up on the hourly billable rate charged to clients for internal fees.

Surveys: A survey of program clients was conducted. The survey was sent to 118 client representatives and there were 26 valid responses, for a response rate of 22%. In addition, the evaluation made use of findings from the client feedback survey carried out by the RPSB in the fall of 2016.

Limitations of the methodology

The planning of the evaluation applied a risk-based assessment to determine the most appropriate approach and level of effort to be applied in order to ensure the availability of timely and objective information to meet the needs of senior management. In the application of the approach, several issues were encountered. The evaluation addressed limitations of specific methods by triangulating findings across multiple lines of evidence. None of the identified issues were significant enough to prevent evaluation reporting.

Comparative review: The review was limited due to a lack of data from comparator organizations, including from the provinces.

Interviews: Planned interviews with senior representatives of OGDs were not carried out due to a concern expressed by senior managers of the RPSB regarding client interview fatigue. The branch had recently carried out a survey of these individuals and they had also recently responded to the department's periodic client barometer. To address this gap, the evaluation carried out a survey of individuals who were project-level clients on recent RPS projects.

Survey: From the sample of projects used as the basis for the client survey, the same individual was often identified as the client representative for several projects. This limited the number of projects available for the survey conducted by the evaluation.

Data analysis: There were a number of issues with the program data for the years 2012 to 2013 to 2016 to 2017 that limited the usefulness of the data and the types of analyses that could be done with it. The expenditure and revenue data contained in the download, although close, was not completely consistent with the financial data contained in the program's annual audited statement of operations. As well, a number of fields pertaining to project characteristics, such as the type of project, the service line under which the work was done, and the funding program are optional were not completed for all projects, limiting the ability to conduct in-depth analysis of program trends and influencing factors.

Reporting

Findings were documented in a director's draft report, which was reviewed by the Office of Audit and Evaluation's Quality Assurance function. The program's director generals were provided with the director's draft report and a request to validate facts and comment on the report. A Chief Audit and Evaluation executive's draft report was prepared and provided to the Assistant Deputy Minister, RPSB, for acceptance as the Office of Primary Interest. The Office of Primary Interest was requested to respond with a Management Action Plan. The draft final report, including the Management Action Plan, will be presented to PSPC's Audit and Evaluation Committee for information in June 2018 and to PSPC's Performance Management, Evaluation and Results Committee for the Deputy Minister's approval in June 2018. The final report will be submitted to the Treasury Board Secretariat and posted on the PSPC website.

Project team

The evaluation was conducted by employees of the Office of Audit and Evaluation, overseen by the Director of Evaluation and under the overall direction of the Chief Audit and Evaluation Executive. The evaluation was reviewed by the quality assessment function of the Office of Audit and Evaluation.

Page details

Date modified: