Public Services and Procurement Canada
Targeted Review of the phased bid compliance process

Introduction

This engagement was included in the Public Services and Procurement Canada (PSPC) 2019 to 2022 Risk-Based Audit Plan following the work conducted by the Office of the Chief Audit Executive in its Strategy for Assurance Engagements related to Defence Programs and Projects.

One of the top priorities per the fall 2015 Mandate letter for the Minister of PSPC was to modernize procurement practices so that they are simpler and less administratively burdensome, and encourage greater competition. Accordingly, the Acquisitions Program established the flexible bid compliance as an objective in support of this Ministerial mandate. In July 2017, PSPC's Acquisitions Program implemented the Policy on the phased bid compliance process (PN-123), supported by a Handbook as additional guidance to enable its implementation. The Strategic Policy Sector of PSPC's Acquisition Program is responsible to monitor and evaluate the use of the phased bid compliance process, while the procurement officers are responsible for applying the process, collecting relevant data, and report it to the Strategic Policy Sector.

The phased bid compliance process is a process that can be incorporated into solicitations involving either a request for proposal or a request for standing offer. As stated in the policy, "it provides bidders or offerors with an opportunity, after the solicitation closing date, to provide missing financial bid information and to correct a finding of non-compliance with respect to Eligible Mandatory Requirements (mandatory requirements for which compliance can be assessed shortly after bid closing). Where a bidder or offeror is evaluated as non-compliant, they will be offered an opportunity to submit additional, or different information to be re-evaluated as compliant with respect to such eligible mandatory Requirement".

The policy sets out a common set of factors that will be the basis of PSPC's decision concerning whether the phased bid compliance process will form part of the terms of a given request for proposal or a request for standing offer. According to the policy, the phased bid compliance process must be included in a request for proposal or a request for a standing offer when the estimated value exceeds $2,000,000 and other relevant factors are present as stipulated in the policy.^{Footnote 1}

Notwithstanding the above, PSPC Directors and above may determine that: the phased bid compliance process not be included in a request for proposal or request for standing offer where the above factors are met; or, the phased bid compliance process be included in a request for proposal or request for standing offer where the above factors have not been met. A justification for the above-noted exceptions must be documented in the procurement file.

Furthermore, the application of the process is divided amongst 3 phases. Phase 1: compliance assessment of the required financial information of a bid; Phase 2: evaluates all other eligible mandatory requirements of a bid; Phase 3: final completion of the evaluation process. Bidders are given opportunities at Phase 1 to provide missing financial information, and 2, to achieve compliance by providing additional documentation.

The phased bid compliance process is expected to support the Government of Canada's procurement objectives of competition and best value to Canada by increasing the number of bids or offers that demonstrate compliance with the eligible mandatory requirements of a given PSPC request for proposal or a request for standing offer.

Definition of review

A review provides a moderate level of assurance by designing procedures so that the risk of an inappropriate conclusion being drawn based on the review procedures being performed is reduced to a moderate level. Procedures are normally limited to inquiry, analytical procedures and discussion. Risk is reduced to a moderate level when the evidence obtained enables us to conclude that the subject matter is plausible in the circumstances.

Focus of the review

The objective of this review was to determine whether the phased bid compliance process is: clear, understood, and consistently applied; contributing to retaining more suppliers; and, updated to integrate lessons learned.

The scope of the review focused on assessing the application of the phased bid compliance process from its implementation in July 2017. The scope period of this targeted review was from July 17, 2017 to October 31, 2019.

More information on the review objective, scope, approach and criteria can be found in the section "About the review" at the end of the report.

Statement of conformance

The review conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Review methodology

The fieldwork for this engagement was substantially completed in October 2019.

Throughout the examination phase of this targeted review, the review team assessed and analyzed documentation; interviewed key individuals within the procurement field; sent a survey to 99 procurement community members (participation rate of 51%); and, conducted procurement file walkthroughs with lead procurement officers, as it pertains to the phased bid compliance process. In addition, the review team selected a sample of 13 procurement files having used the phased bid compliance process, and have had the findings validated by their respective lead procurement officers. Furthermore, a process map of the phased bid compliance process was developed in order to represent and understand the process as a whole (Appendix A: Process map of the phased bid compliance process).

Conclusion

We found that for the sample of procurements in scope of our review, the application of the phased bid compliance process contributed to retaining 42% more suppliers. However, due to limitations in obtaining a complete population of both procurements that applied the phased bid compliance process and those that were eligible to apply the process, we were unable to assess whether the Process was applied consistently and to determine whether the Process as a whole, was successful in retaining more suppliers.

We found the Acquisition Program's ability to measure the success and performance of the phased bid compliance process was hindered by:

• a lack of an effective central tracking mechanism as it pertained to the tracking of procurement data on the phased bid compliance process
• a lack of clearly defined governance and oversight roles and responsibilities in the Policy on the phased bid compliance process to allow for monitoring its performance against outcomes
• a lack of lessons learned in each of the procurement files to feed into a continuous improvement process
  • although, internal consultations on the phased bid compliance process were held to gather broad lessons learned and incorporate them in the handbook

We found that guidelines have been designed and documented to enable the application of the process (such as a Phased Bid Compliance Handbook). However, an opportunity was identified to make the Handbook more accessible to the procurement community.

Finally, the results of our interviews and survey revealed the phased bid compliance process may not be clear to, or well understood by the procurement community. An opportunity was also identified to improve training activities for the application of the phased bid compliance process.

Observations

Effective and appropriate review procedures have been conducted and evidence has been gathered to support the accuracy of the findings and conclusions in this report with a moderate level of assurance. The findings and conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established review criteria that were agreed upon with management. The findings and conclusions are only applicable to the entity examined, and for the scope and time period covered by the review.

Information management of procurement files

Information management is an essential component of effective management across departments. The availability of high-quality information to decision makers supports the delivery of programs, services, and processes, thus enabling departments to be more responsive and accountable to Canadians. Adequate information management allows for subsequent monitoring, performance measurement, and reporting, which supports the development of meaningful lessons learned and continuous improvement.

Throughout our review, we were unable to obtain a complete data set which included all procurement files where the phased bid compliance process was applied, or, the population of procurements that may have been eligible for applying the phased bid compliance process (that is had a value of $2 million, and had at least one other factor as stipulated in the policy).

We found there was no central tracking mechanism that would allow for the identification of procurement files that applied the phased bid compliance process, because the procurement systems used to input procurement information did not track information to the phased bid compliance process specifically. This was also supported by comments provided during our interviews, as 15 out of 15 (100%) of interviewees acknowledged being aware of data monitoring deficiencies as it pertains to the phased bid compliance process. Given these limitations and in absence of other tracking tools, the Strategic Policy Sector implemented a manual tracking mechanism to identify procurements that potentially applied the process by maintaining a list of requests for the Handbook by the procurement officers; we were informed however that the list may have been incomplete and may have not reflected all the procurements that applied the process.

In terms of continuous improvement of the phased bid compliance process, we found that the Strategic Policy Sector conducted 5 information sessions in August 2019 with the procurement community across the regions to obtain feedback on the process. We noted however that no formal process was in place on an ongoing basis to capture lessons learned at the end of each procurement that applies the process, for the purposes of continuous improvement. This was confirmed through file review (no lessons learned were documented in any of the files reviewed) and through the results of the interviews and surveys demonstrating that 36 out of 42 (86%) survey respondents are unware of any lessons learned being developed as per the use of the phased bid compliance process, and 13 out of 15 (87%) interviewees could not identify any form of lessons learned.

Increased competition

Our review of the files selected for testing revealed that the application of the phased bid compliance process resulted in the retention of initially non-compliant bidders. From the reviewed procurement files having applied the phased bid compliance process:

• 12 out of 13 (92%) were successful in awarding a contract for a total value of $419,531,996.52
• the total number of suppliers which provided an initial bid in our file review was 55, from these 55, 27 (49%) were asked to provide additional information; 20 (74%) of the 27 suppliers asked to provide more information were successful in provision of information and retained accordingly
  • as a result, the phased bid compliance process was successful in retaining 20 (74%) suppliers which were initially deemed non-compliant
  • therefore, without using the phased bid compliance process, only 28 suppliers would have been considered
  • in the application of the Phased-Bid Compliance Process, 20 more suppliers were eligible, bringing the number of qualified suppliers to 48, an increase of 42%
• additionally, 4 of the 12 (33%) contracts awarded, were awarded to suppliers who would have otherwise been eliminated without the phased bid compliance process in place. These 4 contracts have a value of $13,375,321.07, demonstrating the magnitude of products and services saved by avoiding a re-solicitation to the government of Canada, should the process have not been used

Due to unavailability of procurement data, we were unable to conduct an in-depth analysis on all procurements that may have been eligible to apply the phased bid compliance process (that is over $2 million, and were subject to one of the factors identified in the Policy on the phased bid compliance process) to assess whether it was applied consistently or identify potential gaps or root causes for not applying the process. Further, because the inventory of the procurement files that went through the phased bid compliance process was incomplete, our review sample may not have been representative of the entire population, impacting the accuracy and relevance of test results. Due to these limitations, we were unable to provide an overall assessment as to whether the phased bid compliance process was applied consistently and was overall successful in retaining more suppliers.

The design and implementation of the policy on the Phased Bid Compliance Process and supporting guidelines

Policies and procedures are important instruments that enable departments to manage risks to operations and achieve expected results and objectives by establishing specific rules to be followed in a consistent manner across identified stakeholders. Well-designed policies should follow a common set of principles that include: clear definition and delegations of the roles, responsibilities, and, performance expectations for all stakeholders. Additionally, policy compliance should be adhered to, and monitored through credible and timely information reports, including performance management frameworks which encompass the necessary processes and oversight for successful implementation.

Roles, responsibilities and accountabilities

The Policy on the phased bid compliance process was developed and implemented by the Strategic Policy Sector within PSPC's Procurement Branch. The phased bid compliance process is one of the many policies, clauses and procedures that is intended to be included as part of the Supply Manual and the Standard Acquisitions Clauses and Conditions (SAAC) manual. The 2 manuals are primarily for the use of PSPC contracting officers acting in PSPC's capacity as a common services provider; when it undertakes procurements on behalf of other Government of Canada organizations. As such, the Policy on the phased bid compliance process is inherently intended for the use and application by the same audience, and its format and content are more similar to a set of mandatory procedures rather than a traditional departmental policy.

Upon review of the policy, we noted that it identifies an overview of the responsibilities for when and how to apply the phased bid compliance process. The policy also refers to the phased bid compliance process Handbook which provides more detailed roles and responsibilities for Acquisition Program procurement officers, officials in client departments and agencies, as well as bidders and offerors. The phased bid compliance process Handbook is an evergreen procedural document that is made available upon request.

We found that while the Policy on the phased bid compliance process was designed to define the roles of the procurement officers in applying the process, it does not clearly define responsibilities and accountabilities for: overseeing the implementation, application and monitoring of the policy; measuring its performance; nor reporting on results and instances of non-compliance. Interviews with the Strategic Policy Sector indicated that their responsibilities are to design and implement the policy. Accordingly, the group is not given any enforcement authority to monitor its compliance once in use, as that is part of the Procurement Directorates' duties under the regular procurement practices. As a result, the Strategic Policy Sector was unable to oversee whether the application of the process was followed as intended.

Performance measurement, monitoring and reporting

The main objective of the Phased Bid Compliance process is to encourage greater competition by increasing the number of eligible bidders. The Policy on the phased bid compliance process includes a section on expected outcomes, which states that "the phased bid compliance process is expected to support the Government of Canada's procurement objectives of competition and best value to Canada by increasing the number of bids or offers that demonstrate compliance with the eligible mandatory requirements of a given Public Works and Government Services Canada (PWGSC) request for proposal^{Footnote 3} or a request for standing offer^{Footnote 4}". As mentioned above however, the policy does not identify who is responsible and accountable to monitor and measure its performance and success in achieving the expected outcomes.

Further, the Policy on the phased bid compliance process includes a section on reporting requirements, which states that procurement officers must provide information related to the usage of the phased bid compliance process in an individual request for proposal or a request for standing offer, back to the Strategic Policy Sector. These reporting requirements are detailed in the phased bid compliance process Handbook, and are to be applied at 5 procurement stages. The purposes of monitoring compliance with the policy on phased bid compliance process are: assessing the effect of the phased bid compliance process on competition; assessing the effect of the phased bid compliance process on the procurement process (for example throughout time); and, to determine the appropriate scope of the policy application. Discussions with the Strategic Policy Sector confirmed that these reporting requirements were purposely included in the policy by the sector as a feedback mechanism on the use of the phased bid compliance process by obtaining data on procurements that applied the phased bid compliance process.

While the Policy on the phased bid compliance process identifies expected outcomes for the process, it does not provide for oversight and monitoring to ensure that the performance and success in achieving the expected outcomes are monitored, measured and reported upon. While reporting requirements are defined in the policy, as per above, no authority is established to enforce compliance. Consequently, reporting back to the Strategic Policy Sector at each procurement stage is at the discretion of the procurement officers. Our file review revealed that for none of the files reviewed did the procurement officers report back to the Strategic Policy Sector the required information at each of the 5 reporting stages, as per the policy and Handbook requirements.

Interpretation and application of the policy, related criteria and supporting guidelines

The baseline for an effective policy is to provide requirements that are clear, understandable and accessible, so to increase the likelihood of success in applying it consistently and as intended. The Policy on the phased bid compliance process is composed of a set of factors which drive its use, and a set of steps related to its application. In addition, the phased bid compliance process Handbook has been developed to document and provide specific details on how to apply the process.

We conducted a survey with members of the procurement community to obtain their feedback on the policy, process and supporting guidance tools. The results from the interviews revealed that 6 out of 7 (86%) interviewees (excluding individuals from the Strategic Policy Sector who manage the process) indicated a lack of clarity and understanding of the Policy on the phased bid compliance process, particularly with respect to the interpretation of the factors which drive the use of the policy. More specifically, comments were provided as to the interpretation of one of the factors that drive the use of the policy, namely factor "that is - requirement is complex", which by its subjective terminology increases the likelihood of different interpretations and hence applications by each individual procurement officer.

In addition, 35 out of 43 (81%)^{Footnote 5} survey respondents indicated that they did not always consider the phased bid compliance process for their procurement files. Through interviews and survey comments received, the main reasons provided for not considering the process were because some weren't aware of it, did not fully understand it or weren't comfortable in applying it. Similarly, 22 out of the 31 (71%)^{Footnote 6} of the respondents did not find the Handbook to be clear.

In terms of accessibility, we noted that the policy was available on the department's Buy and Sell^{Footnote 7} website, but that it had not been incorporated into the Supply Manual nor the Standard Acquisition Clauses and Conditions Manual, as we were informed it was still under review at the time of our engagement. Also, limited awareness activities had occurred since its publication to increase awareness of its existence Furthermore, the phased bid compliance process Handbook was not readily available on the procurement web site, but was rather obtained on demand, by emailing the Strategic Policy Group for a copy.

We inquired with the Strategic Policy Group as to the availability of training as it pertains to the phased bid compliance process. We were informed that during the launch of the phased bid compliance process, training was offered for approximately 5 months to the procurement community to inform them of the new initiative and its application. Since then, no formal training has been offered, however guidance has been provided upon request, by answering questions received in the Strategic Policy Sector's generic mailbox. Similarly, the results of our survey on training revealed that 34 out of 42 (81%)^{Footnote 8} of respondents were unaware if any training regarding the phased bid compliance process existed. Also, 33 out of 42 (79%)^{Footnote 9} of respondents indicated they would be interested in taking training as it pertains to this process. Although no formal training was in place for the application of the Phased Bid Compliance process at the time of the review, we were informed that the Strategic Policy Sector were engaging with the Professional Training and Development Office within PSPC to look at opportunities to develop a standalone course on the phased bid compliance process to improve understanding and application; however no specific timelines were provided.

About the review

Authority

This engagement was included in the Public Services and Procurement Canada (PSPC) 2019 to 2022 Risk-Based Audit and Evaluation Plan.

Objective

The objective of this review was to determine whether the phased bid compliance process is: clear, understood, and consistently applied; contributing to retaining more suppliers; and, updated to integrate lessons learned.

Scope and approach

The focus of this review was to assess the application of the phased bid compliance process from its implementation in July 2017. The scope period of this targeted review occurred from July 17, 2017 to October 31, 2019.

To assess the phased bid compliance process, we examined a sample of 13 procurement files which were subject to the phased bid compliance process. These files were selected from a list of projects provided by the Strategic Policy Sector from the Acquisition Program. The selection of the files to be reviewed occurred during the aforementioned scope period.

This review was conducted in accordance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.

Criteria

The review criteria, as agreed by offices of primary interest (OPI) as part of the Targeted Review Terms of Reference were derived from the 2015 Mandate letter for the Minister of PSPC regarding the modernization of procurement, and the Policy on the phased bid compliance process (PN-123).

The criteria were as follows:

• criteria for applying the phased bid compliance process to procurements are clear, understood, and consistently applied
• policies, processes and procedures related to the phased bid compliance process are clearly documented, understood and consistently applied
• the phased bid compliance process is successful in retaining initially non-compliant suppliers
• lessons learned from the phased bid compliance process are gathered, analyzed and considered for the continuous improvement of the process

Review work completed

Review fieldwork for this engagement was substantially completed in October 2019.

Review team

The review was conducted by members of the Office of the Chief Audit Executive, overseen by the Director of Procurement and under the overall direction of the Chief Audit Executive. Furthermore, the engagement was reviewed by the quality assurance function of the Office of the Chief Audit Executive.

Appendix A: Process map of the phased bid compliance process

Image description

This table is a representation of the phased bid compliance process as it is intended to be applied to solicitations. It is meant to be a descriptive tool to help procurement officers, and others, to better understand the application of the policy.

The flowchart covers the 5 sections of procurement phases:

1. requirement definition
2. procurement strategy
3. solicitation
4. evaluation
5. approval

Section 1: Requirement definition

The first section (requirement definition) requires the user to decide whether the requirement exceeds $2M. If it does not, a procurement director is given the authority to still use the process, if not, the process does not need to be used. Should the director decide to apply the process, an approved director inclusion form is required, which is then submitted to Strategic Policy Sector (Stage 1).

Section 2: Procurement strategy

The second section (procurement strategy) continues should the requirement exceed $2M in the previous section. If so, an assessment of the phased bid compliance process 7 factors need to be considered to confirm if the procurement file is required to use the phased bid compliance process. If not, it is at the Director’s discretion to use the phased bid compliance process or not. If one of the 7 factors from the phased bid compliance process is present, directors are given the authority to decide whether they would like to exclude the process. Should they decide to exclude the process, a director exclusion form is required, and must be documented in the procurement file with a rationale. Following doing so, this form must be reported to the Strategic Policy Sector (Stage 1).

Section 3: Solicitation

The third section (solicitation), continues should the director decide they do not want to exclude the process in the previous section. At this time, the work flows as such: approval of the procurement process, the Strategic Policy Sector records data and provides clauses, report to the Strategic Policy Sector (Stage 2), include the use of the phased bid compliance process clause in the request for proposal and report to the Strategic Policy Sector (Stage 3).

Section 4: Evaluation

The fourth section (evaluation), is the key part of the phased bid compliance process application. This is where the process is intended to give suppliers a second opportunity to remain in consideration should they have missed or forgotten required procurement information. This section is broken down into 3 phases. The first phase is known as the financial portion. Procurement officers will evaluate if the bid contains all necessary financial details, if yes, they can proceed to phase 2. If not, procurement officers must notify bidders of missing financial information and give them a specific timeframe to provide it. After said timeframe, should the bidders remain unsuccessful, they will be eliminated from evaluation. Should they prove successful, their bid proceeds to phase 2. Phase 2 is known as the mandatory requirement phase. If all mandatory requirements are met, the bid proceeds, and the procurement officers issue a Compliance Assessment Report informing compliancy. At such time, the bid then moves to phase 3. However, if the bid does not have all mandatory requirements, procurement officers issue a Compliance Assessment Report and allocate a specific timeframe for response. Should the bidder be unsuccessful at responding, procurement officers are to report to the Strategic Policy Sector (Stage 4), and eliminate the bidder. Should the bidder be successful in responding, procurement officers are to report to the Strategic Policy Sector (Stage 4), and advance to Phase 3. Phase 3 is a complete evaluation of all remaining bids.

Section 5: Approval

The fifth and final section (approval) is where contract award is determined, and then reported to the Strategic Policy Sector for a last time (Stage 5).