Language selection

Government of Canada / Gouvernement du Canada

Search

Future Fighter Capability Project

Fairness monitor final report: February 13, 2023

Addendum to final report: February 15, 2023

Submitted to: Director, Fairness Monitoring Program

Submitted by: HKA Global (Canada), Inc.

On this page

Attestation of assurance

The fairness monitor (FM) hereby provides the unqualified assurance statement below concerning the Future Fighter Capability Project (FFCP).

It is our professional opinion that the procurement process we observed or monitored was carried out in a fair, open, and transparent manner.

Original signed by:

Rick Moffat
Partner Canada
HKA Global (Canada), Inc.
Fairness Monitoring Contractor’s Representative

Andrea Robinson
Fairness Monitoring Team Leader
Fairness Monitor Specialist

Bruce Maynard
Professional Engineer
Fairness Monitor Specialist

 

Geneviève Arcand
Fairness Monitor Specialist

Jean Montplaisir
Fairness Monitor Specialist

Project requirement

HKA Global (Canada), Inc. was engaged on March 29, 2017, as a FM to observe the competitive procurement process for the Future Fighter Capability Project (FFCP) conducted by Public Works and Government Services Canada (PWGSC) on behalf of the Department of National Defence (DND) under solicitations W847A-180210/A and W847A-180210/B. The objective of the procurement process is to replace Canada’s CF-18 fighter fleet with an advanced fighter fleet including associated sustainment support. HKA Global (Canada), Inc. is an independent third party with respect to this activity.

The procurement of advanced fighter aircraft will require the involvement of both the manufacturer of the aircraft as well as the government of the manufacturer’s country or responsible defence organization. The FFCP has adopted the term Supplier to identify a team of a government or defence organization together with at least the fighter aircraft manufacturer, which together have been qualified to submit a proposal in response to the request for proposal (RFP). In view of its distinctive meaning applicable to this project, Supplier is capitalized in this report.

This report covers the activities of the FM during the entire procurement process, including the supplier qualification and engagement phase from April 10, 2017 to July 23, 2019, through the RFP phase from July 23, 2019 to July 31, 2020, the proposal evaluation and selection phase from August 1, 2020 to March 28, 2022, and the finalization phase from March 28, 2022 to November 4, 2022.

We reviewed all of the information provided and observed or monitored all relevant activities.

This report includes our:

Fairness monitoring engagement and observations

The scope and objectives of the assignment was to provide independent observation and monitoring of the procurement process and to submit fairness-related comments to project officials, as early as possible, so that appropriate action could be taken to address the comments before fairness was impacted. The director, Fairness Monitoring Program would be advised of any fairness-related concerns that were not addressed promptly.

In this section

In accordance with the terms of our engagement, we:

Supplier qualification and engagement phase

Activities and findings prior to the posting of supplier list invitation solicitation

During the period April 10 to 20, 2017, we reviewed background information provided by the project team including:

During the period May 11 to June 11, 2017, we reviewed a set of questions that were developed to form the basis for discussions with potential Suppliers at the Paris Air Show to ensure that each potential Supplier was provided with the same information and had the same opportunity to respond. Fairness-related comments were provided and appropriate action was taken by project officials. Subsequently, the activity related to the Paris Air Show was cancelled.

On May 24, 2017, we observed a meeting of FFCP officials and the Canadian Defence and Security Industries Association and a meeting of FFCP officials and the Aerospace Industry Association of Canada. Only publicly released information was provided by FFCP officials. The 2 organizations emphasized the importance of the project to their members. No fairness-related deficiencies were identified.

On June 30, 2017, we were advised that the FFCP project team would be visiting relevant European governments-agencies in July 2017 to collect information to inform planning for the procurement. We reviewed plans for the meetings including reviewing the presentations that would be provided and the questions that would be asked during each meeting. Fairness-related comments were provided and appropriate action was taken by project officials. Following the meetings, we reviewed summaries of each meeting including the information obtained. No fairness-related deficiencies were identified.

On July 30, 2017, we were advised that the FFCP project team would be making a similar visit to the United States in August 2017 for the same purpose and would be returning to Europe in the autumn to meet with additional governments, including governments that were in the process of completing, or had recently completed, similar fighter aircraft acquisitions. The purpose of the visits was similar to the earlier European visit and a similar presentation and questions would be used. We reviewed plans for the meetings and the presentation that would be used, and the subsequent summaries. No fairness-related deficiencies were identified.

On August 11, 2017, we reviewed a high-level outline of the draft value proposition (VP) strategy for the project. On September 27, 2017 we reviewed an outline of a process to select Suppliers that had demonstrated a capability to provide a fighter aircraft that meets Canada’s requirements. The process would use a letter of interest (LOI) solicitation entitled, and referred to in this report, as the Supplier list invitation (SLI) solicitation. Fairness-related comments were provided and appropriate action was taken by project officials.

On October 31, 2017, we monitored a teleconference call between the Ministry of Defence United Kingdom (MOD/UK) and the FFCP project office, the purpose of which was to better understand the contract arrangements between a ministry of defence of a NATO EF 2000 and Tornado Development, Production and Logistics Management Agency (NETMA) member country and NETMA regarding a sale to a non-NETMA country. No fairness-related deficiencies were identified.

On October 26, 2017, we reviewed a new draft of the SLI along with drafts of cover letters to applicable foreign governments. The plan was to forward the letters, with the draft SLI attached, to potential Supplier governments to obtain comments on the draft SLI. Revised versions of the SLI and covering letters were reviewed as they were developed until the letters with the draft SLI were released on December 12, 2017.

On November 6, 2017, we responded to a query from the contracting authority regarding the timing of the finalization of the evaluation plan for responses to the SLI and the release and closing of the SLI solicitation. Fairness-related comments were provided and appropriate action was taken by project officials.

Activities and findings following the posting of Supplier list invitation solicitation

On December 12, 2017, we reviewed the SLI as posted on buyandsell.gc.ca. The SLI invited potential Suppliers that met the specified criteria to submit a response to the SLI. The SLI had no closing date and would remain open until Canada determined it was no longer required, however the SLI encouraged responses by February 9, 2018. No fairness deficiencies were identified.

Eight amendments were issued to the SLI. We reviewed each amendment just prior to it being posted. No fairness-related deficiencies were identified.

On December 12, 2017, we also reviewed a letter to be sent to a foreign government requesting that it respond to the SLI. The foreign government had asked for the letter as its regulations prevented it from responding to the SLI otherwise. A similar letter would be sent to other foreign governments if Canada was requested to do so. No fairness-related deficiencies were identified.

On December 22, 2017, we reviewed a draft agenda for industry day scheduled for January 22, 2018. Up until the event was held, we reviewed emails concerning planning for the event and the one-on-one meetings which would follow. Fairness-related comments were provided and appropriate action was taken by project officials.

On January 22, 2018, we observed the industry day. The agenda was followed, hard copies of the presentations were provided to attendees and questions answered. During the period January 22 to 29, 2018, we observed or monitored by teleconference 1 hour long one-on-one meetings held with 30 firms which had requested meetings. Not all the firms manufacture aircraft. In each case the firm outlined its capabilities and interest. Project officials responded to questions to clarify information provided at the industry day session and did not provide any additional information. No fairness-related deficiencies were identified.

On January 31, 2018, we reviewed a draft evaluation management plan (EMP) for SLI responses. The plan described the evaluation organization and structure, roles and responsibilities, process, and evaluation framework. While the SLI specified basic qualification criteria that would be straightforward to evaluate, the plan ensured a structured, rigorous and well documented evaluation activity. On February 5, 2018, we reviewed a revised EMP. Fairness-related comments were provided and appropriate action was taken by project officials.

On February 2, 2018, we reviewed plans for the development of an industrial technological benefits/value proposition (ITB/VP) framework for the project that would be workable and fair for all potential Suppliers and achieve government ITB/VP policy objectives. No fairness-related deficiencies were identified.

On February 7, 2018, we reviewed the presentation deck for the information session for all the SLI evaluators and on February 8, 2018, we monitored an evaluators’ information session. No fairness-related deficiencies were identified.

On February 13, 2018, we were advised that the FFCP project team was planning a visit to MOD/UK, in the near future, to obtain an understanding of interoperability requirements complementary to discussions held previously with United States Department of Defense (US DOD) on the same subject. On February 21, 2018, we reviewed a summary of a teleconference call between the FFCP project team and MOD/UK outlining the purpose of the upcoming visit. No fairness deficiencies were identified.

On February 16, 2018 we were briefed on the results of the evaluation of responses to the SLI. All results were agreed unanimously by each of the evaluation teams and there were no issues or concerns. No fairness-related deficiencies were identified.

On February 19, 2018, we reviewed draft emails to be forwarded to the respondents to the SLI, one format to the 5 respondents that had been determined to be qualified as Suppliers, and a different format to those respondents that had been determined to be not qualified. The emails were subsequently forwarded to the respondents. We also reviewed an outline of the plan to announce and post the list of qualified Suppliers on buyandsell.gc.ca. No fairness-related deficiencies were identified.

Activities and findings during qualified Supplier engagement

Supplier week 1

On March 2, 2018, we reviewed draft documents providing:

Fairness-related comments were provided and appropriate action was taken by project officials.

On March 6, 2018, we reviewed a draft operations procedure document that provided rules for interaction between the FFCP project office and the next generation fighter capability (NGFC) office in DND. The NGFC office manages Canadian participation in the joint strike fighter program office in which Canada is a member. The procedure document stipulated that there be no communications or information transfer between the FFCP and NGFC offices and all communications by FFCP with the joint strike fighter competitor be direct, identical to communications by the FFCP with the other FFCP competitors. Fairness-related comments were provided and appropriate action was taken by project officials.

On March 9, 2018, we reviewed the approved statement of operational intent (SOI) for the definition stage of the FFCP which provided an outline of how the FFCP will be used. The SOI flowed from approved defence policy and would be the basis for FFCP requirements being specified in the RFP.

During the period March 7 to 25, 2018, we reviewed planning documents for the upcoming group of one-on-one meetings. The documents were presentation decks outlining Canada’s intended requirements including:

There was also a pivotal presentation deck outlining a proposed procurement approach that recognized the unique nature of the acquisition, namely the participation of the government members of the Supplier teams. The intent was to obtain comments and feedback on the procurement process and requirements that would inform the procurement activity so that appropriate adjustments could be made. No fairness-related deficiencies were identified.

During the period March 26, 2018 to April 11, 2018, we observed or monitored the one-on-one meetings with each of the qualified Suppliers. Each Supplier included both government officials and an aircraft manufacturer. The agenda was the same for each of the Suppliers. Each was given identical presentations and were invited to comment, including providing written comments at a later date. Each was allotted the same time, which was sufficient in all cases. After each one-on-one meeting, we observed the post-meeting of the FFCP project team during which the team reviewed the feedback provided by each Supplier and assigned tasking to team members when appropriate. Fairness-related comments were provided and appropriate action was taken by project officials.

Supplier week 2

On April 21 and May 20, 2018, we provided fairness-related comments relative to planned visits by the FFCP project teams to Supplier operational bases to be better informed of infrastructure requirements for the new aircraft. These visits were conducted in accordance with a protocol which we reviewed on May 7, 2018, to ensure the visits were limited to the approved purpose and did not involve any other information exchange. No fairness-related deficiencies were identified.

During the period May 21 to 25, 2018, we reviewed summaries of teleconferences between Canada and each of the Suppliers that provided an update on the development of the economic impact assessment (EIA) criterion which was to be included in the RFP. The EIA was a new criterion which was still being developed. No fairness-related deficiencies were identified.

On May 25 and 28, and June 5, 2018, we reviewed draft presentation decks prepared for the upcoming Supplier engagement meetings. During the period June 13 to July 5, 2018, we observed 5 Supplier week 2 meetings, 1 with each Supplier. The approach was the same for each Supplier. A background presentation on the agenda topic was provided and a dialogue opened with the Supplier to obtain comments to better inform the development of the RFP. The same amount of time was allotted to each. After each meeting the project team held a post-meeting to discuss what was heard and to provide tasking. Fairness-related comments were provided and appropriate action was taken by project officials.

On July 10, 2018, we reviewed a draft LOI that was to be posted to obtain feedback from Canadian industry on their capabilities to support the FFCP to better inform the development of the VP criteria for the FFCP RFP. No fairness deficiencies were identified and the LOI was subsequently posted on July 24, 2018.

During the period July 27 to 30, 2018, we reviewed and provided comments on a draft FFCP Solicitation Approach document. Our comments were subsequently discussed with FFCP officials. On August 17, 2018 we reviewed feedback provided on our comments and we reviewed a revised version of the draft FFCP solicitation approach document and comments on the earlier version. On August 20, 2018, we provided fairness related comments on the revised version and appropriate action was taken by project officials.

On July 31, 2018, we reviewed the objective of a proposed teleconference with 1 Supplier. The meeting would involve a VP matter applicable only to the specific Supplier. No other matters would be discussed. No fairness deficiencies were identified.

On August 7, 2018, we reviewed a document reporting on the visits of the project team conducted in June, 2018 to Supplier operating bases to better inform the development of infrastructure needs for the future fighter capability. No fairness-related deficiencies were identified.

On August 8, 2018, we were informed that FFCP had contracted for the services of a third party reviewer to review and validate procurement documents and process.

On August 29, 2018, we reviewed a proposed amendment to the RE document to allow for an optional preliminary security interoperability assessment step that Suppliers would be encouraged to use. This assessment step, prior to RFP closing, would relate to 2/5 eyes special security requirements (5 eyes refers to Australia, Canada, New Zealand, United Kingdom and United States and 2 eyes refers to Canada and United States in this context.) No fairness-related deficiencies were identified.

On August 30, 2018, we monitored a teleconference between project officials and a Supplier to better understand and discuss the feedback that the Supplier had provided after the recent Supplier engagement meeting. No fairness-related deficiencies were identified.

On September 18, 2018, we reviewed a common agenda for teleconference calls to the Suppliers scheduled to begin the same day. The purpose of the calls was to provide an update on action items from previous meetings and an update on planning for site visits to Royal Canadian Air Force bases scheduled for later in the year. During the period September 18 to 21, 2018, we monitored the conference calls. No fairness-related deficiencies were identified.

Review of draft request for proposal documents

The RFP as it developed grew into approximately 50 documents. Beginning on September 23, 2018, and until the release of the RFP, we progressively reviewed draft versions of RFP documents which were provided to us in advance of being released to the Suppliers. There were multiple draft versions of each document. On November 6, 2018 we were informed that FFCP would be using the AWARD software tool to support the development and review of the draft RFP. Draft RFP parts would be posted in the AWARD data room where they would be available for review by the Suppliers. We were provided on-line access to the data room. We accessed the data room periodically and reviewed new or revised RFP parts as well as any queries received from the Suppliers, along with proposed and final responses. We provided fairness-related comments and appropriate action was taken by project officials.

On October 2, 2018, we were informed that the Suppliers were being invited to site visits at both Canadian main operation bases to view the existing infrastructure for fighter aircraft so that they could assess changes that would be required or appropriate for the new aircraft. We reviewed the plans for the site visits and on October 10, 2018, provided fairness related comments on the plans. On November 30 and December 4, 2018, we monitored one-on-one meetings held with each of the Suppliers to respond to questions concerning the infrastructure that had been viewed during the visits. No fairness-related deficiencies were identified.

On November 8, 2018, we were advised that 1 of the 5 qualified Suppliers was withdrawing from the competition.

On January 8, 2019, we reviewed a presentation deck outlining a methodology for security interoperability. Security interoperability assessment (mentioned previously) is a highly specialized, complex and lengthy process. The proposed methodology provided for an early submission by Suppliers of an interoperability proposal to allow for an initial assessment by Canada and feedback to Suppliers. This would allow for each Supplier’s formal submission at bid closing to be informed by the initial assessment. We reviewed the proposed methodology and identified no fairness-related deficiencies.

On January 23, 2019, we reviewed a deck providing clarification of bilingual requirements for technical publications, training, etc. No fairness-related deficiencies were identified.

Supplier week 3

From February 4 to 15, 2019, we observed 4 Supplier week 3 one-on-one meetings, 1 with each of the Suppliers. Each meeting was scheduled for 2 and a half days and followed the same agenda with Canada providing identical presentations to each Supplier on each agenda item. The project team noted comments provided by the Suppliers and provided background information where appropriate. The same amount of time was allotted to each meeting. After each meeting the project team held a post-meeting to discuss what was heard and to provide tasking. Fairness-related comments were provided and appropriate action was taken by project officials.

On February 26, 2019, we reviewed a key take away report that covered the series of meetings held earlier in the month with the Suppliers, and had been prepared to inform senior government officials of the key findings. The findings were consistent with what we had observed at the meetings. No fairness-related deficiencies were identified.

On April 17, 2019, we met with the project team and were briefed on changes to the procurement process that responded to feedback provided by the Suppliers as well as general improvements. On April 18, 2019, we reviewed the presentation deck from the previous day. No fairness-related deficiencies were identified concerning the changes to the procurement process.

Ongoing teleconferences

After Supplier week 3, engagements with the Suppliers were conducted primarily by sets of teleconferences, with each Supplier given the opportunity to have a teleconference one-on-one meeting during each set of teleconferences. Not all Suppliers accepted the opportunity to have a teleconference for each set. The frequency of teleconferences generally increased as the expected date for the release of the final version of the draft RFP approached. Procedures for the teleconferences were similar to face to face meetings and each Supplier was given a full opportunity to discuss any matters that it wanted to cover.

During the period April 23 to 25, 2019, we monitored 4 one-on-one teleconferences. The purpose of the teleconferences was to brief each Supplier on the upcoming changes to the procurement process, to ensure an understanding of the feedback provided by the respective Supplier on the draft RFP and to provide an opportunity for each Supplier to raise any concerns. There was a standard agenda, the project team used the same script for each. After each teleconference, the project team discussed the concerns expressed by the respective Supplier and action was assigned when appropriate. No fairness-related deficiencies were identified.

On May 9, 2019 we observed a WebEx teleconference with all Suppliers including a pre-meeting and a post-meeting with the project team. The purpose of the teleconference was to brief the Suppliers on the upcoming schedule for the draft RFP engagement period and particularly to brief on a significant change to the RFP VP submission requirements. While all Suppliers were represented, some individuals had difficulties joining the teleconference so the presentation was repeated in full after a few minutes. There were no questions. We monitored one-on-one teleconferences with each Supplier the following day. No fairness deficiencies were identified.

On May 10, 2019 we observed a meeting of the project team with representatives of the Aerospace Industries Association of Canada (AIAC) and the Canadian Association of Defence and Security Industries (CADSI) at which the RFP VP change was briefed. On the same day we observed a teleconference with each Supplier during which questions and comments on the information provided during the WebEx teleconference the previous day were received and clarity provided as required. No fairness deficiencies were identified.

During the period May 23 to 24, 2019, we observed teleconferences with each of the Suppliers. The purpose was to provide background information to ensure an understanding of the FFCP key requirements and to obtain any feedback and comments from the Suppliers. An agenda was followed and each Supplier was allotted the same time period. No fairness deficiencies were identified.

All Suppliers were offered the opportunity to meet during the time period of the Canadian security (CANSEC) Conference 2019 to discuss topics of their choice. Two Suppliers requested meetings. We observed a meeting on May 28, 2019, with 1 Supplier. Another Supplier requested a teleconference one-on-one to clarify questions concerning the revised VP submission requirements. We monitored the latter teleconference on June 6, 2019. No fairness-related deficiencies were identified.

During the period June 7 to 11, 2019, we observed 4 one-on-one meetings with Suppliers. The purpose of the meetings was to give Suppliers the opportunity to make clear and to discuss verbally their concerns with the procurement process particularly the changes that had been provided in recent weeks. The concerns expressed differed from Supplier to Supplier. The project team asked questions to ensure understanding, provided background where applicable and, in the case of some issues, agreed to carry out a review. No fairness-related deficiencies were identified.

Evaluation model

On June 12, 2019, we observed a presentation provided by a consulting team on their detailed assurance review of the FFCP evaluation model. The presentation covered a comprehensive review and a thorough analysis which included, but was not limited to:

There were no findings that the evaluation model was not sound. No fairness-related deficiencies were identified.

On June 14, 2019 we observed a WebEx presentation by the project team provided concurrently to the 4 Suppliers. The purpose of the presentation was to provide background on:

After the presentation, we monitored one-on-one teleconferences with each Supplier to receive and respond to any questions. No fairness-related deficiencies were identified.

On June 17, 2019, we observed a teleconference with a Supplier to provide feedback on a specific matter that the Supplier had raised during a previous teleconference. The feedback was provided to the Supplier and no fairness-related deficiencies were identified.

During the period June 26 to 27, 2019, we observed 3 one-on-one teleconferences. The purpose was to provide an opportunity for Suppliers to express concern with any matter, particularly any that would prevent them from submitting compliant proposals. While all Suppliers were given the opportunity to have a teleconference, 1 Supplier stated that it did not need one. Several concerns were raised and the project team provided appropriate responses. No fairness-related deficiencies were identified.

Beginning on June 25, 2019, we reviewed version 2 of the draft RFP and beginning on July 2, 2019, we reviewed feedback submitted on the version 2 draft by each of the Suppliers. Our fairness review process continued until the release of the formal RFP on July 23, 2019. No fairness-related deficiencies were identified.

Request for proposal phase

Notice of proposed procurement

On July 23, 2019, we reviewed the notice of proposed procurement (NPP) posted on buyandsell.gc.ca that advised of the procurement and provided a general outline of the requirement. It also stated that the competition was open to the Suppliers that had qualified at the qualification stage through the SLI, and any others that could demonstrate that they could meet the qualification requirements. No fairness-related deficiencies were identified.

AWARD

As described, AWARD software was used by the project team to support the development and review of the RFP. Its use continued during the RFP phase. It was used as the repository for the RFP and as the means of distribution of the RFP, including amendments, to the Suppliers. In addition, it was used as the means of receiving questions and clarification requests from the Suppliers, coordinating responses and other actions within the project team, and communicating to the Suppliers. We had access to AWARD that provided us with an effective means of monitoring communications to and from the Suppliers. We received an automatic notification whenever there was AWARD activity and we reviewed the activity. There was almost daily AWARD activity with some days having more than a dozen activities.

Request for proposal and amendments review

As noted we reviewed the RFP prior to it being released on July 23, 2019. During the RFP phase we periodically reviewed RFP documents, not only to review amendments but also to maintain familiarity and to understand the links between various sections and parts. We provided fairness-related comments and appropriate action was taken by project officials.

We reviewed each amendment in draft form prior to its release and the final version after release. Each amendment consisted of the parts of the RFP that had been changed with a brief description of the change. AWARD would include the new version of the RFP parts that had been amended. In most cases, we followed the issue on AWARD as it was identified through to the development of a response or resolution of the identified issue. Often we were also briefed by the project team on the action that was being contemplated. Fairness-related comments were provided and appropriate action was taken.

Presentation to industry

On August 15, 2019, we observed a web-based briefing by project officials to interested Canadian companies concerning the FFCP that had been organized by the CADSI and the AIAC. A notice had also been posted on buyandsell.gc.ca concerning the briefing. The objective was to update Canadian industry on the FFCP project to better prepare them to participate.

The briefing covered:

The presentation was comprehensive and was viewed by more than 175 companies. It was provided in both English and French. No fairness-related deficiencies were identified.

Public Work and Government Services Canada updates

Generally on a monthly basis throughout the RFP period we were briefed by PWGSC project officials on current and anticipated procurement issues that were being analyzed with a view to making improvements and resolving problems. We were informed of the steps that officials intended to take and provided fairness-related comments where appropriate.

The first such update was provided on September 5, 2019, when we met with the PWGSC contracting team during which several matters were discussed, including a request from 1 Supplier, and the proposed evaluation governance and plans. The meeting followed the format that we continued to use for our meetings or telephone communications with project officials. For each item on the agenda, officials outlined the issue and described the steps they intended to take.

Fairness-related comments were provided as applicable during these briefings and appropriate action was taken by project officials.

Withdrawal of 1 qualified Supplier

During the period September 10 to 13, 2019, we reviewed several drafts of a response to 1 Supplier acknowledging their letter to Canada stating that they were withdrawing from the competition. On September 16, 2019, we were informed that the Supplier had been removed from AWARD. No fairness-related deficiencies were identified

Notification of site visits

On September 19, 2019, we reviewed a draft notification to the Suppliers to be posted on AWARD informing them that site visits were planned to the 2 main operating bases, Bagotville and Cold Lake, for late November and early December 2019.

The announcement outlined the facilities that would be viewed and the actions required of the Suppliers to participate.

The purpose of the visits was to provide another opportunity for the Suppliers to view the existing facilities and to view the sites under winter conditions.

Subsequently, project authorities outlined the plans for the visits to us, including the proposed number of attendees and the proposed number of, and composition of, tour groups.

The dates for the visits were subsequently delayed by 1 week.

Fairness-related comments were provided and appropriate action was taken by project authorities.

Preliminary security offer submission and assessment

In view of the special nature of FFCP security and interoperability requirements, the RFP required Suppliers to each submit a preliminary security offer prior to proposal closing date. The security offer would be assessed by Canada and feedback provided on its acceptability. On September 27, 2019, we received a draft of the FFCP first security acceptability assessment plan (FSAAP) for review and an invitation for us to attend a training briefing session for the participants in the security acceptability assessment activity. We provided fairness-related comments and appropriate action was taken by project officials. On October 7, 2019, we reviewed a revised version of the draft FSAAP and provided fairness-related comments. Appropriate action was taken by project officials.

Also, on September 27, 2019, we reviewed broad guidelines for assessment and evaluation consensus meetings provided by project officials. We followed up by drafting and forwarding a description of what, from a fairness perspective, we looked for in a consensus process.

On October 1, 2019, we observed the first security assessment training session for the security subject matter experts (SSMEs) that would be working with the security assessment team (SAT) in reviewing and assessing the preliminary security offers. The security assessment involved an assessment of the acceptability of each security offer. Each Supplier was to be provided with an acceptable/non-acceptable assessment of its offer and, if applicable, a description of the reasons for a non-acceptable result.

The session included an overview of the:

We observed similar sessions on October 3 and 10, 2019, for those SSMEs that were not available for the first session. No fairness-related deficiencies were identified.

On October 7, 2019, we discussed with the PWGSC contracting team the need for continuity of members of the SAT. Fairness-related comments were provided and appropriate action was taken by project officials.

On October 17, 2019, we observed the training session for the members of the SAT. The session followed the format of the earlier sessions for the SSMEs. On October 18, 2019, we reviewed the initial tasking directives that the SAT had authorized to direct their respective SSMEs members to review the applicable portions of each preliminary security offer. No fairness-related deficiencies were identified.

On October 24, 2019, we observed a briefing of SSMEs and SAT at the secure site at which the reviews of the security offers would be conducted. In view of the security classification of the security offers, special procedures had been put in place to ensure proper security of the offers and notes on the offers prepared by SSMEs and SAT members throughout the assessment period. Security arrangements were briefed and the information technology to be used was explained and demonstrated. No fairness-related deficiencies were identified.

Throughout the security offer assessment period until the assessment was completed in late January 2020, we received a daily situation report describing the progress being made in reviewing the security offers.

On October 25, 2019, we reviewed revised versions of the SAT tasking directives to the SSMEs teams. No fairness-related deficiencies were identified.

There were several key steps in the security assessment process. For each of the security and interoperability aspects being assessed, a team of SSMEs reviewed each security offer and prepared a report identifying strengths and deficiencies relative to the RFP specified submission requirement without expressing any acceptable/not acceptable assessment. Each report was provided to members of the SAT and outlined in a briefing to the SAT members, again without any assessment. Each member of the SAT reviewed the offers as well as the reports by the SSMEs teams and made their individual acceptable/not acceptable assessment. The members of the SAT discussed their findings and agreed on a consensus assessment. The process was repeated for each security offer.

We observed the security offer assessment activity until it was completed in late January 2020. For each security offer, we reviewed the SSMEs reports to the SAT, observed the briefing of the SAT by the SSMEs and observed the assessment consensus meetings of the SAT at which the characterization acceptable/not acceptable of each offer was agreed. Best practices similar to those for an evaluation process were followed. Fairness-related comments were provided and appropriate action was taken by project officials.

Following each SAT characterization meeting, we reviewed the written feedback prepared for each Supplier advising of Canada’s characterization of their respective security offer and the reasons for the characterization. In each case the feedback was consistent with the consensus of the SAT and the explanations were tailored to the individual security offers and provided no Supplier with an improper advantage. No fairness-related deficiencies were identified.

Site visits

During the period December 5 to 10, 2019, we observed a 2 day site visit to Bagotville and a 2 day site visit to Cold Lake. Each of the 3 Suppliers was represented by a team. At each base, operations, maintenance, support, and training facilities were viewed in turn with explanations by base personnel. The group was kept together and moved from location to location on a bus. Where possible, questions were answered and the questions and answers were subsequently posted on AWARD. While photographs by the teams were not allowed, a base photographer accompanied the tour and took photographs on request by the teams. Subsequently the photographs were vetted and provided on AWARD to all teams. Fairness-related comments were provided and appropriate action was taken by project officials.

Delivery of proposal submissions

Prior to RFP closing, Suppliers were required to provide a notice of expected proposal. Notices were to include the number of packages expected to be submitted and the identification of the security levels of each package. This information was important to facilitate the receipt and proper handling of the submission packages. In May 2020, we reviewed the notices received. During the subsequent period up to RFP closing, we reviewed follow-on communications with each Supplier to ensure proper safeguarding of the contents of each submission. No fairness-related deficiencies were identified.

International travel restrictions resulting from the COVID-19 pandemic created the need to allow for additional means for proposal submission delivery beyond what had been specified in the RFP and which took into consideration the security classification of the packages. PWGSC project officials recognized the need and worked with the Suppliers within the communications rules of the RFP to identify additional delivery methods. We monitored these activities on AWARD and were briefed by officials as various delivery means were explored. The RFP was amended accordingly providing the same optional means for all Suppliers. Fairness-related comments were provided and appropriate action was taken by project officials.

The RFP closed on July 31, 2020.

Proposal evaluation and selection phase

Evaluation overview

Note: The term evaluation is used in this report to include related activities including:

On June 12, 2020, we monitored a presentation provided by a third-party consulting firm on its review of the proposed FFCP evaluation process. The review led to some refinements to the evaluation process. No fairness-related deficiencies were identified.

Due to the complexity, size and scope of the FFCP, the evaluation of responses required a very large number of resources with specialized knowledge and experience to execute the evaluation in a diligent and fair manner.

FFCP evaluation and selection consisted of 6 activities, each of which we reviewed, observed or monitored:

Proposal Evaluation Management Plan and Governance Structure

A proposal evaluation governance structure (PEGS) document, approved at the Assistant Deputy Minister level on 25 October, 2019, provided FFCP with a high level evaluation governance structure. The composition, roles and responsibilities of those involved in the evaluation and the management of the solicitation process were described in the PEGS. It was the umbrella governance document for the evaluation process. Two amendments to the PEGS were subsequently made.

The PEMP was the directive document for the execution of the evaluation. Whenever an evaluation question arose the PEMP was used to provide direction.

Beginning on February 28, 2020, we reviewed draft versions of the FFCP PEMP. The PEMP was a comprehensive and authoritative document detailing how the evaluation as described in the RFP would be executed and the results documented. It covered:

The PEMP document was finalized and approved at the Senior Director level on August 11, 2020. Three amendments to the PEMP were subsequently made. Fairness-related comments were provided and appropriate action was taken by project officials.

Evaluation teams

There were 13 evaluation teams, each consisting of 3 or more members.

The PEMP specified the process for appointing evaluators. The proposed composition of each evaluation team was considered by an evaluation review board (ERB) and approved by an evaluation management committee. Evaluators were selected based on their expertise in the evaluation area concerned.

The security acceptability assessment was performed by 2 executives from the communications security establishment (CSE) and 2 executives from the DND supported by 4 teams of subject matter experts. The reports from the subject matter experts provided comments on the bidders’ proposals relative to the requirements of the RFP and other aspects allowed by the RFP, but did not make any recommendations concerning the acceptability or compliance of a bidder’s response.

No fairness-related deficiencies were identified concerning the selection of members for, or composition of, the evaluation teams.

Evaluation training and orientation

On August 7, 2020, we reviewed a draft welcome package that was to be sent to all evaluators and assessors informing them of pre-requisite training each was required to complete in advance, as well as forms that each was required to read and acknowledge by signing.

Training included the subjects of:

The forms included:

On August 11, 2020, we reviewed the final version of the welcome package as sent to the evaluators. No fairness-related deficiencies were identified.

Training and orientation of evaluators took place mainly during the latter part of August 2020, and the first half of September 2020. All participants in the evaluation process were given an extensive initial briefing on the project by both PWGSC and DND including an overview of the:

Subsequently, each evaluation team was provided with a follow-on detailed presentation on their specific evaluation area that focused on the evaluation requirement, detailed processes that would be followed, available tools and documentation requirements.

Evaluation managers and team leaders were provided with a more comprehensive presentation on organization and process details and on evaluation management and support. The presentation highlighted specific areas of the PEMP in order to facilitate evaluation activities in accordance with the PEMP, RFP and best practices.

Training also focused on the fact that only electronic copies of the proposals would be used in the evaluation and that many evaluators would have to assess bidders’ proposals electronically from their residences. Many consensus meetings would be conducted via video link, and individual and consensus results would be documented electronically.

Evaluators were briefed on the:

For those evaluators who, for security reasons, had to have physical access to secure facilities, the procedures to do so were briefed.

We monitored all training and orientation sessions. Each session tightly followed a PowerPoint presentation that had been approved in advance by project officials and which we had reviewed in advance. The presentations were consistent with the PEMP. Questions were asked and answered. Fairness-related comments were provided throughout the training and orientation sessions and appropriate action was taken by project officials.

Evaluation activity

The RFP requested that a proposal consist of several elements covering different submission requirements each of which was to be provided in 1 or more separate volumes. In accordance with the RFP, the offers or volumes were evaluated or assessed and the results used in the selection process. As specified in the RFP, other volumes were assessed but the results were not used in the selection process, however feedback on the results were provided to the respective bidder later in the process, prior to contract award, to facilitate the finalization process.

Eight areas of the submitted proposals were evaluated as part of the selection process:

The RFP provided a detailed description of the evaluation process including instructions to bidders for all evaluation areas. No fairness-related deficiencies were identified.

The RFP specified for each of the 8 areas detailed instructions as to what a bidder was asked to provide in its response and how the response would be evaluated.

There were 4 different evaluation approaches specified:

Some evaluation areas used only 1 approach while others used up to 3.

Mandatory criteria and rated criteria evaluations were similar to those commonly used in procurement evaluations. The procedures for pairwise comparison and mandatory requirements subject to assessment evaluations were fully described in the RFP. No fairness-related deficiencies were identified concerning the evaluation approaches used.

In accordance with the RFP, the following areas of the proposals were not evaluated:

These parts of the proposals were reviewed by teams for the purpose of identifying aspects that required improvements. Feedback on these aspects would be provided to the respective bidder to facilitate contract finalization. While not used in the selection process, we monitored the consensus meetings for each. No fairness-related deficiencies were identified.

The evaluation of proposals was conducted in 8 steps. The first 2 steps were completeness and conformity checks which were conducted before the other steps. The other steps were conducted in parallel as allowed by the RFP. The completeness check reviewed each bidder’s proposal to ensure each included all required elements. A bidder would be advised of any missing element(s) and given a period of time to provide the missing element(s). The conformity check reviewed each bidder’s proposal to ensure that certain instructions in the RFP were complied with in its proposal. A bidder would be advised of any instructions that it did not comply with, and was given a period of time to submit correct information.

We did not monitor the completeness and conformity checks. We did, however, review the feedback provided to the Bidders identifying discrepancies and the responses received from the Bidders. The form and level of detail provided in the feedback were consistent amongst Bidders. No fairness-related deficiencies were identified concerning the completeness and conformity checks.

We monitored or observed all evaluation consensus meetings which took place during the remaining 6 evaluation steps except for 1, as described below. Consensus meetings took place over a period of more than a year starting in September, 2020 and ending in September, 2021. During this period, we monitored or observed more than 200 evaluation consensus meetings.

Due to the pandemic lock-down there were 2 types of evaluations based on the classification level of the responses. Evaluations involving information classified as protected B were conducted remotely with each evaluator working from their residence and connected to the government virtual public network (VPN) allowing access to GCdocs, the repository of enterprise content including PWGSC documents. The RFP, PEMP and each bidder’s proposal were available from the government VPN. Evaluators recorded their findings on GCdocs and consensus meetings were held using the Microsoft Teams application.

The other type of evaluation involved information classified higher than protected B. These evaluations, including the consensus meetings, were held at appropriate DND facilities or at the CSE. Bidders’ proposals were available on an internal DND network or a CSE classified network as were the RFP and PEMP and evaluators recorded their findings on the same classified network. Consensus meetings were held face to face except for those evaluators that were not located in Ottawa. The latter evaluators participated using classified DND networks allowing all members of an evaluation team to view and hear each other and view a common screen.

All consensus meetings followed best practices. On a rotational basis each evaluator was given a full opportunity to explain their finding. A general discussion followed after all evaluators had explained their finding. The discussion often was brief if there was agreement on the result. When there was not agreement every effort was made to reach unanimous consensus agreements. In a few cases a majority consensus agreement was reached. We provided fairness-related comments when applicable, and appropriate action was taken by the evaluation team concerned.

As noted, due to the classified nature of the information, there was 1 evaluation consensus meeting that we did not monitor or observe. It was observed by the PWGSC procurement evaluation manager (PEM) and DND evaluation manager (EM). Although we were not able to monitor the meeting, we were made aware of the criterion being evaluated and in broad terms the information required in a bidder’s response. After the consensus meeting we were given a briefing of the consensus discussion and the result. All of the evaluators as well as the PEM and EM confirmed the result. While not knowing the specific issue in question, but having a general knowledge of the subject matter and knowing the evaluation team well, having spent many hours in consensus meetings with the team, we are confident the result was arrived at through a fair consensus process.

The PBCP was executed in accordance with the provisions of the RFP and we reviewed each step in the process. Fairness-related comments were provided and appropriate action was taken by project officials.

The consensus evaluation of bids was completed on September 21, 2021, and on November 25, 2021, we were advised that the bid evaluation and selection process was completed on November 19, 2021. Two of the 3 Suppliers were found to be responsive to the requirements of the RFP.

On December 17, 2021, we reviewed a presentation to be provided to the ERB concerning the dialogue and finalization phases which contained the recommendation that Canada proceed directly to phase 6, finalization, of the solicitation process, along with supporting rationale. No fairness-related deficiencies were identified.

On December 19, 2021, and January 11, 2022, we reviewed a draft presentation deck concerning the dialogue and finalization phases prepared for senior management which contained the recommendation that Canada proceed directly to phase 6, finalization. On January 17 and 18, 2022, we reviewed the final presentation deck. No fairness-related deficiencies were identified.

Finalization phase

On March 11, 2022, we reviewed the draft finalization notice letters to be sent to the selected Supplier and the non-selected Supplier.

On March 24, 2022, we reviewed the proposed correspondence to the 2 Suppliers. No fairness-related deficiencies were identified.

On March 28, 2022, Canada announced that it would now enter into the finalization phase.

On April 6, 2022, we monitored the finalization phase kick-off meeting with the selected Supplier. No fairness-related deficiencies were identified.

During the period April 7 to November 4, 2022, we reviewed documents concerning the finalization phase and proposed communications and meetings with the selected Supplier. Fairness-related comments were provided and appropriate action was taken by project officials.

On October 6 and 12, 2022, we monitored EIA consensus meetings. No fairness-related deficiencies were identified.

On November 4, 2022, we reviewed the draft PWGSC finalization report. No fairness-related deficiencies were identified. We were advised that if there were any substantive changes to the finalization report that we would be provided with an opportunity to review it.

Reference documents

Documents related to solicitations W847A-180210/A and W847A-180210/B can be obtained through the FFCP project office.

Addendum to the final report

February 15, 2023

Addendum to the FM final report dated February 13, 2023, for the FFCP.

On December 9 and 10, 2021, we reviewed the draft debriefing deck to be provided to an unsuccessful bidder.

On December 14, 2021, we monitored the debriefing of the unsuccessful bidder. On December 17, 20, and 22, 2021, we reviewed subsequent communications with the unsuccessful bidder in response to questions received from it. No fairness-related deficiencies were identified.

On May 31, 2022, we reviewed the draft debriefing presentation deck to be provided to an unsuccessful bidder. On June 3, 2022, we monitored the debriefing of the unsuccessful bidder. On June 19, 2022, we reviewed a subsequent communication with the unsuccessful bidder in response to questions received from it. No fairness-related deficiencies were identified.

The contractual arrangement was made with the selected bidder which included a memorandum of understanding with the United States government as well as foreign military sales cases, and 2 economic benefit arrangements. A debriefing was not requested by the selected bidder.

Attestation of assurance

The FM hereby provides the following unqualified assurance statement concerning the FFCP.

It is our professional opinion that the procurement process we observed or monitored was carried out in a fair, open, and transparent manner.

Original signed by:

Rick Moffat
Partner Canada
HKA Global (Canada), Inc.
Fairness Monitoring Contractor’s Representative

Andrea Robinson
Fairness Monitoring Team Leader
Fairness Monitor Specialist

Bruce Maynard
Professional Engineer
Fairness Monitor Specialist

 

Geneviève Arcand
Fairness Monitor Specialist

Jean Montplaisir
Fairness Monitor Specialist

Page details

Date modified: