Audit of inspection activities at Health Canada

Final Report: March 2020

Table of contents

• List of acronyms
• Executive summary
• A - Introduction
  • Background
• B - Findings, recommendations and management responses
  • Compliance and enforcement modernization plan
  • Governance structure and oversight
  • Monitoring and reporting
  • Funding and resources
  • Key elements to support inspection activities
  • Sex- and gender-based analysis plus
• C - Conclusion
• Appendix A - Scorecard
• Appendix B - Responsibility for C&E functions
• Appendix C - About the audit
  • Audit objective
  • Audit scope
  • Audit approach
  • Statement of conformance

List of acronyms

BEC

Branch Executive Committee

C&E

Compliance and Enforcement

DG Forum

Director General Forum Committee

GMP

Good Manufacturing Practices

HC

Health Canada

IM/IT

Information Management and Information Technology

NLU

National Learning Unit

OECD

Organization for Economic Cooperation and Development

QAM

Quality Assurance Management

ROEB

Regulatory Operations and Enforcement Branch

SGBA+

Sex- and Gender-Based Analysis Plus

Executive summary

What we examined

As a regulator, Health Canada (HC) has a responsibility to protect Canadians from unsafe health and consumer products. To fulfill this responsibility, HC conducts inspection activities, as part of its compliance and enforcement (C&E) function, to monitor compliance with applicable Acts and Regulations and identify potential issues that could negatively affect the health and safety of Canadians. However, with a rapidly changing global environment and advancements in technology, HC must ensure a robust framework is in place to modernize its C&E practices and keep pace with these changes.

Over the last number of years, HC has been responding to these changes by updating the regulatory framework for health, food, and consumer products; expanding cooperation with international regulatory agencies; and aligning inspection activities to the updated regulations. It is also addressing gaps in order to stabilize some inspection programs, implementing new inspection programs in response to Government of Canada (GoC) priorities, and setting a high-level path forward to modernize its C&E oversight.

In 2016, Health Canada introduced a dedicated compliance and enforcement branch, which was a critical step to strengthening oversight of the Department's acts and regulations, and improving consistency of program delivery. Since its creation in 2016, the Regulatory Operations and Enforcement Branch (ROEB) has been responsible for the Department's inspection activities, while sharing certain C&E accountabilities with other branches (See appendix B).

This audit focused on the processes and tools in place to support the implementation of the key elements of C&E modernization. Specifically, we examined:

• plans and management practices, at the branch and select program levels (Medical Devices, Biological Products, and Pesticides), that support implementation measures for, and monitoring of progress on the following four key elements of C&E modernization:
  • increased use of risk assessment in inspection planning;
  • more efficient and effective use of technology;
  • greater accessibility to current operating procedures; and
  • establishment of a professional designation and training program for inspectors;
• the identification of funding and resources required to modernize inspection programs; and
• the application of Sex- and Gender-Based Analysis Plus (SGBA+) to ensure that sex, gender, and diversity considerations are integrated into program offerings.

The audit did not include a detailed testing of the work performed by inspectors.

Why it is important

Keeping pace with the rapidly changing global and technological environment is key to protecting the health and safety of Canadians. Otherwise, there is a risk to the health and safety of Canadians and a risk of losing the confidence of Canadians in the safety of health and consumer products and in HC as a regulator. Appropriate implementation of the key elements of C&E modernization will help mitigate these risks.

What we found

The C&E function at HC included 11 different regulated product categories^{Footnote 1} for which ROEB has partial or full C&E responsibility, depending on the product category. Over the past number of years prior to the creation of ROEB, the Department undertook various exercises that focused on C&E modernization, including an Inspection Function Review (2014), and a C&E Modernization Roadmap (2015) that included a C&E Excellence Framework (2016).

These initiatives identified a number of gaps in HC's C&E practices, including in the following areas:

• clear roles, responsibilities, and accountabilities;
• risk-based planning of inspection activities;
• effective use of modern technology;
• up-to-date operating procedures; and
• a professional competency-based training program linked to designation.

Addressing these gaps is a challenge due to the number of modernization elements, the application of these to numerous inspection programs, and modernizing oversight while delivering operationally, which over the last two years has included designing and implementing new inspection regimes for Cannabis and Vaping product lines.

The Department has shown progress in modernizing compliance and enforcement oversight and in addressing the identified gaps, including the following:

• creating a dedicated compliance and enforcement branch: the Regulatory Operations and Regions Branch in 2016 (renamed ROEB in 2019);
• transitioning from a regional to a national product line program delivery model;
• clarifying roles and responsibilities for C&E oversight among the different Health Canada branches;
• analyzing program gaps, priorities, and resource requirements to stabilize, modernize, and transform operations through a Comprehensive Review exercise;
• updating the Cost Recovery Fees for Drugs and Medical Devices;
• making investments to stabilize certain C&E programs, and making modest investments to allow some modernization to begin in horizontal infrastructure like technology, data and analytics, inspector training, and OHS; and
• setting a high-level vision and plan for C&E modernization and transformation with the release of ROEB's Strategic Plan in April 2019.

While it was evident that progress was being made, we found that management could not demonstrate how efforts had been prioritized to address the gaps between their current state and the future desired state of C&E modernization. Other than where investments had been made, management did not clearly indicate how key elements of C&E modernization would be implemented to close gaps in all program areas.

We found that ROEB was in the process of launching a strategic plan that outlines a vision and high-level plan for how it will continue to modernize the C&E approach. Progress made included:

• establishing a number of teams that are well positioned to support C&E modernization, such as:
  • the Technology and Business Innovation Division, to lead the implementation of information management and information technology (IM/IT) strategies;
  • the Quality Assurance Management team, to assist with the updating of standard operating procedures;
  • the National Learning Unit, to assume responsibility for the development and delivery of core curriculum training courses for all inspectors;
  • the Project Management Office to promote strong project management practices, and consistency in monitoring and reporting on investments to date for modernization.
• incorporating, in some programs, elements of risk assessment in inspection planning;
• establishing a number of projects and project charters to address the IM/IT modernization needs of some program areas; and
• revamping elements of the core training curriculum while prioritizing support to cannabis inspector training and designation.

We found that there was consideration of the four key elements of C&E modernization (i.e., risk in inspection planning, IM/IT, operating procedures, training). However, we noted that there were deficiencies that required enhanced focus and oversight to ensure the needs of all inspection programs were addressed. These included:

• IM/IT strategies that addressed the needs of only some of the 11 regulated product categories (responsibility for IM/IT strategies to support C&E is shared between ROEB and partner branches for some of the inspection programs - see appendix B);
• limited progress in addressing backlogs of outdated standard operating procedures and guidance documents (responsibility for C&E standard operating procedures and guidance documents is shared between ROEB and partner branches for some of the inspection programs - see appendix B);
• a lack of policy on designation and mandatory training to advance the national training approach, and the provision of specialized training required by individual inspection programs; and
• no Sex- and Gender-Based Analysis Plus (SGBA+) was conducted, nor training provided.

Overall, we recognize the significant work undertaken to establish a pathway to success within ROEB; however, we found instances where improvements could have been made to further strengthen the implementation of the four key elements of C&E modernization. The areas for improvement noted in this audit report, along with the associated recommendations, will collectively strengthen the management practices, at the branch and program levels, to support appropriate implementation of key elements of C&E modernization.

A - Introduction

Background

1. One of Health Canada's (HC) key responsibilities as a regulator is to protect Canadians from unsafe health and consumer products. To fulfill this responsibility, HC conducts inspection activities to monitor and enforce compliance with six different federal Acts (the Food and Drugs Act, the Canada Consumer Product Safety Act, the Cannabis Act, the Controlled Drugs and Substances Act, the Tobacco and Vaping Product Act and the Pest Control Products Act) and eleven regulatory frameworks unique to each product line. Ultimately, this oversight helps to ensure that health and safety are paramount in the Canadian marketplace.

2. Inspection activities are an important component of the compliance and enforcement (C&E) function that contributes to the protection of the health and safety of Canadians. These activities can vary depending on the type of product and their corresponding regulatory framework (e.g., Human and Veterinary Drugs, Medical Devices, Pest Control Products, Biological Products), and can cover an array of activities, such as manufacturing, importation and sales, and labelling. HC conducts inspection activities on a planned basis (cyclical or as warranted), or on a responsive basis (e.g., follow-up on a complaint or incident or support for a product recall). HC has also has taken a risk-based approach to Drug Good Manufacturing Practices (GMP) inspections. These activities allow the Department to verify if regulatory requirements are being met, and to identify potential issues that could negatively affect the health and safety of Canadians.

3. One of the primary challenges that the Department faces in protecting the health of Canadians is the changing nature of the global supply chain, resulting from the rapid pace of innovation and technological advancement. To keep pace, HC must ensure a robust framework is in place, including updates to existing regulations and enhanced inspection practices that are in alignment with the new regulations. If it does not, the health and safety of Canadians may be at risk.

4. Over the past number of years, the Department has made progress in updating the regulatory system for health products and consumer products, and in enhancing cooperation with regulatory agencies in other countries.

5. It has also been working to strengthen accountability and improve the effectiveness and efficiency of C&E program delivery. Key initiatives included:

2014 -

conducting an Inspection Function Review to examine governance, tools, and processes in place, and to make recommendations for enhancing accountability and strengthening the inspection function. The review identified a lack of a departmental vision and mission for C&E, inconsistent roles and responsibilities, outdated systems, ambiguous guidance documents, and inconsistencies in the conduct of inspections. It pointed to the need to modernize the inspection function;

2015 -

developing the Compliance and Enforcement Modernization Roadmap (CEMR) as an action plan to address issues identified through the Inspection Function Review. Key actions included establishing mandatory obligations for regulated parties to disclose global risk information; increasing integration with international regulatory authorities; updating policies, guidance, service standards, and standard operating procedures; investing in information technology to build capacity for surveillance and analytics; and enhancing training programs;

2016 -

establishing the Compliance and Enforcement Excellence Framework, as part of the CEMR, to set out expectations and priorities for compliance programs, in terms of policy, risk, governance, planning, quality and performance, Information Management and Information Technology (IM/IT), and technical expertise;

2016 -

launching a five-year Drug GMP Inspection Program transformation initiative includingimplementing risk-based inspection planning for the Drug GMP Inspection Program

2017 -

clarifying roles and responsibilities for compliance and enforcement oversight among the different Health Canada branches;

2017 -

undertaking a Comprehensive Review, including identification of program funding requirements to address gaps in C&E inspection capacity, system and tools, training, and intelligence gathering;

2017-18 -

developing and launching a new inspection regime to support the legalization and strict regulation of Cannabis;

2017-18 -

developing and launching a new Inspection regime for Vaping Products, and transform-ing inspector roles by designing an innovative program model that uses cross-designated inspectors across multiple acts and regulations;

2018 -

beginning the implementation of commitments made through Comprehensive Review, including the stabilization of some inspection programs and modest modernization activities in horizontal infrastructure, like technology, data and analytics, inspector training, and Occupational Health and Safety;

2018 -

developing a ROEB Strategic Plan for 2019-22 to set the high-level vision and direction for C&E modernization.

6. Through these initiatives, the Department has identified some key elements that need to be in place to better support regulatory and organizational changes. These include the needs for clear roles, responsibilities, and accountability; risk-based planning of inspection activities; effective use of modern technology; up-to-date operating procedures; and a professional competency-based training program linked to designation. These initiatives also led to the creation of plans C&E modernization.

7. As ROEB shares some accountability with other product branches, making progress on these elements requires ongoing collaboration. In terms of health products (i.e., Therapeutic and Non-Prescription Drugs, Natural Health Products, Veterinary Drugs, Blood/ Cells/Tissues/Organs, Semen/Ova, and Medical Devices), ROEB is responsible for the development of high-level C&E policies and guidance documents, conducting inspection and compliance verification activities, maintaining IT systems, and creating a Quality Management System. For all other regulated product categories, these responsibilities are shared between ROEB and other branches (refer to appendix B for a breakdown of responsibilities across regulated product lines).

Appendix C provides details on the audit objective, criteria, scope, and approach.

B - Findings, recommendations and management responses

Compliance and enforcement modernization plan

1. 8. The creation of the Regulatory Operations and Regions Branch (later renamed the Regulatory Operations and Enforcement Branch - ROEB) in 2016 represented a new approach to the management of inspection activities. The centralization of responsibility for different inspection programs to one branch allowed the Department to move forward with the transition from a regional to a national delivery model and ensure consistency across programs. The Branch has taken steps to organize and clarify its roles and responsibilities with partner branches in transitioning to a national delivery model (See appendix B).
2. 9. While ROEB leveraged the Comprehensive Review exercise to highlight gaps in its inspection programs, it was not intended as a comprehensive modernization plan for all ROEB programs. In alignment with funding levels and organizational priorities, the Branch developed and implemented plans to stabilize Drug and Medical Device inspection operations. In addition, horizontal infrastructure was developed in the areas of training, occupational health and safety, and IT to allow some modernization to begin. Management also indicated that significant efforts were made to the development and implementation of new inspection regimes for Vaping and Cannabis product lines, in order to support GoC priorities.
3. 10. At the time of the audit, we found that there was no comprehensive plan for C&E modernization that spanned the breadth of the Branch's inspection activities, nor was there a supporting action plan with clear deliverables, timelines, and milestones.
4. 11. A comprehensive plan for C&E modernization would enable the Regulatory Operations and Enforcement Branch (ROEB) to communicate its goals and coordinate the work required to modernize its inspection programs. This would serve as a tool to guide modernization efforts, as well as support oversight, prioritization, and reprioritization of its implementation, as needed.
5. 12. ROEB is responsible for carrying out inspections in 11 regulated product categories, so a comprehensive plan that identifies how it intends to proceed with C&E modernization over time and across all regulated product categories would bring consistency and enable management to monitor the implementation of these efforts more effectively.
6. 13. At the time of the audit, ROEB had developed a Strategic Plan that outlined a high-level vision and plan for the next three years to modernize and transform the C&E approach, and strengthen the needed infrastructure to equip the workforce. As it was a high-level Strategic Plan, it did not include specificity in terms of expected deliverables and timeframes. The Branch indicated that the ROEB Strategic Plan was released in April 2019, and that work was underway to ensure they had implementation plans in place to achieve their vision.
7. 14. Overall, while some plans existed, there was no comprehensive plan to support C&E modernization in place that covered all eleven regulated product categories at the time of the audit.

Recommendation 1

Management response

Governance structure and oversight

1. 15. When the ROEB was created in 2016, a governance structure was put in place to support its strategic and operational goals. Since then the governance structure continued to evolve to the support organizational requirements.
2. 16. We expected to find a governance structure in place in support the implementation of C&E modernization. However, in the absence of a comprehensive plan for C&E modernization and corresponding governance, we examined the ROEB governance components and their relationship to Comprehensive Review initiatives.
3. 17. We found that the Branch had five senior management committees in place. We reviewed the terms of reference, agendas, records of decisions, and meeting minutes for four of these committees. These were the Branch Executive Committee (BEC), the Branch Executive Sub-Committee on Compliance and Enforcement Transformation (BEC-CET), the Director General Quadrilateral Committee, and the Director General Forum Committee (DG Forum). The BEC-CET was mandated specifically to manage ROEB's transformative C&E agenda. The DG Forum's mandate was to oversee the implementation of Comprehensive Review specific to cross-branch initiatives, which included progress on staffing, accommodations, training, occupational health and safety, IT, data and analytics, and fleet management. In addition, ROEB's ADM held monthly meetings to monitor the progress of comprehensive review commitments.
4. 18. We found that efforts to stabilize health product programs as a result of Comprehensive Review were monitored through existing mechanisms, like monthly operational dashboards.
5. 19. We found that BEC-CET discussed modernization elements of some C&E programs, which supported information exchange and learning from best practices between C&E Directors. We found that it was beginning to address C&E issues common to all programs, like repeat non-compliance. However, because there was no comprehensive plan for C&E modernization in place, the scope of the committee mandate was limited.
6. 20. Oversight is important for ensuring the effective and efficient realization of strategic and organizational goals for C&E modernization. This requires an appropriate governance structure to monitor implementation progress, with defined roles, responsibilities, and accountabilities for overseeing the implementation of C&E modernization.
7. 21. Given the importance of mitigating risks in a rapidly changing global environment, a consolidated oversight function within ROEB would add efficiency to the implementation of its modernization agenda. (See recommendation 2).

Monitoring and reporting

1. 22. Monitoring and reporting are key to ensuring successful implementation of C&E modernization. They support management oversight and help identify issues or areas for management attention.
2. 23. We expected to find effective monitoring and reporting practices in place to support the implementation of C&E modernization. However, in the absence of a comprehensive plan, we examined the monitoring and reporting in relation to Comprehensive Review.
3. 24. We found that ROEB had established a Project Management Office (PMO) to support the implementation and oversight of a subset of Comprehensive Review Projects, and to extend project management disciplines more broadly across ROEB.
4. 25. In terms of strengthening project management practices, the PMO is responsible for:
   • promoting project management standards and consistency;
   • providing advice and coordination for the governance and approvals processes;
   • reviewing project gating documentation for consistency and content; and
   • maintaining a project registry, project data, project management tools, templates, and lessons learned.
5. 26. The PMO provided oversight for a number of horizontal comprehensive review initiatives, including progress on staffing, accommodation projects, an IM/IT strategy, a National Learning Unit, and the Occupational Health and Safety Program.
6. 27. We reviewed a sample of the dashboards that were prepared for senior management and presented to the DG Forum and Branch Executive Committee (BEC), to assess the adequacy of the information to support oversight and decision making.
7. 28. We found that, while the dashboards promoted consistency in reporting on each of the projects, there were opportunities for improvement. For example, the dashboard for training initiatives was missing target end dates and a clear explanation of the planned or likely scope and operations of a designation program. This dashboard also only identified requirements in terms of person days and not dollars. Similarly, dashboards for IM/IT initiatives lacked target end dates. We also found that the dashboards focused on the current year's activities and could have provided a more complete picture of the overall status of these initiatives, thus enhancing monitoring and better supporting decision making. While there were detailed project management plans for all Investment Plan projects, there were opportunities to improve the value of dashboards.
8. 29. In conclusion, we found that there was monitoring of, and reporting on, Comprehensive Review projects; however, reporting dashboards could have been enhanced to better support monitoring of Comprehensive Review projects.

Recommendation 2

Management response

Funding and resources

1. 30. Successful implementation of the modernization of C&E efforts is dependent on having the necessary financial and human resources.
2. 31. We expected that the funding and resources required to address key components of C&E modernization had been identified.
3. 32. At the time of the audit, we found that funding requirements for some modernization activities had been identified, including inspector training, increased inspections of foreign sites, and IT requirements through the ROEB IM/IT Strategy. However, this was not done for all inspection programs.
4. 33. Through a Comprehensive Review exercise, ROEB developed a plan that aimed at stabilizing, modernizing, and transforming C&E oversight. In alignment with funding levels and organizational priorities, the Branch focused on investments to stabilize Drug and Medical Device inspection operations and made investments to allow some modernization to begin in the areas of training, occupational health and safety, and IT.
5. 34. In conclusion, current funding levels have enabled some investments to begin modernization. However, other levels of funding will need to be identified to complete modernization across all of the Branch's inspection programs. (See recommendation 1).

Key elements to support inspection activities

1. 35. The modernization of C&E activities is important for mitigating the risks that the rapid pace of innovation and the resultant changing nature of the global supply chain may pose to the health of Canadians.
2. 36. This is a complex undertaking that involves multiple inspection programs, multiple regulatory frameworks, and many elements within each program. The implementation of the key elements to support inspection activities is central to the vision of a national inspection function. These elements include:
   • risk-based inspection planning;
   • IM/IT infrastructure;
   • up-to-date operating procedures; and
   • training.
3. 37. The HC 2014 Inspection Review identified these elements as important to an effective regulatory regime, as does the Organization for Economic Cooperation and Development's (OECD) Regulatory Enforcement and Inspections Toolkit.
4. 38. Specific action plans for C&E modernization would ensure a high degree of importance and prominence for the following:
   • strengthening risk-based inspection planning;
   • developing or improving IM/IT systems that would better support inspection planning and reporting;
   • updating operating procedures and guidance; and
   • ensuring that inspectors had received appropriate training.
5. 39. Branch-wide oversight of modernization initiatives would ensure greater consistency between inspection programs. While ROEB is responsible for carrying out inspection activities across the 11 regulated product categories, it is also dependent on other branches (See appendix B) for developing and implementing a cohesive and consistent approach to inspection modernization.
6. 40. Due to the range of product categories that are subject to the Department's inspection activities, the audit focused on the product categories of Medical Devices, Biological Products (Blood, Cells/Tissues/Organs, and Semen/Ova), and Pesticides.

I. Risk-Based Inspection Planning

1. 41. A risk-based approach to inspection planning enables the Department to use resources more efficiently by directing inspection activities to areas of greatest risk. The OECD Regulatory Enforcement and Inspections Toolkit notes that a key element in enforcement and inspections is an approach that is risk-based and proportionate "…the frequency of inspections and the resources employed should be proportional to the level of risk and enforcement action should be aimed at reducing the actual risk posed by infractions."
2. 42. We found that there was no common approach to risk assessment practices across ROEB's 11 regulated product categories. In its Toolkit document, the OECD notes that common approaches, understanding, and practices for risk assessment and risk management across regulatory domains can make coordination of actions between different inspection structures more effective, and allow for better allocation of resources across different fields and regulatory areas. With 11 regulated product categories and corresponding unique regulatory frameworks, it is recognized that risk assessment approaches will need to be tailored. However, a Branch-level approach would serve to guide and ensure consistency across individual programs as they develop their own risk-based approaches to inspection planning.
3. 43. Management indicated that the Drug Good Manufacturing Practices (GMP) Inspection Program had already transitioned to a risk-based approach to inspection planning.
4. 44. Of the programs that were examined, we found that some inspection programs had started the process of assessing how strengthened risk-based inspection planning could lead to more effective resource use, by moving away from cyclical approaches towards more risk-based profiles for products, applications, or sites. Specifically, the Biological Products program's Semen/Assisted Human Reproduction product line and the Medical Devices product line had project charters that included completion of reviews of risk-based inspection planning by December 2019 and May 2021 respectively. The Pesticides inspection program's approach shifted from a cyclical approach to a sector-based one. However, this sector-based approach had not changed significantly from 2014, reflecting a combination of past practice, resource availability, and regional preferences and suggestions. There was no indication that this approach was going to be reassessed.
5. 45. We noted that the draft Data Analytics and Business Solutions Project Charter included the establishment of a Data Analytics and Business Solutions unit to provide guidance and support to business intelligence subject matter experts within ROEB. The Branch planned for this unit to increase the effectiveness of risk-based decision making and business intelligence activities, and was in the early stages of scoping its strengths and gaps at the time of the audit.
6. 45. Without a clear and consistent focus on risk-based planning, inspector resource allocation is at risk of being less efficient and inspection activities may not be as effective as possible in mitigating risks to human health and the environment.
7. 47. Overall, even though efforts are underway, there is still a need to improve inspection planning through a common understanding and application of risk assessment and risk management.

II. Information Management and Information Technology

1. 48. IM/IT are integral to the modernization of C&E functions at HC. They serve as important tools to support risk-based inspection planning and consistent application of inspection practices. However, since IM/IT projects can be significant investments, an enterprise approach must be considered. This requires assessing whether existing systems can be used to address the program needs. If a new system is required, efforts should be made to ensure that other programs with similar needs can use the new system as well.
2. 49. The 2014 Inspection Review identified that current IM/IT systems did not adequately support inspection functions. We found that this situation persists. For example, in the Biological Products sector, inspectors were responsible for carrying out inspections of three different product categories (Blood, Cells-Tissues-Organs, and Semen); however, they must use two different systems to record the inspection results. The Medical Devices inspection program relies on a system that the vendor no longer supports, with a number of separate systems to post reports online, to store incidents and recalls, and to store inspection reports.
3. 50. Having plans in place to guide the implementation of IM/IT systems and tools would contribute to the modernization of those C&E programs responsible for the Branch's 11 regulated product categories.
4. 51. We found that ROEB had established a Technology and Business Innovation Division to lead the implementation of IM/IT strategies that align with HC's enterprise approach to IM/IT.
5. 52. We found that, although ROEB had developed a five-year IT Strategy (2018-23), it did not address the needs of all 11 regulated product categories. As indicated in appendix B, ROEB is not the lead on all C&E programs' technology projects. ROEB is the lead for health product C&E functions, works in collaboration with HPFB where solutions are needed for the full program, and depends on other branches to lead investment planning projects, including PMRA for pesticide C&E, HECSB for consumer products, and CSCB for Cannabis, Controlled Substances, and Tobacco/Vaping. ROEB has been applying a "proof of concept" approach in certain areas, including IT, with the hope of applying lessons learned to other inspection programs. The extent to which proven systems are implemented in other inspection programs can be dependent on the other programs having funding available and being prepared to move forward.
6. 53. A number of investment plans were under development that addressed the needs of some program areas. These included:
   • The Establishment Licensing and Inspection System, which was intended to become the system for a number of departmental programs that provide licences to entities, but it would not support compliance verifications and was not expected to address inspection planning. The Project Charter identified six business programs as being within its scope. The first two programs that are slated to be onboarded are Medical Devices and Controlled Substances, in April 2021 and July 2021 respectively. The Charter did not provide expected onboarding dates for the other business programs (Blood, Cells, Tissues and Organs, Drugs, and Natural and Non-prescription Health Products). At the time of the audit, the high-level business requirements of this project were being developed;
   • An Enterprise Quality Management System, with the Cannabis program scheduled to be onboarded first. The Branch indicated that an onboarding schedule will be developed for other HC programs once they are ready to move to this enterprise system;
   • Medical Devices Post-Market Renewal would enhance HC's ability to receive, process, and analyze an increasing volume and complexity of problem and recall reports in a timely manner to support medical device monitoring and improve transparency;
   • Modernizing the ROEB E-Toolkit will provide inspectors with tools like tablets, smart phones, and data plans to support the conduct of inspections; and
   • The Compliance and Enforcement Program Delivery project is made up of six sub-projects, including the development of a learning management system. The project is intended to be scaled up to other programs, following the initial onboarding of the Cannabis program and the Tobacco and Vaping program, but a target date has not been established.
7. 54. We found that none of the projects for which investment plans existed specifically addressed the need to replace the aging Electronic Pesticides Regulatory System (ePRS). The ePRS is a PMRA information management system used primarily for pre-market activities. As noted above, the current system was only supporting the tracking of non-compliant inspection results. Solutions to improve ROEB inspection functions for pesticides would be a shared responsibility between ROEB and PMRA.
8. 55. There may be a gap between the IM/IT systems developed and the operational needs of inspectors, unless discussion and monitoring take place for an established stabilization and modernization plan for IM/IT systems that addresses each of the 11 regulated product categories. This could impair efficiency and effectiveness. The ROEB Strategic Plan 2019-22 identifies "Implementing a new IM/IT strategy, including modernizing tools for inspectors, analysts, and specialists…" as a key initiative.
9. 56. In conclusion, modernization efforts for C&E included considerations of IM/IT. The current projects, however, are not likely to address the IM/IT needs of all ROEB inspection programs.

III. Operating Procedures

1. 57. Up-to-date operating procedures and guidance documents are needed to ensure that inspectors have clear roles and responsibilities, and conduct their inspections consistently and fairly. They also serve as an important source of information for the development of training courses and materials for inspectors. As previously noted, the Inspection Review conducted in 2014 identified that there were too many operating procedures, many of which were outdated or in draft form, and while document management systems existed for some programs, there was no central repository to manage and disseminate documents for all programs.
2. 58. Although a Quality Assurance Management (QAM) team was in place within ROEB to assist in updating standard operating procedures, the scope of its activities was limited to the documents of only two directorates within ROEB: the Health Products Compliance Directorate, and the Medical Devices and Clinical Compliance Directorate. QAM did not address the updating of guidance documents for the other directorates within ROEB that are responsible for conducting inspections, but share the responsibility for updating documents with other program branches (See appendix B).
3. 59. A three-year cycle had been established as a benchmark for the review of quality documents, like standard operating procedures and other guidance documents. We reviewed documentation prepared by QAM for presentation to management for Medical Devices and Biological Products, and found that both areas had a number of quality documents that had not been updated for at least three years. Management has indicated that higher-risk priorities meant a less aggressive timeline for updating the operating procedures in the Quality Management System.
4. 60. While oversight was in place for the Medical Devices and Clinical Compliance Directorate and the Health Products Compliance Directorate, this was not the case for all product categories.
5. 61. A key to assuring consistency of inspection practices is to have up-to-date operating procedures. Given their importance, updated standard operating procedures should be in place for all inspection programs to support the work of inspectors in all regulatory programs.
6. 62. In conclusion, greater efforts are required to ensure operating procedures and guidance documents are up-to-date for all the regulated product categories.

IV. Training

1. 63. As with operating procedures, training is needed for the development and maintenance of a professional inspection workforce that can ensure consistency of practice and mitigate risks resulting from the rapidly changing global environment. It also supports the vision of creating a national inspection program.
2. 64. The 2014 Inspection Review identified the need for enhanced training to ensure that the inspection workforce can meet the complex requirements of HC's regulated environment.
3. 65. We found that ROEB had recently established the National Learning Unit (NLU) to lead the modernization of training for inspectors. NLU developed a project charter that indicated it was responsible for developing and delivering the core courses of the curriculum for all inspectors, and would assist each program in the development of program-specific training. While progress was made in revamping the core curriculum, management indicated that there needs to be a clear designation management framework in place, including an approved designation policy and training standards, in order to develop a designation training program. For example, the NLU was focusing its efforts on helping the Cannabis Directorate analyze designation requirements, prepare a Cannabis Designation Directive, identify training needs, and set a learning path. The development of a designation training program will proceed following the approval of the Certification standard and the Cannabis designation policy. It was intended that the work to support the Cannabis Inspection program would be directly leveraged for other programs. The NLU was then expecting to assist, with dates to be determined, the Pesticides, Tobacco Control and Vaping, Health Products, Medical Devices, and Controlled Substances programs in conducting analyses of their designation and training needs. The project charter does not identify when, or in which order, the program-specific training would be made available. As noted above, under the "proof of concept" approach, adoption by other program areas can be dependent on the availability of funding and the state of program preparedness.
4. 66. While none of the three inspection programs examined (Medical Devices, Pesticides, and Biological Products) had a formal plan for modernizing training, they were addressing their current specific training needs through less formal means, such as on-the-job shadowing, structured supervision, WebEx seminars, and face-to-face meetings or conferences.
5. 67. We found that other key elements of training had yet to be developed. These include a national training policy, a policy that addresses consistent approaches to designation of inspectors, and the inclusion of conflict of interest training, and fraud prevention and detection training in the core curriculum. These elements are key to informing the training content, as well as driving the mandatory training curriculum and certification process to support designation. Consequently, although inspectors were supposed to take all the core courses within the first year of work, there was no requirement for mandatory training before designation^{Footnote 2}, and inspector designations were issued independently of formal training. In addition, no repository existed to track the courses that inspectors had taken. A lack of formal training infrastructure could raise questions about inspection activities.
6. 68. In conclusion, while modernization efforts for C&E have included considerations of better training infrastructure for inspectors, more effort is required to advance the development of a training curriculum based on designations and competencies.
7. 69. Overall, we found that there was consideration of the four key elements of C&E modernization (risk in inspection planning, IM/IT, operating procedures, and training). However, deficiencies were noted that require enhanced focus and oversight to ensure that the needs of all inspection programs are addressed.

Recommendation 3

Management response

Sex- and gender-based analysis plus

1. 70. Sex- and Gender-Based Analysis Plus (SGBA+) is an analytical tool used to assess how various groups of women, men, and gender-diverse people may experience policies, programs, and initiatives. SGBA+ also considers many other identity factors like race, ethnicity, religion, age, and mental or physical disability.
2. 71. We expected that ROEB had determined how SGBA+ would be incorporated in inspection activities. We expected specifically that training was given to employees, SGBA+ was based on appropriate information sources, and SGBA+ was applied to appropriate aspects of C&E activities.
3. 72. We found no documented evidence to demonstrate that ROEB had incorporated SGBA+ into its inspection activities. We also found that there had been no offering of SGBA+ training at the program levels.
4. 73. Management indicated that the lack of SGBA+ and related training at the program levels was based on an understanding that the populations served by the ROEB regulatory environment are entities that have no sex or gender, such as manufacturers, wholesalers, and retailers.
5. 74. In conclusion, the C&E approach, including inspection planning, had not considered Sex- and Gender-Based Analysis Plus.

Recommendation 4

Management response

C - Conclusion

1. 75. The compliance and enforcement (C&E) function at HC includes 11 different regulated product categories for which the Regulatory Operations and Enforcement Branch (ROEB) has partial or full C&E responsibility, depending on the category. Over the past number of years, prior to the creation of ROEB, the Department undertook various exercises that focused on C&E modernization, including an Inspection Function Review (2014) and a C&E Modernization Roadmap (2015), which included a C&E Excellence Framework (2016).
2. 76. These initiatives identified a number of gaps in HC's C&E practices. These included the need for clear roles, responsibilities and accountabilities, risk-based planning of inspection activities, effective use of modern technology, up-to-date operating procedures, and a professional competency-based training program linked to designation.
3. 77. Addressing these gaps is a challenge due to the number of elements and inspection programs that are involved. This must be done while delivering on the Branch's other responsibilities, which at the time of the audit included work to support the legalization of cannabis, and a new regime for vaping products.
4. 78. The Department has shown progress in stabilizing operations to address the identified gaps, including the following:
   • creating a new C&E branch in 2016: the Regulatory Operations and Regions Branch (renamed ROEB in 2019);
   • implementing a national program delivery model;
   • clarifying roles and responsibilities for C&E oversight among the different Health Canada branches;
   • advancing a risk-based approach for Drug Good Manufacturing Practices (GMP) inspection;
   • planning the analysis of program gaps, priorities and resource requirements;
   • updating the Cost Recovery Fees for Drugs and Medical Devices;
   • stabilizing certain C&E programs;
   • making modest investments to allow some modernization to begin in horizontal infrastructure, like technology, data and analytics, inspector training, and occupational health and safety; and
   • setting a high-level vision for C&E modernization and transformation with the release of ROEB's Strategic Plan in April 2019.
5. 79. While it was evident that progress was being made, we found that management could not demonstrate how efforts had been prioritized to address the gaps between their current state and the future desired state of C&E modernization. Other than where investments had been made, management did not comprehensively indicate how key elements of C&E modernization would be implemented to close gaps in all program areas.
6. 80. We found that ROEB was in the process of launching a strategic plan that outlines a vision and high-level plan for how it will continue to modernize the C&E approach. Progress made included:
   • establishing a number of teams that are well positioned to support C&E modernization, such as:
     • the Technology and Business Innovation Division, to lead the implementation of information management and information technology (IM/IT) strategies;
     • the Quality Assurance Management, team to assist with the updating of standard operating procedures;
     • the National Learning Unit, to assume responsibility for the development and delivery of core curriculum training courses for all inspectors;
     • the Project Management Office, to promote strong project management practices, and consistency in monitoring and reporting on investments to date for modernization;
   • incorporating, in some programs, elements of risk assessment in inspection planning;
   • establishing a number of projects and project charters to address the IM/IT modernization needs of some program areas; and
   • revamping elements of the core training curriculum while prioritizing support to cannabis inspector training and designation.
7. 81. We found that there was consideration of the four key elements of C&E modernization (i.e., risk in inspection planning, IM/IT, operating procedures, training). However, we noted there were deficiencies that require enhanced focus and oversight to ensure the needs of all inspection programs were addressed. These included:
   • IM/IT strategies that addressed the needs of only some of the 11 regulated product categories (responsibility for IM/IT strategies to support C&E is shared between ROEB and partner branches for some of the inspection programs – see appendix B);
   • limited progress in addressing backlogs of outdated standard operating procedures and guidance documents (responsibility of for C&E standard operating procedures and guidance documents is shared between ROEB and partner branches for some of the inspection programs – see appendix B);
   • a lack of policy on designation and mandatory training to advance the national training approach and the provision of specialized training required by individual inspection programs; and
   • no Sex- and Gender-Based Analysis Plus was conducted, nor training provided.
8. 82. Overall, we recognize the significant work undertaken to establish a pathway to success within ROEB. However, we found instances where improvements could have been made to further strengthen the implementation of key elements of C&E modernization. The areas for improvement noted in this audit report, along with the associated recommendations, will collectively strengthen the management practices, at the branch and program levels, to support appropriate implementation of key elements of C&E modernization.

Appendix A - Scorecard

Audit of Inspection Activities

Criteria	Risk RatingFootnote 3	Risk to the Compliance and Enforcement program without Implementing Recommendation	Rec #
A comprehensive plan is in place to support Compliance and Enforcement (C&E) modernization.	4	The C&E plan did not cover all aspects of C&E modernization for ROEB's inspection programs. Without a comprehensive plan that clearly identifies all aspects of C&E modernization, as well as the resource requirements, milestones, and deliverables, there is a risk that some aspects of the C&E modernization are being missed or not being considered by management.	1
Oversight of the implementation of C&E modernization is in place.	3	While elements of a governance structure were in place, overall responsibility for oversight of compliance and enforcement modernization initiatives was not clear due to oversight committees having broad and overlapping mandates. Due to the complexity and magnitude of the initiatives, oversight is required to ensure timely and appropriate implementation. Otherwise, successful implementation of modernization efforts may be at risk.	2
Plans related to C&E modernization are monitored and reported on to ensure their implementation.	2	Dashboards presented to management did not contain sufficient information to provide senior management with a complete picture of the overall status of modernization initiatives. There is a risk that management does not have sufficient information to support decision making, which can negatively affect its ability to exercise effective oversight and ensure consistent implementation of the key elements of compliance and enforcement modernization.	2
Plans related to C&E modernization have sufficient funding and resources to achieve their objectives.	3	Current funding levels have enabled some investments to address certain identified gaps. However, other levels of funding will need to be identified to complete implementation of key elements of compliance and enforcement modernization across all of the Branch's inspection programs. Gaps in compliance and enforcement might not be addressed consistently across all inspection programs.	1
Plans for inspection activities consider risk in inspection planning, IM/IT, operating procedures, and training.	3	Action plans for inspection activities included some consideration of the four key elements of C&E modernization (risk in inspection planning, IM/IT, standard operating procedures, and training). However, greater efforts are required to ensure the needs of all inspection programs are addressed. This may affect management's ability to ensure consistency in the implementation of the key elements across all the inspection programs.	3
Sex- and Gender-Based Analysis Plus (SGBA+) is considered in the C&E approach.	2	The compliance and enforcement approach, including inspection planning, does not consider Sex- and Gender-Based Analysis Plus. Compliance and enforcement activities might not sufficiently mitigate risks to vulnerable segments of the population.	4
Risk Rating Minimal Minor Moderate Significant Major

Appendix B - Responsibility for C&E functions

Appendix B - Responsibility for C&E functions: Text description

Table showing responsibilities for key compliance and enforcement activities for products regulated by Health Canada.

Therapeutic and Non-Prescription Drugs - ROEB is responsible for C&E policy, guidance, plans, and procedures; conduct of inspections and investigations; quality management system (QMS); inspector recruitment; and IT to support C&E Strategy. ROEB and another branch share responsibility for the training of inspectors.

Veterinary Drugs - ROEB is responsible for C&E policy, guidance, plans, and procedures; conduct of inspections and investigations; quality management system (QMS); inspector recruitment; and IT to support C&E Strategy. ROEB and another branch share responsibility for the training of inspectors.

Blood, Semen/Ova, Cells, Tissues & Organs - ROEB is responsible for C&E policy, guidance, plans, and procedures; conduct of inspections and investigations; quality management system (QMS); inspector recruitment; and IT to support C&E Strategy. ROEB and another branch share responsibility for the training of inspectors.

Natural Health Products - ROEB is responsible for C&E policy, guidance, plans, and procedures; conduct of inspections and investigations; quality management system (QMS); inspector recruitment; and IT to support C&E Strategy. ROEB and another branch share responsibility for the training of inspectors.

Clinical Trial - ROEB responsible for the conduct of inspections and investigations; quality management system (QMS); inspector recruitment; and IT to support C&E Strategy. ROEB and another branch share responsibility for C&E policy, guidance, plans, and procedures; and training of Inspectors.

Medical Devices - ROEB is responsible for C&E policy, guidance, plans, and procedures; conduct of inspections and investigations; quality management system (QMS); inspector recruitment; and IT to support C&E Strategy. ROEB and another branch share responsibility for the training of inspectors.

Cannabis – ROEB is responsible for inspector recruitment; and the training of inspectors. ROEB and another branch share responsibility for C&E policy, guidance, plans, and procedures; and IT to support C&E Strategy. Responsibility to be determined for the conduct of inspections and investigations. Quality Management System (QMS) not applicable.

Controlled Substances – ROEB responsible for the conduct of inspections and investigations; inspector recruitment; and the training of inspectors. ROEB and another branch share responsibility for C&E policy, guidance, plans, and procedures; quality management system (QMS); and IT to support C&E Strategy.

Tobacco – ROEB responsible for the conduct of inspections and investigations; and inspector recruitment. ROEB and another branch share responsibility for the training of inspectors. Another branch is responsible for C&E policy, guidance, plans, and procedures; and IT to support C&E Strategy. Quality Management System (QMS) not applicable.

Vaping - ROEB responsible for the conduct of inspections and investigations; and inspector recruitment. ROEB and another branch share responsibility for C&E policy, guidance, plans, and procedures; training of Inspectors; and IT to support C&E Strategy. Another branch is responsible for quality management system (QMS).

Consumer Products Safety & Cosmetics - ROEB responsible for the conduct of inspections and investigations; and inspector recruitment. ROEB and another branch share responsibility for C&E Policy, Guidance, Plans, and procedures; and the training of Inspectors. Another branch is responsible for quality management system (QMS); and IT to support C&E Strategy.

Pest Control Products - ROEB responsible for the conduct of inspections and investigations; and inspector recruitment. ROEB and another branch share responsibility for C&E policy, guidance, plans, and procedures; training of Inspectors; and IT to support C&E Strategy. Another branch is responsible for quality management system (QMS).

Table is based on the ROEB variability chart in the implementation of C&E function across regulated product lines – updated October 2018

Appendix C - About the audit

Audit objective

The objective of this audit was to provide reasonable assurance that key elements of compliance and enforcement (C&E) modernization were being implemented appropriately.

The audit criteria were as follows:

• A plan is in place to support C&E modernization.
• A governance structure is in place that provides oversight on the implementation of C&E modernization.
• Plans related to C&E modernization are monitored and reported on to ensure its implementation.
• Plans related to C&E modernization have sufficient funding and resources to achieve the plans' objectives.
• Plans for C&E modernization consider risk in inspection planning, IM/IT, operating procedures, and training.
• The C&E approach, including inspection planning, considers Sex- and Gender-Based Analysis Plus.

Audit scope

The audit focused on the plans and management practices, at the branch and program levels, to monitor progress and support implementation of key elements of C&E modernization. The audit also included an examination of detailed action plans in the areas of Biological Products, Medical Devices, and Pesticides. The three inspection programs were selected based on an evaluation of the complexity of the activity, recent changes in the external environment, materiality, and risk to human health.

The scope did not include detailed testing of the work performed by inspectors, nor compliant and non-compliant inspection report ratings.

In addition, inspection of radiation emitting devices was excluded from the scope of the audit, as the Healthy Environments and Consumer Safety Branch performs these inspections. Other ROEB inspection activities related to consumer products and cosmetics, tobacco, drugs, clinical trials, controlled substances, cannabis, and border operations were excluded from the scope of the audit to better focus on the scoped-in areas identified above.

Audit approach

The audit approach included, but was not limited to:

• interviews with key officials responsible for the design and implementation of compliance and enforcement modernization initiatives;
• review of relevant documentation, including plans, policies, standards, guidelines, and frameworks related to compliance and enforcement modernization;
• detailed analysis of a sample of action plans; and
• analysis of observations arising from interviews, inquiries, document reviews, and detailed testing.

Statement of conformance

This audit was conducted in conformance with the International Standards for the Professional Practice of Internal Auditing and is supported by the results of the Office of Audit and Evaluation's Quality Assurance and Improvement Program.