ARCHIVED - Evaluation Report of the Canadian Food Inspection Agency's Dairy Program Activities Related to Imported Cheese Made from Pasteurized Milk

Bureau of Food Safety Assessment, Health Canada
July 2006
ISBN:0-662-43556-7
Cat. H164-21/2006E
HC Pub. No: 8020

1.0 Introduction
1.1 Evaluation Scope and Objectives

2.0 Evaluation Approach
2.1 Evaluation Questions and Methodology
2.2 Evaluation Challenge - Data Limitations

3.0 Description of Dairy Program Activities Related to Imported Cheese Made from Pasteurized Milk
3.1 Legislation, Policies, Standards
3.2 Program Description
3.3 Dairy Program Logic Model Related to Imported Cheese Made from Pasteurized Milk

4.0 Observations, Conclusions and Recommendations
4.1 Rationale/Relevance of the Dairy Program's Import Activities
4.2 Design and Management of Dairy Program's Import Activities
4.2.1 Dairy Program Objectives
4.2.2 Program Design
4.2.3 Science and Risk-Based Management
4.2.4 Integration of the CFIA Import Control Policy
4.2.5 Program Review
4.3 Program Delivery
4.3.1 Delivery of Inspection Activities as Designed
4.3.2 Follow-up and Enforcement Activities
4.3.3 Evidence of Training Activities
4.3.4 Evidence of Adjustments to Program Delivery
4.3.5 Trends in the Rate of Compliance of Imported Cheese Made from Pasteurized Milk and Rate and Trend of Compliance for Products Previously Identified as Non-compliant
4.3.6 Effective Management of Food Safety Emergencies and Incidents
4.4 Stakeholder Engagement
4.4.1 Facilitating the Delivery of Dairy Program Import Requirements
4.4.2 Industry Responsibility/Commitment for the Safety of Imported Cheese

Appendix 1 - Dairy Program Logic Model Related to Cheese Made from Pasteurized Milk
Appendix 2 - Narrative For The Logic Model
Appendix 3 - Evaluation Methodology and Design
Appendix 4 - Guiding Principles and Elements of the Import Control Policy

Glossary of Terms

Dairy Assessment Committee Members

Health Canada Evaluation Team Members

References

Exhibits

Exhibit 1
Exhibit 2
Exhibit 3
Exhibit 4

Copyright

"Our mission is to help the people of Canada maintain and improve their health"
Health Canada

Contents may not be reproduced for commercial purposes, but any other reproduction, with acknowledgements, is encouraged.

Également offert en français sous le titre :

"Rapport d'évaluation des activités de l'Agence canadienne d'inspection des aliments reliées au Programme des produits laitiers concernant les fromages importés faits de lait pasteurisé".

For further information or to obtain additional copies, also in alternate formats, please contact:

Ashwani Wadhera, Director
Bureau of Food Safety Assessment
Sir Frederick Banting Building
4 East - 2204D
Ottawa, Ontario K1A 0L2

Telephone: (613) 954-2996
FAX: (613) 954-0149
E-mail: BFSA_BESA@hc-sc.gc.ca

^© Her Majesty the Queen in Right of Canada, represented by the Minister of Health, 2006

ISBN:0-662-43556-7
Cat. H164-21/2006E
HC Pub. No: 8020

Executive Summary

This report is the result of an evaluation of the Canadian Food Inspection Agency's (CFIA) Dairy Program's activities related to imported cheese made from pasteurized milk by the Bureau of Food Safety Assessment of Health Canada. This evaluation stems from the Minister of Health's responsibility under the Canadian Food Inspection Agency Act to assess the effectiveness of CFIA's food safety activities. The Dairy Program was chosen for evaluation by a joint Health Canada and CFIA committee and covers the period between the fiscal years 2000-2001 until 2003-2004.

The mandate of CFIA's Dairy Program is to ensure that dairy products leaving federally inspected establishments or being imported into Canada are safe, wholesome, accurately labelled to avoid misleading consumers, and are eligible to be traded inter-provincially or internationally. The primary monitoring activity for imported dairy products focusses on product inspection. CFIA employees inspect imported dairy products to ensure that minimum requirements such as microbiological and chemical standards, labelling requirements and that the health and safety requirements prescribed in the Food and Drugs Act and Regulations and the Dairy Products Regulations are met.

A result of the evaluation was the development, in collaboration with CFIA Dairy Program staff, of a logic model of the Dairy Program activities related to imported cheese made from pasteurized milk (see Appendix 1 and 2). This logic model, which describes Program activities, outputs, and desired outcomes, was used to formulate evaluation questions concerning program rationale, design and management, delivery and outcomes, based on Treasury Board Secretariat guidelines. These evaluation questions form the basis for assessing the effectiveness of the Dairy Program's food safety activities related to cheese made from pasteurized milk. The Evaluation Framework contains 11 evaluation questions and 29 indicators. In order to provide a concise report the evaluation questions (see Appendix 3) have been consolidated into four areas of reporting: program relevance / rationale; program design and management; program delivery; and stakeholder engagement.

The evaluation used a range of methodologies including: literature reviews; environmental scans; document and file reviews; collection and analysis of available electronic and hard copy data; interviews (using developed interview guides); and a case study of importers through interviews (using developed interview guides). For each evaluation methodology employed, multiple lines of evidence were evaluated. Interview subjects' responses have been kept confidential.

For some of the evaluation questions that addressed the delivery of the program activities and progress toward its desired outcomes, the available data was not sufficient to reach sound conclusions. In such instances, the evaluation questions are discussed to the extent made possible by the data.

All enforcement programs evolve over time in response to lessons learned and changing conditions. The evaluation team concluded, with respect to the evaluation questions, that the Dairy Program could take steps to improve their current level of effectiveness. Accordingly, recommendations for improvements were suggested and are listed below. In this regard, CFIA already has several initiatives underway. In 2002, in response to changing global trade, CFIA provided an overall strategic approach (Import Control Policy) to the management of imported commodities by all CFIA Programs. The Dairy Program is planning to enhance its import control system with a new activity, Good Importing Practices inspections. These inspections, which take place at importers' premises that are not registered dairy establishments, assess the importers' ability to ensure the safety and compliance of the dairy products they import.

Recommendations were formulated based on the data provided and analysed in the context of this evaluation. The recommendations focus on: enhancing information systems; improving program design and oversight; enhancing knowledge and understanding of performance measurement; and regularly engaging importers to promote compliance.

Recommendations:

Enhancing Information Systems

1. In order to enhance their science-based approach, the Dairy Program should centrally collect information from environmental scans, product monitoring and surveillance or other sources.

2. The Dairy Program should increase its capacity to collect data to monitor workplan delivery.

Improving Program Design and Oversight

3. The Dairy Program should consider appointing a manager to oversee its import control system in accordance with the Import Control Policy.

4. There should be more formal collaboration between Programs and Operations to ensure that the Dairy Import Program design and workplans are both science-based and feasible, that their implementation is monitored and that continuous improvements are made as required.

5. The Dairy Program should design and implement a surveillance system to ensure that products previously identified as non-compliant are monitored on a national basis and to better target (risk-based) product inspections.

Enhancing Knowledge and Understanding of Performance Measurement

6. The Dairy Program should periodically review the effectiveness of their import activities in order to enhance Program design and delivery, and take steps to improve their capacity to collect and analyse information so that systematic performance measurements of their activities can be carried out.

7. The Dairy Program should collect reliable data to track the compliance status of imported cheeses and analyse trends in compliance and disseminate this information to staff for use in program design, delivery and reporting.

Regularly Engaging Importers to Promote Compliance

8. The Dairy Program may want to consider and implement initiatives designed to regularly engage importers or their associations to promote compliance.

9. The Dairy Program should design a strategy that best enhances importers' understanding and commitment to regulations and policies, and develop performance indicators to measure progress towards their outcomes.

10. The Dairy Program should be more proactive in providing guidance (i.e. give guidance before problems occur) to enhance importers' ability to effectively trace products and manage food safety emergencies and incidents related to imported cheese made from pasteurized milk.

11. The Dairy Program should clearly document the roles and responsibilities of its CFIA and other federal government partners involved in facilitating the delivery of the Dairy Program's import activities and occasionally monitor the effectiveness of its partners' delivery of the Dairy Program's import control system.

CFIA Management Response

Enhancing Information Systems

1. In order to enhance their science-based approach, the Dairy Program should centrally collect information from environmental scans, product monitoring and surveillance or other sources.

CFIA Response:

CFIA actively participates in various Codex Alimentarius meetings and the International Dairy Federation standing committees which provide current information with respect to environmental scans of dairy products including cheese. As well, reviews of issues and recalls from the Issues Management System are carried out for this purpose.

The CFIA has adopted Performance Management Framework (PMF) reporting which will contribute to key data capturing and analysis for import product monitoring. The PMF initiative will include the Dairy Program for the 2005/06 fiscal year.

The CFIA will include the Dairy Program in the Agency's Import Control Tracking System (ICTS) initiative. The Import Control Tracking System program is currently used by three programs in the Agency (Meat, Eggs and Fish). Adopting the system for dairy programs will enhance the monitoring and surveillance activities for imported products.

2. The Dairy Program should increase its capacity to collect data to monitor workplan delivery.

CFIA Response:

As a result of the Agency adopting the PMF approach, the Dairy Program has identified various indicators which will track delivery of dairy import activities for 2005/06.

Improving Program Design and Oversight

3. The Dairy Program should consider appointing a manager to oversee its import control system.

CFIA Response:

The Food of Animal Origin Division has established and staffed on a term basis, a Chief position to cover Dairy Program import issues.

CFIA Response:

The PMF will be used to develop and track workplan delivery as well as the monitoring and reporting of compliance. Formal consultations will occur between Programs and Operations Branches to validate and analyse this data on a regular basis as well as recommend improvements.

The Dairy Program is considering modifying program design through the implementation of a statistically-based inspection frequency strategy. Adopting the new frequency strategy will increase the Agency's confidence in workplan development and forms the basis for drawing conclusions on the safety and quality of dairy products marketed in Canada.

CFIA Response:

The CFIA is evaluating the Dairy Program including the Agency's ICTS initiative. The system will track level of compliance and trigger timely enforcement intervention as needed.

Enhancing Knowledge and Understanding of Performance Measurement

CFIA Response:

The PMF and ICTS initiatives contained in recommendations 1,4,5 and 7 will address this issue.

In addition, the Dairy Products Inspection Manual, as related to the Dairy Import Program (Chapter 16), is being revised to address observations contained in the Health Canada Assessment.

CFIA Response:

The PMF and ICTS initiatives contained in recommendations 1, 4, 5, and 6 will address the CFIA capacity to collect and analyse information.

In addition, the Dairy Program is assessing the introduction of new mobile computing technology, to include a database system for capturing inspection compliance data. Adopting such technology would contribute streamlined data entry and provide access to compliance information for program design, delivery and reporting purposes.

Regularly Engaging Importers to Promote Compliance

8. The Dairy Program may want to consider and implement initiatives designed to regularly engage importers or their associations to promote compliance.

CFIA Response:

Overall compliance reporting is presently carried out and annual meetings with the industry association takes place. The introduction of mobile computing data capture will greatly increase the quality of compliance data and enable reporting to the individual importers.

The Dairy Program will develop greater presence on the CFIA website.

CFIA Response:

The CFIA will engage the International Cheese Council, which represents the greatest number of cheese importers, in order to develop a collaborative approach to train their members on the revised requirements of Chapter 16 of the Dairy Products Inspection Manual. Progress will be monitored on an ongoing basis by the Dairy Program.

PMF data will indicate overall industry compliance and when mobile computer technology is implemented, individual importer data will measure specific attainment of these outcomes.

CFIA Response:

Chapter 16 of the Dairy Product Inspection Manual will specify requirements for effectively dealing with food safety emergencies including defining roles and responsibilities. As well, it will reference established CFIA policies and procedures for handling food safety emergencies.

CFIA Response:

Chapter 16 of the Dairy Products Inspection Manual will include roles and responsibilities of the various federal partners involved in the Dairy Import Program.

Audits and evaluations of the Dairy Import Program will encompass the impact of federal partners' delivery of the Dairy Import Program on achievement of results.

1.0 Introduction

1. Health Canada's Food Directorate is Canada's primary food safety regulator and policy-maker. It establishes policies, regulations and standards related to the safety and nutritional quality of food, which are enforced by the Canadian Food Inspection Agency (CFIA). The Food Directorate's Bureau of Food Safety Assessment (BFSA) has a clear mandate stemming from the Canadian Food Inspection Agency Act to assess the effectiveness of the activities of CFIA related to food safety. The Bureau of Food Safety Assessment uses an evaluation approach, in accordance with the Appendix to the Memorandum of Understanding between Health Canada and the Canadian Food Inspection Agency on the Food Safety Assessment Provision Contained in Subsection 11(4) of the Canadian Food Inspection Agency Act, and the Health Canada Policy, Food Safety Assessment Program to evaluate the effectiveness of the Canadian Food Inspection Agency's activities related to food safety and nutritional quality. BFSA also provides feedback to Health Canada with respect to developing policies and standards.

1.1 Evaluation Scope and Objectives

2. The Assessment Committee (comprised of Health Canada and CFIA representatives listed on page 37) reviewed and approved the proposed scope of the evaluation, namely, the food safety activities related to imported cheese made from pasteurized milk. Cheese is a food that is potentially susceptible to conditions and circumstances that may result in a food safety risk. An evaluation focussing on imported cheese made from pasteurized milk and the Dairy Program's import control systems might provide beneficial information on the effectiveness of a number of Dairy Program health and safety activities. An Evaluation Framework was developed and key evaluation questions were identified that relate to program design, management and delivery, and outcomes. The time frame for this evaluation targeted the period between the fiscal years 2000-2001 to 2003-2004. The objective of this evaluation was to provide information to Health Canada and CFIA on the effectiveness of the Dairy Program's food safety activities related to imported cheese made from pasteurized milk. CFIA is responsible for implementing an appropriate response to the evaluation recommendations.

2.0 Evaluation Approach

2.1 Evaluation Questions and Methodology

3. A Dairy Program logic model was used to formulate evaluation questions concerning program rationale, design and management, delivery and outcomes, based on Treasury Board Secretariat guidelines. These evaluation questions form the basis for assessing the effectiveness of the Dairy Program's food safety activities related to cheese made from pasteurized milk. The evaluation questions, as well as indicators (also in the form of questions) needed to answer them are listed in Appendix 3. The evaluation used a range of qualitative and quantitative methodologies that are further described in Appendix 3. These included: literature reviews; environmental scans; document and file reviews; collection and analysis of available electronic and hard copy data; interviews using developed interview guides; and a case study of importers through interviews using interview guides. For each evaluation methodology employed, multiple lines of evidence were evaluated. Interviewees' individual responses have been kept confidential in this report.

2.2 Evaluation Challenge - Data Limitations

4. For some of the evaluation questions related to program delivery and desired outcomes, the available data was not sufficient to reach sound conclusions. In such instances, the evaluation questions are discussed to the extent made possible by the data. Overall, it was a challenge to obtain data that distinguished between imported cheese made from raw milk and imported cheese made from pasteurized milk. The capability to do so is important in assessing or developing program priorities and activities based on risk. Only the Quebec Area tracking system distinguishes samples of imported cheese made from pasteurized milk from imported cheese made from raw milk (see section 4.3.1). Dairy Program data on the delivery of the Program's activities, received from different CFIA Area and Headquarters sources, did not always correlate between sources. As a result, analysis of the fulfilment of workplans and trend analysis was not reliable or in some cases feasible. The Dairy Program is progressing towards identifying performance indicators and monitoring their performance in program delivery and the achievement of program goals.

5. There is a limited ability to measure changes in public health resulting from regulatory oversight of the food supply because food-borne illnesses is greatly under-reported¹, and it is difficult to attribute changes or trends in food-borne illness or public health to regulatory programs and activities. For these reasons and the data limitations noted above, the evaluation team could not conclude on the evaluation question "How effective are the Dairy Program's import activities in contributing to the safety of imported cheese made from pasteurized milk sold in Canada?".

3.0 Description of Dairy Program Activities Related to Imported Cheese Made from Pasteurized Milk

3.1 Legislation, Policies and Standards

6. The mandate of CFIA's Dairy Program is to ensure that dairy products leaving federally-inspected establishments or being imported into Canada are safe, wholesome, accurately labelled to avoid misleading consumers, and are eligible to be traded inter-provincially or internationally. The overall objective of the Dairy Program is to ensure that dairy products meet the requirements of Canadian legislation pertaining to dairy products such as the Food and Drug Act and Regulations and the Dairy Products Regulations (Canada Agricultural Products Act - Dairy Products), as outlined in the Program's reference manual (Dairy Product Inspection Manual, chapter 16.3).

7. The primary monitoring activity for imported dairy products is product inspection, although inspection is not obligatory for dairy products. In accordance with workplans, CFIA may inspect imported dairy products to ensure that minimum requirements are met, that is, that they:

meet minimum grade or standard (sampling for microbiological and chemical compliance);
are correctly labelled; and
are accompanied by an Import Declaration that states that the dairy product meets the health and safety requirements prescribed in the Food and Drugs Act and Regulations and the Dairy Products Regulations.

Any non-compliant product is detained until it meets regulatory requirements, or it may be ordered out of the country or destroyed.

3.2 Program Description

8. In October 2000, CFIA began an initiative to enhance and guide the integration of the various import control systems (ICS) within the Agency, including the Dairy Program's import control system. The Dairy Program (as part of the Food of Animal Origin Division) plans to develop an import control system that will be consistent with the guiding principles of CFIA's Import Policy.

9. The Agency is divided along the functional lines of Programs and Operations. Programs Branch has the primary responsibility for program design while Operations Branch is responsible for program delivery. The Dairy Program's import activities are delivered in four CFIA Operational Areas: Western, Ontario, Quebec and Atlantic. The Dairy Program Network Specialists provide a link between Programs and Operations Branches in the Areas.

10. The Dairy Program depends on the cooperation of its internal and external partners to implement its import requirements. The CFIA has established three regional Import Service Centres (ISC): Eastern Region - served from Montreal; Central Region - served from Toronto; and Western Region - served from Vancouver. These Import Service Centres process import request documentation/data (e.g. Import Declarations) sent by the importing community across Canada. The Import Declarations (ID) are reviewed initially by CFIA Import Service Centre staff for completeness and accuracy of all accompanying documents and for the collection of fees, before recommending release for entry to the Canada Border Services Agency (CBSA). Once reviewed and approved, the IDs are then faxed back to the broker or importer, who then submits the release package to CBSA. In addition, Dairy Program Inspectors also review all IDs related to dairy products to track imported products and identify which products will be sampled and/or inspected to fulfill workplan requirements.

11. Inspectors conduct product inspections or sample products for random monitoring, targeted surveillance or compliance as deemed necessary. Samples may be submitted for microbiological, chemical or composition analysis. Tracking the compliance of imported cheeses and their associated manufacturers and importers is done and maintained by Areas in databases that each Area has designed. Follow-up activities are conducted when inspection results are non-satisfactory and in response to consumer complaints on imported products. Inspections can also result in enforcement activities such as product detention and recall. Product inspection or sampling is currently the main import control activity, however the Dairy Program has plans for a new activity, Good Importing Practices Inspections. These inspections will assess the importer's ability to ensure the safety and compliance of the dairy products they import.

3.3 Dairy Program Logic Model Related to Imported Cheese Made from Pasteurized Milk

12. A program logic model was developed in collaboration with CFIA Dairy Program staff (see Appendix 1) . The Dairy Program logic model related to imported cheese made from pasteurized milk is a visual diagram that outlines:

activities (how the program is carried out);
outputs (what is produced by these activities); and
outcomes (what does the program want to achieve and why).

The logic model activities and outputs should be related to the outcomes. A written description of each component of the logic model can be found in Appendix 2.

4.0 Observations, Conclusions and Recommendations

13. The Dairy Program evaluation framework contains 11 evaluation questions and 29 indicators. In order to provide a concise report, the evaluation questions (Appendix 3) have been consolidated into four areas of reporting as follows.

Rationale/Relevance
- Are the Dairy Program's activities related to imported cheese made from pasteurized milk needed to protect the health of Canadians?
Program Design and Management
- To what extent are the strategies and activities of the Dairy Program (related to imported cheese made from pasteurized milk) based on science and risk management?
- To what extent are the objectives of the Dairy Program's import activities clearly articulated and reflected in the program design?
- To what extent does the Dairy Program's import activities integrate appropriate CFIA import control policy approaches that would enhance the program design?
- To what extent does the Dairy Program review import activities in order to enhance the design and management of the program to address food safety issues?
Program Delivery
- To what extent is the implementation of the Dairy Program's import activities (related to imported cheese made from pasteurized milk) consistent with the program design and its intended food safety goals?
- How effective is the Dairy Program in attaining compliance of imported cheeses (made from pasteurized milk) with the food safety related regulations / policies?
- How effective is the Dairy Program in managing food safety emergencies and incidents related to imported cheese made from pasteurized milk?
Stakeholder Engagement
- To what extent is the Dairy Program facilitating delivery of its import requirements by CFIA partners and other government partners?
- To what extent and how has the Dairy Program's import activities helped industry assume greater responsibility / commitment for the safety of imported cheese made from pasteurized milk?

4.1 Rationale/Relevance of the Dairy Program's Import Activities

14. This section addresses the evaluation question "Are the Dairy Program's activities related to imported cheese made from pasteurized milk needed to protect the health of Canadians?" The evaluation team examined the rationale and relevance of the Dairy Program's import activities for cheese made from pasteurized milk based on the following indicators:

legislative and policy base;
a potential food safety risk;
food safety hazards associated with these products; and
food safety related incidents, illnesses or outbreaks.

15. The Dairy Program's activities related to imported cheese made from pasteurized milk are consistent with the requirements of Canadian Acts and Regulations (e.g. the Canadian Food Inspection Agency Act and the Dairy Product Regulations, the Food and Drug Act and Regulations and the Consumer Packaging and Labelling Act). These activities are also in accordance with the Agency's food safety priorities as set out in the 2003-2004 CFIA Food Safety Business Line Workplan and the 2002-2003 and 2003-2004 CFIA Reports to Parliament on Plans and Priorities. The key results as outlined in the Report on Plans and Priorities include:

food meets domestic and trading partner requirements;
food safety emergencies and incidents are effectively managed;
industry complies with regulations; and
stakeholders understand and are committed to regulations and policies.

16. A literature review and a review of the annual CFIA Food Science Committee Meeting minutes and Health Risk Assessments conducted by Health Canada confirmed that imported cheese made from pasteurized milk is potentially susceptible to conditions and circumstances that may result in a food safety risk. Environmental scan and food safety related outbreak data confirm that food safety hazards have been and continue to be associated with cheese made from pasteurized milk²^, ³^, ⁴ .

17. The evaluation team, based on the above information, concludes that the Dairy Program's activities related to imported cheese made from pasteurized milk are relevant and there is an ongoing need for the Dairy Program to design and implement activities to ensure that legislative and policy requirements are met and the health of Canadians is protected.

4.2 Design and Management of the Dairy Program's Import Activities

18. This section addresses the following evaluation questions:

To what extent are the objectives of the Dairy Program's import activities clearly articulated and reflected in the program design?
To what extent are the strategies and activities of the Dairy Program (related to imported cheese made from pasteurized milk) based on science and risk management?
To what extent does the Dairy Program's import activities integrate appropriate CFIA Import Control Policy approaches that would enhance the program design?
To what extent does the Dairy Program review import activities in order to enhance the design and management of the Program to address food safety issues?

4.2.1 Dairy Program Objectives

19. The evaluation team reviewed key Dairy Program documents (e.g. inspection manuals and workplans) and interviewed CFIA staff to determine if the Dairy Program's objectives are:

clearly written, available and understandable; and
reflected in the Program activities and outcomes.

20. There is one objective for the Dairy Program's import activities available in the draft Chapter 16 of the Dairy Product Inspection Manual, "To ensure that imported dairy products meet the requirements of the Acts, Regulations and policies as indicated in Section 16.3". Section 16.3 of the Chapter lists five Acts and Regulations that apply to dairy import activities. This objective was understood and clear to CFIA staff interviewed during the evaluation.

21. While this objective is consistent with the 2003-2004 National Workplan objective (key result) that the industry comply with Regulations, it puts the onus on CFIA for the safety of imported dairy products. During a review of the 2003-2004 CFIA National WorkPlan, the CFIA Food Safety Business Line, and the Dairy Program Logic Model (see Appendix 1), it was noted that these documents contain outcomes that can be considered as more specific objectives for the Dairy Program's import activities than the single objective cited above. Consolidating these import related outcomes or objectives, and clearly documenting them in the Program manual, could enhance the guidance provided to staff and serve as a starting point for designing performance indicators.

4.2.2 Program Design

22. This section addresses the evaluation question "To what extent are the strategies and activities of the Dairy Program (related to imported cheese made from pasteurized milk) based on science and risk management?" The evaluation team reviewed the design of the Dairy Program's import activities for cheese made from pasteurized milk based on the following indicators:

program design promotes compliance, identifies non-compliance and enforces compliance when appropriate;
program design addresses identified hazards based on risk; and
an effective process to manage food safety risk is in place.

23. As defined by its objective (see section 4.2.1), the Dairy Program, as it relates to import activities, is an enforcement program, that is, a program dedicated to achieving compliance with legislative requirements. Enforcement is the set of actions taken to achieve compliance within the regulated community and to correct or halt situations that might endanger public health. In general, an effective enforcement program involves several components⁵ that may include:

knowing who is subject to the requirements and setting program priorities;
promoting compliance in the regulated community;
monitoring compliance;
responding to violations; and
evaluating the success of the program.

The evaluation team examined the program design contained in the key documents such as manuals, workplans, policies and reports, and conducted interviews with CFIA staff to identify the design elements contained in the Dairy Program's import activities related to imported cheese made from pasteurized milk.

24. The Dairy Program is designed to identify non-compliance and enforce compliance through a variety of tools which range from product sampling to refusal of entry into Canada or destruction of the unsatisfactory product. The Dairy Product Inspection Manual is the key reference for the use of these tools by inspection staff. The evaluation team found that the Dairy Product Inspection Manual is not up-to-date with revised enforcement procedures and guidelines (such as CFIA's Food Emergency Response Manual, the reference to be used by inspectors and managers in food emergency situations).

25. With respect to the design element of compliance, the Dairy Program is designed to promote compliance through enforcement actions taken in response to a violation. Compliance can also be facilitated by engaging importers or their associations through educational activities and the sharing of information on regulatory matters. Consulting importers and/or their associations is a Dairy Program activity identified in the Logic Model (see Appendix 1) that might be better utilized on a regular basis. The planned activity, Good Importing Practices Inspections, may be another means of promoting compliance in the future. This design element might contribute to the Dairy Program's effectiveness in achieving their goals:

stakeholders understand and are committed to regulations and policies;
importers assume greater responsibility for the safety of imported cheese; and
imported cheese made from pasteurized milk complies with regulations and policies.

Recommendation:

The Dairy Program may want to consider and implement initiatives designed to regularly engage importers or their associations to promote compliance.

4.2.3 Science and Risk-Based Management

Science-based Approach

26. The Dairy Program, as identified in the Logic Model (see Appendix 1), collects information to identify emerging food safety issues and produce trend analysis reports for use in program design and management. This activity is linked to the immediate outcome "effective science-based programming". A science-based approach can be defined as the collection, analysis and use of scientific information for the design and management of an effective program. The evaluation team evaluated key CFIA documents and conducted interviews with Dairy Program staff using structured interview guides to determine whether a science-based approach is in place.

27. The scientific approach used in the design of the Dairy Program's import activities is linked to the approach used by the Agency's Science Committee. The hazards identified through the Science Committee, such as Listeria in raw and pasteurized soft and semi-soft cheese, are well documented. However, the Dairy Program's contribution to the Science Committee process is limited as the hazards identified by the Dairy Program are not well documented and are communicated to the Science Committee informally. Dairy Program staff indicated that they are not satisfied with the current system's (Resource Management System) ability to collect and report scientific information from product inspection activities for use in program design and management.

Recommendation:

In order to enhance their science-based approach, the Dairy Program should centrally collect information from environmental scans, product monitoring and surveillance or other sources.

Risk-Based Management

28. In the Dairy Program Logic Model, "Dairy Program resources are focussed on imported cheese of highest risk" is an immediate outcome of their design, management and delivery activities. The evaluation team visited three Areas (Quebec, Ontario and BC), reviewed key Dairy Program documents, interviewed Dairy Program staff and observed the processes in place to manage food safety risks associated with imported cheese made from pasteurized milk.

29. The Dairy Program currently monitors imported cheese in order to establish a product compliance history as outlined in draft Chapter 16 of the Dairy Product Inspection Manual. In order to focus on imported cheeses of highest risk, the guideline for selecting imported cheeses for analysis should also take into consideration risk priority, in addition to the development of a compliance history. For example, the high risk rating assigned to the presence of Listeria monocytogenes in cheese by the Science Committee could be better reflected in the Dairy Program sampling guidelines.

30. Additionally, imported cheeses made from pasteurized milk that were previously identified as non-compliant should be considered as higher risk. The assessment team observed that imported cheeses previously identified as non-compliant are addressed through surveillance activities. However it was noted that surveillance procedures differ from Area to Area and are not coordinated between Areas because they are implemented and administered independently by each Area. Thus each Area may not be aware of, nor monitor, import shipments for cheeses previously identified as non-compliant in other Areas.

Recommendation: (see also 4.3.2 Recommendation)

The Dairy Program should design and implement a surveillance system to ensure that products previously identified as non-compliant are monitored on a national basis and to better target (risk- based) product inspections.

4.2.4 Integration of the CFIA Import Control Policy

31. In early 2002, CFIA finalized an Agency-wide Import Control Policy providing each food safety Program with a framework for strengthening their import control approach while enhancing consistency and effectiveness in managing imported foods. The Dairy Program identified the following immediate outcome in their Logic Model: "Dairy import program is consistent with CFIA's Import Control Policy".

32. The evaluation team conducted document reviews and interviews with Dairy Program staff to determine if the Dairy Program:

implements the guiding principles of the Import Control Policy;
developed a risk analysis framework for their import control system; and
used risk analysis to integrate elements of the Import Control Policy framework.

33. The Dairy Program participated in the Food of Animal Origin Division meeting to discuss the implementation of the Import Control Policy. The meeting's action plan generally reflects the guiding principles (see Appendix 4) of the policy, with the exception that a formal risk analysis framework was not developed as the Import Control Policy requires. The development of a risk analysis framework and a specific action plan for its own import control program initiatives could enhance the Dairy Program's design and program development.

34. The Import Control Policy identifies six elements (see Appendix 4) which constitute the import control systems for all Agency programs. These are used to guide each program in its development and implementation of enhancements to its import control system. Based on the review of program documents and interviews, it was noted that of the six elements of an import control system listed in the policy, the following are part of the Dairy Program's import activities related to imported cheeses made from pasteurized milk:

Point of Entry Control (e.g. Import Declaration reviews and participating in Import Blitzes at the point of entry);
Tracking and Informatics (e.g. updating dairy import requirements in the Automated Import Reference System and tracking Import Declarations by Area until the Dairy Program is included in the Agency-wide Import Control and Tracking System); and
Inspection Program (e.g. product inspection sampling).

35. The elements of Foreign Equivalency/Certification, New Technologies and Importer Quality Management Systems are currently not part of the Dairy Program's import control system for cheese made from pasteurized milk, but need to be considered. Although some of these elements may be included in the Food of Animal Origin Division's list of priorities, they may not be the same as the Dairy Program's own priorities with respect to import controls. The Import Control Policy recommends the use of a recognized risk management approach when considering possible import control system enhancements.

36. Health Canada supports a Hazard Analysis Critical Control Point (HACCP) approach to the assurance of safety of imported foods. The safety of products should be assured by the application and implementation of HACCP principles at the source in the country of origin. However, when it is difficult to determine if HACCP principles were correctly applied and implemented, inspection and analysis of imported lots may be indicated⁶.

37. In the context of an integrated risk management approach, section 13.2 of the Import Control Policy notes that each Program import control system should have a manager, and each Program import control system should be fully documented. This position might address the need to create a Dairy Program action plan; develop a formal risk analysis framework; identify and document clearly the roles and responsibilities of staff, internal and external partners and other stakeholders; and perhaps participate in developing and managing information systems related to imports.

Recommendation:

The Dairy Program should consider appointing a manager to oversee its import control system in accordance with the Import Control Policy.

4.2.5 Program Review

38. This section addresses the evaluation question "To what extent does the Dairy Program review import activities in order to enhance the design and management of the program to address food safety issues?" The evaluation team assessed the Dairy Program's review or audit program for import activities related to cheese made from pasteurized milk, based on the following indicators:

periodic reviews and performance measurement;
revisions are used to enhance the Dairy Program's import activities; and
the Program monitors emerging issues and incorporates these issues into program design and delivery.

39. Evaluating the success in reaching its objectives is an important aspect for any program. The Dairy Program workplans include activities such as verification of program delivery in each Area and evaluation of program effectiveness by Headquarters. The evaluation team, through file reviews and interviews, concluded that Dairy Program verifications take place at the Area and Regional levels, however import activities have yet to be reviewed and the Dairy Program does not yet conduct performance measurement. For these reasons, the assessment team was not able to evaluate the enhancement of the import program through program revisions. Although not necessarily a result of periodic reviews or performance measurement, the Dairy Product Inspection Manual's draft Chapter 16 Import Activities is evidence of an initiative to enhance the Dairy Program's import activities and will be fully implemented after Dairy Program staff have been trained on the requirements of the import activities such as Good Importing Practices inspections.

Recommendation:

The Dairy Program should periodically review the effectiveness of their import activities in order to enhance Program design and delivery, and take steps to improve their capacity to collect and analyse information so that systematic performance measurements of their activities can be carried out.

4.3 Program Delivery

40. This section of the report addresses the following evaluation questions:

To what extent is the implementation of the Dairy Program's import activities (with regard to imported cheese made from pasteurized milk) consistent with the program design and its intended food safety goals?
How effective is the Dairy Program in attaining compliance of imported cheeses (made from pasteurized milk) with the food safety related regulations / policies?
How effective is the Dairy Program in managing food safety emergencies and incidents related to imported cheese made from pasteurized milk?

4.3.1 Delivery of Inspection Activities as Designed

41. The evaluation team assessed the consistency of the Dairy Program's implementation of import activities (for cheese made from pasteurized milk) with its program design and its intended food safety goals. This was done based on the following indicators:

inspection activities are conducted as designed;
follow-up and enforcement activities are conducted as designed;
training activities; and
adjustments to program delivery are made as needed and are effective.

42. The evaluation team examined the extent to which the Dairy Program implemented its import inspection activities with regard to imported cheese made from pasteurized milk and examined if the implementation was consistent with program design and the Dairy Program's immediate outcome, "Imported cheeses made from pasteurized milk comply with regulations and policies", as identified in the Logic Model (see Appendix 1). The Dairy Program's workplans, databases, methods of monitoring delivery and summary reports of laboratory analysis were reviewed. Where possible, the assessment team conducted trend analysis. Three regional offices and Headquarters were visited and CFIA Managers and Inspectors were interviewed.

43. The Dairy Program uses the Resource Management System (RMS) tracking and reporting tool to track delivery of its workplans. Through a review and analysis of RMS reports, it was noted that RMS reports are not a reliable tool to track delivery of the Dairy Program sampling workplans because RMS tracks the work conducted by available resources and not the fulfilment of the sampling workplans. In addition, the RMS-reported sampling numbers do not provide a description of the workplans for which samples are taken (e.g. microbiological workplan, chemical residues workplan), but rather lumps the samples taken for all workplans into one total. The RMS sample numbers includes those samples taken by inspectors on behalf of importers who subsequently have these imported dairy products analysed at their expense by an accredited laboratory.

Microbiological Sampling Workplan

44. The evaluation team received data from Headquarters and from the Areas summarizing the delivery of microbiological sampling plan D-210 (Imported Dairy Products) for 2001-2002 and 2002-2003 as follows:

Exhibit 1

Microbiological Sampling Summary from Laboratory Services Tracking System ^*

Areas	2001-2002			2002-2003
	Planned	Delivered	%	Planned	Delivered	%
Western Ontario Quebec Atlantic	150 300 400 25	74 83 357 16	49.3% 27.6% 89.3% 64%	75 300 400 25	48 70 263 20	64% 22.3% 65.8% 80%

* Source - Dairy Summary from LSTS.xls

Exhibit 2

Microbiological Plans Performance Summary Tables for 2001-2002 and 2002-2003.^**

Areas	2001-2002			2002-2003
	Plan.	Del.	%	Plan.	Del.	%
Western Ontario Quebec Atlantic	150 300 400 25	75 87 370 16	50% 30% 92.5% 64%	75 300 400 25	N/A 57 173 14	_ 19% 43.3% 56%

N/A - Not Available

** Source -National Micro Sampling Summary for 2001-02 - dairy import only.xls ; Dairy - Micro sampling result summary - 2002-03 - Atlantic.xls; Dairy - Micro sampling result summary - 2002-03 - Ontario.xls; Dairy - Micro sampling result summary - 2002-03 - Quebec.xls; Dairy - Micro sampling result summary - 2002-03 - West.xls

45. The Dairy Program data on the delivery of the Program's activities, received from different Area and Headquarters sources, did not always correlate between sources. As a result, analysis of the fulfilment of workplans and any trend analysis was not reliable or in some cases feasible. This is illustrated in the following examples from the Quebec Area and the Toronto Region. Microbiological sampling data provided by the Dairy Regional Technical Specialist in the Quebec Area indicates that 113 cheeses made from pasteurized milk and 337 cheeses made from raw milk were sampled in 2002-2003 for a total of 450 samples. These 450 samples include samples taken on behalf of importers for analysis by accredited laboratories at the importers' expense and are not exclusively related to sampling project D210. During the visit to the Toronto Region, the microbiological sampling 2002-2003 logbook was reviewed and it indicated that the Toronto Region was assigned 284 of the 300 total samples assigned to the Ontario Area. According to the logbook, 50 samples were picked-up, however, when compared to the data received from Headquarters, the reported samples delivered by the Ontario Area was 70.

46. All of the data obtained from Headquarters, above, does not separate the number of samples dedicated to imported cheese made from pasteurized milk versus imported cheese made from raw milk. Only the Quebec Area tracking system distinguishes samples of imported cheese made from pasteurized milk from imported cheese made from raw milk. The ability to collect data which identifies the imported cheese as originating from either raw or pasteurized milk cheese is important when setting priorities based on risk and conducting data analysis.

Chemical Residues Sampling Workplan

47. Delivery data for the Chemical Residues Sampling Workplan D-CHE-02 were assessed in the following documents:

National Chemical Residues Analytical Results (Reports 2001-2002 and 2002-2003 from Headquarters): From these reports the extent of workplan delivery is unclear because the number of samples actually delivered is not identified and, as a result, a comparison to the number of samples originally requested cannot be made.
Inspectors' sampling logbooks and chemical residues sampling workplans available during Ontario and Western Area visits: 11 (8.5%) of 129 samples assigned to the Toronto Region were sampled in 2002-2003. In the Western Area, 2 (6%) of 36 samples assigned to the BC Coastal Region were sampled in 2002-2003. No data was available for the Quebec Area.

48. The data presented in this section illustrates that workplans are not fulfilled in accordance to the workplan design which means that the Dairy Program's intended food safety monitoring goals will not be fulfilled.

Recommendation:

The Dairy Program should increase its capacity to collect data to monitor workplan delivery.

49. The 2002-2003 Resource Management System, Planned vs Delivery Report for All Areas - All Quarters Summation, indicates that 0.736 Full Time Equivalents are available to deliver the 1282 planned samples, in addition to other tasks that include follow-up and enforcement actions as well as the review of close to 4000 Import Declarations annually (based on a 2002-2003 RMS report). RMS reports indicate reviewing import declarations is the activity that takes up most of an Inspector's time. As illustrated in Exhibit 3 the number of samples planned may not match the resources assigned.

Exhibit 3

Analysis of 2002-03 RMS Sampling Plan Requirement versus Resource Allocation

Planned Number of Samples^*	Planned Number of Full Time Equivalents (FTE)	Number of Available Work Hours	Time Available per Sample
1282 Samples¹	0.736 FTE¹	37.5 h / week X 48 weeks X 0.736 FTE = 1324.8 hours	Approximately 1 hour/sample²

* Includes Micro, Chemical Residues & Composition Samples
1 Based on a 2002-2003 RMS yearly report
2 This includes identifying the product to sample, preparations prior to sampling, driving, shipping or delivery to the laboratory and the reporting aspects.

50. It was noted that the Dairy Program needs to improve its planning and monitoring of workplan delivery. Workplans are designed based on science by the Programs, while delivery by Operations is based on Area/Regional available resources and priorities. At the Area or Regional level workplans may not be implemented as originally designed because of other high priority activities (e.g. food safety emergencies) or the lack of available resources. As such, workplans are not implemented according to design. More formal collaboration between Programs and Operations to ensure that the Dairy import program design and workplans are both science-based and feasible would enhance program delivery.

Recommendation:

There should be more formal collaboration between Programs and Operations to ensure that the Dairy Import Program design and workplans are both science-based and feasible, that their implementation is monitored and that continuous improvements are made as required.

4.3.2 Follow-up and Enforcement Activities

51. The Dairy Program's import follow-up and enforcement activities were reviewed to determine whether they are implemented as designed and if they contribute to achieving the food safety goals of the Logic Model. The evaluation team found that Inspectors have documentation to refer to when implementing compliance and enforcement actions. For example, the Dairy Product Inspection Manual contains enforcement guidelines in Chapters dealing with Sampling Procedures (2), Environmental Sampling (10), Detention and Release (12), and Importing Activities (16).

52. The evaluation team reviewed 20 imported cheese product files related to microbiological or extraneous matter analysis which involved 12 importers. In the files reviewed, which were obtained during visits to the Areas, follow-ups and enforcement activities were implemented in each case. Inspectors may consult Area or Regional Dairy Specialists to ensure appropriate actions are taken. The follow-up actions taken by the Inspectors contribute to the safety of imported cheese sold in Canada by controlling identified hazards and thus limit risk to Canadian consumers.

53. Intensified inspection (surveillance) is an enforcement strategy that requires mandatory sampling, by CFIA Inspectors, of the next ten lots of imported products previously found to be non-compliant (as illustrated in the diagram below). The next ten lots of this product imported by any importer in the Region/Area, will be sampled by the CFIA Inspector. The analysis of eight of these lots at an accredited laboratory is conducted at the expense of the importer. Copies of all results of analysis from accredited laboratories must be sent directly to the CFIA office in addition to the importer. To return to normal sampling frequency level, ten consecutive lots must be satisfactory.

Exhibit 4

Intensified Inspection Strategy

54. The evaluation team observed that each Area has its own (and different) tracking system in place for products under intensified inspection (surveillance). Inspectors are not always made aware of high risk products under surveillance in other Areas. Nothing precludes products under surveillance in one Area from being imported into another Area.

55. Few unsatisfactory samples for chemical residues have been identified since 2001-2002, however the chemical residues sampling workplans (D-CHE-02) have been delivered at a low rate. In 2001-2002 four samples were found to be unsatisfactory for presence of mycotoxins (1 Greece, 2 Italy, 1 Spain). One sample was found to be unsatisfactory for presence of arsenic in 2002-2003. No follow-up or enforcement actions have been implemented by the Dairy Program for these unsatisfactory chemical residue samples. (The analysis of chemical residue samples may occur long after the time of sampling, which may preclude follow-up by Inspectors.)

Recommendation: (see also 4.2.3 Recommendation)

4.3.3 Evidence of Training Activities

56. The evaluation team reviewed training related documents and conducted interviews with CFIA staff and determined that the Dairy Program has training activities planned for their Inspectors. Inspectors may regulate both the domestic industry and imports, and other commodities, in addition to dairy products. Accordingly, their training includes basic courses such Hazard Analysis Critical Control Points, recall procedures, and allergen training. Currently, Inspectors who regulate dairy imports, including cheese made from pasteurized milk, also obtain their training through on-the-job shadowing of an experienced Inspector. A Dairy Program Certification Protocol is under development and should ensure that all Inspectors receive required training and have equivalent understanding of procedures to implement. The evaluation team noted that phosphatase testing for imported dairy products was not understood or used in an equivalent way in all Areas. This inspection procedure should be addressed in the Dairy Program's training plans. Training on the Dairy Product Inspection Manual's draft Chapter 16 -- Importing Activities -- is planned for the future.

4.3.4 Evidence of Adjustments to Program Delivery

57. Delivery of workplan activities is affected by emergencies that Regional Inspection Managers may address by redirecting resources to tasks where the risk is higher. Other workplan activities may not be accomplished because of lack of resources to address both. The Dairy Program does not have an effective national system in place to monitor the delivery of dairy import workplans, assess the need for national adjustments to workplans, and implement them in order to optimize the delivery of workplans. Such a system can better integrate the activities conducted in the Areas/Regions with the planning responsibilities at Headquarters, and form an integral part of a performance measurement system.

4.3.5 Trends in the Rate of Compliance of Imported Cheese Made from Pasteurized Milk and Rate and Trend of Compliance for Products Previously Identified as Non-compliant

58. This section addresses the evaluation question "How effective is the Dairy Program in attaining compliance of imported cheeses (made from pasteurized milk) with food safety related regulations / policies?". The evaluation team assessed the Dairy Program's compliance data for imported cheese made from pasteurized milk, based on the following indicators:

trends in the rate of compliance; and
rates and trends in the rate of compliance for previously non-compliant products.

59. Measuring the success of an enforcement program is not easy. Many parameters can be used to evaluate program effectiveness. Compliance rates are one of the best overall measures of enforcement success. High compliance rates are the ultimate goal of most enforcement programs. However, this measure is dependant on the reliability of the data obtained to measure the compliance rate:

a lower compliance rate may mean that the Program is doing a good job of detecting violations, or that importers are not adhering to food safety requirements; and
a high compliance rate can be misleading if the inspection activities are not implemented as designed or according to established frequency, or may be indicative of adherence of importers to food safety requirements.

60. A trend analysis in the rate of compliance for imported cheese made from pasteurized milk was difficult to conduct because:

the Dairy Program does not deliver its workplans as established by Headquarters from year to year and from Area to Area;
there is no centralized or common compliance tracking system used by Areas and the data used to determine compliance rate varies from Area to Area.

Furthermore, national summary data on microbiological compliance of imported cheese made from pasteurized milk are not systematically produced. As a result, a valid rate of compliance trend analysis nationally and between Areas for imported cheese could not be determined.

61. Following the identification of non-compliant imported cheese made from pasteurized milk, importers have the option to cease import of those products or to take measures to ensure that subsequent shipments are in compliance with the food safety regulatory requirements. The collaboration of importers in taking appropriate actions to ensure that subsequent import shipments (previously identified as non-compliant) are in compliance with the food safety related regulations and policies may be another indicator of the success of an enforcement program.

62. The evaluation team assessed the data available for a trend analysis in the rate of compliance for imported cheese made from pasteurized milk previously identified as non-compliant. Data limitations made it difficult to determine a reliable rate and trend of compliance. The Laboratory Sample Tracking System (LSTS) follow-up summary reports were examined and found to be incomplete when compared to the information obtained in the Areas. For example, none of the surveillance samples recorded in the Quebec Area were recorded in those reports.

Recommendation:

The Dairy Program should collect reliable data to track the compliance status of imported cheeses and analyse trends in compliance and disseminate this information to staff for use in program design, delivery and reporting.

4.3.6 Effective Management of Food Safety Emergencies and Incidents

63. This section addresses the evaluation question "How effective is the Dairy Program in managing food safety emergencies and incidents related to imported cheese made from pasteurized milk?" The evaluation team reviewed key food safety emergency documents and conducted interviews with CFIA staff. The following indicators were used:

facilitation with internal and other federal government partners; and
guidance and enhancement of importers ability to manage food safety emergencies.

64. The management of emergencies and incidents related to imported cheese made from pasteurized milk may involve different groups, including the Dairy Program, the Office of Food Safety & Recall (OFSR), Dairy operational staff, the Bureau of Food Safety & Consumer Protection, the National Import Sector and provincial partners, such as the Ministère de l'agriculture, pêcheries et alimentation du Québec (MAPAQ), depending on the risk management options implemented.

65. Clearly defining the roles of the individuals involved in the management of food safety emergencies and incidents provides a basis for efficiency and cooperation. For example, the role of the Dairy Program is to gather information at the request of the OFSR. The evaluation team found that each Area, because of their unique situation or structure, has its own specific set of roles and responsibilities in the management of food safety emergencies or incidents. For example, MAPAQ does the investigation at the retail level and does the recall checks in Quebec; in the Toronto and BC Coastal Regions, Food Safety Inspectors, not Dairy Program inspection staff, take the lead in food safety emergencies and incidents. Details about the roles and responsibilities specific to each Area are located in the OFSR procedural manuals, developed by the Area OFSR.

66. Document reviews and interviews indicate that the roles, responsibilities and procedures to follow in the management of food safety emergencies and incidents related to imported cheese made from pasteurized milk are well documented in the Area OFSR manuals and understood by the Dairy operational staff. The Dairy Program may want to assess the value of including or making reference to the roles, responsibilities and procedures documented in the OFSR manual, in their Program manuals, as mentioned earlier in this report (see section 4.2.2).

67. Activities connected to the promotion of compliance provides the Dairy Program with an opportunity to educate and provide importers with guidance pertaining to managing food safety emergencies and incidents including tracing suspect products. Once the Dairy Program has provided importers with this information, importers are aware of what is required and are responsible for establishing trace-back, consumer complaint and recall systems. If a food safety emergency should occur, importers with established trace-back, consumer complaint and recall systems are more likely to implement an effective emergency management strategy. Successfully managed food safety emergencies or incidents is a desired outcome for the Dairy Program (see Logic Model Appendix 1) and industry.

68. The evaluation team found through document reviews and interviews of CFIA staff that the Dairy Program provides guidance to industry through various activities. For example, guidance may be provided by Inspectors on a one-on-one basis during visits to importer premises for product inspection or follow-up purposes. Presentations to import associations, such as the Good Importing Practices presentation (in which recall is included) to the International Cheese Council of Canada, provide an excellent opportunity to promote compliance. Also, importers can access the CFIA website for more information (e.g. importers may consult the "Food Recalls - Importers Guide" web section). Implementation of the new activity, Good Importing Practices inspections, is an opportunity for the inspection staff to provide one-on-one guidance to enhance the industry's ability to manage food safety emergencies and incidents.

69. To determine the extent to which the Dairy Program's guidance has enhanced importers' ability to effectively trace products and manage food safety emergencies and incidents related to imported cheese made from pasteurized milk, the evaluation team examined and reviewed all eight recall files on imported cheese made from pasteurized milk that occurred between 2001-2004. All recall files were examined for implemented recall activities and overall effectiveness based on criteria described in the OFSR manual of procedures. Three of eight recalls were assessed by CFIA staff as effective in removing the product from the marketplace, another three of eight recalls were rated by CFIA staff as ineffective because either the product was still on the market or because of process deviations (e.g. Notice of Recall was not received by all consignees within the established time frames), and two of eight recall files contained no summary of effectiveness checks and/or no follow-up report. The evaluation team concludes from this analysis that there is a continuing need to provide importers guidance to enhance their ability to manage food safety emergencies.

Recommendation:

The Dairy Program should be more proactive in providing guidance (i.e. give guidance before problems occur) to enhance importers' ability to effectively trace products and manage food safety emergencies and incidents related to imported cheese made from pasteurized milk.

4.4 Stakeholder Engagement

70. This section of the report addresses the following evaluation questions:

To what extent is the Dairy Program facilitating delivery of its import requirements by CFIA partners and other federal government partners?
To what extent and how has the Dairy Program's import activities helped industry assume greater responsibility / commitment for the safety of imported cheese made from pasteurized milk?

4.4.1 Facilitating the Delivery of Dairy Program Import Requirements

71. The Dairy Program's import control strategy involves internal CFIA partners and external government partners. In order to import cheese into Canada the cheese importing community deals with the Canada Border Services Agency (customs), CFIA's Import Service Centres (document review and enforcement), and International Trade Canada (import permits, tariffs) in addition to the Dairy Program (inspection). Constructive cooperation can strengthen the Dairy Program's import control system through increased efficiency and by identifying gaps in regulatory programs.

72. The evaluation team reviewed and analysed documents and conducted interviews and found that, at the Area and Regional level, the Dairy Program collaborates with the Import Service Centres through individual initiatives to improve or maintain the delivery of Dairy Program requirements. The main interface with other federal government partners such as the Canada Border Services Agency and International Trade Canada, is the Import Section at CFIA Headquarters and the Import Coordinator in the Areas, who reports to the Import Sector. The level of effectiveness of the Dairy Program's partners in delivering its import requirements is not monitored but could form a part of routine program reviews or verifications in the future. This might indicate if the level of collaboration between the Dairy Program and its partners is sufficient.

73. A key element in any strategy is defining the roles and responsibilities of those who either deliver, or facilitate the delivery of, the Dairy Program's requirements in order to establish clear lines of communication, mechanisms and procedures for handling areas of mutual interest. Draft Chapter 16 of the Dairy Product Inspection Manual would benefit from a more detailed description of the roles and responsibilities of all of the partners who play a role in facilitating the delivery of the Dairy Program's import requirements, including the Import Sector and the Import Coordinators ⁷.

Recommendation:

The Dairy Program should clearly document the roles and responsibilities of its CFIA and other government partners involved in facilitating the delivery of the Dairy Program's import activities and occasionally monitor the effectiveness of its partners' delivery of the Dairy Program's import control system.

4.4.2 Industry Responsibility / Commitment for the Safety of Imported Cheese

74. The evaluation team used key document and report reviews, CFIA staff interviews and an importer case study using structured interview questions to evaluate the following indicators:

communication of Dairy Program requirements to importers; and
importers assuming greater responsibility for the safety of the cheeses imported.

The importer needs to know what steps to take to achieve compliance and must implement the necessary technology to monitor, control, prevent or correct food safety issues and thus assume greater responsibility in this area. A lack of knowledge or technology can be a significant barrier to compliance. This barrier can be removed by providing education, outreach, and technical assistance. These activities can ultimately be linked to the measured progress of the cheese importers in implementing Quality Management Systems.

75. The main methods by which the Dairy Program currently helps or plans to help industry assume their responsibility for food safety are in the draft Chapter 16 of the Dairy Product Inspection Manual and include: Good Importing Practices Inspections (planned), intensified product inspection (implemented but not tracked nationally), new product or variety sampling (currently partially implemented, but planned for full implementation). These activities are to be fully implemented when Inspectors have received training on the draft Chapter 16 of the Dairy Product Inspection Manual.

76. In order to obtain the importers' perspectives on the Dairy Program's import activities as well as other aspects of the Dairy Program, the team conducted a case study of importers. Focussed questions were posed to the importers who volunteered to be interviewed. In a case study situation there is no statistical significance linked to the case study group. This information cannot be extrapolated to a larger population. The importers' responses were kept confidential and were reported in a generic fashion. The evaluation team noted that the level of knowledge of the Dairy Program's import requirements and new initiatives (e.g. Good Importing Practices inspection, intensified product inspection) varied widely between importers who took part in the case study. Also, many case study participants expressed that their confidence in the safety of the cheese they import came from a variety of factors other than their own Quality Management System.

77. The Dairy Program Logic Model identifies "importers assume greater responsibility for the safety of imported cheese" as an immediate outcome. The Dairy Program has yet to develop indicators to track the importers' performance in achieving this outcome, and the evaluation team found little evidence (e.g. inspection compliance and trend analysis reports, manuals, guidelines and workplans) to illustrate the level of responsibility for food safety assumed by importers. In order to measure the Dairy Program's progress towards this immediate outcome, measures of technical assistance provided to importers (e.g. inspections, meetings, or other educational activities), and the corresponding level of implementation of Quality Management Systems by importers, can be useful information to collect.

Recommendation:

The Dairy Program should design a strategy that best enhances importers' understanding and commitment to regulations and policies, and develop performance indicators to measure progress towards their outcomes.

Appendix 1 Dairy Program Logic Model Related to Imported Cheese Made from Pasturized Milk

Appendix 2 Narrative for the Logic Model

A logic model is a diagram that captures information about the main elements of a program being examined and describes in concise terms how the program works. It includes program activities, outputs and outcomes. Components for the Dairy Program Logic Model are described below.

Program Design and Management

CFIA's National Manager Dairy and Honey and the Director of Food of Animal Origin Division have the primary responsibility for the design and management of the Dairy Program's import activities. The Dairy Program collects information to enable routine internal evaluation and reviews and revises its design as necessary. Program revision can also arise as a result of a risk assessment, data obtained through product or establishment inspection or through tracking activities (e.g. trend analysis of chemical residue and microbiological test results, environmental scans identifying new issues). The outputs of these activities are new and revised regulations and policies which are in agreement with international standards and meet the requirements of CODEX. New or revised guidelines, procedures, workplans or sampling plans may also result from these activities.

Program Delivery

The Dairy Program's import activities are delivered in four CFIA Areas: Western, Ontario, Quebec and Atlantic. Three Import Service Centres control those four Areas.

Import Declarations (ID) are reviewed initially by the CFIA Import Service Centre staff for completeness and accuracy of all accompanying documents and for the collection of fees before recommending release for entry to the Canada Border Services Agency. This is a document review, not a technical assessment of product. IDs are also reviewed by Dairy Program Inspectors in order to track importing products and identify products to be sampled and/or inspected to fulfill workplan requirements.

Product Inspection consists of sampling products for random monitoring, or targeted surveillance or compliance. Samples may be submitted for microbiological, chemical or composition analysis. Samples are also selected to assess the compliance of the product's label to mandatory labelling requirements.

Follow-up Activities are conducted when inspection results are non-satisfactory and in response to consumer complaints on imported products. Inspections can also result in enforcement activities such as product detention and recalls.

Establishment Inspection includes visual on-site inspections of the importers' premises, as well as an assessment of either Good Importing Practices and/or Good Manufacturing Practices in the case of federally registered establishments that repackage and distribute imported cheese and may include environmental sampling to verify the effectiveness of sanitation programs and practices.

Import Blitzes are random and targeted inspections of import shipments at land borders, seaports, airport cargos, couriers and post office international mail.

Program Verification is performed by Program Network Specialists by observing activities and conditions in selected areas and assessing the findings for conformity to program requirements.

Compliance Tracking of imported cheese, their suppliers and importers is done systematically and may be maintained in a database.

Partner / Stakeholder Engagement

Another component of the Dairy Program is Partner / Stakeholder Engagement. The Dairy Program works in collaboration with internal and external partners and stakeholders to design, manage and deliver effective dairy import controls. Internal CFIA partners include Enforcement and Investigations Services, Import Service Centres, Animal Health, Office of Food Safety and Recall, Bureau of Food Safety and Consumer Protection, Laboratories Directorate, and International Affairs. Other federal government partners include Health Canada, International Trade Canada, the Canada Border Services Agency, and the Canada Revenue Agency. CFIA provides technical advice to partners and stakeholders and liaises with stakeholders including industry and importer associations such as the Canadian Association of Importers and Exporters. Also, there is on-going participation with international organizations such as the CODEX Alimentarius Commission.

Outcomes

The Dairy Program's desired impacts or successes are reflected in the Logic Model's Immediate (short-term) Outcomes, Strategic (intermediate) Outcomes and Benefit for Canadians (or Ultimate Outcome).

Immediate (Short-term) Outcomes

The Dairy Program's stakeholders understand and are committed to the Dairy Import Regulations and Policies.
Emergencies and incidents related to the Dairy Program are managed in an effective manner.
The Dairy Program is consistent with CFIA's Import Policy.
Imported cheese made from pasteurized milk comply with regulations and policies.
Dairy Program resources are focussed on imported cheese of highest risk.
Dairy importers assume greater responsibility for the safety of imported products.
The Dairy Program is effective and science-based.

Strategic (Intermediate) Outcomes

Through implementation of Dairy import controls and activities, imported cheese are safe and suitable for consumption.
The Dairy Program contributes to public and market confidence in the safety of imported cheese.
Imported and domestic cheese are treated equitably within the regulatory regime.

Benefit for Canadians (Ultimate Outcome)

The Dairy Program contributes to the protection of Canadians from preventable food safety risks related to imported cheese.

Appendix 3 Evaluation Methodology and Design

The evaluation was conducted using the following methodologies:

Program Document Review

This included a review of relevant background documents and information gathered on program results and performance measurement data, which demonstrated program effectiveness and impacts. Other relevant documents from key stakeholders, partners and CFIA were also reviewed.

Literature Review and Environmental Scans

A literature review of both published and unpublished academic and scientific papers was conducted and information analysed to answer particular questions related to the evaluation. This was useful information in addressing topics related to the design and delivery of the Program and the Program's effect on human health.

Key Informant Interviews

The evaluation team interviewed appropriate CFIA managers and staff responsible for the Dairy import activities at Headquarters and the Areas, as well as Health Canada staff, industry and academia for this evaluation. The interviews with key informants provided critical information and perspective on the program.

Trend Analysis

This involved the evaluation team's review of existing CFIA trends analyses on such issues as the compliance rate of imported cheese made from pasteurized milk, and the team conducting a trend analysis exercise to obtain information.

Case Study

In order to obtain the importers' perspectives on the Dairy Program's import activities as well as other aspects of the Dairy Program, the team conducted a case study of importers. Focussed questions were posed to the importers who volunteered to be interviewed. In a case study situation there is no statistical significance linked to the case study group. The case study is meant to obtain detailed information on focussed areas of interest. This information cannot be extrapolated to a larger population.

Review of Databases

This involves the examination of any database information that is available and would benefit the evaluation team in answering the evaluation questions. Some of the CFIA databases that were important to the team included: Incidence Management System database which contains incident and recall information, and the Import Retrieval System. The evaluation team also reviewed information from the disease surveillance database that is maintained by the Public Health Agency.

Eleven evaluation questions were developed and addressed to gather extensive descriptive data on program rationale, program design and management, program delivery and program successes. The evaluation questions and evaluation indicators, previously described in the evaluation framework report, are also listed in the table that follows.

Program Rationale/Relevance

Evaluation Questions	Indicators
1. Are the Dairy Program's activities related to imported cheese made from pasteurized milk needed to protect the health of Canadians?	1.1 Evidence of a legislative and policy base for imported cheese made from pasteurized milk
	1.2 Evidence that food safety risk is identified
	1.3 Evidence that food safety hazards associated with imported cheese made from pasteurized milk exist
	1.4 Evidence of food safety related incidents, illness and outbreaks associated with imported cheese made from pasteurized milk

Program Design & Management

Evaluation Questions	Indicators
2. To what extent are the objectives of the Dairy Program's import activities clearly articulated and reflected in the program design?	2.1 Objectives for the Dairy Program's import activities are clearly written, available and understandable
	2.2 Evidence that objectives are reflected in the Program activities and outcomes
3. To what extent are the strategies and activities of the Dairy Program (related to imported cheese made from pasteurized milk) based on science and risk management?	3.1 Evidence that hazards associated with imported cheese have been identified and the program is designed to address hazards based on risk
	3.2 Evidence of an effective process in place to manage the food safety risk
	3.3 Evidence that the Program is designed to promote compliance, identify non-compliance and enforce compliance when appropriate
4. To what extent does the Dairy Program's import activities integrate appropriate CFIA import control policy approaches that would enhance the Program design?	4.1 Extent to which the Dairy Program implements the guiding principles of the Import Control Policy
	4.2 Evidence of Dairy Program Risk Analysis Framework as per the Import Control Policy
	4.3 Evidence that the integrated Import Control Policy elements are driven by risk analysis
5. To what extent does the Dairy Program review import activities in order to enhance the design and management of the Program to address food safety issues?	5.1 Evidence of periodical reviews and performance measurement
	5.2 Evidence that revisions are used to enhance the Dairy Program's import activities
	5.3 Evidence that the Dairy Program monitors emerging issues and incorporates these issues into program design and delivery

Program Delivery

Evaluation Questions	Indicators
6. To what extent is the implementation of the Dairy Program's import activities (related to imported cheese made from pasteurized milk) consistent with the program design and its intended food safety goals?	6.1 Inspection activities are conducted as designed
	6.2 Follow-up and enforcement activities are conducted as designed
	6.3 Evidence of training activities
	6.4 Evidence that adjustments to program delivery are made as needed and are effective
7. To what extent is the Dairy Program facilitating delivery of its import requirements by CFIA and other federal government partners?	7.1 Evidence of effective collaboration established between the Dairy Program and its partners

Immediate Outcomes

Evaluation Questions	Indicators
8. How effective is the Dairy Program in attaining compliance of imported cheese (made from pasteurized milk) with the food safety related regulations / policies?	8.1 Trends in the rate of compliance of imported cheese made from pasteurized milk
	8.2 Rate and trend of compliance for imported cheese made from pasteurized milk previously identified as non-compliant
9. How effective is the Dairy Program in managing food safety emergencies and incidents related to imported cheese made from pasteurized milk?	9.1 Extent to which the Dairy Program facilitates effective management of incidents, including the work it does with internal and other federal government partners
	9.2 Extent to which the Dairy Program provides guidance and enhances importers' ability to effectively trace products and manage food safety emergencies and incidents related to imported cheese made from pasteurized milk
10. To what extent and how has the Dairy Program's import activities helped industry assume greater responsibility / commitment for the safety of imported cheese made from pasteurized milk?	10.1 Evidence that the Dairy Program's food safety related requirements have been effectively communicated to the cheese import industry
	10.2 Evidence that importers are assuming greater responsibility for the safety of imported cheese made from pasteurized milk

Strategic Outcome

Evaluation Questions	Indicators
11. How effective are the Dairy Program's import activities in contributing to the safety of imported cheese made from pasteurized milk sold in Canada?	11.1 Evidence that the Dairy Program's approach and design is comprehensive at enhancing food safety of imported cheese made from pasteurized milk
	11.2 Extent to which imported cheese are in compliance with food safety regulations
	11.3 Extent to which importers comply with the food safety regulations and related requirements such as Good Importing Practices, Good Manufacturing Practices, when applicable
	11.4 Extent to which imported cheese made from pasteurized milk is implicated in recalls, incidents and food-borne illnesses

Appendix 4 Guiding Principles and Elements of the Import Control Policy

Five Guiding Principles

The approach shall be Agency-wide with a management framework that cuts across individual programs and supports the development of corresponding Agency tools to support staff at all levels to provide a basis for a more informed discussion of Agency risks, the strategic allocation of resources and the monitoring of outcomes.
The approach shall be driven by a risk analysis framework that demonstrates (a) an assessment of risk using an appropriate risk management methodology, (b) a clear assessment of the options for intervention including efficacy, costs and impacts, (c) a clear specification of the intervention required, its risk parameters and other relevant standards and (d) a plan for periodic review of the need for continued intervention.
The approach shall clearly reflect the responsibilities of importers to import food, plants and animals that comply with the Canadian requirements.
The approach shall clearly reflect the accountability of the CFIA to implement the import control policy respecting the vision and the guiding principles.
The approach shall respect Canada's international rights and obligations.

Six Elements of the Import Control Policy Framework

The CFIA recognizes the following six elements in its import control policy framework, which constitute the import control systems for all Agency programs. These are:

foreign equivalency / certification;
Point of Entry control;
tracking and informatics;
Importer Quality Management Systems (QMS);
inspection program; and
new technology.

Appendix 5 Glossary of Terms

Automated Import Reference System - (AIRS) provides accurate and timely information on import requirements. The application uses a question and answer approach to guide the user through a series of questions about the Harmonized System (HS) Codes, origin, destination, end use and miscellaneous qualifiers of the product they wish to import.

CFIA Areas - with its headquarters in the National Capital Region, the CFIA is organized into four Operations Areas (Atlantic, Quebec, Ontario, and Western) that are subdivided into 18 regional offices, 185 field offices (including border points of entry), and 408 offices in non-government establishments, such as processing facilities.

CODEX Alimentarius Commission - a subsidiary body of the United Nations World Health Organization (WHO) and the Food & Agriculture Organization (FAO) of the United Nations.

Consumer Complaint - see definition for incident.

Effectiveness - the extent to which a program achieves its objectives. In the case of the Dairy Program, effectiveness includes the extent to which the Program's activities are contributing to the level of compliance of imported cheese in Canada.

Enforcement Actions - the Dairy Program provides a compliance regime which includes a range of enforcement activities that are employed when necessary to ensure compliance with federal regulations and standards. These include warning letters, product detentions, and suspension or revocation of certificates of federal registration. These activities may lead to product recalls or prosecutions, which are coordinated respectively by CFIA's Office of Food Safety and Recall (OFSR) and Enforcement and Investigations Services.

Food Emergency - is broadly defined as any situation involving or potentially involving food which may pose a health and safety concern to humans. Emergencies usually involve significant resources and require the coordination of a timely and/or extraordinary operational response.

Hazard - According to the CODEX Alimentarius Commission of WHO and FAO, a hazard is a biological, chemical or physical agent in, or condition of, food with the potential to cause an adverse health effect.

Incident - an issue that requires documentation and/or investigation into food safety, quality, labelling or fraud concerns within the CFIA's Programs' priorities and mandate. An incident can be generated by a variety of sources including consumer complaints.

Incident Management System (IMS) - CFIA's informatics system for tracking information regarding incidents which could include consumer complaints, emergencies and recalls.

Industry - for this evaluation, "industry" denotes all Canadian importers of cheese.

Program Clarification Requests - a mechanism to allow CFIA Operations and Program staff to seek national policy guidance.

Recall - action where an establishment removes from further sale or use, or corrects, a marketed product that contravenes legislation administered and/or enforced by CFIA.

Stakeholders - The Dairy Program's primary stakeholders are the importers of dairy products and their industry association, the Canadian Importers and Exporters Association.

Dairy Assessment Committee Members

Health Canada

Ashwani Wadhera, Acting Director, Bureau of Food Safety Assessment, Food Directorate (FD), Health Products and Food Branch (HPFB) (co-chair)
John Lytwyn, Food Safety Evaluator, Bureau of Food Safety Assessment, FD, HPFB
Luciano Silicani, Food Safety Evaluator, Bureau of Food Safety Assessment, FD, HPFB
Theresa Paolasini, Regulatory Policy Officer, Bureau of Food Policy Integration, FD, HPFB
Hélène Couture, Section Head, Policy and Method Development, Bureau of Microbial Hazards, FD, HPFB
Bernard Fortier, Scientific Evaluator and Program Officer, Bureau of Chemical Safety, FD, HPFB
Catherine Fothergill-Payne, Evaluation Manager, Departmental Program Evaluation Division, Information, Analysis and Connectivity Branch

Canadian Food Inspection Agency

Gail Daniels, Chief, Dairy Programs (co-chair)
Debra Fishbein, Program Officer, Performance Management and Program Support - Food of Animal Origin Division
Sonia Bernardi, Senior Analyst, Corporate Audit and Performance Measurement
Scott Rattray, Senior Analyst, Corporate Audit and Performance Measurement
Don Raymond, National Manger, Internal Affairs and Retail
Merv Baker, Special Advisor, Vice-President Programs

Health Canada Evaluation Team Members

Luciano Silicani
John Lytwyn
France Lacroix
Gilles Carreau
Michael Andre
Irene Roberts
Jason Pelletier
Michel Cloutier

References

1. Microbial Food Safety Risk Assessment Unit, Public Health Agency of Canada

2. Microbiological Safety of Cheese Made from Heat-Treated Milk, Part 1. Executive Summary, Introduction and History: Eric A. Johnson, John H. Nelson, and Mark Johnson: Journal of Food Protection Vol. 53, No. 5, Pages 441-452, May 1990)

3. U.S. Food and Drug Administration, Import Program, Current Import Alerts http://www.fda.gov/ora/fiars/ora_import_list_12.HTML

4. Outbreaks and Recalls Attributed to Cheese and Pasteurized Milk. Judy Greig: Microbial Food Safety Risk Assessment Unit, Public Health Agency of Canada
IFST Current Hot Topic: Food Safety and Cheese. IFST website http://www.ifst.org/hottop15.htm

5. Principles of Environmental Enforcement. International Network for Environmental Compliance and Enforcement, 1992. http://www.inece.org/enforcementprinciples.html

6. Guidance Concerning Microbiological Criteria, Microbiological Testing and Associated Methods for the Food Industry and Regulatory Agencies in Canada. J. Harwig, A. N. Sharpe, K. Dodds. Polyscience Publications. April 1998.

7. CFIA Import Control Policy, section 13.1. January 15, 2002
Principles of Environmental Enforcement. International Network for Environmental Compliance and Enforcement, 1992. http://www.inece.org/enforcementprinciples.html

Page details

Date modified:: 2006-07-19

ARCHIVED - Evaluation Report of the Canadian Food Inspection Agency's Dairy Program Activities Related to Imported Cheese Made from Pasteurized Milk

We have archived this page and will not be updating it.

We have archived this page and will not be updating it.

Table of Contents

Copyright

Executive Summary

CFIA Management Response

1.0 Introduction

1.1 Evaluation Scope and Objectives

2.0 Evaluation Approach

2.1 Evaluation Questions and Methodology

2.2 Evaluation Challenge - Data Limitations

3.0 Description of Dairy Program Activities Related to Imported Cheese Made from Pasteurized Milk

3.1 Legislation, Policies and Standards

3.2 Program Description

3.3 Dairy Program Logic Model Related to Imported Cheese Made from Pasteurized Milk

4.0 Observations, Conclusions and Recommendations

4.1 Rationale/Relevance of the Dairy Program's Import Activities

4.2 Design and Management of the Dairy Program's Import Activities

4.2.1 Dairy Program Objectives

4.2.2 Program Design

4.2.3 Science and Risk-Based Management

4.2.4 Integration of the CFIA Import Control Policy

4.2.5 Program Review

4.3 Program Delivery

4.3.1 Delivery of Inspection Activities as Designed

4.3.2 Follow-up and Enforcement Activities

4.3.3 Evidence of Training Activities

4.3.4 Evidence of Adjustments to Program Delivery

4.3.5 Trends in the Rate of Compliance of Imported Cheese Made from Pasteurized Milk and Rate and Trend of Compliance for Products Previously Identified as Non-compliant

4.3.6 Effective Management of Food Safety Emergencies and Incidents

4.4 Stakeholder Engagement

4.4.1 Facilitating the Delivery of Dairy Program Import Requirements

4.4.2 Industry Responsibility / Commitment for the Safety of Imported Cheese

Appendix 1 Dairy Program Logic Model Related to Imported Cheese Made from Pasturized Milk

Appendix 2 Narrative for the Logic Model

Appendix 3 Evaluation Methodology and Design

Appendix 4 Guiding Principles and Elements of the Import Control Policy

Appendix 5 Glossary of Terms

Dairy Assessment Committee Members

Health Canada Evaluation Team Members

References

Page details