Appendix 1: Evaluation of the Blood Safety Contribution Program at the Public Health Agency of Canada 2008–09 to 2013–14 – Logic model
Appendix 1 – Logic Model
Text equivalent for Appendix 1 – Logic Model
Blood Safety Contribution Program Logic Model
Inputs: Contribution funding, human resources, facilities/infrastructure, policies/regulations/international collaboration, and surveillance system
All inputs lead to three activities:
- Systematic data collection; coordination, analysis and interpretation of data; and dissemination of data
All activities lead to the following outputs:
- Surveillance data, surveillance reports, guidance documents, conference presentations, and stakeholder meeting presentations.
These outputs reach health system stakeholders responsible for monitoring, preventing, and controlling transfusion and transplantation adverse effects.
The logic model identifies the Program's immediate and intermediate outcomes as follows: timely and useful surveillance evidence for health system stakeholders and health system stakeholders utilize surveillance evidence to inform public on health risks and threats, to inform planning, programming and policy which contributes to regulation and legislation development.
Long-term outcome: Improved patient safety
Appendix 2 – Evaluation Description
Evaluation Issues
The specific evaluation questions used in this evaluation were based on the five core issues prescribed in the Treasury Board of Canada's Policy on Evaluation (2009). These are noted in the table below. Corresponding to each of the core issues, evaluation questions were tailored to the program and guided the evaluation process.
| Core Issues | Evaluation Questions |
|---|---|
| Relevance | |
| Issue #1: Continued Need for Program |
Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians
|
| Issue #2: Alignment with Government Priorities |
Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes
|
| Issue #3: Alignment with Federal Roles and Responsibilities |
Assessment of the role and responsibilities for the federal government in delivering the program
|
| Performance (effectiveness, economy and efficiency) | |
| Issue #4: Achievement of Expected Outcomes (Effectiveness) |
Assessment of progress toward expected outcomes (incl. immediate, intermediate and long-term outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes
|
| Issue #5: Demonstration of Economy and Efficiency |
Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes
|
Data Collection and Analysis Methods
The scope of the evaluation included an assessment of the relevance and performance of the Agency's biosecurity program between 2008-2009 and 2013-2014. Evaluators collected and analyzed data from multiple sources. Sources of information used in this evaluation included:
Literature review: A review of the literature using GoogleScholar was performed. Material was also provided by the Program and stakeholders.
Document and file review: The Program provided financial and nonfinancial administrative records, surveillance data, supporting documents, presentations, relevant articles, meeting schedules and reports. The Web was used to collect surveillance reports from relevant countries to determine surveillance reporting characteristics (e.g., frequency of reporting, lag time between data collection and reporting, scope of surveillance).
Key informant interviews: The first wave of interviews took place with Program staff after a preliminary review of documents and literature were conducted. Findings from these reviews were used to construct the Program questionnaire and focus the interviews. This phase of interviewing focused mainly on program activities and outputs. A second wave of interviews was done with stakeholders (e.g., provincial and territorial funded recipients, Health Canada's MHPD and BGTD, CBS and Hema-Québéc and blood researchers). A total of 27 individuals were interviewed. The questions posed in these interviews addressed relevance and performance of the Program.
Data analysis: A number of data matrices were created to organize and analyse the data by core evaluation issues and questions. Overall, Program outcomes provided the analytical lens. Central themes of timely surveillance data, utilization and patient safety were used to develop lines of questioning, data extraction and analysis. Data were analyzed by triangulating information gathered from the different sources and methods listed above. Contact was maintained with the Program throughout the data collection and analysis process in order to address emerging questions, for data requests and for technical and analytical assistance.
Appendix 3 – International Comparisons
| Country | Scope (most recent reports) | Donor/ Recipient Safety | Mandatory/ Voluntary Reporting |
Incidents/ Errors | Morbidity/ Mortality |
Publication Year | Schedule | Lag (years) |
|---|---|---|---|---|---|---|---|---|
| Australia | Transfusion-related adverse events between July 2009 and June 2011 and donation-related adverse events between July 2011 and June 2012. | D/R | I/E | MT/MR | 2013 | Annual | 1 | |
| Canada | 2006 - 2010 2004 - 2005 TTISS 2005 - 2007 TESS |
R | V | ? I E |
MT/MR | 2014 + 2008 2012 |
Sporadic | 3 - 7 3 5 - 7 |
| France | Rapport d'activité biovigilance 2012 | D/R | M | I/E | MT/MR | 2013 | Annual | 1 |
| New Zealand | 2007-11 | R | I/E | MT/MR | 2012 | Annual | 1 | |
| United Kingdom | 2009-2012, transfusion errors and adverse events (SHOT) | D/R | V | I/E | MT/MR | 2013 | Annual | 1 |
| United States of America | Fiscal year 2012, Fatalities Reported to Food and Drug Administration (FDA) Following Blood Collection and Transfusion | R | M | I/E | MT | 2013 | Annual | 1 |
| Center for Disease Control_ National Health Safety Network (Blood, tissues and organs) Under development |
R | V | I/E | MT/MB | N/A | N/A | N/A |
Data sources:
Australia: National Blood Authority Haemoviligance Advisory Committee. (2013). Australia haemovigilance report, data for 2009-10 and 2010-11. Retrieved from http://www.blood.gov.au/haemovigilance-reporting on February 10, 2014.
France: Creusvaux, H., Rigault, G, & Roche, M. (2012). Biovigilance in organ procurement and transplantation: French experience and results 2007 – 2011. Organs, Tissues and Cells, 15, 179-185.
New Zealand: Dagger, J., Dinesh, D. National Haemovigilance Programme Annual Report 2011. Retrieved from http://www.nzblood.co.nz/content/download/3066/38247/file/Haemovigilance%20Annual%20Report%202011.pdf on February 10, 2014.
United Kingdom: PHB Bolton-Maggs (Ed), D Poles, A Watt, D Thomas and H Cohen on behalf of the Serious Hazards of Transfusion (SHOT). Steering Group. The 2012 Annual SHOT Report (2013). Retrieved from http://www.shotuk.org/shot-reports/report-summary-and-supplement-2012/ on February 10, 2014.
United States: US Food and Drug Administration. (2012). Fatalities reported to FDA following blood collection and transfusion. Retrieved from http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/ReportaProblem/TransfusionDonationFatalities/ucm346639.htm on February 10, 2014.
+ Report has not been released at the time of this writing.
Appendix 4 – References
Alexander, P. E., Hyson, C., & Robillard, P. (n.d.). The development of a Canadian cells tissues and organ surveillance system (CTOSS) by the Public Health Agency of Canada. Draft, program document.
Creusvaux, H., Rigault, G, & Roche, M. (2012). Biovigilance in organ procurement and transplantation: French experience and results 2007 – 2011. Organs, Tissues and Cells, 15, 179-185.
Dagger, J., Dinesh, D. National Haemovigilance Programme Annual Report 2011. Retrieved from http://www.nzblood.co.nz/content/download/3066/38247/file/Haemovigilance%20Annual%20Report%202011.pdf on February 10, 2014.
Ditomasso, J., Liu, Y., & Heddle, N. M. (2012). The Canadian transfusion system: What is it and how can the data be used? Transfusion and Apheresis Science, 46(3).
Dzik, W. H. (2003). Emily Cooley Lecture 2002: Transfusion safety in the hospital. Transfusion, 43(9), pp 1190-1199.
Ganz, P. R. & Wu, J. (2012) Regulatory, Public Health, and International Aspects of Hemovigilance in Canada, in Hemovigilance: An Effective Tool for Improving Transfusion Safety (eds R. R. P. De Vries and J.-C. Faber), Wiley-Blackwell, Oxford, UK.
Heddle, N. M., Liu, Y., Barty, R., Eckert, K., Hsia, C., Tinmouth, A., … Ferguson, D. (2013). Ontario data collection strategy: Proof of concept project – final report. Ontario.
Kleinman, S., Caulfield, T., Chan, P., Davenport, R., McFarland, J., McPhedran, S., … Slinger, P. (2004). Toward an understanding of transfusion-related acute lung injury: Statement of a consensus panel. Transfusion, 44, pg 1774 – 1789.
Lacroix, J., Hébert P. C., Hutchison, J.S., et al. Transfusion strategies for patients in pediatric intensive care units. N Engl J Med 2007;356(16):1609-19.
MacDonald, N. E., O'Brien, S. F., & Delage, G. (2012). Transfusion and risk of infection in Canada: Update 2012. Retrieved from http://www.cps.ca/documents/position/transfusion-and-risk-of-infection-Canada on February 10, 2014.
Maskens, C., Downie, H,. Wendt, A., Lima, A., Merkley, L., Lin,Y., & Callum, J. (2014). Hospital-based transfusion error tracking from 2005 – 2010: Identifying the key errors threatening patient transfusion safety. Transfusion, 54, pp 66 – 73.
National Blood Authority Haemoviligance Advisory Committee. (2013). Australia haemovigilance report, data for 2009-10 and 2010-11. Retrieved from http://www.blood.gov.au/haemovigilance-reporting on February 10, 2014.
Norris, S. (July 10, 2008). Canada's Blood Supply Ten Years After The Krever Commission. Library of Parliament. PRB -08-14E.
PHB Bolton-Maggs (Ed), D Poles, A Watt, D Thomas and H Cohen on behalf of the Serious Hazards of Transfusion (SHOT). Steering Group. The 2012 Annual SHOT Report (2013). Retrieved from http://www.shotuk.org/shot-reports/report-summary-and-supplement-2012/ on February 10, 2014.
Public Health Agency of Canada. (2012). Public Health Agency of Canada Surveillance Strategic Plan 2013-2016. Ottawa, ON.
Public Health Agency of Canada. (2012). Surveillance Strategic Plan, 2013-2016, Public Health Agency of Canada, http://intranet.phac-aspc.gc.ca/surveillance/ssp-pss-eng.html.
US Food and Drug Administration. (2012). Fatalities reported to FDA following blood collection and transfusion. Retrieved from http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/ReportaProblem/TransfusionDonationFatalities/ucm346639.htm on February 10, 2014.
Weirsom-Osselton, J. C., Faber, J. C., Politis, C., Brand, A., van der Bom, J. G. & Schipperus, M. R. (2013). Quality validation of data in national haemovigilance systems in Europe: Report of a survey on current state of practice. Vox Sanguinis, 104, 214 – 217.
Page details
- Date modified: