Audit of the Management of HIV/AIDS Programs

June 2014

Table of Contents

• Executive summary
• A - Introduction
  • 1. Background
  • 2. Audit objective
  • 3. Audit scope
  • 4. Audit approach
  • 5. Statement of conformance
• B - Findings, recommendations and management responses
  • 1. Governance
    • 1.1 Committee governance
    • 1.2 Roles and responsibilities
    • 1.3 Internal coordination
  • 2. Risk management
    • 2.1 Risk management
  • 3. Internal controls
    • 3.1 Program management
    • 3.2 Sex and gender-based analysis
    • 3.3 Privacy
    • 3.4 Knowledge development
    • 3.5 Global engagement
    • 3.6 Communications and social marketing
    • 3.7 Program intervention
    • 3.8 Canadian HIV Vaccine Initiative
    • 3.9 Monitoring and reporting
• C - Conclusion
• Appendix A - Lines of enquiry and criteria
• Appendix B - Scorecard
• Appendix C - 2012-13 Financial Summary for the Federal Initiative on HIV/AIDS
• Appendix D - 2013-14 Grants and Contributions Expenditure Summary

Executive summary

The focus of the audit is the HIV/AIDS programs. The Public Health Agency of Canada (the Agency) is the federal lead for two initiatives: the Federal Initiative to Address HIV/AIDS (Federal Initiative) and the Canadian HIV Vaccine Initiative (Vaccine Initiative). These two initiatives represent a continued focus and investment by Canada to combat HIV/AIDS. The Federal Initiative is designed to prevent new infections, slow the progression of the disease, improve the quality of life of those affected, reduce the social and economic impact of HIV/AIDS and contribute to the global effort. The Vaccine Initiative is a transfer payment designed to accelerate the development of a vaccine and contribute to the achievements of the global HIV enterprise's scientific strategic plan.

The objective of the audit was to assess the effectiveness of the management control framework for HIV/AIDS programs including controls to ensure that program transfer payments are made in compliance with Treasury Board policy. The audit examined governance, risk management and internal control considerations related to HIV/AIDS program activities undertaken within the Infectious Disease Prevention and Control Branch (IDPCB).

In 2005, in collaboration with its government partners, the Agency designed a strategic approach to addressing HIV/AIDS through the Federal Initiative. The investment in the program remains substantial, as the number of new HIV/AIDS infections is persistently high. In 2011, the incidence of HIV (occurrence of new cases of disease that develop in a population in a one-year period) was 7.6 per 100,000 in the population. For the coming fiscal year, the Agency reports that it intends to reduce the incidence number to 6.41 per 100,000 in the population.

Over the last nine years, there have been relatively few changes to program delivery (with the exception of the launch of the Vaccine Initiative in 2006). In light of this, the Agency acknowledges that new prevention approaches are required. The program is currently reviewing its existing strategy towards addressing HIV/AIDS in a more holistic manner. While this approach is expected to build upon collective efforts to date, the Agency has yet to completely articulate the full nature of this strategic change, its impacts and the transition planning required to effect the change. To guide the existing programs, processes and practices into a coherent approach, the IDPCB would benefit from renewing the Federal Initiative strategic plan, using an implementation strategy as a roadmap. This would reflect the branch's commitment to bringing greater coherence to the program's activities. The implementation plan should include key activities, milestones and timelines, as well as a responsible lead.

The Agency has implemented governance structures for the Federal Initiative and the Vaccine Initiative to deliver on its HIV/AIDS program commitments. However, the Agency's shift towards a more holistic approach will require an examination of the impacts on the Federal Initiative's external and internal governance environment, including the clarification of intra-Agency roles and responsibilities related to collaboration and the impacts associated with the ongoing grants and contributions transformation. As well, for a large scale initiative such as the HIV/AIDS programs, it would be important for management to identify overall program risks, particularly as they move to a larger program framework.

The Agency will benefit from having additional internal controls related to operational planning and resource allocation. The audit found that there was a need to integrate operational work plans to support interdivisional collaboration and financial resource allocation. As well, the Agency will benefit from reviewing privacy aspects related to the HIV/AIDS programs and refining and fully implementing performance measurement frameworks and reporting approaches.

The Agency has integrated sex and gender-based analyses into its program operations, is meeting its international commitments towards strengthening the response to HIV/AIDS, and supports the implementation of communications and social media approaches. As well, the grants and contributions agreements were found to be in compliance with the Policy on Transfer Payments. Sampled contribution agreements provided evidence that the Agency received accountability documents, in accordance with the terms and conditions of the contribution agreements, and received required reporting documents.

Management has agreed with the four recommendations and has provided a detailed action plan that, once implemented, will strengthen the effectiveness of the management control framework supporting the delivery of HIV/AIDS program activities within the Agency.

A - Introduction

1. Background

Several years after its onset, the HIV/AIDS pandemic remains an enormous challenge in Canada and around the world. In 2011, the World Health Organization reported that 34 million people were living with HIV/AIDS and 1.7 million people died of AIDS-related illnesses worldwide. In Canada, according to 2011 national HIV estimates, 71,300 Canadians were living with HIV, an 11-percent increase over three years. In 2011, an estimated 24,300 people died due to an HIV-related illness.

In 1983, the federal government committed $3.2 million annually towards the fight against HIV/AIDS. In 1986, the Federal Centre for AIDS was established. This was followed by two phases of the National AIDS Strategy (1990-93 and 1994-97). In 1998, Canada renewed the strategy and launched the Canadian Strategy on HIV/AIDS to support the development of a coherent, national framework for addressing the epidemic. Continuing support was delivered in 2005 when the Government of Canada established the Federal Initiative to Address HIV/AIDS (Federal Initiative), at an ongoing funding level of approximately $70.5 million to strengthen domestic action and to support a global health response.

By 2006, the Government of Canada had also joined forces with the Bill and Melinda Gates Foundation and established a second program authority, the Canadian HIV Vaccine Initiative (Vaccine Initiative) ($20.6 million). The Vaccine Initiative is a key element in the Government of Canada's commitment to a comprehensive, long-term approach to addressing HIV/AIDS domestically and internationally.

These two initiatives represent a continued focus and investment by Canada to prevent new infections, slow the progression of the disease, improve the quality of life of those affected, reduce the social and economic impact of HIV/AIDS and contribute to the global effort.

The Federal Initiative to Address HIV/AIDS in Canada

The Federal Initiative was designed to build a coordinated Government of Canada approach. It focuses on prevention and access to diagnosis, care, treatment and support for those populations most affected by HIV/AIDS in Canada. The Public Health Agency of Canada (the Agency) is the federal lead for issues related to HIV/AIDS in Canada.

It is responsible for the overall coordination, communication and reporting, as well as national and regional programs, policy development, surveillance, laboratory science and leadership on international health policy, program and technical issues. In addition, the Agency leads the Responsibility Centre Committee governance body for the Federal Initiative and is expected to promote policy and program coherence among the participating departments and agencies (Canadian Institutes for Health Research, Health Canada and Correctional Service Canada).

The Canadian HIV Vaccine Initiative

In 2006, the Government of Canada and the Bill and Melinda Gates Foundation signed an agreement formalizing the collaboration to further strengthen global efforts to accelerate the development of HIV vaccines and to contribute to the achievement of the Global HIV Vaccine Enterprise's Scientific Strategic Plan.

The Canadian HIV Vaccine Initiative was established by the Government of Canada to serve as the vehicle for implementing the agreement. Participating federal organizations are the Agency, Health Canada, Foreign Affairs, Trade and Development Canada, Industry Canada and the Canadian Institutes of Health Research. In 2010, the agreement was renewed with the primary objective of collaborating to accelerate the development of a safe and effective HIV vaccine by building on Canada's scientific excellence for the benefit of those most in need in low- and middle-income countries (particularly in Africa) but also in Canada. The renewed initiative introduced changes to the program governance structure and program activities and is to be implemented over six years, with combined total expenditure of $139 million. The total expenditure for the Agency over six years is $18 million.

Organizational Summary

Table 1 provides a summary of the key Agency organizations supporting the delivery of HIV/AIDS Programs.

Table 1: Summary of key Agency organizations supporting the delivery of HIV/AIDS Programs

2. Audit objective

The objective of the audit was to assess the effectiveness of the management control framework for the HIV/AIDS programs and to provide assurance that the transfer payments are in compliance with Treasury Board's Policy on Transfer Payments.

3. Audit scope

The scope of the audit included an examination of the Agency's management processes and commitments in the funding agreements related to the planning and delivery of HIV/AIDS programs, the leadership of which falls primarily under the Centre for Communicable Diseases and Infection Control and the National Microbiology Laboratory. The audit scope also included relevant HIV/AIDS program management activities undertaken by resources within the Regional Operations.

The scope of the audit covered program activities undertaken during fiscal year 2012-13 and 2013-14. It included the management and administration of both transfer payments and operating funds related to the programs. The scope of the audit was restricted to Agency activities and resources, and did not include an examination of activities undertaken by HIV/AIDS program delivery partners.

4. Audit approach

The audit approach included a review of documentation, policies, standards, guidelines and frameworks, interviews, observation and inquiry, testing and analyses. Regional field work was conducted via telephone and documentation review. The audit criteria, outlined in Appendix A, were derived from the Treasury Board Secretariat Policy on Privacy Protection, the program proposal submitted in 2004, the renewed agreement between the Government of Canada and the Bill and Melinda Gates Foundation and the Treasury Board Secretariat Policy on Transfer Payments and related Directive. The audit was conducted in accordance with the Government of Canada's Policy on Internal Audit and examined sufficient relevant evidence and obtained sufficient information and explanations to provide a reasonable level of assurance in support of the audit conclusion.

5. Statement of conformance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

B - Findings, recommendations and management responses

1. Governance

Audit criterion: The HIV/AIDS programs have an effective governance structure to support the strategic direction and program objectives.

1.1 Committee governance

Effectively governing and managing horizontal initiatives is important to garner public service effectiveness from the collective results achieved, for the efficient use of resources and to prevent or minimize program overlap and duplication. The federal government's purpose for the horizontal Federal Initiative to Address HIV/AIDS in Canada (Federal Initiative) was to achieve a coherent approach to HIV/AIDS.

Federal Initiative external governance

External governance and advice related to national HIV/AIDS multi-sectoral and multi-jurisdictional activities continues to be provided by the Ministerial Advisory Council on the Federal Initiative and the National Aboriginal Council on HIV/AIDS. Membership includes representation from a cross-section of researchers, health care professionals, community-based front-line representatives and people living with HIV/AIDS. Meetings take place up to three times a year. The Programs and Partnership Division performs the secretariat role and provides input into meeting agendas, but it is not an active participant.

In 2013, the Agency initiated a renewal process for the governance committees, with a view to aligning with a new integrated delivery model for sexually transmitted blood-borne infections that consolidates programs such as HIV/AIDS, Hepatitis C and other sexually transmitted infections. The Agency proposed timelines to have the committees renewed by 2014; however, this has yet to be done. As well, current membership and program contacts are not up-to-date on the website. The Agency's website also reports on a Government of Canada Assistant Deputy Minister Committee on HIV/AIDS. The website notes that the committee was created to achieve greater coherence, complementarity and collaboration within federal HIV/AIDS policy and programming. It is composed of the assistant deputy ministers from 13 federal departments and agencies whose mandate impacts the determinants of health or is related to Canada's response to HIV/AIDS. Management reports that this committee does not exist, that updates to the website have been identified, and that new content has been submitted for posting on the website.

Interdepartmental governance

Given the complexity of the Federal Initiative, it was determined that an interdepartmental coordinating committee would be established by the Agency to promote policy and program coherence among the four participating departments and agencies, to maximize the use of available resources and to guide results-based work plans designed to achieve the outcomes of the Federal Initiative. The Responsibility Centre Committee was to be chaired by the Director General, Centre for Infectious Diseases Prevention and Control and to include representatives from the four participating departments and agencies. The committee was tasked with the following functions: policy conflict resolution; agreements toward meeting targets; defining roles and responsibilities with regard to HIV prevention; and access to care, treatment and support.

The Agency established the interdepartmental committee with the appropriate partners, which meets three times a year, as expected. Through a review of meeting agendas, minutes and interviews, it was determined that the committee is primarily a forum for partners to share information, to discuss priority areas and to identify overlaps or gaps between and among partners. Each of the partner departments and agencies makes its own resourcing decisions and develops its own work plans. The Agency consolidates initiative results, which are shared through presentations at committee meetings and through annual performance summaries. All reporting related to the Federal Initiative is vetted through the committee. Currently, the committee is chaired by a director and the majority of the participants are at a managerial level. As with the external governing bodies, the Agency will need to update the mandate and terms of reference for the Responsibility Centre Committee, so that they align with the Agency's new strategic direction of a holistic approach to addressing HIV and other sexually transmitted and blood-borne infections.

Vaccine Initiative external governance

One of the key requirements related to the 2010 renewal of the Canadian HIV Vaccine Initiative (Vaccine Initiative) was the establishment of an external advisory board. The Agency launched an open solicitation for the selection of a coordinating office. The Agency reports that the process found one eligible option and after a comprehensive review, the option was recommended by the advisory board. In 2011, the Agency signed a contribution agreement with the International Centre for Infectious Diseases to operate the CHVI Research and Development Alliance Coordinating Office. This effectively transitioned the Agency's support role to the new Alliance Coordinating Office, which provides the secretariat function for the advisory board. The contribution agreement is administered by the Agency. Management notes that the roles and responsibilities of the board, Agency and Alliance Coordinating Office are established.

As planned, governance for the Vaccine Initiative is provided by an advisory board that is mandated to provide recommendations to the Ministers of Health, Industry and International Cooperation and to the Bill and Melinda Gates Foundation on projects to be funded and the agreement between the Government of Canada and the Bill and Melinda Gates Foundation, as well as to monitor that funded activities continue to align with the key areas of focus identified in the agreement.

The board comprises up to three representatives from the Bill and Melinda Gates Foundation, the Director of the Alliance Coordinating Office (the secretariat), three external experts and one (non-voting) representative from each of the Vaccine Initiative departments and agencies, including the Agency. The board is co-chaired by a representative of the Bill and Melinda Gates Foundation and an appointed external expert nominated by the Minister of Health.

Strategic direction

In 2005, the strategic direction for the HIV/AIDS programs was defined in the Federal Initiative funding agreement. Between 2005 and 2014, there were no significant changes to the strategic direction, with the exception of the launch of the Vaccine Initiative in 2006. After nine years, the Agency is looking to renew the strategic direction by broadening its HIV/AIDS approach. It will be important for the Agency to define and document the comprehensive nature of the change in strategic direction for the Federal Initiative, including the impacts on the existing governance structure.

Recommendation 1

It is recommended that the Assistant Deputy Minister, Infectious Disease Prevention and Control Branch, update the strategic plan based on current risks (with an implementation strategy) for the HIV/AIDS programs, including governance, roles and responsibilities, surveillance, research, plans and priorities, grants and contributions and performance measurement frameworks, and the relevant website pages.

Management response

1.2 Roles and responsibilities

Audit criterion: The HIV/AIDS programs' roles and responsibilities are documented and operating effectively.

In order to meet the objectives of the Federal Initiative and the Vaccine Initiative, it is critical to have an aligned inter- and intra-departmental approach, with clearly defined roles and responsibilities. For the Federal Initiative, a results-based management accountability framework was developed that broadly describes the roles, responsibilities and accountabilities of the various players. As well, there are interdepartmental agreements established to define specific roles and responsibilities on discrete projects.

Figure 1: Organizational chart for Agency responsibility centres involved in the Federal Initiative and the Vaccine Initiative

Enlarge Figure 1

The Programs and Partnerships Division ($13.6 million) is the focal point for the coordination of the Federal Initiative. The division is expected to work towards increased collaboration and engagement, increased coherence of the federal response at the global and national levels and increased knowledge and awareness. The division has recently taken the lead on all grants and contribution spending ($24.9 million) under the Federal Initiative, including community funds previously managed by Regional Operations. The regions will continue to support the monitoring of recipient activity (see Section 3.7).

The Professional Guidelines and Public Health Practice Division ($0.265 million) works to enhance the practices of public health professionals and clinicians to prevent and control communicable diseases in Canada. The division develops guidelines and public health intervention research, identifies and disseminates promising practices, and supports technical knowledge transfer and exchange, including modelling, estimations and projections.

The Surveillance and Epidemiology Division ($2.3 million) supports the management of a national communicable disease surveillance system, leading epidemiological research and providing technical epidemiological support to provincial and territorial governments, international institutions and foreign government institutions.

The National HIV and Retrovirology Laboratories ($5 million) provide a comprehensive range of laboratory science services and expertise related to HIV and emerging retroviruses. The provision of national laboratory services for the purpose of enhancing diagnosis, care, treatment and diagnosis of HIV-infected Canadians is mandated under the Federal Initiative. It includes the National Laboratory for HIV Genetics, the National Laboratory for HIV Reference Services and the National Laboratory for HIV Immunology.

For the Vaccine Initiative, the roles and responsibilities of the initiative participants are defined in its Performance Measurement Strategy. The Strategic Issues and Integrated Management Division ($1.6 million) is responsible for the overall coordination of the Vaccine Initiative. In this role, the division serves as the focal point for initiative implementation and is responsible for the coordination and oversight of the initiative's governance and management structures, the facilitation of collaboration with domestic and international stakeholders and the provision of secretariat support.

1.3 Internal coordination

The Agency's internal activities for HIV/AIDS prevention and control have to work together to be effective. The Agency's five responsibility centres include the Program and Partnership Division, the Surveillance and Epidemiology Division, National HIV and Retrovirology Laboratories, the Professional Guidelines and Public Health Practice Division and Regional Operations.

It was noted through interviews and documentation review that the work of the Surveillance and Epidemiology Division informs the work of the HIV Retrovirology Laboratory and vice versa. This work is also linked to the work of the Professional Guidelines and Public Health Practice Division. Management of divisions noted that the current roles and responsibilities are based on historical inter-collaboration, and that a more formalized approach would be beneficial in clarifying how the responsibility centres are to collaborate on joint areas of focus. Given the renewal of the strategic direction, it would be important to formally update all the roles and responsibilities (see Recommendation 1).

2. Risk management

2.1 Risk management

Audit criterion: Internal and external risks related to the delivery of the HIV/AIDS programs are identified, assessed and have mitigating strategies.

For a large scale initiative such as the HIV/AIDS programs, it would be expected that management would identify overall program risks, that the risks would be assessed for likelihood of occurrence and magnitude of impact, and that a roll-up of the individual risks would be conducted to form a complete program risk analysis. It would be expected that this systematic exercise would be conducted at least annually.

The 2005 funding agreement documents a risk-based approach for designing the HIV/AIDS programs, in order to reach priority populations (those at greatest risk). The allocations were developed based on evidence of HIV incidence and prevalence. Analysis was applied to identify the different risk factors for the eight priority populations, and approaches were built based on these factors and the epidemiological evidence. As well, a strategy to manage HIV/AIDS program risks was described in the funding agreement and the vaccine-related risks were included in the 2010 Vaccine Initiative funding agreement. The Agency committed to an ongoing process to improve its risk management and committed to making efforts for the development of a systematic risk assessment approach, including the development of risk indicators for each of the key risks identified. The Agency also committed to monitoring annually the project risk; however, an analysis of the documentation noted that the risks have not been updated since the original dates of the funding agreements.

The funding agreement has a requirement to produce an annual report on risk. This was done in the past but is no longer being produced. The program indicates that it is addressing this requirement through the Agency Corporate Risk Profile. However, over the last few years, HIV/AIDS has not been included in the Corporate Risk Profile or the Departmental Performance Report. It was also noted that interdepartmental risks have yet to be identified or assessed by the Responsibility Centre Committee.

Given the Agency's lead on the file combined with fact that the Agency intends to administer a portion of the HIV/AIDS programs to address HIV/AIDS and other sexually transmitted blood-borne infections in an integrated manner, it will be important to conduct a risk analysis to support the strategic planning exercise (see Recommendation 1).

3. Internal controls

3.1 Program management

Audit criterion: The HIV/AIDS programs have operational plans that demonstrate the use of their resources to support program delivery.

Federal Initiative program management

The Programs and Partnership Division plays an overall Government of Canada coordination role function for the Federal Initiative that includes supporting joint planning, monitoring and reporting. The division promotes the integration of all HIV/AIDS program elements within and between the Agency and its partners. It was expected that within the Agency, priority planning would be developed based on the programs' objectives and expected results, and that the plans would include the budget necessary to carry out the activity, to ensure that the expected results are achieved.

Planning

The audit requested the strategic and operational plans prepared at the corporate, branch and directorate level, as well as the documents used to make decisions on resource allocation. While there is no specific operational plan for the HIV/AIDS programs, it was found that planning and budgeting for the programs is dispersed among several documents. For example, the 2013-14 Agency Operational Plan and Branch Operational Plan captured all of the grant and contribution funds ($22.9 million), but did not include objectives or priorities for the operating funds ($17.9 million). The National Microbiology Laboratory has an operational plan that details some laboratory activities and full-time equivalents for three labs receiving Federal Initiative funding (approximately $5 million). The Treasury Board of Canada Secretariat horizontal database notes expected results and results achieved for each of the program elements, but it does not indicate the resources required or used.

The audit analyzed the divisional work plans from each of the responsibility centres. The work plans vary significantly in format and depth of detail and for the most part, do not include resource allocations for work to be performed. The programs would benefit from having the divisional work plans integrated with resource allocations, to provide a more complete operational plan. The Centre for Communicable Diseases and Infection Control has identified this integration challenge and has taken steps to implement a standard work planning template for each of its divisions, with the intent of developing an integrated work plan for the centre starting in April 2014 (see Recommendation 2).

Financial management

In 2012-13, the Agency received $45.4 million for the Federal Initiative, of which the funding agreement sets aside $1.4 million for corporate support. Over the last eight years, the corporate support transfer has gradually increased to $6.5 million. In fiscal year 2012-13, $36.6 million was expended, resulting in a surplus of $3.4 million for the year, which was either lapsed or re-profiled within the Agency. Internal corporate re-allocations of funds are done to offset costs associated with other programs or operating expenditures. Following the financial analysis, the audit notes that it would be important for the Agency to use its budget/banking days and operational planning process to examine the alignment between program funding priorities established in the funding agreement and corporate requirements.

At the beginning of each fiscal year, HIV/AIDS resource allocations are made notionally to each branch, centre and division, based on historical funding levels rather than on plans. At mid-year and year-end, the division produces a consolidated budget to actual expenditure analysis, which summarizes expenditures by key activity area and by responsibility centre. This activity costing is currently supported by the use of internal order codes as a tracking mechanism within the Agency's financial system. However, the internal order codes were not used by all and when used, they were not used consistently, requiring the Agency to allocate costs on the basis of a detailed review of cost centre actuals, supplemented by management estimates. This made it difficult for the program to complete a mid-year review. Program staff confirmed that they are revising the method for tracking program-related costs for 2014-15, in order to simplify the financial coding structure and provide for more consistency in tracking expenditures.

Canadian HIV Vaccine Initiative

In 2012-13, the Agency received $2.06 million for the Vaccine Initiative. As a result of various allocations and program funding transfers, planned spending for the year was $1.54 million. Spending was less in one area as management reports that the vaccine efforts have not advanced as expected to be able to disburse the funds as originally intended. In the interim, the Strategic Issues and Integrated Management Division has been analyzing options, meeting with stakeholders and preparing materials to brief senior management, in order to present recommendations to the Advisory Board on how the funds will be dispersed going forward. The Vaccine Initiative notes that the money was reallocated to offset costs associated with other programs or operating expenditures. More specifically, $500,000 was transferred interdepartmentally to one of the partner departments, and the remaining $250,000 was transferred to the World Health Organization (WHO). Management reports that this funding was used to support the implementation of an HIV Drug Resistance Surveillance and Monitoring Strategy. Following this financial analysis, the audit notes that the Vaccine Initiative, if extended, would benefit from an updated strategic plan to align with vaccine development progress (see Recommendation 1).

In conclusion, without an Agency-wide perspective of the staff resources required and the resources used, the program has difficultly determining the costs associated with the HIV/AIDS programs. The 2012-13 evaluation of the programs also identified the tracking and reporting of financial and human resources engaged in HIV/AIDS activities as an area for improvement. To more effectively manage the initiatives, it would be important to monitor the usage of total financial and human resources, to understand better the overall requirements needed to meet the planned commitments.

Recommendation 2

It is recommended that the Assistant Deputy Minister, Infectious Disease Prevention and Control Branch, develop and implement an integrated work plan, including appropriate financial and non-financial information.

Management response

3.2 Sex and gender-based analysis

Audit criterion: The HIV/AIDS programs have integrated sex and gender-based analysis into their programs.

Sex and gender-based analysis is an analytical tool used to examine the differential impacts on both women and men of government policies, programs and legislation. Applying this analysis to specific diseases highlights how risk factors, symptom patterns and outcomes differ for women and men. Therefore, it is important to integrate gender impacts into health-based policy and program development.

In 2009-10, the Health Portfolio approved a Sex and Gender Analysis Policy and a five-year performance measurement framework. In addition, the Agency drafted a five-year sex and gender-based analysis policy implementation plan. The Social Determinants and Science Integration Directorate in the Health Promotion and Chronic Disease Prevention Branch is responsible for supporting the implementation of the policy.

The directorate in charge provides guidance to the programs on including sex and gender-based analysis when drafting funding agreements. The 2004 funding agreement on the Federal Initiative and the 2010 funding proposal for the Vaccine Initiative include gender-based analysis requirements.

In 2013, the Agency approved a Plan to Advance Health Equity, which lays out a roadmap to achieve specific results over the next three years. Transformation activities include modernizing grants and contributions, strengthening surveillance and strengthening science capacity and the science-policy interface. The Centre for Grants and Contributions is currently working to revise funding descriptions to include more specific wording around sex and gender-based analysis of populations. The funding descriptions will use language to describe health equity and diversity that is consistent with pre-approved language.

The Agency produces HIV/AIDS surveillance reports that identify trends by sex, age group, race/ethnicity, exposure category and geographic location. In 2011, under the umbrella of the Federal Initiative, the Agency, in consultation with an external working group, developed a Population-Specific HIV/AIDS Status Report: Women.

The Social Determinants and Science Integration Directorate has begun developing peer review support for research conducted with the Health Promotion and Chronic Disease Prevention Branch. There are plans to work with the Chief Science Officer at the Agency to integrate gender equity into the science plans. The Agency recognizes that there is a need for this collaboration and is in the process of developing a plan.

The Agency has made good progress in developing the policy to advance health equity and sex and gender-based analyses, and has integrated sex and gender-based analysis into its grants and contributions and surveillance systems.

3.3 Privacy

Audit criterion: The HIV/AIDS programs have privacy practices that are in compliance with Government of Canada expectations.

The HIV/AIDS programs collect substantial amounts of personal information for the purpose of monitoring, researching and providing accurate information pertaining to infectious disease records. This information is used by the Agency and stakeholders to inform decision making, policies and practices for the prevention and management of health risks and determinants associated with HIV/AIDS. Personal information is collected pursuant to section 4 of the Department of Health Act. For example, the Surveillance and Epidemiology Division may collect personal information such as date of birth, medical information and patient's initials, specimen id code, ethnicity, province of residence at time of diagnosis, country of birth, year of arrival into Canada, date of first positive HIV test and date of diagnosis for diseases indicative of AIDS.

Privacy impact assessment

Privacy impact assessments (PIA) are used to identify the potential privacy risks of new or redesigned federal government programs or services. They also help eliminate or reduce those risks to an acceptable level. They take a close look at how government departments protect personal information as it is collected, used, disclosed, stored and ultimately destroyed. These assessments help create a privacy-sensitive culture in government departments. Under the Treasury Board of Canada Secretariat's Directive on Privacy Impact Assessment (effective April 1, 2010), government departments must conduct a PIA in a manner that is commensurate with the level of privacy risk identified, before establishing any new or substantially modified program or activity involving personal information.

Neither the Federal Initiative nor the Vaccine Initiative has completed a PIA. In the past, some draft PIAs were started for some of the surveillance activities, but they were never completed. The programs note that these were not finished because the Agency had not yet finalized a protocol with respect to what was required for a PIA of its surveillance databases. The programs acknowledge the need for a PIA for the HIV/AIDS programs, in order to ensure that the personal information they collect and the recipients are safeguarded (see Recommendation 3).

Personal information banks

Personal information banks are descriptions of personal information that are maintained by government institutions about individuals, in support of specific programs and activities. The Privacy Act requires that personal information banks include all personal information that is organized and retrievable by a person's name or by an identifying number, symbol or other particular identifier assigned only to that person. Personal information banks must also include personal information that has been or is being used or is available for use for an administrative purpose.

The Agency's personal information holdings related to HIV/AIDS are published in Info Source through six personal information banks related to the Canadian Needle Stick Surveillance Network, the Enhanced Surveillance of Canadian Street Youth, the National HIV/AIDS Surveillance System, the Strain and Drug Resistance Surveillance, Extraordinary Assistance Plan, I-Track Surveillance and M-Track Surveillance. The programs should update InfoSource, since the Canadian Needle Stick Surveillance Network has been discontinued. As well, it was noted that there are no personal information banks for the E-Track Surveillance or A-Track Surveillance (see Recommendation 3).

Privacy clauses

The audit conducted an analysis of the grant and contribution agreements and noted that they all contain the required privacy clause specifying that "the Parties shall comply with applicable laws pertaining to privacy and confidentiality in dealing with information and records related to the project." The audit notes that personal information was found to be collected by recipients through certain contribution funds. For example, Community-Based Organizations and the National Voluntary Sector Response Fund collect personal information. The privacy protection section includes a clause stating that, to protect the data, it will be stored on a database while the analysis of the data is being conducted. Given that it is the responsibility of the Agency to monitor recipients' compliance with the terms and conditions of the agreements, including compliance to privacy requirements, it would be important to include the transfer payment program in a PIA.

In conclusion, a PIA documents how personal information flows in a program. It analyzes personal information flow in the context of applicable privacy policies and legislation, with the objective of identifying, evaluating and documenting the major privacy risks or vulnerabilities, as well as the actions or options to mitigate privacy risks. While there were no privacy breaches reported, the programs would benefit from conducting a PIA.

Recommendation 3

It is recommended that the Assistant Deputy Minister, Infectious Disease Prevention and Control Branch, conduct a privacy impact assessment for the HIV/AIDS programs.

Management response

3.4 Knowledge development

Audit criterion: The HIV/AIDS programs conduct approved scientific research, operate surveillance and have testing strategies, including integrated public health information systems.

Laboratory services and research

The National HIV and Retrovirology Laboratories provide a comprehensive range of laboratory services and scientific expertise related to HIV and emerging retroviruses. The laboratories were the first in Canada to be accredited by the Standards Council of Canada for the period from 2008 to 2016. The laboratory consists of three working divisions that deliver core activities.

The Agency reports that all funding for the laboratories is from the Federal Initiative funding agreement. However, management reports that activities performed in the laboratory are not easily tied back to the funding agreement because many priorities have changed since it was first signed. Also reported is that there is some non-HIV retrovirology work being done to satisfy "safety of blood supply" requirements. The laboratories receive a notional budget at the start of the year of approximately $5 million.

The laboratories provided the 2011-12 to 2012-13 work plans that generally delineate core diagnostic and testing activities (for example, testing lab samples from provincial laboratories), supplemented by research activities. Research is typically focused on evaluating new testing methods and is considered "applied" research. However, there is currently no formal approach to the selection and approval of research related to new technologies. The Agency did provide a listing of research publications for 2013, which substantiates that HIV research is being published and tracked for the Federal Initiative. However, the laboratories do not have a compendium of research projects undertaken, ongoing or planned. Management reports that there is no required research and that research is based on a combination of internal (surveillance and epidemiology) and external (provincial laboratories, WHO, etc.) stakeholder requirements, although the actual experimental design is left largely to the laboratories' research scientists.

In 2012, the Agency developed and presented to the Executive Committee a proposed Science and Research Priorities and Prioritization Criteria Methodology. It was acknowledged at that time that there is a long-standing need for the Agency to prioritize better its science and research activities, in order to maximize effectiveness and efficiency and ensure that efforts are directed to generating the evidence base that supports policy priorities. As a result, as the programs renew their strategic direction for the Federal Initiative, it would be important that they explicitly integrate the research priorities to expected results HIV/AIDS (see Recommendation 1). As well, it will be important for this responsibility centre to update its work plan to demonstrate annual priorities, resources required and timelines, in order to align with other partners (see Recommendation 2).

Surveillance

Systematic and ongoing HIV/AIDS surveillance is needed to monitor disease rates and detect changes in the occurrence of the disease and its complications. Surveillance data allows governments and stakeholder groups to plan, implement and evaluate disease prevention and control programs.

The Agency has three surveillance systems: HIV and AIDS Core Surveillance, HIV Strain and Drug Resistance Surveillance and HIV Enhanced Surveillance. The Surveillance and Epidemiology Division also sponsors research through contracts with private sector groups and associations, and with government bodies through memorandum agreements. Data collection on a person's confirmed HIV infection, referred to as positive HIV test reports, as well as newly diagnosed AIDS cases, are submitted to the Agency's HIV/AIDS and Tuberculosis Section (HATS) by all provinces and territories. The Agency requests provincial/territorial HIV/AIDS surveillance data twice a year, at scheduled intervals. This information is sent either electronically or in hard copy case reports, depending on the province/territory. Once data is received, the information is entered into one of two secure national databases, one for HIV and one for AIDS. The Agency reports that the data is cleaned by the surveillance officers and the senior research analyst to ensure consistency and accuracy during the import process.

Additional sources of data, such as research and targeted studies, supplement the surveillance data, allowing the Agency to use an estimation process to determine approximate number of all people living with HIV in Canada. The Agency publishes its national estimates of HIV prevalence and incidence every three years. In 2011, the incidence of HIV (occurrence of new cases of disease that develop in a population in a one-year period) was 7.6 per 100,000 in the population. In 2011, the United Kingdom reported incidence at 10 per 100,000 and Australia reported incidence at 4.6 per 100,000. For the coming fiscal year, the Agency reports a target to reduce the incidence number to 6.41 per 100,000.

Surveillance data quality assessments (DQA) involve an assessment of the common data management practices. The criterion assesses data dimensions related to accuracy, timeliness, serviceability, usability and relevance. DQAs were provided for all three key surveillance activities and included recommendations supported by management action plans and a status of implementation.

The division measures the usefulness of its surveillance system through surveys and webinars (following which participants are asked to take a survey). The Agency provided the results of the 2010 HIV and AIDS in Canada: User Satisfaction Survey, in which the response rate by province was between five and twenty percent. As the response rate for this survey was low, the Agency reports that it is looking into different ways to obtain feedback. The division provided one example, namely the presentation of the reports via webinars (and the accompanying participant feedback survey), which was completed for the 2012 report.

The audit analyzed documentation related to surveillance streamlining. The branch expects that efficiencies could be achieved by streamlining all surveillance related to sexually transmitted blood-borne infections. The program areas will benefit from having a systematic approach to acquiring data and effectively sharing data among these programs. The division will be developing a five-year sexually transmitted and blood-borne infections surveillance strategy. The surveillance strategy should be a subset of the larger HIV/AIDS strategy and implementation plan. Both strategies should consider the need for improved alignment between research and surveillance, to take into account priorities and operational capacity (see Recommendation 1).

Integrated public health information system

In the 2005 Federal Initiative funding agreement, it is noted that an HIV/AIDS module for the integrated public health information system was under development, leading to the improved transmission of HIV/AIDS surveillance data from the provinces and territories. The Agency reports that over the past decade, programs have leveraged available expertise, technologies and resources to implement data management tools to support daily surveillance activities. These data management systems, although functional, are extremely labour intensive, personnel dependent, expensive to maintain and evolve and vulnerable to data errors and loss.

Provinces and territories report surveillance data to the Centre for Communicable Diseases and Infection Control in a number of different formats including Excel, Access, text files, etc. These data sets need to be consolidated by the Agency for analysis and reporting at the program level. Over time, the Centre for Communicable Diseases and Infection Control developed a case-by-case solution for provincial and territorial data collection, but determined that this approach was not cost effective. The Agency reports that currently, there is no national public health electronic data standard for data submission. As a result, dataset column names and field names or formats change over time and data formats vary within the same data column.

In 2012, the Agency worked with an external contractor to develop and implement a generic data management software solution. Management notes that the application is approximately 70 percent complete but requires an experienced development team to finalize efforts in order to deploy the application into a production environment. While the application currently under development is not an Integrated Public Health Information System, it is a surveillance data management platform that is configurable and adaptable to meet the varying and unique needs of the Centre for Communicable Diseases and Infection Control's public health surveillance programs.

It was noted that in its current state, the application does not comply with current Health Canada/Public Health Agency of Canada application development methodologies. To reduce the maintenance and overhead required for the installation, support and upgrades of custom desktop solutions, they are required to be developed as web applications. In addition, the Government of Canada Windows 7 initiative will limit the installation of custom-rich client applications onto desktop workstations.

Recommendation 4

It is recommended that the Assistant Deputy Minister, Infectious Disease Prevention and Control Branch, implement a surveillance solution that will allow for the efficient importation of data submissions.

Management response

3.5 Global engagement

Audit criterion: The Agency meets its international commitments towards strengthening the response to HIV/AIDS.

The Federal Initiative calls for Canada's leadership, participation and sharing of best practices with international bodies, to strengthen the global response. Specifically, authorities reference the Health Portfolio's role in coordinating the Government of Canada's participation in international AIDS conferences. Through the Federal Initiative, the federal government is to be in a stronger position to meet its international commitments, including reporting back to the United Nations General Assembly on HIV/AIDS on the state of the Canada-wide response.

One of the regular global engagement activities is the Biennial International AIDS Conference organized by the International AIDS Society. The upcoming 20^th conference, to be held in Melbourne, Australia in 2014, is expected to have over 20,000 delegates, including the Ministerial and Health Portfolio representatives. The Agency has an interdepartmental working group to share information about the Government of Canada's engagement in the conference, which is supported by the Federal Secretariat within the Programs and Partnerships Division.

The Vaccine Initiative represents a significant Canadian contribution to the global efforts to develop a safe, effective, affordable and globally accessible HIV vaccine. It supports activities that span the continuum of HIV vaccine research and development efforts in the private and public sectors, including in academic institutions. A contribution recipient for this initiative reports that it is involved in holding satellite meetings at targeted national and international conferences.

As a result of the 2011 Agency internal audit on international activities, the Agency now has a global health framework for international activities (2012-17). The framework is designed to improve the coherence and coordination of the Agency's global health activities. In 2012, the programs outlined a global engagement framework that includes global engagement opportunities, proposed strategic priorities and strategic objectives for global engagement within the centre. The programs monitored and reported its global activities for 2012-13, listing the name of the event with the level of collaboration. The programs are planning on compiling a global engagement framework this fiscal year.

In conclusion, through global engagement, the Agency meets its international commitments towards strengthening the response to HIV/AIDS.

3.6 Communications and social marketing

Audit criterion: The Agency has approved national public awareness campaigns and targeted social marketing initiatives, with professional guidelines and practices.

Public awareness and social marketing activities are important to improve Canadian HIV knowledge, to address the community and societal attitudes that contribute to individual risk behaviours and to reduce the stigma and discrimination that promote the epidemic. The 2004 funding agreement for the Federal Initiative outlined that the Agency would be responsible for raising awareness in the general populations of the seriousness of HIV through national, gender-sensitive public awareness campaigns.

Communications planning and implementation is a shared responsibility between the Communications and Public Affairs Branch and the programs. The programs receive guidance on e-communications, social media marketing, media lines and coordinating ministerial events and speeches on HIV/AIDS (see text box). The Centre for Communicable Diseases and Infection Control has a communications plan that is aligned with the branch communications strategy for 2013-16. However, the budget section was not completed, was reported as "to be determined", and did not link to the Federal Initiative performance measurement template (see Recommendation 1).

In 2009, a targeted community-based approach was developed and implemented to address the needs of those most-at-risk for HIV infection. Working with a contractor, training webinars have been developed to enhance the social marketing knowledge and skills of community-based organizations. These training webinars are provided to contribution recipients working with communities vulnerable to HIV/AIDS. It is expected that the training will be fully delivered in the first quarter of the fiscal year, and management indicates that results will be captured through performance reporting and evaluations.

According to the Evaluation of the Building Excellence in HIV Knowledge Exchange project, the National HIV/AIDS and Knowledge Exchange Fund reaches front-line workers through conferences, education and training sessions and web conferencing. This organization reports that it is now Canada's largest distributor of HIV print resources, disseminating more than one million resources a year.

The Vaccine Initiative developed a Communications Engagement Strategy to establish stakeholder and public communication measures regarding the implementation of the initiative. This was a deliverable under the CHVI Formative Evaluation Management Response and Action Plan, which was approved in April 2012. A non-governmental organization, funded as a recipient of the Vaccine Initiative is responsible for developing communications products that include website maintenance, news releases and a review of progress reports.

In conclusion, the Agency has been involved in various methods of communicating with the public that include community-based social marketing campaigns, to provide HIV/AIDS awareness to targeted populations.

3.7 Program intervention

3.7.1 Strategy

Audit criterion: The HIV/AIDS programs have a strategy to identify recipients and a process for the strategic allocation of funds for front-line programs for priority populations.

The program intervention area of action focuses on the delivery of community-based programs, Aboriginal health programs and targeted interventions. The grants and contributions administered through the Federal Initiative have specific funds geared towards priority populations (see Appendix D).

In 2013-14, the Agency funded one grant and 147 contribution agreements within the Federal Initiative (approximately $24 million) and three contribution agreements were funded through the Vaccine Initiative ($1 million). The three national funds administered for the Federal Initiative are the National HIV/AIDS and Knowledge Exchange Fund, the National HIV/AIDS Voluntary Sector Response Fund and the Non-Reserve First Nations, Inuit and Métis Communities HIV/AIDS Project Fund. The AIDS Community Action Program funding stream included in the Federal Initiative supports community-based organizations, to reach people at risk and those living with HIV/AIDS. In all jurisdictions, the funded organizations collaborate or partner with provincially/territorially funded regional or local public health and social service agencies (see Appendix D).

Recently, the Agency has centralized its grants and contributions program. The Centre for Communicable Diseases and Infection Control now makes all program funding decisions, and the Centre for Grants and Contributions in the Office of the Chief Financial Officer provides financial monitoring services. The role of the regions, which has recently been clarified in a memorandum of agreement, will be to monitor and provide support for funded projects including teleconferences and site visits with funding recipients.

In addition to the change noted above, the Agency is currently transforming its grants and contributions programs. As part of this effort, HIV/AIDS and Hepatitis C transfer payment programs are being consolidated. The Agency reports that the new HIV/AIDS and Hepatitis C Community Action Fund will be in place by 2017 and that the funding amount will remain stable at $26.2 million annually. The number of funds and the delivery mechanism for the new model have yet to be finalized. Many of the agreements were up for renewal in the last fiscal year. Given the transformation process underway, the Agency decided to extend the agreements for an additional three years. The change to the grants and contributions should also form part of the renewed HIV/AID strategy and implementation plan (see Recommendation 1).

3.7.2 Compliance

Audit criterion: Grant and contribution agreements comply with the Policy on Transfer Payments, including funding agreement provisions.

The objective of the 2008 Policy on Transfer Payments is to ensure that transfer payments are managed with integrity, transparency and accountability in a manner that is sensitive to risks, are citizen and recipient-focused, and is designed and delivered to address government priorities in achieving results for Canadians. The policy provides a list of requirements that are applicable to the contribution agreements under review. A sample was tested for compliance and to validate whether the Agency has captured all of the required terms and conditions within the contribution agreements and documented and established processes to identify, select and monitor recipients and release payments.

The audit selected 26 contribution agreements from both of the initiatives for testing. Fifteen samples were selected among the national funds. All national recipients were ranked as medium risk (except one which is low), and all AIDS Community Action Program funding recipients were ranked as low risk.

*This contribution agreement has funds that are devoted to both HIV/AIDS and Hepatitis C.

The results of the testing indicated that the contribution agreements captured all of the required terms and conditions.

Recipient monitoring - risk assessment

Health Canada, the Agency and the Canadian Institutes of Health Research implemented the Health Portfolio Integrated Risk Management Framework for grants and contributions. The framework reflects senior management's risk tolerance and establishes the mandate for risk management tools and processes, including using the automated tool, the Enterprise Risk Management Agreement/Recipient Risk Assessment Tool (ERM-ARRAT).

The automated tool allows program officers to assess the risks associated with grants and contributions. The aim is to assist program areas in managing risks associated with funding agreements (projects) and recipients. The automated tool calculates risk scores for the funding agreement based on an assessment (rating) of risk factors. These risk scores determine which risk mitigation activities will be carried out to manage agreements, based on the assessed level of risk. The complete set of risk mitigation activities is referred to as the Risk Tolerance Strategy. This exercise classifies risks in three categories: low, medium or high. As a result, various actions may be undertaken such as restrictions in advances, the withholding of payments or even a recipient audit of high risk recipients. The audit found that a risk assessment was documented in all of the sampled contributions agreements.

Recipient monitoring – accountability

Each contribution agreement has reporting requirements that the recipient must provide in a timely manner to allow program managers to keep track of the recipient's activities. If a recipient's reporting is late beyond an agreed-upon timeframe (generally 45 days), the program may withhold funding on non-mandatory programs. In the sampled contribution agreements, reporting requirements included interim and annual financial statements and the completion of the Project Evaluation and Reporting Tool (PERT) and cash flows. The tool is an annual exercise that must be completed by recipients, in accordance with their contribution agreement.

The recipients' reporting requirements vary based on the risk assessment result. Sample testing noted that all but one of the national fund recipients was ranked as medium risk. Testing also noted that all AIDS Community Action Program contributions were ranked as low risk. The Vaccine Initiative agreements were ranked as medium risk. Recipients with low risk are only required to provide an annual report and two cash flows, while a medium risk recipient is required to provide a progress report, an annual report and three cash flows.

Through testing, it was noted that the sampled contribution agreements provided evidence that the HIV/AIDS programs received accountability documents in accordance with the terms and conditions of the contribution agreements, as well as the required reporting documents, as per the risk ranking.

Release of payments

A sample of 26 payment transactions from 26 contribution agreements was tested to provide assurance that the payments were made in compliance with the requirements of the Financial Administration Act and in compliance with the terms and conditions of the contribution agreements. The test was to determine that the documentation submitted was in accordance with the requirements established in the agreements and to determine that the documentation was sufficient to support payment. Also, payments were tested for accuracy and compliance with Section 32 (commitment of funds) and Section 34 (certification that goods and services have been delivered), as per the funding agreement and the Financial Administration Act.

All sampled payment transactions showed evidence that the process and controls were operating consistently and that payments were released in accordance with the Financial Administration Act and the funding agreements. In addition, approval was performed by an authorized officer. It was noted, however, that the review of cash flows was often not fully documented. Cash flows were reviewed using checkmarks, but there was a lack of signature providing evidence that the cash flow had in fact been reviewed. Release of payments is based on the recipient's reporting plan, which is part of the contribution agreement. Prior to receiving payment, recipients often have to provide progress reports, cash flows or financial statements (whichever is in keeping with the reporting requirements).

Eligible expenses

The audit found evidence that eligible expenditures had been reported by the recipients and approved through the examination of cash flows by the Centre for Grants and Contributions. Supporting documentation included a work plan prepared by the recipients, describing what the recipient intends to achieve with the funding, and the assessment by program managers.

3.8 Canadian HIV Vaccine Initiative

Audit criterion: The Vaccine Initiative has met commitments related to clinical trial capacity and regulatory approaches.

In 2006, the Government of Canada signed an agreement with the Bill and Melinda Gates Foundation, with the primary goal of accelerating the development of HIV vaccines. Since 2006, there has been enhanced capacity for clinical trials. As well, there has been a significant increase in private sector HIV vaccine pilot scale manufacturing facilities in North America and Europe. Thus, the agreement renewal was approved in 2010. Its primary purpose was to accelerate the development of a safe, effective HIV vaccine by building on Canada's scientific excellence, for the benefit of the most-in-need, particularly in Africa but also in Canada.

The Strategic Issues and Integrated Management Division within the Centre for Communicable Diseases Infection and Control is the Agency lead for this program. Its role is to support coordinating efforts (contribution agreement in place), translate basic science into clinical trials (contribution agreement to be developed) and address enabling conditions (contribution agreement in place).

Funds for the program are used through two established contribution agreements: the Global HIV Vaccine Enterprise and the Alliance Coordinating Office. To address enabling conditions, a contribution agreement is in place with the Global HIV Vaccine Enterprise. The objectives of this activity are to strengthen the regulatory capacity, including the development of national regulatory authorities, to promote the exchange of best practices and protocols in health policy approaches and to strengthen existing mechanisms to support community involvement in vaccine research, clinical trials and activities related to public awareness and education. For supporting coordinated efforts, the Agency has a contribution agreement in place with the Alliance Coordinating Office.

The Agency has one non-voting representative who participates in Vaccine Initiative Advisory Board meetings and in Outcome meetings. To validate the status of the Initiative, the agreement states that semi-annual progress reports are to be assessed by the Government of Canada and the Bill and Melinda Gates Foundation, for planning and monitoring purposes. The Alliance Coordinating Office prepares quarterly progress reports to the Agency and reports to the Advisory Board on activities at each board meeting. The Agency's Departmental Performance Report provides annual reporting on the overall Vaccine Initiative on behalf of all Government of Canada partners.

3.9 Monitoring and reporting

Audit criterion: HIV/AIDS progress is measured and reported, to inform overall program effectiveness.

Performance measurement efforts within the Federal Initiative are guided by the Horizontal Results-based Management and Accountability Framework, which was last revised in 2010. The framework provides a structure to help demonstrate and measure performance against the desired outcomes of the Federal Initiative, and includes a logic model that articulates activities, outputs and outcomes, as well as a performance strategy and a data collection and reporting plan. Key monitoring and reporting deliverables highlighted in the framework and produced by the Agency include the Report on Plans and Priorities, the Departmental Performance Report and the Global AIDS Response Progress Report to the United Nations.

The Report on Plans and Priorities and Departmental Performance Report are submitted annually to the central agency to satisfy accountability requirements, as defined in the funding agreement. The Departmental Performance Report was last completed for the 2012-13 fiscal year, while the Report on Plans and Priorities was completed for 2013-14. The Departmental Performance Report produces a consolidated annual budget-to-actual analysis that summarizes expenditures by key activity area and by Federal Initiative responsibility centre. The Federal Initiative Performance Report consolidates information from all Federal Initiative responsibility centres and provides an account of annual progress. This report was last produced in January 2014 for 2012-13.

A Government of Canada Report to the Secretary General of the United Nations on the United Nations General Assembly Special Session on HIV/AIDS Declaration of Commitment on HIV/AIDS is a biennial report on progress in meeting the targets agreed-upon by the United Nations. The Agency reports that it submitted an interim report in April 2013 (new UNAIDS requirement) and the 2014 report (for 2012 and 2013) was submitted to UNAIDS.

In addition, the funding agreement references expectations for the production and publishing on the Agency website, of a collective annual report on the Federal Initiative activities and on the outputs and progress towards meeting the expected outcomes for World AIDS Day. It has since been decided to use the Departmental Performance Report for this purpose.

While it has made progress in implementing a performance measurement framework for the Federal Initiative, the Agency would benefit from improving the alignment between defined performance indicators and the current activities of the Federal Initiative and improving the consistency of measurement within responsibility centres; from addressing measurement gaps, including those identified at an "outcome" level; and from introducing performance targets and comparative baseline data to better establish performance expectations.

Program representatives have recognized a need to revise the performance measurement framework, to better respond to the Agency's strategic shift to a more holistic approach to addressing HIV/AIDS. This could increase the complexity of central agency reporting should funds continue to be reported against separate performance measurement approaches.

The Vaccine Initiative

As part of the 2010 renewal, the Agency committed to establishing a performance measurement framework, which was completed in 2011. The performance measurement framework includes a logic model that articulates activities, outputs and outcomes and defines performance indicators and a data collection and reporting plan. However, as of April 2014, the performance measurement framework had not yet been fully implemented. While performance indicators have been defined in relation to program "outputs" and "outcomes", performance information is not regularly collected or reported beyond reporting through the parliamentary reporting process. The framework references an expectation that the Agency will develop annual progress reports to summarize the progress achieved against the outcomes identified in the program logic model. Because this information is not being made available on a timely basis, it is unclear what progress is being made against defined initiative objectives, and whether that progress meets management expectations.

In conclusion, there is a need to update the performance measurement framework supporting the management of both the Federal Initiative and the Vaccine Initiative. These efforts should be identified in a strategic plan and an implementation plan (see Recommendation 1).

C - Conclusion

The Agency's strategy in addressing HIV/AIDS under the Federal Initiative is evolving towards a more holistic approach that encompasses HIV/AIDS and other sexually transmitted and blood-borne infections and related health factors such as chronic diseases and mental health. This approach is expected to build upon collective efforts to date in these various areas. While progress is underway in support of this transition, the Agency has yet to completely articulate the full nature of this strategic change, its impacts and the transition planning required to effect the change.

The Agency's shift towards a more holistic approach should continue to examine the impacts on the external and internal governance structures, including a clarification of intra-Agency roles and responsibilities related to collaboration. As well, surveillance, reference services and testing should be reviewed to ensure alignment with the new direction. A large part of the HIV/AIDS programs is delivered through contribution agreements. This allows the Agency to target priority populations in the delivery of front-line programs. It is the intent of the programs to consolidate their HIV/AIDS and Hepatitis C transfer payments. This initiative should also be integrated into the strategic plan. Finally, given the changes noted, it would be important to revise the performance measurement framework. Leading up to the development of the strategic plan, the program should update the program risks to inform the renewal process and strategic plan.

The Agency has integrated sex and gender-based analysis into its program operations and is meeting its international commitments towards strengthening the response to HIV/AIDS and supporting the implementation of communications and social media approaches. As well, the grant and contribution agreements were found to be in compliance with the Policy on Transfer Payments. Sampled contribution agreements provided evidence that the Agency received accountability documents in accordance with the terms and conditions of the contribution agreements, and received required reporting documents, in line with the risk ranking.

The Agency recognizes that it must continue to evolve in order to ensure its relevance in a complex public health environment. To guide the movement of the responsibility centres from a wide variety of existing processes and practices to a coherent approach, the Infectious Disease Prevention and Control Branch will benefit from implementing the four recommendations. This will demonstrate the branch's commitment to bringing greater coherence to the activities.

Appendix A - Lines of enquiry and criteria

Appendix B - Scorecard

Appendix C - 2012-13 Financial Summary for the Federal Initiative on HIV/AIDS

This annex summarizes the expenditures by activity for the Agency and other Government of Canada departments in delivering the Federal Initiative for 2012-13 (* see note 1).

*note 1 – unaudited by Portfolio Audit and Accountability Bureau

Appendix D - 2013-14 Grants and Contributions Expenditure Summary