Language selection

Government of Canada / Gouvernement du Canada

Search

Evaluation of Ministerial Instructions (Implementation)

Executive summary

Purpose of the evaluation

This report presents the findings of the implementation evaluation of the first set of Ministerial Instructions (MI), which were issued in November 2008 on the basis of a legislative amendment made to the Immigration and Refugee Protection Act (IRPA) earlier that year. In keeping with the requirements of the Directive on the Evaluation Function (TBS, 2009) and the Treasury Board commitment for an implementation evaluation of Ministerial Instructions during the second year of operation, the purpose of the evaluation was to provide an objective assessment of:

  • the continued relevance of MI as a legislative tool;
  • the design and implementation of the first set of Ministerial Instructions (MI1);
  • the performance of MI1 in achieving selected results; and
  • the cost-effectiveness of the Centralized Intake Office (CIO).

It is important to note that although the original subject of this evaluation included MI1 and the CIO, it was expanded to include an assessment of the initial impact of MI2, which came into effect on June 26, 2010. The fact that MI2 had been in effect for almost a year at the time the data collection was taking place, and that it resolved some of the major issues with MI1, made it very relevant to this study.

While the plan for the evaluation was to look only at the cost-effectiveness of the CIO, some of the research conducted was also relevant to the question of whether the CIO is achieving its intended results.

Because the evaluation was conducted approximately two years following the announcement of MI1, it looks only at initial and intermediate results, and not at the longer-term outcome of labour market responsiveness. However, to the extent that applications can be shown to have been processed more quickly, wait times reduced and the eligible NOC codes drawn from high demand occupations, we would expect MI to be more responsive to the Canadian labour market in the long term.

Ministerial instructions

The federal government elected to respond to the backlog of FSW applications by introducing amendments to the Immigration and Refugee Protection Act (IRPA), through Bill C-50, the 2008 Budget Implementation Act. Bill C-50, which came into effect on February 27, 2008, made a number of fundamental changes to the way in which immigration applications are managed: it eliminated the (previous) obligation to process all applications received; and authorized the Minister to issue instructions (Ministerial Instructions) to immigration officers regarding which applications were eligible for processing, based on the government’s overall goals for immigration. Under these “MI authorities,” the Minister had the power to limit the numbers of applications processed, accelerate some applications or groups of applications, and return applications without processing them to a final decision.

The first set of Ministerial Instructions (MI1), which articulated how the Minister would operationalize the authority inherent in the changes to IRPA, were announced as part of CIC’s Action Plan for Faster Immigration on November 28, 2008, and were directed to the FSW Program. MI1 had three primary objectives:

  • to control FSW application intake; and
  • reduce the backlog of FSW applications by 50% by 2013; while
  • remaining responsive to the Canadian labour market.

Centralized Intake Office (CIO)

In conjunction with MI, and as part of the department’s Modernization Agenda, CIC decided that the initial eligibility assessment of new FSW applications would be centralized in Canada. The Centralized Intake Office (CIO) was established in the Case Processing Centre (CPC) in Sydney, Nova Scotia, which was already undertaking citizenship and permanent resident card processing, and so had the infrastructure in place to administer the receipt and initial processing of FSW applications.

The goals of the CIO were:

  • To expedite the front-end processing of applications, thereby increasing the efficiency of the overall process;
  • To reduce the workload in the missions, freeing mission staff to focus on non-clerical tasks;
  • To provide consistency in implementing the Ministerial Instructions and assessing FSW applications; and
  • To facilitate the management of fees.

There were subsequent administrative changes made at the CIO to coincide with MI2, the most important of which was that the final eligibility decision was made at the CIO, rather than at missions. Under this new process, missions could not reverse a positive eligibility decision, although they could refuse the application on other grounds. In addition, the CIO also awarded points as per the FSW grid and refused cases that failed to attain 67 points.

Evaluation methodology and limitations

Data collection for this evaluation took place between March and June, 2011. The time period that is covered by the evaluation extends from the time that the 2008 Budget came into effect (June 18, 2008) until the end of the data collection phase.

Several lines of enquiry, including both quantitative and qualitative lines of evidence, were used for the evaluation:

  • document review;
  • interviews;
  • review of administrative data;
  • survey of Canadian Visa Offices Abroad (CVOA); and
  • CIO site visit

Although the evaluation included a good balance of quantitative and qualitative lines of enquiry, and allowed for the triangulation of results, there were two notable limitations to the methodology:

  • The potential for confusion between MI1 and MI2, given the timeframe in which data collection took place. This limitation was mitigated by ensuring that the discussions with key informants always distinguished between the two sets of Instructions. With respect to the survey, where it was not possible to probe, contextual evidence was used to determine if there was some confusion between MI1 and MI2 and, where this was the case, responses were not included in the analysis.
  • CIC maintains an activity-based financial system, the Cost Management Model (CMM), that is generally very useful for an analysis of processing costs. However, with respect to MI, it was not sufficiently detailed to be able to conduct a cost-effectiveness analysis of the introduction of the CIO. In order to address this problem, the evaluation expanded its assessment of the CIO to determine whether it is achieving its other objectives, all of which would contribute to the efficiency of the overall process.

Key findings

Relevance

  • All interviewees and two-thirds of CVOA survey respondents agreed that MI continues to be relevant as a legislative tool. The fact that MI has been used to make subsequent processing changes, both for FSW applications and for those in other immigration programs, supports this finding.

Design and implementation

  • The majority of study informants rated the flexibility of the MI authorities as “excellent” or “good”. The fact that, when the limitations of the first set of Instructions were identified, the Minister was able to introduce a second set relatively quickly to address these issues, indicates that the design of MI is very responsive.
  • Under MI1, CIC established a number of monitoring tools, which were seen as being very effective by virtually every key informant. The timeliness and quality of the information gathered from the monitoring processes made it possible for CIC to make relevant adjustments in a timely manner to policy and/or operations.
  • Both the proportion of incomplete applications and the proportion that ended in negative eligibility assessments decreased fairly quickly following the announcement of MI1, suggesting that immigrants and immigration consultants generally understood the criteria. The majority of NHQ informants and mission survey respondents were of the opinion that immigrants understood MI1.

Performance: Achievement of intended results

  • The initial processing time for MI1 applications fell within the target timeframe of 6-12 months. However, as applications continued to flow in at a higher rate than forecasted, the processing time increased. For the typical MI1 applicant whose case had been decided by May 2011, the time between submitting an application and the final positive case decision at CVOA was 412 days, about a month and a half above the upper limit target. However, while MI1 did not maintain the specific 6-12 month processing time objective, it was still faster than the previous FSW system: an average of 13.6 months, in comparison to 25.5 months for the period from January 2006 to February 2008 (pre-MI).
  • Most interviewees at NHQ and those surveyed in the missions thought the consistency in CIO decision-making, from one decision to the next, was high. With respect to consistency between the CIO and CVOA decisions, the assessment by key informants was somewhat lower. However, the percentage of CIO decisions that were reversed at the missions due to a differing assessment of eligibility was fairly low, between 5 and 10%.
  • The intent, under MI1 was to limit the number of applicants who could apply as FSWs, thereby freeing up time to process the pre-MI backlog. Although the number of applications received was initially quite low, it rose steadily over the delivery period of MI1, from 5,000 applications in the first quarter (12 weeks) of 2009, to 25,000 in the second quarter of 2010. This represented the highest intake seen prior to the introduction of Bill C-50.
  • With the implementation of MI1, CIC set a target of reducing the pre-MI backlog by 50% by 2013. By April, 2011, they had achieved this goal, two years ahead of schedule. However, because of the increasing volume of applications under MI1, a backlog of MI1 applications had developed during this time. This new MI1 backlog, together with the pre-MI backlog, represented a total reduction of 23% since February 2008, when the first set of Instructions were issued.
  • Nearly every key informant mentioned at some point during the interview that the major strength of MI1 was that it provided essential information that informed the development of MI2, which was viewed as superior to MI1 in controlling application intake, increasing processing efficiencies and reducing the clerical workload at missions.
  • Additional administrative changes were made at the CIO to coincide with MI2, the most important of which was that the final eligibility decision is now made at the CIO, rather than at missions. Many key informants — most CVOA interviewees and survey respondents, many CIO interviewees, and some at NHQ — identified this as potentially problematic because CIO staff don’t have sufficient local knowledge to be able to detect fraudulent applications.

Performance: Economy and efficiency

  • Although it was not possible to quantify the impact of the introduction of the CIO on FSW processing costs, there is some evidence, related to the CIO’s original objectives, that centralization of the initial assessment of applications is more efficient than the previous model: the application processing time and clerical workload in missions were reduced; and consistency in decision-making and record-keeping were improved.

Conclusions

  • There is a continued need for CIC to be able to manage the intake and processing of immigration applications in a timely, efficient and responsive manner.
  • The design of MI is both flexible and responsive, which is the key to its success as a policy tool.
  • Although MI1 temporarily reduced the intake of FSWP applications, the numbers received rose very quickly to the pre-MI1 level. However, the data systems established as part of MI1 allowed for the early identification of this increase, and led to MI2, which has been much more successful in limiting intake.
  • While MI2 has been more successful than MI1 in limiting applications, the transfer of the eligibility decision to the CIO under MI2 was viewed as potentially problematic, particularly by staff in the missions. The scope of this evaluation did not allow for an assessment of the impact of this change.
  • The implementation of MI1 contributed to a substantial reduction in the backlog, although there was a subsequent, and unanticipated, development of an MI1 backlog.
  • While it was not possible to conclude that CIC costs were reduced as a result of centralizing the front-end processing of FSW applications, the CIO did achieve a number of objectives that contribute to improving the efficiency of the overall process.

Recommendations

Recommendation 1: CIC should conduct a more formal assessment of the impact of further centralization of the processes historically undertaken in missions. Specifically, this assessment should examine the risks associated with centralizing decision-making, particularly in relation to the potential of not detecting fraud. This study should also include the identification of mitigation strategies, as required.

Recommendation 2: As part of its overall approach to program performance measurement, CIC should ensure that there is financial data that is sufficiently robust and detailed to support the on-going analysis, and periodic evaluation, of program costs.

The research for the evaluation identified a number of findings that were not, by themselves, sufficient to support a recommendation, but were suggestive of particular actions or further considerations by the department. These are presented below.

Recommendation 3: The department should consider each of the following observations, investigate further, as required, and decide on how best to proceed:

  • Communications
    CIC should permit and facilitate direct communications between missions and the CIO. The implementation of a feedback loop could help to identify any systematic errors and improve decision-making. Lessons learned through interaction with individual missions should be shared across the network.
  • Electronic Application
    The department should expedite the move to an e-application for the FSW Program, particularly now that GCMS has been fully implemented.
  • Fee Payment
    The electronic application platform should facilitate fee payment through such means as PayPal, etc. Also, consideration should be given to charging a fee for processing applications that are determined to be ineligible. This will help to cover some of the associated costs and deter applicants who know they are unlikely to be successful.
  • CIO Pilot Status
    Given that it appears to have a continued and increasing role in processing FSW applications, the CIO should be designated a permanent operation.

Page details

Date modified: