Evaluation of the Canada-Northwest Territories Labour Market Development Agreement

The LMDA investment

In fiscal year 2020 to 2021, Canada transferred approximately $4.8 million (including nearly $1.5 million in administration funds) to the Northwest Territories.

Table i provides an overview of the share of funding allocated to EBSMs and the average cost per participant. The average cost per participant is calculated based on the 2009 to 2012 data from the EI Monitoring and Assessment Reports. The 2009 to 2012 period corresponds with the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

• Sources: EI Monitoring and Assessment Reports for fiscal years^{Footnote 4} 2010 to 2011 and 2011 to 2012.

Compared to the 2010 to 2012 period, the LMDA budget allocation varied for some programs and services in 2020 to 2021. For example, investments in Building Essential Skills decreased from 51% to 26%, and investments in Training-on-the-Job increased from 5% to 20% of EBSM expenditures in the Northwest Territories.

Evaluation objectives

Building on the success of previous LMDA evaluation cycles, the aim of this evaluation is to fill in knowledge gaps about the effectiveness, efficiency, as well as the design and delivery of EBSMs in the Northwest Territories.

Evaluation methodology

The findings in this report are drawn from 8 separate evaluations. These studies examine issues related to program effectiveness, efficiency, and design and delivery. A mix of qualitative and quantitative methods are used, including:

• incremental impact analysis for participants who began an intervention between 2009 and 2012
• outcome analysis
• cost-benefit analysis (including savings to health care)
• key informant interviews with 19 territorial program officials, caseworkers, service providers, and project holders
• questionnaires
• document and literature reviews

The incremental impacts are estimated for 2 types of EI claimants:

• active claimants are participants who started an EBSM intervention while collecting EI benefits
• former claimants are participants who started an EBSM intervention up to 3 years after the end of their EI benefits^{Footnote 5}

Key findings

Across the Northwest Territories, nearly 1,670 EI active and former claimants began participating in LMDA programs and services between 2009 and 2012.

Effectiveness and efficiency of EBSMs

• overall, incremental impacts and labour market outcomes demonstrate that participation in some EBSMs improves labour market attachment and reduces use of government income supports
• labour market outcomes analysis for Training-on-the-Job found that annual earnings were higher in the post-participation period and that use of EI benefits was lower
• a subgroup analysis of labour market outcomes shows that all sub-groups of participants in Building Essential Skills and EAS had higher annual earnings and used EI benefits less in the post-participation period
• the social benefits of participating in EBSMs exceeded initial investment costs for most interventions over time

Chart i presents the incremental impacts on the incidence of employment for active and former claimants by EBSM. The estimates can be interpreted as a chance in the probability of being employed following participation. For example, participation in Building Essential Skills increases the probability of being employed by 7 percentage points for former EI claimants relative to non-participants.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart i

Program name	Incidence of employment for active claimants (percentage points)	Incidence of employment for former claimants (percentage points)
Building Essential Skills	3.4*	7
Employment Assistance Services	-5.2*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Chart ii presents the annual average in employment earnings for active and former claimants over the post-participation period. For example, active EI claimants who participated in Building Essential Skills, on average, earned $5,870 more than similar non-participants.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart ii

Program name	Employment earnings of active claimants	Employment earnings of former claimants
Building Essential Skills	$5,870	$8,125
Employment Assistance Services	$1,148*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

As shown in Chart iii, active claimants who participated in Building Essential Skills reduced their use of government income supports by 4.1 percentage points post-participation.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart iii

Program name	Use of government income support for active claimants (percentage points)	Use of income support for former claimants (percentage points)
Building Essential Skills	-4.1	-2.8*
Employment Assistance Services	-2.6*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Table ii presents the number of years required for the social benefits to exceed program costs. Social benefits to participation exceed initial investment costs over a period ranging from within the program start year to 1.6 years.

Table ii. Number of years for the benefits to exceed program costs^{Footnote 6}

Indicator	Building Essential Skills active claimants (10 years post-program)	Employment Assistance Services active claimants (5 years post-program)	Building Essential Skills former claimants (10 years post-program)
Payback period (years after end of participation)	1.3	<11	1.6

• ¹Paid back within the program start year.

Recommendations

Since 2012, 15 qualitative and quantitative studies addressed issues and questions related to EBSM design, delivery and effectiveness. The quantitative studies successfully assessed the effectiveness and efficiency of EBSMs by producing incremental impacts and cost-benefit analysis. The qualitative studies help to contextualize the findings from the quantitative studies and to identify specific challenges, lessons learned and best practices associated with the design and delivery of EBSMs. Each study included key considerations for program and policy development or recommendations

In addition, the recently completed evaluation of the Workforce Development Agreements complements the LMDA qualitative studies. This comprehensive evaluation provided unique insights into challenges and lessons learned to assist persons with disabilities, immigrants and those further removed from the labour market.

Most results from this evaluation stem from the conduct of advance causal analysis whereby impacts found could be attributed to a specific EBSM. These analyses are predicated on having access to high quality administrative records, thereby confirming the importance of the capacity to leverage and integrate relevant administrative data.

From these main findings, 2 key recommendations for the Northwest Territories emerge.

Evaluation methodology

The findings in this report are drawn from 8 separate evaluation studies carried out at the territorial level. These studies examine issues related to program effectiveness, efficiency, and design and delivery.

The 2009 to 2012 timeframe was selected in order to assess the impacts of EBSMs in the years following participation. Impacts were assessed over a period of at least 4 years after program completion up to the 2017 calendar year (most recent available information at the time of this evaluation).

Programs and services examined in this study include:

• Employment benefits:
  • Building Essential Skills
  • Training-on-the-Job
  • Job Creation Partnerships
  • Self-Employment Option

• Support measures
  • Employment Assistance Services^{Footnote 7}
  • Strategic Workforce Initiatives
  • Research and Innovation

From these main findings, 2 key recommendations for the Northwest Territories emerge.

Recommendation #1

The Northwest Territories are encouraged to share and discuss lessons learned, best practices and challenges associated with the design and delivery of program and services. Discussions are encouraged with ESDC, at the bilateral or multilateral levels as well as with the territorial service delivery network if necessary.

Recommendation #2

The Northwest Territories are encouraged to pursue efforts to maintain and strengthen data collection provisions in support of reporting, performance measurement and data-driven evaluations at the national and territorial levels.

Employment benefits

Building Essential Skills provides direct financial assistance to individuals to select, arrange, and pay for training. Training is tailored to the needs of participants through counselling and career orientation. It can include adult-based education, literacy and essential skills, language training, short-term training and occupational training leading to certification from an accredited institution.

Training-on-the-Job participants obtain on-the-job work experience by providing employers with a wage subsidy.

Job Creation Partnerships (JCP) provide participants with opportunities to gain work experience that will lead to ongoing employment. Employment opportunities are provided by projects that contribute to developing the community and the local economy.

Self-Employment Option aims to assist participants in creating employment for themselves by providing them with a range of services.

Support measures

Employment Assistance Services (EAS) supports individuals as they prepare to enter or re-enter the workforce or assists them to find a better job. Services can include job search services, career development and counselling, and résumé writing assistance. These services are referred to as ‘light touch interventions’ due to their very short duration. They can be provided on a one-on-one basis or in a group setting.

Strategic Workforce Initiatives aim to support community partners in undertaking labour market activities that promote labour force development, workforce adjustments and effective human resources planning.

Research and Innovation aims to fund previously untried solutions to specific labour market issues or problems and create strategic investment mechanisms to fund experiments that could inform the development of policies, programs or industry practices.

Eligible participants covered in this study

The incremental impacts are estimated for active and former EI claimants:

• active claimants are participants who started an EBSM intervention while collecting EI benefits
• former claimants are participants who started an EBSM intervention up to 3 years after the end of their EI benefits^{Footnote 14}

Average EBSM share of funding and cost per Action Plan Equivalent

Table 1 provides an overview of the share of funding allocated to EBSMs and the average cost per Action Plan Equivalent for active and former claimants in the Northwest Territories. It is noted that the average cost per participant is calculated based on the 2009 to 2012 data from the EI Monitoring and Assessment Reports. The 2009 to 2012 period corresponds with the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

From the 2010 to 2012 time period to the 2020 to 2021 fiscal year, investments in Building Essential Skills decreased by 25 percentage points. The largest increases in funding are noted for Training-on-the-Job (+15 percentage points) and the Strategic Workforce Initiatives program (+8 percentage points).

• Sources: EI Monitoring and Assessment Reports for fiscal years 2010 to 2011, 2011 to 2012, and 2020 to 2021.

Incremental impacts analysis^{Footnote 17}

Program effectiveness is assessed by estimating incremental impacts from EBSM participation on participants’ labour market experience. That is, earnings from employment and self-employment, incidence of employment, use of EI, use of social assistance SA and dependence on government income supports after participation.

In the Northwest Territories, incremental impacts were estimated for active and former EI claimant participants in Building Essential Skills, Training-on-the-Job and active EI claimant participants in EAS. Incremental impacts for JCP were not produced due to the small samples or due to the fact that the models used to estimate the incremental impacts did not balance. Having balanced models is an essential condition to ensure that participants and non-participants are similar.

The role of the incremental impact analysis is to isolate the effects of participation from other factors. In order to achieve this, the incremental impact analysis compares the labour market experience of participants before and after their participation with that of similar non-participants. Figure 1 presents an example of incremental impact calculation.

Text description – Figure 1

Incremental impacts are the difference in the pre-post participation outcomes between participants and similar non-participants.

For example:

• if the average annual earnings of participants are $30,000 before participation and $38,000 after participation, the change in earning for participants is an increase of $8,000
• if the average annual earnings in the comparison group are $31,000 before participation and $36,000 after participation, the change in earnings for the comparison group is an increase of $5,000
• the incremental impact, that is the change attributed to program participation, is $8,000 minus $5,000, which equals an increase of $3,000

The main estimator used is propensity score kernel matching technique combined with difference-in-differences estimator. Moreover, 3 different state-of-the-art estimation techniques (Inverse Probability Weighting, Nearest Neighbour and Cross-sectional Matching) were carried out separately for each type of EBSMs and EI claimants in order to validate the impact estimates.

As for previous LMDA evaluation studies, the Action Plan Equivalent is the unit of analysis used. Action Plan Equivalents regroup all EBSMs received by an individual within less than 6 months between the end of 1 EBSM and the start of the next. Action Plan Equivalents are categorized based on the longest EBSM they contain, except for EAS-only Action Plan Equivalents which include only EAS interventions.

The analysis includes Action Plan Equivalents that consist only of LMDA interventions. Action Plan Equivalents that include a combination of LMDA and other labour market programs funded by ESDC, were excluded from the participant sample.

The matching of participants and comparison group members used up to 75 socio-demographic and labour market variables observed over 5 years before participation. Two different comparison groups were used to measure impacts for active and former EI claimants.

For active claimants, incremental impacts were measured relative to a comparison group of active claimants who were eligible to, but did not, participate in EBSMs during the reference period.

For former claimants, the comparison group was created using individuals who participated in EAS only during the reference period.^{Footnote 18} In other words, the experience of former claimants in Building Essential Skills interventions is compared to the experience of former claimants who received EAS only. This is a conservative approach given the fact that participation in EAS can lead to limited effects on labour market outcomes.

Due to this difference in measurement, incremental impacts estimated for active claimant participants should not be directly compared to those of former claimant participants.

Impacts are generated over 4 years for Building Essential Skills, while a fifth year is estimated for participants in EAS.^{Footnote 19}

Medium-term outcomes analysis

The study reports on the labour market outcomes for active and former EI claimants EAS, and former EI claimant participants in Training-on-the-Job. When the number of participants was sufficient, outcomes were examined for active and former EI claimants who were female, male, youth (30 years old and under), and Indigenous.

The study includes up to 100% of participants who began their EBSM participation between April 1, 2009, and December 31, 2012. The timeframe was selected in order to assess the outcomes of EBSMs in the years following participation.

The analysis of outcomes provides descriptive statistics on the labour market experience of participants before, during and after participation. For example, it shows the average annual earnings of active claimants before, during and after participation, and presents what changes were observed from before to after participation. Overall, the analyses were conducted over a 10 to 11 year period:

• a pre-participation period of 5 years for Building Essential skills, Training-on-the-Job and EAS
• a participation period of 2 years for Building Essential Skills, and of 1 year for Training-on-the-Job and EAS
• a post-participation period of 4 years for Building Essential Skills, Training-on-the-Job and EAS

It is important to note that the labour market outcomes regarding employment earnings, EI and SA benefits are presented in constant 2010 dollars.

Factors accounted for in the cost-benefit analysis^{Footnote 20},^{Footnote 21}

Building on the results of the incremental impacts, program efficiency is assessed through a cost-benefit analysis. The analysis compares the participants’ cost of participating and the government’s cost of delivering the program to the benefits associated with the program. Overall, this analysis provides insights on the extent to which the program is efficient for society (that is, for both participants and the government).

Strengths and limitations of the studies

One of the key strengths from the studies is that all quantitative analyses are based on administrative data rather than survey responses. Compared to survey data, administrative data are not subject to recall errors or response bias.

The propensity score models used to match participants and non-participants for the incremental impact analyses are judged to be robust. In part this is because they were based on 5 years of pre-participation data. Moreover, these models are based on a vast array of variables including sociodemographic characteristics, location, skill level related to last occupation, and indicators of labour market attachment.

However, the matching process can be further refined for specific subgroups if the following information is available in the future:

• persons with disabilities: the type and severity of the disability, and the capacity/willingness to work full-time
• recent immigrants: the country of origin, the proficiency in English or French, and the relevance of credentials and work experience
• visible minorities: place of birth; individuals who are born outside of Canada face different challenges compared to those born in Canada

Refining the matching process for population subgroups could broaden the scope for greater Gender-based Analysis Plus.

Sensitivity analysis and the use of alternative estimation methods have increased confidence in the incremental impact estimates. However, one limitation with the propensity score matching techniques is that no one can be fully sure the impacts are not influenced by factors not captured in the data.

The cost-benefit analysis accounted for all quantifiable costs and benefits directly attributable to the EBSMs and could be estimated with the available administrative data. It is furthered strengthened by incorporating one of the indirect benefits, which is the change in public health care expenditures associated with program participation. However, the analysis did not account for non-quantifiable factors that can lead to an understatement of the benefits (for example, positive spillovers to other family members) and factors that can lead to an overstatement of the benefits (for example, effects on skill prices or displacement).

In some studies that use qualitative data collection methods, the number of key informants interviewed is relatively small in some the Northwest Territories. Responses provided by key informants reflect their own experience and may not be fully representative of the Northwest Territories.

Overview of the studies summarized in this report

The findings in this report are drawn from 8 separate studies carried out at the territorial level. These studies examine issues related to program effectiveness, efficiency, design/delivery and used a mix of qualitative and quantitative methods. Appendix A presents an overview of these studies. The studies are:

• Examination of the medium-term outcomes from 2009 to 2017
• Estimation of the medium-terms incremental impacts from 2009 to 2017
• Cost-benefit analysis of Employment Benefits and Support Measures in the Northwest Territories
• Cost-benefit analysis: incorporating public health care costs savings in the context of the labour market programs evaluation in the Northwest Territories
• Design and delivery of the Job Creation Partnerships program in the Northwest Territories
• Design and delivery of the Self-Employment Option program in the Northwest Territories
• Design and delivery of the Strategic Workforce Initiatives program in the Northwest Territories
• Design and delivery of the Research and Innovation support measure in the Northwest Territories

Main findings

• Overall, where results are statistically significant, incremental impacts demonstrate that Building essential Skills improves labour market attachment and reduces use of governmental income supports
• Medium-term outcomes show that following program participation, former EI claimants in Training-on-the-Job and active and former claimants in EAS:
  • had higher earnings
  • used EI benefits less, but used SA benefits more
  • had a lower incidence of employment, which could partially be explained by participants’ retirement decisions ^{Footnote 23}

• With few exceptions, a subgroup analysis of medium-term outcomes shows that female, male, youth, and Indigenous participants in Building Essential Skills and EAS generally:
  • had higher earnings
  • used EI benefits less, but used SA benefits more
  • had a lower incidence of employment, which could partially be explained by participants’ retirement decisions

• The social benefits of participating in Building Essential Skills and EAS exceeded the costs of investment over time

Profile of participants

In the Northwest Territories, nearly 1,670 EI active and former claimants participated in LMDA programs and services between 2009 and 2012.

The profile of participants is presented in Table 2 by gender, age, sociodemographic group, and marital status. Information about their educational attainment, occupation and industry is based on the last job they held prior to applying for EI benefits. Information about sociodemographic groups is self-reported.

• Note: Values may not equal 100% due to rounding or missing information.

Top 3 occupational groups for EBSM participants

Active claimants:

• other manual workers = 17%
• semi-skilled manual workers = 17%
• skilled crafts and trades = 15%

Former claimants:

• other manual workers =20%
• skilled crafts and trades = 17%
• semi-skilled manual workers = 14%

Top 3 industries for EBSM participants^{Footnote 24}

Active claimants:

• construction = 21%
• public administration = 17%
• mining, quarrying, and oil and gas extraction; and retail trade = 7% each

Former claimants:

• construction = 24%
• public administration = 19%
• accommodation and food services = 9%

As presented in Table 3, in the year before program participation, former claimants have lower levels of employment and earnings than active claimants. A greater percentage of former claimant participants used SA benefits.

Incremental impacts for active and former EI claimant participants in Building Essential Skills and active claimant participants in EAS

The incremental impact results presented below are generally consistent with those found as part of the second LMDA evaluation cycle.

Incidence of employment

Chart 1 presents the incremental impacts on the incidence of employment for active and former claimants by type of program.^{Footnote 25} The estimates can be interpreted as a change in the probability of being employed following participation.

Former EI claimant participants in Building Essential Skills increase their incidence of employment relative to similar non-participants.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart 1

Program name	Incidence of employment for active claimants (percentage points)	Incidence of employment for former claimants (percentage points)
Building Essential Skills	3.4*	7
Employment Assistance Services	-5.2*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Employment earnings

Chart 2 presents the average annual change in employment earnings for active and former EI claimants over the 4 years post-participation.

Active and former EI claimants in Building Essential Skills increase their employment earnings compared to similar non-participants.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart 2

Program name	Employment earnings for active claimants	Employment earnings for former claimants
Building Essential Skills	$5,870	$8,125
Employment Assistance Services	$1,148*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Use of EI benefits

The results presented in Chart 3 are not statistically significant over the entire post-program period. However, active claimants who received EAS had statistically significant decreases in their use of EI benefits of $1,269 and $863 in years 1 and 2 post-program.

• ^*The impact is not statistically significant over the entire post-participation period.
• ¹The impact is not statistically significant. However, participants decreased the use of EI by $1,269 and $863 in years 1 and 2 post-program participation respectively.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart 3

Program name	Use of EI benefits for active claimants	Use of EI benefits for former claimants
Building Essential Skills	-$665*	-$32*
Employment Assistance Services	-$3451	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• ¹The impact is not statistically significant. However, participants decreased the use of EI by $1,269 and $863 in years 1 and 2 post-program participation respectively.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Use of SA benefits

The results presented in Chart 4 are not statistically significant.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart 4

Program name	Use of SA benefits for active claimants ($)	Use of SA benefits for former claimants ($)
Building Essential Skills	-$128*	$346*
Employment Assistance Services	$184*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Use of government income support

As shown in Chart 5, active EI claimants in Building Essential Skills reduced their use of government income supports compared to similar non-participants.

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Text description – Chart 5

Program name	Use of government income support for active claimants (percentage points)	Use of government income support for former claimants (percentage points)
Building Essential Skills	-4.1	-2.8*
Employment Assistance Services	-2.6*	n/a

• ^*The impact is not statistically significant over the entire post-participation period.
• Note: Impacts are estimated over 4 post-program years (or 5 years in the case of EAS).

Outcomes for former EI claimant participants in Training-on-the-Job and active and former EI claimant participants in EAS ^{Footnote 26}

Incremental impacts for Training-on-the-Job were not produced due to low participant numbers or because the models used to estimate the incremental impacts not balancing. Having balanced models is an essential condition to ensure that participants and non-participants are similar. However, medium-term outcomes were estimated for active and former EI claimant participants in Training-on-the-Job and EAS. The outcomes describe the changes in participants’ earnings, use of EI and SA benefits, and incidence of employment that occur from before to after program participation.

The labour market outcomes are based on individuals who began their participation during the 2009 to 2012 period. Statistics focus on 5 years before and 4 years after program participation.

Between 2009 and 2012:

• 78 former EI claimants participated in Training-on-the-Job
• 286 active EI claimants and 513 former EI claimants received EAS

The outcomes for former claimants who received Training-on-the-Job should be interpreted with caution given the small number of participants (n=78).

Employment earnings

As shown in Chart 6, active and former EI claimants who participated in EAS and former claimants who participated in Training-on-the-Job had higher annual employment earnings during the post-participation period when compared to the 5 years before participation. For example, the average annual earnings of former EI claimants who participated in Training-on-the-Job were $9,826 higher in the post-participation period.

• Note: The outcomes for former claimants who participated in TOJ should be interpreted with caution given the small number of participants (n=78).

Text description – Chart 6

Program name	Employment earnings for active claimants	Employment earnings for former claimants
Training-on-the-Job	n/a	$9,826
Employment Assistance Services	$4,717	$6,201

• Note: The outcomes for former claimants who participated in TOJ should be interpreted with caution given the small number of participants (n=78).

Use of EI and SA benefits

As shown in Chart 7, compared to the 5 years before participation, the average proportion of participants who used EI benefits is lower in the post-participation period, while use of SA benefits is higher. For example, former claimants who participated in Training-on-the-Job decreased their use of EI benefits by 6 percentage points but increased the use of SA benefits by 7 percentage points post-participation.

Text description – Chart 7

Program name	Proportion of active claimant participants who received EI benefits (percentage points)	Proportion of former claimants- participants who received EI benefits (percentage points)
Training-on-the-Job	n/a	-6
Employment Assistance Services	-5	-18

Program name	Proportion of active claimant participants who received SA benefits (percentage points)	Proportion of former claimant participants who received SA benefits (percentage points)
Training-on-the-Job	n/a	7
Employment Assistance Services	2	7

Incidence of employment

As shown in Chart 8, participants’ incidence of employment was lower in the post-participation period. The lower incidence of employment can be attributed, in part, to participants’ retirement decisions. For example, while the incidence of employment of employed former claimant participants in Training-on-the-Job decreased by 4 percentage points, the percentage of participants receiving Canada Pension Plan and Old Age security benefits increased by 3 and 1 percentage points, respectively.

Text description – Chart 8

Program name	Change in incidences of employment for active claimants (percentage points)	Change in incidences of employment for former claimants (percentage points)
Training-on-the-Job	n/a	-4
Employment Assistance Services	-8	-8

Population subgroup profiles and labour market outcomes

Cost-benefit analysis

This analysis is based on the EBSM medium-term incremental impacts previously described in this report. Costs and benefits are examined over the participation period of 1 or 2 years and 5 or 10 years after the end of participation.^{Footnote 31}

The cost-benefit analysis addresses the following questions:

1. Are the benefits from EBSMs exceeding the costs within 5 years (for EAS), 10 years (for Building Essential Skills) after the end of participation?
2. How much is the benefit for the government and society if the government spends $1 in EI part II funding?
3. How many years does it take the benefits to recover the costs?

Table 12 presents the cost-benefit results for active and former EI claimant participants.
Results are presented from the social perspective, that is, the government and individual combined. This allows for a sound assessment of program effectiveness in achieving its objectives of helping unemployed individuals to obtain and maintain employment and to generate EI savings.

• ¹ Paid back within the program start year.

The information below provides examples of the net present value, the benefit-cost ratio, the payback period, the social rate of return and savings to health care costs.

Building Essential Skills^{Footnote 32}

During the 2009 to 2012 period, Building Essential Skills represented almost 51% of EBSM expenditures under the LMDAs in the Northwest Territories. The average duration of a Building Essential Skills Action Plan Equivalent is 33 weeks for active claimants and 34 weeks for former claimants.

Table 12 presents the cost-benefit analysis results for active and former EI claimant participants in BES. For example, over the 10 year post-program period, the benefit for active claimants is $54,080 higher than the costs, yielding a social return of 627% on investment. This means that if the government spends $1 on Building Essential Skills for active EI claimants, it generates a benefit of $7.27 for society. It takes 1.3 years for the benefits to recover the costs of programming. Overall, there are savings in health care costs of $485 per participant.

The benefit for former claimants is $82,730 higher than the costs, yielding a social return of 896% on investment. This means that if the government spends $1 on Building Essential Skills for active EI claimants, it generates a benefit of $9.96 for society. It takes 1.6 years for the benefits to recover the costs of programming. Overall, there are savings in health care costs of $492 per participant.

Self-Employment Option^{Footnote 34}

Job Creation Partnerships (JCP)^{Footnote 37}

Strategic Workforce Initiatives^{Footnote 39}

The Strategic Workforce Initiatives program aims to support community partners in undertaking labour market activities that promote labour force development, and support workforce adjustments and effective human resources planning. It includes a wide range of funded activities, such as:

• labour market research
• development of strategic plans to address community economic and employment issues
• promotion of employment supports to employers and unemployed individuals
• communications to share best practices or to initiate discussions regarding community challenges and opportunities
• coordination to facilitate community problem solving related to labour market/employment challenges; and to support the development of local planning groups
• coordination of workforce adjustment to help businesses set up support mechanisms to assist employees who are at risk of losing their jobs

The Strategic Workforce Initiatives program is one type of support measures delivered under the LMDAs. In 2019 to 2020, the Northwest Territories spent nearly $75,000 or nearly 2% of total expenditure under the Canada-Northwest Territories Labour Market Development Agreement.

Research and Innovation^{Footnote 40}

Research and Innovation projects in the Northwest Territories support the design and development of new initiatives and programs. Funded activities can include, for example, research, stakeholder engagement, and proposal development.

The Northwest Territories used the Research and Innovation program in 5 out the 7 fiscal years between April 2013 and March 2020. Research and Innovation spending varied between 0% ($0) to 3% ($119,000) of annual LMDA expenditures in the territory.

Summary of findings

Overall, where results were statistically significant, incremental impacts demonstrate that:

• active claimant participants in Building Essential Skills had improved their earnings and decreased their use of government income supports
• former claimant participants had increased their incidence of employment and earnings
• active claimant participants who received EAS decreased their use of EI by $1,269 and $863 in years 1 and 2 post-program participation respectively.

A subgroup analysis of medium-term outcomes shows that female, male, youth, and Indigenous participants in Building Essential Skills and EAS generally:

• had higher earnings
• used EI benefits less, but used SA benefits more
• had a lower incidence of employment, which could partially be explained by participants’ retirement decisions

While no incremental impact analysis was conducted for Training-on-the-Job, outcomes were identified for active claimant participants in EAS, and for former claimant participants in Training-on-the-Job and EAS. Outcomes for these participants were consistent with those for subgroup participants in Building Essential Skills and EAS.

A series of supplemental studies address information gaps previously identified in LMDA evaluations for Self-Employment Option, JCP, Strategic Workforce Initiatives and Research and Innovation. Overall, it was found that the Northwest Territories use:

• the Self-Employment Option program to assist participants in creating employment for themselves by providing them with a range of services
• JCP to collaborate with community organizations and employers to develop the territorial labour market by providing participants with the opportunity to gain work experience or training that improves their employment prospects
• Strategic Workforce Initiatives to support activities that promote labour force development, support workforce adjustments and facilitate effective human resources planning
• Research and Innovation to support the design and development of activities that support community labour market development

Recommendations

Since 2012, 15 qualitative and quantitative studies addressed issues and questions related to EBSM design, delivery and effectiveness. The quantitative studies successfully assessed the effectiveness and efficiency of EBSMs by producing incremental impacts and cost-benefit analysis. The qualitative studies help to contextualize the findings from the quantitative studies and to identify specific challenges, lessons learned and best practices associated with the design and delivery of EBSMs. Each study included key considerations for program and policy development or recommendations.

In addition, the recently completed evaluation of the Workforce Development Agreements complements the LMDA qualitative studies. This comprehensive evaluation provided unique insights into challenges and lessons learned to assist persons with disabilities, immigrants and those further removed from the labour market.

Advanced causal analysis permitting the attribution of impacts to specific EBSMs was conducted as part of this evaluation. These analyses are predicated on having access to high quality administrative records, thereby confirming the importance of the capacity to leverage and integrate relevant administrative data.

From these main findings, 2 key recommendations for the Northwest Territories emerge: