Evaluation of the Canada – Alberta Labour Market Development Agreement

From: Employment and Social Development Canada

3. Comparison of key findings by program type
4. Conclusions
5. Recommendations
References
Acronyms
Appendix A - Methodology
Appendix B - Detailed results Skills Development
Appendix C - Detailed results Skills Development-Apprentices
Appendix D – Detailed results Targeted Wage Subsidies
Appendix E - Detailed results Self-Employment
Appendix F - Detailed results Job Creation Partnerships
Appendix G - Detailed results Employment Assistance Services only
Appendix H - List of nine studies included in the synthesis report

Alternate formats

Evaluation of the Canada – Alberta Labour Market Development Agreement Synthesis Report [PDF - 1.15 MB]

Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.

Executive summary

Introduction

Employment and Social Development Canada (ESDC) worked jointly with Alberta and 11 other Provinces and Territories (P/Ts) to undertake the 2012 to 2017 second cycle for the Labour Market Development Agreement (LMDA) evaluation. The first cycle of LMDA evaluation was carried out between 1998 and 2012 and involved conducting bilateral formative and summative evaluations in all provinces and territories. Under the second cycle, the evaluation work consisted of conducting two to three studies per year on the Employment Benefits and Support Measures (EBSMs) similar programming delivered under these agreements. The studies generated evaluation evidence on the effectiveness, efficiency and design/delivery of EBSMs for Canada overall, for Alberta and for the 11 other P/Ts.

Under LMDAs, Canada transfers $2.14B Employment Insurance (EI) Part II funds to P/Ts for the design and delivery of programs and services to help unemployed individuals, mainly those eligible for EI, to find and maintain employment.

Programs and services delivered by P/Ts have to correspond to the EBSM categories defined under the EI Act. The following is a short description of the five programs and services examined in the evaluation:

Skills Development (Work Foundations / Occupational Training) helps participants obtain employment skills by giving them financial assistance in order to attend classroom training. This program also includes apprenticeship training.
Targeted Wage Subsidies (Workplace Training) help participants obtain on-the-job work experience by providing employers with a wage subsidy.
Self-Employment provides financial assistance and business planning advice to participants to help them start their own business.
Job Creation Partnerships (Integrated Training Program and Immigrant Bridging) provide participants with opportunities to gain work experience that will lead to ongoing employment. Employment opportunities are provided by projects that contribute to developing the community and the local economy.
Employment Assistance Services includes services such as counselling, job search skills, job placement services, the provision of labour market information and case management.

Three additional programs and services are available under the LMDA and they are: Labour Market Partnerships, Research and Innovation and Targeted Earnings Supplements. They were not evaluated as part of this evaluation. The Targeted Earnings Supplements program is not implemented in Alberta while Labour Market Partnerships, and Research and Innovation will be evaluated at a later stage.

Table i provides an overview of the share of funding allocated to the five EBSMs examined under the second cycle for LMDA evaluation in Alberta and the average cost per participant.

**Table i. Share of LMDA funding and average cost per participant**
Program and service	Share of funding 2014 to 2015	Average cost per participant 2002 to 2005
Skills Development	69%	$5,081
Employment Assistance Services	22%	$252
Targeted Wage Subsidies	1%	$7,440
Self-Employment	1%	$7,038
Job Creation Partnerships	7%	$8,790
Total	100%	–

Sources: EI Monitoring and Assessment Reports 2002 to 2003 to 2014 to 2015.

This report presents a summary of the findings from nine studies. Results are presented for active and former EI claimants as well as for long-tenured workers^{Footnote 1}, youth (under 30 years old) and older workers (55 years old and over) when the number of participants was sufficient to conduct quantitative analyses. Active EI claimants were clients that received EI at the time of their EBSM participation. Former EI claimants received EI up to three years before starting their EBSM participation.

Key findings

Effectiveness and efficiency of EBSMs

Incremental impacts and cost-benefit analyses assessed EBSM effectiveness and efficiency. Overall, incremental impacts demonstrate that LMDA programs and services are improving the labour market attachment of active and former EI claimant participants in Alberta. As well, social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first eight weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Figure i presents the incremental impacts on the incidence of employment for active and former claimants by type of program. The estimates can be interpreted as a change in the probability of being employed following participation. For example, participation in Skills Development increases the probability of being employed by 9.6 percentage points for active EI claimants relative to unemployed non- participants.

**Figure i. Change in probability of being employed in participants relative to non-participants^{Footnote 2}**

	Percentage Points
Active Claimants	Former Claimants
Skills Development	9.6	5.8
Targeted Wage Subsidies	7.3	3.9
Job Creation Partnership	10.8	5.2
Employment Assistance Services	2.2

Figure ii presents the cumulative increase in employment earnings for active and former claimants over the 5 years post-participation. While the estimates for Employment Assistance Services are not statistically significant, providing these services earlier during an EI claim (first 8 weeks) produced larger impacts on earnings and employment and facilitated earlier return to work.

**Figure ii. Increased cumulative earnings of participants relative to non-participants^{Footnote 3}**

	Cumulative Gains in Earnings
Active Claimants	Former Claimants
Skills Development	16952	9464
Targeted Wage Subsidies	12700	10665
Job Creation Partnership	17283	4832^a
Employment Assistance Services	2485^a

^a The incremental impacts on earnings for former claimants who participated in Job Creation Partnerships and active claimants who participated in Employment Assistance Services are not statistically significant at the 95% confidence level.

Table ii presents the number of years required for the social benefits to exceed program cost. Social benefits exceeded investment costs in a period ranging between the first year of program participation to about 14 years after participation.

**Table ii. Number of years for the benefits to exceed program costs**
	Skills Development	Targeted Wage Subsidies	Job Creation Partnerships	Employment Assistance Services
Active claimants	5.1	4.3	6.5	5.2
Former claimants	5.5	0.8	14.1	n/a

Lessons learned about program design and delivery

Key informants interviews^{Footnote 4} with service providers and program managers as well as the documents reviewed and the questionnaires filled by Alberta representatives identified:

Lessons learned and best practices as well as features of program design and delivery that work well
Challenges related to program design and delivery
Some external factors that influence program design and delivery

Skills Development

A key feature in the application process for Skills Development is to ensure that prospective participants are choosing a training that will meet labour market demand. As well, Alberta plans training investment to target labour market trends. This may contribute to program effectiveness. Other key features included:

Undertaking a comprehensive assessment of the participants in order to assist them in making informed choices and to commit to their training
Having access to disability-related employment and financial supports

In terms of lessons learned and best practices, key informants indicated the importance of:

Providing long-term assistance and to have an interdisciplinary team of professionals that can provide a continuum of services and supports to individuals facing multiple barriers
Monitoring participants during training and following up after the training
Coordination among key labour market stakeholders and service providers
Offering coordinated foundational learning programs that assess basic skills at the outset of the program, address weaknesses and integrate basic skills development into professional program would be beneficial
Providing intensive case management for individuals facing multiple barriers to employment

The main challenges related to Skills Development design and delivery included:

The level and duration of the financial support being too low for individuals facing multiple barriers to employment
Lack of instrument to test literacy level
Identifying learning disability too late in the process
Lack of awareness around available support for those with physical and mental disabilities

Skills Development – for Apprentices

Key informants identified the following best practices and lessons learned about the program:

Having close connections with the industry helps to understand where the government can make better investments and offer training closer to the participant’s home. For example, it has been possible to increase the training offered in some locations based on employers input.
Providing accommodations to individuals facing barriers to employment or to completing training. For example, providing disability related supports or help to individuals with learning barriers in order to help them complete their exams.
Existing Canadian labour market literature showed that there is a fairly high non-completion rate among apprentices; however it is not possible with available data to generate a reliable estimation of the completion rate of Skills Development – Apprentices participants in Alberta. According to key informants, the drop-out from the apprenticeship process may be due to external factors such as:
- The apprentice realises that he/she is not suited for the job
- Employers are not able to let the apprentice go on training

Targeted Wage Subsidies

Key informants identified best practices or the lessons related to Targeted Wage Subsidies’ design and delivery. These included:

Having a client-focused approach to the program application to ensure the participant is ready, willing and able to pursue participation
Ensuring a good match between employer and employee helps to guarantee participant retention after the end of the subsidy
Ongoing case management is key for proactively addressing barriers faced by participants (for example, mental health issues, addictions and social barriers)
Maintaining a good relationship with employers and providing them support during the subsidy is important

While evaluation results have demonstrated program effectiveness, its use has been falling in recent years (from 261 new interventions in 2003 to 2004 to 41 in 2013 to 2014). According to key informants potential reasons to explain this decline may include:

Changes in the labour market and the low unemployment rate affect the program
Mismatch between employers needs and participants skills
Lack of capacity among career employment consultants or service providers due to high turnover and lack of resources
Employers may not be inclined to use the program because of:

Time and resource constraints on the part of the employers to apply, receive approval and meet reporting requirements
A lack of awareness from employers about the program
The stigma associated with hiring someone with a subsidy

Challenges related to program design and delivery included:

Some career employment consultants cannot follow-up with participants as often as needed because they lack capacity. In some situations, the caseload of career employment consultants may involve approximately 300 individuals.
High staff turnover results in loss of knowledge and expertise.
It can be difficult to find participants and employers who meet the program guidelines. Eligibility criteria are also seen as being too restrictive by key informants. For example, the program has a limitation where only one employee can be subsidised with one employer.

Employment Assistance Services

Key informants identified some best practices and lessons learned related to the provision of Employment Assistance Services:

It would be beneficial to provide a one-stop-shop for participants instead of relying on referrals made to other organizations
Long-term interventions and follow-up are required for individuals facing multiple barriers since those barriers are often inter-related and participants need a long time to address them
Conducting adequate and detailed assessments of participants’ needs to help determine training needs
Job maintenance services and job placements are helpful. Holding job fairs also help to increase awareness about available services
Targeting contracts for specialized services for participants with specific needs would be beneficial
Having adequate labour market information and having well trained staff is important
Having the capacity to tailor services to community needs is an important consideration
It is important for service providers to build relationships with employers, specialized service providers and community organizations. This helps with participant referrals and facilitates the integration of participants in the local labour market.
Access to EI Part I data is important

Challenges related to the provisions of Employment Assistance Services included:

Program guidelines are perceived as being rigid. This can restrict service provider’s ability to address the needs of participants with multiple barriers. As well, the focus on quicker return to employment as the sole indicator of success does not adequately account for the extensive barriers faced by these participants.
Program awareness varies greatly depending on the various participant groups.
Having one year contracts with service providers can limit their capacity to retain qualified staff and to access stable funding.

Recommendations

Recommendations that emerge from the evaluation findings presented in this report are as follows:

The study on the timing of Employment Assistance Services participation showed that receiving assistance early after starting an EI claim can lead to better labour market impacts. However, key informants reported a lack of awareness about the program.

Recommendation 1: Consideration should be given to providing Alberta with timely access to data on new EI recipients for supporting targeting and increasing awareness.

Key informants reported that lack of essential skills, learning disabilities and literacy issues may be barriers to accessing and completing training.

Recommendation 2: Consideration should be given to remove barriers to accessing and completing training such as literacy/essential skills training and learning disability assessments. The measures would help individuals with multiple barriers to prepare for vocational training and to reintegrate the labour market. The measures should be reported separately from other Skills Development interventions given their unique objectives.

Incremental impact results show that the Targeted Wage Subsidies program is improving the earnings and employment of participants. However, program use has declined over the years. According to key informants, the decline may be related to factors such as a lack of awareness about the program, the commitments required from employers in terms of time and resources as well as the stigma associate with hiring someone with a subsidy.

Recommendation 3: Depending on labour market priorities and budget allocation across the various EBSMs, Alberta may wish to explore ways of removing barriers to employer participation in Targeted Wage Subsidies.

Key informants interviewed in the evaluation confirmed the necessity of having labour market information to support program delivery. They, however, pointed to the potential difficulty of accessing labour market information at the regional or local level.

Recommendation 4: Consideration should be given to enhance the capacity of service providers to access, when needed, relevant labour market information.

The evaluation was not able to produce a conclusive assessment of Self-Employment effectiveness and efficiency since the data used to assess impacts on earnings may not be the best source of information available to reflect the financial wellbeing of the participants. Overall, it is not clear whether participant’s success in improving their labour market attachment through self-employment is more closely associated with their business idea and their entrepreneurship skills than the assistance provided under Self-Employment.

Recommendation 5: Consideration should be given to examine in more detail the design and delivery of Self-Employment and whether the performance indicators for this program are appropriate.

Job Creation Partnerships was found to be particularly effective at improving earnings and incidence of employment of active claimants. However, the evaluation has not yet examined the design and delivery of the program’s two components: Integrated Training and Immigration Bridging. Therefore, it is not clear what factors contribute to program success.

Recommendation 6: Consideration should be given to examine the design and delivery of the program components (Integrated Training and Immigration Bridging) in order to better understand how this program operates.

Overall, the LMDA evaluation was able to produce a sound assessment of EBSM effectiveness and efficiency because the team had access to rich data on EI claimants, EBSM participation data and Canada Revenue Agency taxation files. While the available data allowed the production of robust evidence on impacts and cost-benefits, some data gaps limited the evaluation’s ability to assess how EBSMs operate. For example:

Having access to complete data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants would be useful to inform policy development and program design.
Having access to complete data on the cost of programming per participant would also provide a refined assessment of how much participation cost for active claimants as compared to former claimants.
The evaluation was not able to provide a comprehensive description of the type of training funded under Skills Development (for example, literacy, essential skills, adult basic education and vocational training). As a result, it is currently not possible to separate the effects of adult basic education from those of vocational training although both types of training may lead to very different labour market outcomes.
In Alberta, the Integrated Training and Immigrant Bridging are separate programs that are recorded as Job Creation Partnerships in the LMDA data. As a result, it is currently not possible for ESDC to separate the effects of Integrated Training from those of Immigrant Bridging although both programs may lead to very different labour market outcomes for different target groups.
There are various types of Employment Assistance Services provided under the LMDAs. These services can be very different in nature and it is possible that some may be more effective than others at helping participants to return to employment. For example, having access to a computer for researching jobs on its own may yield different impacts than receiving counselling and assistance to develop a return-to-work action plan.

Recommendation 7: Improvements in the data collection is recommended to address key program and policy questions of interest to the federal and provincial/territorial governments. Specifically:

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. Alberta, ESDC and other P/Ts should work together to define common categories for both EBSMs.
Examine the added value in reporting separately on participants in the Integrated Training and Immigrant Bridging programs
Collect detailed data on the cost of interventions

Management response

Introduction

The Government of Alberta would like to thank all those who participated in the Cycle II Evaluation of the Canada-Alberta LMDA. In particular, the Government of Alberta acknowledges the contributions of key informants, ESDC and the work of the LMDA Evaluation Steering Committee in conducting the assessment of EBSMs in Alberta, funded under the LMDA.

Alberta Labour, on behalf of the Government of Alberta, considered the evaluation findings in this report and provided the following Management Response.

Alberta management response

The Bilateral Evaluation Report indicated that Alberta was successful in meeting the objectives of the EBSMs in that Albertans were benefiting from the programs and services provided by Alberta under this agreement. Alberta is committed to continue to achieve strong results and look for ways to improve on the employment and training programs it provides under the LMDA and is appreciative of the new evidence and information that is available. Information from the Bilateral Evaluation Report will be helpful and considered as program and service changes occur.

Alberta is already addressing some of the recommendations in the current labour market programming.

Alberta is receiving data on new EI applicants as part of the Targeting, Referral and Feedback Pilot program.
Ministry of Community and Social Services has contracts for specialized assessments if Learning Disability is identified as a barrier to accessing and completing training.
Literacy and essential skills training is also built in to several program designs (for example, Integrated Training, a program offered under Job Creation Partnership)
In recent years, 12 new Workplace Training contracts have started in Alberta. The new contracts focus on the client priorities and have eliminated a lot of the administrative work that previously existed for the employers and providers. Early feedback on the new process from the employers and providers is very positive.
Ministry of Labour makes labour market information available on its website. Since the period of evaluation, Alberta labour market information has grown extensively. Efforts are underway to connect service providers with the latest labour market information. For example, the Alberta Short Term Employment Forecast prepared by Ministry of Labour is made available to the public. Regional coordinators are also engaged more in recent times.
Alberta recently met with ESDC to discuss the ongoing work on Service-based Mappings between Mobius, the Enterprise Database, and IMAGIS. As new Service Types are added to Mobius, the mappings allow updating the downstream processes including (but not limited to) reporting. This will allow/improve collection of data on types of trainings and services under broad LMDA groupings.

Alberta notes that while the recommendations are non-binding, some of the recommendations such as improved data collection; understanding of program design and delivery for Self-Employment and Job Creation Partnerships; and development of improved performance indicators for Self-Employment may be better suited for addressing in the joint evaluation work conducted by ESDC and provinces and territories in future evaluation cycles and thus may not be specific to Alberta.

1. Introduction

Employment and Social Development Canada (ESDC) worked jointly with Alberta and 11 other Provinces and Territories (P/Ts) to undertake the 2012 to 2017 second cycle for the Labour Market Development Agreement (LMDA) evaluations. The evaluation work consisted of conducting two to three studies per year on the Employment Benefits and Support Measures (EBSMs) delivered under these agreements. The studies aimed to generate evaluation evidence on the effectiveness, efficiency and design/delivery of programs and services for Canada overall and for the 12 P/Ts that opted for a joint evaluation process with the Government of Canada.

This report presents a summary of the findings from the studies conducted for Alberta. The report is organised as follows:

Introduction with an overview of the studies summarized in this report including their scope and methodology, and contextual information on the LMDAs
Findings section with a discussion around the rationale for investing in labour market programming and summary of evaluation evidence
Conclusions and lessons learned
Recommendations that emerge from the evaluation findings and areas for future investigation

1.1 Labour Market Development Agreement background

LMDAs are bilateral agreements between Canada and each P/T, and were established under Part II of the 1996 Employment Insurance (EI) Act. As part of these agreements, Canada transfers EI part II funding to P/Ts to design and deliver programs and services to assist individuals to prepare for, obtain and maintain employment.

The Canada-Alberta LMDA was signed on December 6, 1996. The agreement transferred responsibility to Alberta for the design and delivery of programs and services classified under two categories: 1) Employment Benefits and 2) Support Measures.

Employment Benefits

Employment Benefits are offered to unemployed individuals who 1) are actively on EI (in other words, active claimants); 2) ended their benefit period within three years before participating (former claimants); or 3) established a claim for maternity or parental benefits within the past five years and are returning to the labour force for the first time (former claimants)^{Footnote 5}. Employment Benefits include the following categories:

Skills Development (Work Foundations / Occupational Training) helps participants obtain employment skills by giving them financial assistance that enables them to select, arrange and pay for classroom training.
Targeted Wage Subsidies (Workplace Training) help participants obtain on-the-job work experience by providing employers with financial assistance to help with the wages of participants.
Self-Employment provides financial assistance and business planning advice to EI-eligible participants to help them start their own business. This financial assistance is intended to cover personal living expenses and other expenses during the initial stages of the business.
Job Creation Partnerships (Integrated Training and Immigrant Bridging) provide participants with opportunities to gain work experience that will lead to ongoing employment. Employment opportunities are provided by projects that contribute to developing the community and the local economy.
Targeted Earnings Supplements encourage unemployed persons to accept employment by offering them financial incentives.

Support Measures

Support Measures are available to all unemployed individuals including those not eligible to receive EI and include:

Employment Assistance Services such as individual counselling, action planning, help with job search skills, job-finding clubs, job placement services, the provision of Labour Market Information (LMI), and case management and follow-up.
Labour Market Partnerships provide funding to help employers, employee and employer associations, and communities improve their capacity to deal with human resource requirements and implement labour force adjustments. These partnerships involve developing plans and strategies, and implementing labour force adjustment measures.
Research and Innovation supports activities that identify better ways of helping people prepare for or keep employment and be productive participants in the labour force. Funds are provided to eligible recipients to enable them to carry out demonstration projects and research for this purpose.

Table 1 provides an overview of the share of funding allocated to the five programs and services examined under the second cycle for LMDA evaluation and the average cost per participant. It is noted that the average cost per participant was calculated based on the 2002 to 2005 data from the EI Monitoring and Assessment Reports for Alberta. The 2002 to 2005 period corresponds to the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

**Table 1. Share of LMDA funding and average cost per participant**
Program and service	Share of funding 2014 to 2015	Average cost per participant 2002 to 2005
Skills Development	69%	$5,081
Employment Assistance Services	22%	$252
Targeted Wage Subsidies	1%	$7,440
Self-Employment	1%	$7,038
Job Creation Partnerships	7%	$8,790
Total	100%	–

Sources: EI Monitoring and Assessment Reports 2002 to 2003 to 2014 to 2015.

1.2 Methodology

This section presents key aspects of the quantitative analyses carried out as part of the LMDA studies, while a more detailed description of the methodology is provided in Appendix A.

All quantitative analyses were based on administrative data from the EI part I (EI claim data) and part II (EBSM participation data collected by Alberta and transferred to ESDC) linked to the T1 and T4 taxation files from the Canada Revenue Agency. Incremental impact analyses and the cost-benefit analyses were based on up to 100% of participants in the reference period selected.

Incremental impacts analysis

Five studies assessed program effectiveness by estimating incremental impacts from EBSM participation on participants’ labour market experience (for example, earnings from employment/self-employment, incidence of employment, use of EI or Social Assistance and dependence on income support) after participation. The role of the incremental impact analysis is to isolate the effects of participation from other factors such as the economic cycle. In order to achieve this, the incremental impact analysis compared the labour market experience of participants before and after their participation with that of the non-participants (see the example of incremental impact calculation in Figure 1).

The matching of participants and comparison group members used up to 75 socio-demographic and labour market variables observed over five years before participation. Two different comparison groups were used to measure impacts for active and former EI claimants. For active claimants, the incremental impacts were measured relative to a comparison group of active claimants who were eligible to, but did not, participate in EBSMs during the reference period.

Former claimants can be underemployed and unable to requalify for EI, out of the labour force for various reasons or on social assistance. Based on previous evaluation methodologies, on expert advice and given the difficulty in generating a suitable comparison for former claimants using administrative data alone, the comparison group for former claimants was created using individuals who participated in Employment Assistance Services only during the reference period. This is a conservative approach given the fact that participation in Employment Assistance Services can lead to limited effects on labour market outcomes. In other words, the experience of former claimants who received Employment Benefits (Skills Development, Targeted Wage Subsidies, Self-employment and Job Creation Partnerships) was compared to the experience of former claimants who received low intensity employment services (Employment Assistance Services only). Due to this difference in measurement, incremental impacts estimated for active claimant participants should not be directly compared to those of former claimant participants^{Footnote 6}.

**Figure 1. Example of incremental impact calculation**

Factors accounted for in the cost-benefit analysis

Program efficiency was assessed through a cost-benefit analysis which compared the cost of participating in the program for the participants and the cost of delivering the program for the government to the benefits generated by the program. Overall, this analysis provided insights on the extent to which the program is efficient for the society (in other words, for both the participants and the government). The costs and benefits accounted for in the calculations were as follows (see detailed definitions in Appendix A):

Program cost: includes program and administration costs paid by the government.
Marginal social costs of public funds: represent the loss incurred by society when raising additional revenues such as taxes to fund government programs.
Employment earnings: consist of incremental impacts on participants’ earnings during and after participation. The calculation accounts for the participant’s forgone earnings during participation (in other words, opportunity cost). Employment earnings were also increased by 15% to account for fringe benefits such as the employer-paid health, life insurance and pensions contributions.

Strengths and limitations of the studies

One of the key strengths of the studies is that all quantitative analyses were based on administrative data rather than survey responses. Compared to survey data, administrative data are not subject to recall errors or response bias.

The propensity score models used to match participants and non-participants for the incremental impact analyses are judged to be robust because they were based on five years of pre-participation data and on a vast array of variables including socio-demographic characteristics, location, skills level related to last occupation and indicators of labour market attachment. Sensitivity analysis and the use of alternative estimation methods have further increased confidence in the incremental impact estimates. However, one limitation with the propensity score matching techniques is that no one can be fully sure the impacts were not influenced by factors not captured in the data.

The cost-benefit analysis accounted for all quantifiable costs and benefits that are directly attributable to the EBSMs and could be estimated with the available administrative data. The analysis did not account for non-quantifiable benefits such as improvements in participant’s wellbeing or for the multiplier effect of increased spending on the economy.

It should be noted that it is not possible to produce an analysis of incremental impacts from Skills Development – Apprenticeship participation. Assessing program impacts for apprentices poses a methodological challenge because apprentices are already employed and are expected to return to their employment after completing their training. Therefore, the expected labour market impacts cannot be examined using a similar approach as for assessing impacts of other EBSMs, which are expected to help participants return to employment. As well, the data available does not permit the identification of a proper comparison group since apprentices alternate between work and training and no other potential counterfactuals have similar employment and training patterns. In this context, it is not possible to produce a reliable assessment of program effectiveness.

When interpreting qualitative findings, readers should keep in mind that those are based on the perception of a small number of key informants who are directly involved in the design or delivery of the program. Their perception may be representative of their own region or community but not necessarily of the entire province. Since the number of key informants interviewed in each study is small (ranging between 3 and 8), the number of informants who reported a specific finding is not indicated in the report. However, the report does note when there was a clear disagreement between key informants.

1.3 Overview of the studies summarized in this report

Findings presented in this report were drawn from nine separate studies carried out in Alberta. These studies examined issues related to program effectiveness, efficiency, design/delivery and used a mix of qualitative and quantitative methods. Each study examined evaluation issues in relation to active and former EI claimants.

Table H1 in Appendix H presents an overview of these studies, including the type of evidence generated, the methods used, the reference period and the length of the post-program period over which program effects were observed.

2. Evaluation findings

2.1 Rationale and labour market context

LMDA investments align with provincial government priorities

Active labour market programs aim to help unemployed or underemployed individuals find and maintain employment. These programs are fairly similar across the Organization for Economic Co-operation and Development countries and consist of skills training in a classroom setting, work experience with employers (often subsidized) or in the public/non-profit sector, return-to-employment assistance and self-employment assistance.

In Alberta, the Ministries of Labour, Community and Social Services (formerly Human Services), and Advanced Education administer a range of active labour market programming targeted at various groups of individuals including EI claimants and social assistance recipients. Programs and services offered to active and former EI claimants are mainly funded under the LMDA transfer.

At the time of writing this report, Alberta was experiencing an economic downturn. In this context, the Ministry of Labour Business Plan for 2016 to 2019 stressed the need to provide occupation-related skills training, work experience and employment programs to help Albertans to fully participate in the economy and get back to work. The plan also stated the need to provide training and re-training to Albertans in order to help addressing current labour and skills shortages.

Overall, impacts found in the LMDA evaluation and discussed in this report demonstrated that LMDA funded programs and services delivered in Alberta are generally helping participants to improve their labour market experience after participation. As such, evaluation evidence suggests that LMDA funded programming contributes to achieve Alberta’s priorities and the following Ministry of Labour outcome: Albertans have the skills required by Alberta’s labour market.

2.2 Skills Development

2.2.1 Program description

Based on a document review and 5 key informant interviews completed in summer 2015

In Alberta, the Skills Development program is called Work Foundations / Occupational Training. The program aims to 1) increase opportunities for Albertans to make successful transitions from school to work, unemployment to employment, and from one career path to another, and 2) increase the capacity of Albertans to respond to changing skills, knowledge, and abilities required by the economy.

The program provides financial assistance to eligible learners to cover expenses related to training including tuition, books, mandatory fees and supplies, student association fees and monthly living allowance. Key informants also indicated that the program may fund child care, transportation and medical benefits (for example, prescription drugs, health care and emergency dental services). The amount of the funding is determined based on the financial need of the individuals and rates set by current legislation and program guidelines.

Table 2 below shows the proportion of expenditures and interventions by type of training supported. More than half of the interventions delivered in 2013 to 2014 provided assistance to individuals pursuing adult basic education and essential skills training as compared to 13% for vocational and occupational training. However, 49% of expenditures are invested in vocational and occupational training.

**Table 2. Types of training supported 2013 to 2014**
Types of training	Expenditures	Interventions
Types of training	Proportion	Proportion
Adult basic education/essential skills	25%	51%
English/French as a second language	26%	36%
Vocational/occupational training, including college or university	49%	13%
Total	100%	100%

Source: Information submitted by the Ministry of Labour

2.2.2 Program delivery

Based on a document review and 5 key informant interviews completed in summer 2015

The case management of participants is usually contracted to service providers. In rural areas, the program is usually delivered by provincial staff while service providers may be used in rare occasions.

A prospective participant has to undergo the following steps in order to apply for the program:

The individual is assessed through a 15-minute Service Needs Determination. The purpose is to gather information, identify issues and needs, and clarify personal and environmental resources available. The assessment helps the individuals to increase self-awareness; understand his/her career issues; establish work, learning and/or life balance goals; and provide a foundation for taking actions and establishing indicators of ongoing progress.
If training is identified as a possible intervention, the individual goes through an Employability Assessment to address one or more of the following: career/occupational decision-making, skill development and/or enhancement, work search and/or maintenance supports.
The individual has to fill a career investigation report which requires him or her to research his/her occupational goals.
A financial need assessment is completed in order to determine what additional funding the individual may need on top of the EI benefits.
A Service Plan is developed in order to describe and record the mutual agreement between the Government of Alberta and the individual about the specific activities to be undertaken to obtain employment. The Service Plan is required for the provision of services and payments to the individual and service providers.

The application process takes 2 to 3 weeks on average but may take around 4 to 6 weeks in the summer since it is the busiest time of the year.

2.2.3 Targeting to labour market demand

Based on a document review and 5 key informant interviews completed in summer 2015

Overall, the application process aims to ensure that prospective participants are choosing training that will meet labour market demand. In particular, they are required to fill a career investigation report to examine the requirements for the job, the projected demand, the salary and location of the job. They may also have to meet with employers to talk about job prospects and to investigate schools. Four of the key informants indicated that they may refuse to fund someone who is choosing training that does not have good job prospects.

At the program level, Alberta developed the Skills Investment Strategy to target labour market trends such as a tight labour market and continued skill shortages. As well, program funding is divided across regions which determine and target the priority areas. Labour Market Information (LMI) is used to help determine how to allocate funding to training providers in order to meet regional labour market demand.

2.2.4 Profile of Skills Development participants

Socio-demographic characteristics

As shown in Table B1 in Appendix B, the majority of active claimants who participated in the program in 2002 to 2005 and 2006 to 2008 were female (52% and 58% respectively) and between 25 and 44 years old (64% in each cohort). As well, participants in both cohorts most frequently had secondary or occupational training prior to participation (40% in each cohort). Active claimants who participated in the program in 2002 to 2005 had slightly lower employment earnings in the year before participation ($21,589) compared to those who participated in 2006 to 2008 ($22,384).

Former claimants who participated in the program in 2002 to 2005 and 2006 to 2008 were mostly female (61% and 70% respectively) and between 25 and 44 years of age (66% each). Former claimants most frequently occupied jobs requiring secondary or occupational training before participation (39% for 2002 to 2005 participants and 40% for 2006 to 2008 participants). Participants in both cohorts had similar earnings in the year before participation ($9,357 and $9,445 respectively).

Barriers faced by participants

Based on a document review and 5 key informant interviews completed in summer 2015

Key informants were asked to identify the main barriers to employment or to participation faced by participants. By order of importance, the most frequently identified barriers were the following:

Financial constraints/hardship
Challenges related to living in remote location (for example, lack of employment opportunities on reserves, having to work in a remote location while the family remains in the community, lack of training providers in certain area, lack of transportation and low internet connection).
Learning disabilities
Lack of essential skills and life management skills
Personal issues (for example, addictions, family or health issues)
Lack of ability to speak English as a second language
Immigrants face difficulties navigating the provincial and federal training and employment system
Difficulty of accessing childcare
Not knowing what career path they would like to pursue

2.2.5 Incremental impacts

Active claimants

As shown in Table B2 in Appendix B, active claimants who started their participation between 2002 and 2005 had incremental gains in earnings and incidence of employment in each of the five years after participation. As shown in Figure 2, earnings gains continuously increased over time and ranged from $887 in the first year after participation to $5,680 in the fifth year. Similarly, the increases in incidence of employment ranged between 8.3 percentage points in the first year and 10.8 percentage points in the fifth year.

**Figure 2. Increased earnings of active and former Skills Development participants relative to non-participants^{Footnote 7}**

	Gains in Earnings
Active Claimants	Former Claimants
Year 1 Post-Program	887	1382
Year 2 Post-Program	2204	1536
Year 3 Post-Program	3400	1871
Year 4 Post-Program	4781	2114
Year 5 Post-Program	5680	2561

EI use generally increased after participation while social assistance use generally decreased. All results for dependence on income support were not statistically significant. The increase in EI use may indicate the inability for some participants to maintain the employment secured in the short-term.

The vast majority of the results for 2006 to 2008 participants were not statistically significant. However, as for the 2002 to 2005 cohort, these participants had an increase in earnings in the third year after participation and increases in incidence of employment in all three post-program years.

Results for sub-groups of active claimants were as follows:

Youth (under 30 years old) who participated between 2002 and 2005 improved their earnings in the fourth and fifth years of the post-program period and incidence of employment in all years after participation. Results for EI and social assistance use and the level of dependence on income support were generally not statistically significant.
Long-tenured workers who participated in between 2007 and 2009 had a decrease in earnings in the first year after participation and an increase in the third year. They increased their incidence of employment in all three post-program years. They decreased their dependence on income support in the second and third years after participation. Results for other indicators were not statistically significant.
Impacts were not produced for older workers as the number of participants was too small.

Former claimants

The 2002 to 2005 cohort of former claimants had incremental gains in earnings in all years after program participation (see Table B3 in Appendix B). As shown in Figure 2, those gains grew over time from $1,382 in the first year post-program to $2,561 in the fifth year. As well, they had increases in incidence of employment ranging between 5.2 and 6.3 percentage points over the five-year post-program period. Their use of social assistance decreased in all years by annual averages ranging between $306 and $387 while their overall dependence on income support decreased by averages ranging between 2.2 and 4.2 percentage points. Their use of EI generally increased after participation indicating the inability for some participants to maintain the employment secured in the short-term.

Former claimants who started their participation in the 2006 to 2008 period had comparable increases in earnings to the 2002 to 2005 participants in the second and third years post-program. They had, however, larger increases in the incidence of employment compared to the 2002 to 2005 participants. As well, their use of EI decreased in the first year after participation and their use of social assistance decreased in all years. These participants also had decreases in their dependence on income support in all years after participation and the magnitude of the reductions was larger than those observed for 2002 to 2005 participants.

For former claimants, it can also be argued that the increase in EI use is an indication of increase labour market attachment (or at least intermittent attachment) for this client group since they did experience increases in employment earnings and incidence of employment, and a decrease in the use of social assistance. As a reminder, former claimants are participants for whom the EI benefit period ended up to three years pre-participation.

Youth who participated between 2002 and 2005 had increases in earnings in most post-program period years and increases in incidence of employment in all years. Their use of EI generally increased after participation but their use of social assistance decreased. Overall, their level of dependence on income support decreased in most post-program years.

Impacts for older workers and long-tenured workers were not estimated as the number of participants was too small.

2.2.6 Cost-benefit results

As shown in Table B4 in Appendix B, for active claimants, the benefits of the program for society were $4,640 higher than the costs six years after participation. The benefits matched the costs 5.1 years after participation. For former claimants, the benefits were $818 above the costs six years after the end of participation. As such, the benefits matched the costs 5.5 years after participation.

2.2.7 Challenges and lessons learned about Skills Development design and delivery

Based on a document review and 5 key informant interviews^{Footnote 8} completed in summer 2015

Key informants identified a number of lessons learned in relation to the design and delivery of the program:

It is important to provide long-term assistance and to have an interdisciplinary team of professionals that can provide a continuum of services and supports to individuals facing multiple barriers.
Undertaking a comprehensive assessment of the participants is important as it helps them to make informed choices and to commit to their training.
Having access to disability-related employment and financial supports is a best practice. The financial assistance for medical care is also helpful.
Monitoring participants during training and following up after the training appears beneficial.

Coordinating with stakeholders is key and this can take the form of:
Holding regular meetings with all the regional staff and with provincial stakeholders to discuss provincial and regional concerns, best practices and solutions.
Involving all labour market actors in the design of the program and delivery to ensure that a manageable number of qualified individuals are trained.
Working closely with training providers.

There is a need to look at program enhancement to better support the participants (for example, offering resiliency training; increasing accessibility to training through more flexible modes of studies or; recognizing skills through recognition of prior learning and the use of competence-based examination).
Offering coordinated foundational learning programs that assess basic skills at the outset of the program, address weaknesses and integrate basic skills development into professional program would be beneficial.
There is a need for more intensive case management of individuals facing multiple barriers to employment.

Key informants also identified the following challenges in relation to the design and delivery of the program (order according to the number of key informants from the highest to the lowest):

The level of financial support is too low for individuals facing multiple barriers to employment. As well, the duration of the support can be an issue especially for slow learners who need more than 30 months to complete training. Participants cannot access the program for four years once they have received funding for 30 months.
There is no instrument to test literacy level.
Finding out too late during the process that the participant has a learning disability.
There is a lack of awareness around available support for physical and mental disabilities.

2.3 Skills Development-Apprentices

2.3.1 Program description

Based on a document review and 4 key informant interviews completed in summer 2015

The Skills Development-Apprentices program is called Apprenticeship Technical Training and is a component of the Occupational Training program. The objective of the program is to assist apprentices in achieving the goal of becoming a skilled tradesperson, and learn a trade through a combination of on-the-job training and technical training.

Apprentices in Alberta complete the majority of their learning (80 to 85%) during on-the-job paid employment. They are also required to participate in technical training, which can range from six to eight weeks per year. Apprentices can receive their EI benefits while in technical training.

The program is provided to apprentices who are EI eligible to help them offset the apprenticeship cost. This is calculated as the difference between the cost of providing the apprenticeship courses at publicly funded training institutions and the tuition fees paid by EI eligible apprentices. In addition, some apprentices qualify for supplemental income support benefits while in training to cover expenses such as, books and supplies, living expenses and additional living costs, such as a second residence. The financial assistance is provided as a grant. The level of support is determined based on demonstrated need and legislated Income Support rate.

During the evaluation period, the Alberta Ministry of Advanced Education was responsible for the apprenticeship training system while the Ministry of Community and Social Services (formerly Human Services) provided funding and benefits for eligible apprentices attending full-time technical training. Since October 2015, registered apprentices seeking financial assistance during technical training have to apply to Student Aid Alberta of the Ministry of Advanced Education^{Footnote 9}.

2.3.2 Profile of Skills Development-Apprentices participants

The vast majority of active claimants who started their participation in 2003 to 2005 and 2013 to 2014 were male (94% and 95% respectively) and below 34 years old (86% and 71% respectively) (see Table C1 in Appendix C). As well, 91% and 92% respectively of these participants had employment requiring college or apprenticeship training in the last job they held before participation. Active claimants who participated in 2003 to 2005 had lower employment earnings in the year before participation ($28,554) compared to those who participated in 2013 to 2014 ($39,761).

Eighty nine percent of former claimants who participated from either 2003 to 2005 or 2013 to 2014 were male. Most of these participants were below 35 years of age (82% for the 2003 to 2005 cohort and 77% for the 2013 to 2014 cohort). Former claimants had occupations requiring college or apprenticeship training before participation but the proportion was lower for 2003 to 2005 participants (58%) compared to the 2013 to 2014 participants (89%). As well, those who participated in 2003 to 2005 had lower employment earnings before participation ($23,468) compared to the 2013 to 2014 cohort ($38,470).

2.3.3 Labour market outcomes

Tables C2 and C3 in Appendix C presents the labour market outcomes for active and former EI claimants who started participation during the 2003 to 2005 period.

Active claimants

As shown in Figure 3, the earnings of active claimants who started the program between 2003 and 2005 grew from $15,190 to $71,012 between the fifth year pre-program and the seventh year after the participation start year. At the same time, their incidence of employment increased from 92% to 100% over the five pre-program years and decreased from 100% to 94% over the participation start year and the following seven years. Their incidence of EI use was generally higher in the years that followed the start of participation when compared to the pre-program period. It varied between 17% and 21% in the five pre-program years. Then, it decreased from 99% in the program start year to 16% in the seventh year after the program start year.

**Figure 3. Average earnings for active claimant participants in Skills Development-Apprentices**

	Employment Earnings
5 yrs pre-program	15190
4 yrs pre-program	18590
3 yrs pre-program	22367
2 yrs pre-program	25974
1 yrs pre-program	31105
Program start year	30895
1 yrs post-program	39945
2 yrs post-program	47597
3 yrs post-program	55268
4 yrs post-program	62487
5 yrs post-program	64277
6 yrs post-program	66510
7 yrs post-program	71012

Former claimants

As shown in Figure 4, the employment earnings of former claimants increased from $15,804 in the fifth year before participation to $58,193 in the seventh year following the program start year. Their incidence of employment ranged between 95% and 99% in the five years before participation and decreased from 99% to 90% over the participation start year and the seven years after the participation start year. The proportion of participants in receipt of EI increased from 24% to 49% over the five years before participation and decreased from 47% to 27% over the seven years after the participation start year.

**Figure 4. Average earnings for former claimant participants in Skills Development-Apprentices**

	Employment Earnings
5 yrs pre-program	15804
4 yrs pre-program	19281
3 yrs pre-program	23059
2 yrs pre-program	25033
1 yrs pre-program	24416
Program start year	26920
1 yrs post-program	33744
2 yrs post-program	42781
3 yrs post-program	46336
4 yrs post-program	51764
5 yrs post-program	52234
6 yrs post-program	60034
7 yrs post-program	58193

2.3.4 Challenges and lessons learned about Skills Development-Apprentices design and delivery

Based on a document review and 4 key informant interviews^{Footnote 10} completed in summer 2015

Key informants identified some best practices about the program including:

Having close connections with the industry helps to understand where the government can make better investments and offer training closer to the participant’s home. For example, it has been possible to increase the training offered in some locations based on employers input.
Providing accommodations to individuals facing barriers to employment or to completing training. For example, providing disability related supports or help to individuals with learning barriers in order to help them complete their exams.

Key informants suggested the following improvements to Skills Development-Apprentices or apprenticeship training in general:

More financial support could be beneficial.
Getting the apprentice to apply early for the financial support would be helpful as sometimes, the participants receive the support after they started their training.

Existing literature has shown that there is a fairly high non-completion rate among apprentices in Canada (40-50%)^{Footnote 11}. Furthermore, subject matter literature revealed that despite the growth in apprenticeship registrations in Canada, there has not been a corresponding increase in completions^{Footnote 12}. While available data do not provide reliable information on completion and non-completion rates for participants, key informants interviewed in the evaluation identified reasons that could lead apprentices to drop-out from the apprenticeship process. These include:

The apprentice realises that he/she is not suited for the job
The supply of training is lower than the demand
Employers are not able to let the apprentice go on training

Key informants were also asked about challenges in relation to the design and delivery of Skills Development-Apprentices or apprenticeship training in general. The only challenge identified was that it can be hard to predict the right level of supply and what are the needs in terms of training modules that are common to different trades.

2.4 Targeted Wage Subsidies

2.4.1 Program description

Based on a document review and 3 key informant interviews completed in summer 2015

The Targeted Wage Subsidies program is called Workplace Training in Alberta. The program provides individual-focused training or work experience to enable participants to find and maintain employment. Participating employers agree to train or provide work experience for a predetermined period of time. Participants are expected to engage and maintain employment with the same employer in most cases. In this regard, key informants mentioned that most employers retain their participants at the end of the subsidy. They attributed this to the successful matching of participants and employers.

Participants have two training options^{Footnote 13} :

On-the-Job training which occurs at the worksite of an employer and is based on an employability assessment and service plan. An employer agrees to hire and train an individual for a period of time. The individual becomes an employee from the beginning of the placement. On-the-Job training provides up to $390 per week based on the individual’s training plan. The maximum duration is 26 weeks but may be extended to 30 weeks in some situations. The length of the subsidy is determined on a case-by-case basis based on the needs of the individual and the complexity of the On-the-Job training and/or work exposure requirement. The program may also provide full wage reimbursement up to $12.00 per hour plus mandatory payroll deductions to non-profit organizations to hire individuals facing significant barriers to employment and who are not able to find an employer. The subsidy is also provided up to 26 weeks.
Paid Work Experience which is short-term paid work experience that helps Albertans gain employment quickly and maintain it. Paid Work Experience is intended to support individuals to perform specific work tasks, to become familiar with employer worksite standards, safe work practices and to demonstrate their competencies and skills. No training plan is required. The individual is expected to be hired after the subsidy ends. Paid Work Experience supports the employer to a maximum amount of $1,440. The amount is based on 40 hours of work at $12.00 per hour for 3 weeks.

Funding may be granted for supplemental training (for example, courses in computer software training, office administration, accounting, English language training, and workplace safety training) with up to a maximum of $1,500 per individual. The supplemental training must be included in the individual’s Training Plan based upon assessed needs and is not intended to replace training that the employer has agreed to undertake as part of the On-the-Job training. This does not include the normal technical training required for apprentices.

2.4.2 Program delivery

Based on a document review and 3 key informant interviews completed in summer 2015

Targeted Wage Subsidies is delivered by both government staff and service providers depending on whether the delivery takes place in an urban or rural area.

As explained by key informants, the application process is typically driven by the individuals and, in general, employers do not have to apply to the program. However, there are situations where employers contact delivery offices and seek assistance in determining whether a potential hire is eligible under the program.

The application process starts with a career employment consultant or service provider conducting an employability assessment with the individual. Then, they work with the individual to develop a service plan which would include Targeted Wage Subsidies.

Factors considered when deciding whether Targeted Wage Subsidies is a suitable program include barriers to employment, work history, skill levels, needs of the individual and the employer and job readiness.

Key informants explained that service providers give assistance to the individual to help them find a potential employer; however the level and means used vary across service providers or employment consultants. Key informants also explained that they try to ensure a good match between the individual and the employer to help with the retention and proper hiring of the individual by the employer after the end of the subsidy.

Once the individual finds an employer, the next step of the application process is to develop a contract with the employer. Each contract is negotiated based on the needs of the individual.

Key informants explained that from the moment the individual is deemed eligible for the program, the process to become a participant may take between 3 and 8 weeks depending on the region and the resources and staff available within the region.

2.4.3 Profile of Targeted Wage Subsidies participants

As indicated in Table D1 in Appendix D, active claimants who started their participation between 2002 and 2005 were evenly split between male and female while those who participated in 2006 to 2008 included a majority of female (55%). Sixty percent of the 2002 to 2005 participants were between 25 and 44 years of age compared to 47% for the 2006 to 2008 participants.

Participants in the 2006 to 2008 period were older as 50% of them were 45 years old and over compared to 25% for the 2002 to 2005 participants. Participants in both cohorts most frequently held jobs requiring secondary or occupational training before participation (41% of the 2002 to 2005 participants and 49% of the 2006 to 2008 participants). Those who participated in 2002 to 2005 had lower employment earnings ($22,254) in the year before participation compared to active claimants who participated in 2006 to 2008 ($29,075).

Former claimants in both the 2002 to 2005 and 2006 to 2008 cohorts were mainly female (53% and 56%, respectively) and between 25 and 44 years of age (64% and 63%, respectively). Before participation, they most frequently had occupations requiring secondary or occupational training (39% and 36% respectively) followed by occupations requiring college or apprenticeship training (31% and 27% respectively). Those who participated in the 2002 to 2005 period had lower earnings in the year before participation ($12,523) compared to $15,635 for the 2006 to 2008 participants.

2.4.4 Incremental impacts

Detailed incremental impact results for active and former claimants are presented in Tables D2 and D3 in Appendix D.

Active claimants

As shown in Figure 5, active claimants who participated between 2002 and 2005 had incremental gains in earnings after participation but the results were statistically significant only in the fourth ($3,654) and fifth year ($4,827). Overall, they had a total gain of $12,700 over the post-program period. Their incidence of employment also increased after participation by annual averages ranging between 5.4 and 8.4 percentage points.

**Figure 5. Increased earnings of active and former Targeted Wage Subsidies participants relative to non-participants**

	Gains in Earnings
Active Claimants	Former Claimants
Year 1 Post-Program	976^a	2114
Year 2 Post-Program	554^a	1855
Year 3 Post-Program	2690^a	2387
Year 4 Post-Program	3654	2057^a
Year 5 Post-Program	4827	2252^a

^a The incremental impact on earnings for active claimant participants in the 1st, 2nd and 3rd year after the program and Former claimant participants in the 4th and 5th year after the program are not statistically significant at the 95% confidence level.

Most annual results pertaining to the use of EI benefits were not statistically significant but, over the five year post-program period, active claimants had a total increase of $1,281 in the amount of EI benefits collected. Increase in EI use post-program is common for the program. Participants build entitlements to EI when working and may re-apply if they are unable to maintain the job occupied during or after the subsidy. Results for social assistance use and dependence on income support were not statistically significant.

Impacts were not produced for the 2006 to 2008 participants and for the other sub-groups as the number of participants were too small.

Former claimants

As shown in Figure 5, former claimants who participated in the 2002 to 2005 period had incremental gains in earnings in all years after participation but only the results for the first three years were statistically significant at the 95% confidence level or higher. Overall, their earnings increased by $10,665 over the post-program period. Similarly, the incidence of employment increased in all years after participation but only the results for the first (5.5 percentage points), second (4.6 percentage points) and third year (5 percentage points) were statistically significant.

These participants also had increases in EI benefit collected after participation ranging between $278 and $659. As previously indicated participants build entitlements to EI when working and may re-apply if they are unable to maintain the job occupied during or after the subsidy. Their use of social assistance decreased by annual averages ranging between $275 and $397. Most results for dependence on income support were not statistically significant.

For former claimants, it can also be argued that the increase in EI use is an indication of increase labour market attachment for this client group since they did experience increases in employment earnings and incidence of employment, and a decrease in the use of social assistance. As a reminder, former claimants are participants for whom the EI benefit period ended up to three years pre-participation.

It was not possible to estimate impacts for 2006 to 2008 participants and for other sub-groups as the number of participants was too small.

2.4.5 Cost-benefit results

For active claimants, as shown in Table D4, the benefits of the program recovered the costs within 4.3 years after participation from the society perspective. The total benefits exceeded the costs by $5,256 six years after program end. Similarly, for former claimants, the benefits of the program exceeded the costs during the first year of the participation period. The benefits exceeded the costs by $10,578 six years after the end of participation.

2.4.6 Challenges and lessons learned about Targeted Wage Subsidies design and delivery

Based on a document review and 3 key informant interviews^{Footnote 14} completed in summer 2015

Key informants were asked about the best practices or the lessons they learned in terms of design and delivery of the program. These included:

Having a client-focused approach to the program application to ensure the participant is ready, willing and able to pursue participation. This helps the participant to succeed
Ensuring a good match between employer and employee helps to guarantee participant retention after the end of the subsidy
Service providers would like to have more referrals from employment counsellors at Alberta Works Centres
Having employers register in a database which could be used to identify possible employers in the future
Ongoing case management is key for proactively addressing barriers faced by participants (for example, mental health issues, addictions and social barriers) rather than waiting until the employer views the individual as no longer being a viable employee
Maintaining a good relationship with employers and providing them support during the subsidy is important
There is a need to look at support or mentorship that could be offered to the individuals after the subsidy
It would be beneficial if the application process was online. It is currently primarily paper based.

Despite being effective at helping participants to find and maintain employment, there has been a steady decline in the number of new interventions delivered per year. The number of new interventions decreased from 261 in 2003 to 2004 to 41 in 2013 to 2014.

Key informants identified the following reasons to explain the decline in the use of the program:

Changes in the labour market and the low unemployment rate affected the program.
Mismatch between employers’ needs and participants’ skills. For example, a lot of subsidised positions are entry-level or blue collar positions and this is not where the demand is in some regions.
Lack of capacity among career employment consultants or service providers due to high turnover and lack of resources.

They also identified the following reasons to explain why employers may not be inclined to use the program:

Time and resource constraints on the part of the employers to apply, receive approval and meet reporting requirements
It can be difficult to find employers and employees that have the perfect combination of needs/barriers as prescribed in the program guidelines
There is a lack of awareness about the program among employers
There is a stigma associated with hiring someone with a subsidy

Other challenges with the design and delivery of the program include:

Some career employment consultants cannot follow-up with participants as often as needed because they lack capacity. In some situations, the caseload of career employment consultants may involve approximately 300 individuals.
High staff turnover results in loss of knowledge and expertise.
It can be difficult to find participants and employers who meet the program guidelines. Eligibility criteria are also seen as being too restrictive by key informants. For example, the program has a limitation where only one employee can be subsidised with one employer.

2.5 Self-Employment

2.5.1 Program description

Based on description submitted by Alberta

The intent of Self-Employment training is to facilitate entry into self-employment by offering individuals formal instruction, business plan development, one-to-one business counseling, coaching, guidance and follow-up during business plan implementation. It is designed for unemployed and marginally employed Albertans who have a viable business idea, personal attributes necessary to be successful at self-employment, and who are in need of self-employment training. The standard practice in Alberta is to provide a viability assessment of the business proposal before accepting a participant into the program.

The objectives of the program are:

to assist unemployed and marginally employed Albertans to create employment opportunities for themselves, and
to create additional employment opportunities in the community that might not otherwise be available.

Priority is given to eligible individuals that own 51% of their proposed business, which must not be considered as part of a franchise company. Funding is provided only to cover training, and not to be used as capital for business start-up or initial investment. Program duration is a maximum of 26 weeks, but can be extended to 52 weeks for exceptional circumstances. During the formal instruction and business plan development activities, it is expected that individuals will attend a minimum of 25 hours per week.

Training components and their approximate duration are as follows:

Formal instruction and business plan development (approximately 8-10 weeks)
Implementation of business plans (approximately 14-16 weeks)
Ongoing support for up to 180 days after completion of training. Business advisors, consultants, financiers, and lawyers are brought on as specialized instructors.

2.5.2 Profile of Self-Employment participants

As shown in Table E1 in Appendix E, active claimants who started their participation between 2002 to 2005 were mainly male (55%) while those who started in 2006 to 2008 were mainly female (52%). About one third (35%) of participants in both cohorts were between 35 and 44 years of age while 32% of the 2002 to 2005 participants and 36% of the 2006 to 2008 participants were 45 years of age or older. Participants in both cohorts most frequently had occupations requiring college or apprenticeship training (38% and 37% respectively) followed by secondary or occupational training (25% and 29% respectively) prior to participation. Both cohorts had similar earnings in the year before participation ($31,358 and $31,370).

The majority of former claimants who participated in 2002 to 2005 and 2006 to 2008 were female (54% and 65% respectively) and between 25 and 44 years of age (64% and 74% respectively). Former claimants who participated in the program between 2002 and 2005 most frequently had occupations requiring college or apprenticeship (32%) followed by secondary or occupational training (30%) prior to participation. In comparison, those who participated in the 2006 to 2008 period most frequently had occupations requiring secondary or occupational training (37%) followed by college or apprenticeship (29%). Both cohorts had similar levels of employment earnings in the year before participation ($12,750 and $12,222).

2.5.3 Incremental impacts

Like other EBSMs, incremental impacts were estimated for participants in the 2002 to 2005 and 2006 to 2008 periods. Results showed large decreases in employment/self-employment earnings and decreases in the incidence of employment. As well, compared to similar non-participants, Self-Employment participants decreased their use of EI and social assistance and reduced their dependence on government income support.

Detailed estimates are presented in Tables E2 and E3 in Appendix E. However, they are not discussed in the report since they may not provide an accurate depiction of the financial well-being of the participants in the post-program period. Impacts were examined using individual earnings reported in the T1 and T4 taxation files from Canada Revenue Agency, and measured relative to active claimants who did not participate in the program and may have been in any employment/unemployment situation following participation (for example, unemployed, paid employee or self-employed).

According to a study from Statistics Canada, self-employed individuals in Canada have a lower average annual income than paid employees ($46,200 versus $52,400 in 2009), but the average net worth of their households is 2.7 times greater than that of the paid employee households, which indicates that some self-employed individuals may leave funds within their business for reinvestment purposes^{Footnote 15}. Overall, this suggests that looking at individual earnings alone, without taking the net worth into consideration, may not provide a fair assessment of how well participants are doing financially after participation.

It is also noted that data from the Work Outcomes Reporting Program survey conducted by Alberta between January 2013 and February 2016 also showed that the mean wage per hour of participants was lower after participation for those who used to work full time before entering the program (n=161). It decreased from $27.30 before participation to $23.13 after participation. At the same time, 53% of those who worked full time before participation continued to work full time after participation.

There is a lack of understanding around the role played by this program in helping future entrepreneurs to implement viable business plans and to develop their entrepreneurship skills. Overall, it is not clear whether participant’s success in improving their labour market attachment through self-employment is more closely associated with their business idea and their entrepreneurship skills than the assistance provided under the program.

2.6 Job Creation Partnerships

2.6.1 Program description

Based on description submitted by Alberta

Job Creation Partnerships is a financial assistance program. Its objective is to enable unemployed persons to obtain employment by providing them with employment opportunities through which they can gain work experience to improve their long-term employment prospects. In Alberta, Job Creation Partnerships-similar programming is delivered through the Integrated Training Program and its component program known as Immigrant Bridging.

Integrated Training is a competency-based occupation-related training program intended to provide unemployed or marginally employed adult Albertans with a unique blend of employability and/or essential skills, applied academics and/or English as a Second Language, occupation-related skills, and work experience placement(s). The objectives of Integrated Training are to:

Enable unemployed or marginally employed adult Albertans to secure and maintain employment in an occupation that is in-demand;
Provide training and occupation-related skills recognized by industry/employers; and,
Help immigrants gain employment in their original occupation or a related occupation.

Immigrant Bridging programs are a sub-set of Integrated Training that focus on training skilled immigrants with prior education and/or experience in a specific occupation, so they can bridge gaps in knowledge or skills in order to gain employment in that occupation or a related occupation. Immigrant Bridging programs usually have a language component.

The following training components are incorporated into all Integrated Training programs:

Occupation-related skills
One or more work experience placement(s)
Employability and/or essential skills

Where necessary for the occupational destination, the following components are also offered:

Academic competencies relevant to the occupation/occupational outcome, and/or
Workplace English as a Second Language.

The work experience placement is not a paid employment opportunity (or a wage subsidy program). Instead the work experience placement is intended to be an extension of the training program. Work experience competencies for each individual vary for each work experience placement. The required competencies that an individual will be expected to demonstrate as a result of each work experience placement is decided by training providers after consultation with each employer. Program duration is a maximum of 52 weeks.

Some of the program examples under Immigrant Bridging are: Enhanced Language Training Program, Internationally Educated Veterinary Bridging, Career Connections: an Introduction to Business, and Business Communication for Professionals. Training Providers may offer some basic English-language, computer, workplace safety, and job search training. Program duration is a maximum 52 weeks.

2.6.2 Program delivery

Based on description submitted by Alberta

Integrated Training programs are delivered in a classroom setting or through synchronous e-learning. Integrated Training is primarily provided at educational institutions or through 3rd party contracts and is delivered through either contract or tuition based procurement.

While it is recognized that the design and delivery of Integrated Training will vary to meet the needs of the individuals and labour market demand, it is strongly recommended that a minimum of 60% of the training should occur in the classroom, or by synchronous e-learning and a maximum of 40% of the training should occur during one or more work experience placements. If more than 40% of work experience is required for a specific individual group, the training provider must provide rationale for the length of the work experience. The delivery mechanism of Immigrant Bridging is the same as that of Integrated Training.

2.6.3 Profile of Job Creation Partnerships participants

Active claimants who participated in 2002 to 2005 and 2006 to 2008 were mostly male (60% and 59%). As shown in Table F1 in Appendix F, sixty one percent of those who participated in 2002 to 2005 and 59% of those who participated in the 2006 to 2008 period were aged between 25 and 44 years old. Participants in both the 2002 to 2005 and 2006 to 2008 cohorts most frequently had occupations requiring secondary or occupational training before participation (38% and 40%). Those who participated in the 2002 to 2005 period had lower employment earnings in the year before participation ($19,084 compared to $23,071).

A little more than half (52%) of former claimants who started participation in 2002 to 2005 were male while 52% of those who started participation in 2006 to 2008 were female. Sixty percent of those who started participation in 2002 to 2005 and 58% of those who started participation in 2006 to 2008 were aged between 25 and 44 years old. Participants in both cohorts most frequently had occupations requiring secondary school or occupational training (38% and 41%) prior to participation. Participants in the 2002 to 2005 period had lower earnings in the year before participation ($8,712) compared to those in the 2006 to 2008 period ($11,533).

2.6.4 Incremental impacts

Active claimants

As shown in Table F2 in Appendix F, active claimants who participated in 2002 to 2005 had incremental gains in earnings in most post-program years. As shown in Figure 6, these gains in earnings ranged between $2,429 and $4,720. They also had gains in incidence of employment ranging between 8.4 and 12.6 percentage points. Their use of EI increased starting the second post-program year by annual averages ranging between $440 and $760. Results for social assistance use and the level of dependence on income support were not statistically significant. The increase in EI use reflects the inability for some participants to maintain the employment secured in the short-term after participation.

**Figure 6. Increased earnings of active and former Job Creation Partnerships participants relative to non-participants**

	Gains in Earnings
Active Claimants	Former Claimants
Year 1 Post-Program	2429	1159
Year 2 Post-Program	2126*	727*
Year 3 Post-Program	3584	1064*
Year 4 Post-Program	4424	872*
Year 5 Post-Program	4720	1009*

^a The incremental impact on earnings for Former participants in the 2nd, 3rd, 4th and 5th year after the program and Active participants in the 2nd year after the program are not statistically significant at the 95% confidence level.

Active claimants who participated in 2006 to 2008 had gains in incidence of employment in the first two years after participation. They had increases in EI use in the second and third year. Results for other indicators were not statistically significant at the 95% level or above.

Former claimants

As shown in Table F3 in Appendix F, results for former claimants who participated in the 2002 to 2005 period were often not statistically significant. However, results showed increases in incidence of employment ranging between 3.8 and 5.9 percentage points in all years after participation and an increase of $1,159 in earnings in the first year after participation (see Figure 6). While all annual impacts for EI use were not statistically significant, a total increase of $700 was found over the five-year post-program period. As indicated above, the increase in EI use reflects the inability for some participants to maintain the employment secured in the short-term after participation.

Similarly, most results for former claimants who participated in the 2006 to 2008 period were also mostly statistically non-significant. Like the 2002 to 2005 participants, they also increased their incidence of employment and EI use after participation.

2.6.5 Cost-benefit results

The benefits of the program were $1,440 less than the costs six years after participation from the society perspective. As shown in Table F4 in Appendix F, the benefits would need to be maintained over 6.5 years after the end of participation to match the costs. For former claimants, six years after program end, the benefits of the program were $5,094 less than the costs. The benefits would need to persist over 14.1 years after the end of participation to match the costs.

Overall, when interpreting cost-benefit results for Job Creation Partnerships, it should also be acknowledged that program funding helps to develop the community and the local economy and none of those benefits were accounted for in the calculations as they are difficult to quantify.

2.7 Employment Assistance Services

2.7.1 Program description and delivery

Based on a document review and 8 key informant interviews completed in summer 2013

Employment Assistance Services assist individuals to prepare for, obtain and maintain employment. Services are accessible to all unemployed persons seeking employment-related supports.

The delivery model is based on three essential components:

Information Services includes the provision of labour market, occupational, training and educational information which helps participants choose training or a learning path, to find and keep a job and to plan or change careers. These services can be accessed through a variety of ways such as visiting Alberta Works Centres, calling the Career Information Hotline, accessing the Alberta Learning Information Service website and by attending events.
Needs Determination Services assist individuals by providing advice, guidance and access to services that help people make successful labour market transitions. This service includes a needs determination assessment, career advising, workshops, exposure courses, financial and non-financial supports and job placements.
Employability Services includes employability assessments to help individuals increase their self-awareness, career counselling and service management, which is similar to individualized case management support.

In Alberta, Employment Assistance Services are predominantly delivered through service providers but can also be delivered through provincial government offices.

Key informants confirmed that they are using LMI products to support the delivery of the program. In particular, LMI products produced by Alberta provide information to staff, service providers, immigrants, professional associations, educators, employers, participants and the general public. However, managers interviewed mentioned that there is room to improve the way LMI is used. Service providers also raised concerns about the unavailability of information about recent EI claimants.

Alberta is devoting efforts to incorporate LMI with broad statistics at the local level. However, it is difficult to access LMI available at the local level. The province is committed to producing more and more tailored reports based on individual needs.

2.7.2 Profile of Employment Assistance services participants

Socio-demographic characteristics

The socio-demographic profile was produced for individuals who only received Employment Assistance Services without participating in other Employment Benefits.

As shown in Table G1 in Appendix G, active claimants who started their participation in 2002 to 2005 were mainly male (54%) while those who started in 2006 to 2008 were almost evenly distributed between male and female. Thirty (30%) and 28% of participants in both cohorts were aged between 35 and 44 years old while 30% and 35% of participants in both cohorts were aged 45 and older. Active claimants who participated in the program most frequently had occupations requiring secondary or occupational training before participation (37% in each cohort). They also had similar employment earnings in the year before participation ($24,638 and $24,321).

Fifty-three percent of former claimants who started in the 2002 to 2005 period were male while 53% of those who started in the 2006 to 2008 period were female. Sixty one percent of the 2002 to 2005 participants and 59% of the 2006 to 2008 participants were aged between 25 and 44 years of age. Participants in both cohorts most frequently had occupations requiring secondary or occupational training prior to participation (37% each). Those who participated in 2002 to 2005 had lower earnings in the year before participation ($12,863) compared to the 2006 to 2008 participants ($15,229).

Labour market barriers faced by Employment Assistance Services participants

Based on a document review and 8 key informant interviews completed in summer 2013

According to key informants, the main labour market barriers experienced by participants in Alberta are:

Lack of adequate skills, poor problem solving skills and non-transferable skills
Skills mismatches, especially for those moving to Alberta from other provinces/territories
Low educational attainment
Lack of affordable housing
Lack of child care spaces
Addiction issues
Mental health issues
Low self-esteem and self-confidence/motivation issues
Lack of adequate marketing skills
Type of work available is different: a lot of work is project based (with a start and end date), which is different than working continuously at the same job
Immigrants face issues with recognition of their foreign credentials and language barriers
Long tenured workers face skills mismatches and often have non-transferable skills

2.7.3 Incremental impacts

As shown in Table G2 in Appendix G, active claimants who participated during the 2002 to 2005 period had increases in incidence of employment ranging between 1.5 and 3 percentage points in all years after participation (see Figure 7). However, their results for earnings were mostly not statistically significant. Their use of EI benefits decreased in the first three years after participation by averages ranging between $109 and $417 while their use of social assistance benefits increased in all post-program years by averages ranging between $43 and $150. Impacts on dependence on income support were mixed.

**Figure 7. Incidence of employment for active claimant participants in Employment Assistance Services**

	Percentage Points
Year 1 Post-Program	3
Year 2 Post-Program	2.3
Year 3 Post-Program	2.2
Year 4 Post-Program	1.5
Year 5 Post-Program	2

The 2006 to 2008 participants also had increases in incidence of employment in the first two years after participation. Their use of EI also decreased after participation. However, these participants experienced decreases in earnings in all three post-program years. They also increased their use of social assistance and their level of dependence on income support over most years after participation.

The results varied based on the sub-groups examined:

Youth (under 30 years old) who participated between 2002 and 2005 had gains in incidence of employment and decreases in EI use. However, the earnings impacts were generally not statistically significant. Their use of social assistance generally increased after participation as well as their dependence on income support.
Older workers (55 years old and over) who participated in 2002 to 2005 increased their earnings and incidence of employment in each of the five post participation years. Their use of EI decreased in the first two years post-program while results for other years and most impacts on dependence on income support were not statistically significant at the 95% level or above.
Long-tenured workers who participated in 2007 to 2009 improved their incidence of employment and decreased their EI use after participation but increased their use of social assistance benefits. Results for earnings and dependence on income support were not statistically significant.

Earlier participation in Employment Assistance Services improves participants’ labour market outcomes

The study on the effects related to the timing of participation showed that incremental impacts on earnings and employment were larger for individuals who received Employment Assistance Services early during their EI claim compared to individuals who never participated and to those who remained on EI a longer time before participating in the program (see Figure 8 below and Table G3 in Appendix G). Specifically, individuals who started their participation within four weeks after the start of their EI benefit period had a total increase of $16,164 in their earnings over five years post-program. They also had increases in their incidence of employment ranging between 1.3 to 2.2 percentage points per year in the first three years after participation.

Participants who started between 5 and 8 weeks and those who started between 9 and 12 weeks after the start of their EI claim had increases in their earnings totalling $12,418 and $11,609 respectively over the post-program period. The increases in earnings for participants who started between 5 and 8 weeks were accompanied gains in incidence of employment of 2.1 and 2.2 percentage points in the first two post-program years. Participants who started between 9 and 12 weeks had statistically non-significant impacts on incidence of employment.

The participants who started their participation during the second and third quarters of their EI claim generally had lower gains in earnings (total of $5,399 and $8,342 respectively) and gains in incidence of employment of 1.4 and 1.6 percentage points in the first and second year after participation. Participants who started in the fourth quarter had statistically non-significant impacts on earnings.

**Figure 8. Cumulative incremental impacts on earnings related to the timing of participation in Employment Assistance Services**

	Cumulative Gains in Earnings
1-4 weeks	16164
5-8 weeks	12418
9-12 weeks	11609
2nd quarter	5399
3rd quarter	8342
4th quarter	2469^a

^a The cumulative incremental impact on earnings for participants in the 4th quarter is not statistically significant.

As well, the study looked at the difference between the number of EI weeks unused by participants and the number of EI weeks unused by their comparison groups in order to determine the effect of the timing of participation on the return to employment. Those who received assistance within the first eight weeks of their claim returned to employment more quickly than the comparison group. Specifically, those who started within 4 weeks returned to employment 4.1 weeks earlier than the comparison group while those who started between the 5th and 8th week returned to employment 1 week earlier.

Among all participants in the 2002 to 2005 period, 13,319 received assistance within the first 4 weeks of establishing an EI claim and 4,714 received assistance in weeks 5 to 8. With an average weekly EI benefits of $327 during this period, the 13,319 participants did not use $17,856,783 in EI benefits (13,319 * $327 * 4.1 weeks). As well, the 4,714 participants did not use $1,541,478 in EI benefits (4,714 * $327 * 1 week). The average cost of Employment Assistance Services interventions in the 2002 to 2005 period was $252. This represents a cost of $4,544,316 [(13,319 + 4,714) * $252] for a net saving of $14,853,945.

2.7.4 Cost-benefit results

Six years after the end of participation, the benefits of the program from the society perspective exceeded the cost by $944 (as shown in Table G5 in Appendix G). It took 5.2 years after the end of participation for the benefits to recover the costs.

2.7.5 Challenges and lessons learned about Employment Assistance Services design and delivery

Based on a document review and 8 key informant interviews^{Footnote 16} completed in summer 2013

Key informants identified best practices and lessons learned related to program design and delivery:

It would be beneficial to provide a one-stop-shop for participants instead of relying on referrals made to other organizations
Long-term interventions and follow-up are required for individuals facing multiple barriers since those barriers are often inter-related and participants need a long time to address them
Conducting adequate and detailed assessments of participants’ needs to help determine training needs
Job maintenance services are essential
Job placements are helpful. Holding job fairs also help to increase awareness about available services
Having more flexibility within program guidelines would allow to try new approaches such as innovation in services to participants
Targeting contracts for specialized services for participants with specific needs would be beneficial
Having adequate LMI and having well trained staff is important
Having the capacity to tailor services to community needs is an important consideration
Participants often require assistance in order to stabilize their life in terms of housing, child care, personal management skills, etc.
It would be useful if a system was developed to facilitate the tracking of participant progression. It is important to have progressive success indicators and not simply focus on the employment outcome alone.
It is important for service providers to build relationships with employers, specialized service providers and community organizations. This helps with participant referrals and facilitates the integration of participants in the local labour market.
Access to EI Part I data is important

Key informants identified a number of challenges with respect to the design and delivery of the program:

Program guidelines are perceived as being rigid. This can restrict service providers’ ability to address the needs of participants with multiple barriers. It is not possible to “fit” participants with multiple barriers into a prescribed program.
Focusing on quicker return to employment outcomes as the sole indicator of success does not adequately account for the extensive barriers faced by these participants.
The awareness about the program varies greatly depending on the various participant groups (for example, new immigrants face challenges around language) and participants are often coming to the service providers through word of mouth.
Some service providers face rigid criteria about what type of career material they can display.
Having one year contracts with service providers creates challenges for some. For example, it can limit their capacity to retain qualified staff and to access stable funding. Staff turnover causes loss of corporate memory about programs and services and their delivery.

3. Comparison of key findings by program type

This section provides an overview of the key findings from the incremental impact analysis for Skills Development, Targeted Wage Subsidies, Job Creation Partnerships and Employment Assistance Services for both active and former EI claimant participants who started participation in the 2002 to 2005 period.

Program participants have a higher probability of being employed than comparison group members

As shown in Figure 9, active EI claimants who participated in Skills Development, Targeted Wage Subsidies, Job Creation Partnerships and Employment Assistance Services had higher probability of being employed (in other words, increased their incidence of employment) compared to similar non-participants. As well, former EI claimants who participated in Skills Development, Targeted Wage Subsidies and Job Creation Partnerships had higher probability of being employed compared to former EI claimants who received low intensity interventions under Employment Assistance Services.

It is noted that Employment Assistance Services are relatively modest activities such as counselling, job search assistance and case management. By themselves, they are not expected to lead to substantial effects on labour market outcomes. However, as demonstrated in the report, providing Employment Assistance Services earlier during the EI claim (first 8 weeks) generates significantly greater returns (see Figure 8 in sub-section 2.7.3).

**Figure 9. Change in probability of being employed in participants relative to non-participants^{Footnote 17}**

	Percentage Points
Active Claimants	Former Claimants
Skills Development	9.6	5.8
Targeted Wage Subsidies	7.3	3.9
Job Creation Partnership	10.8	5.2
Employment Assistance Services	2.2

Increased earnings for participants compared to comparison group members

As shown in Figure 10, active EI claimants who participated in Skills Development, Targeted Wage Subsidies and Job Creation Partnerships increased their employment earnings compared to similar non-participants. As well, former EI claimants who participated in Skills Development, Targeted Wage Subsidies and Job Creation Partnerships increased their employment earnings compared to former EI claimants who received Employment Assistance Services exclusively.

As already noted, Employment Assistance Services are relatively modest activities and, by themselves, are not expected to lead to substantial effects on labour market outcomes. In other words, the program aims to support the return to work of unemployed participants and not necessarily to secure a better paying job than pre-participation. However, as demonstrated in the report, providing Employment Assistance Services earlier during the EI claim (first 8 weeks) generates significantly greater returns.

**Figure 10. Increased cumulative earnings of participants relative to non-participants**

	Cumulative Gains in Earnings
Active Claimants	Former Claimants
Skills Development	16952	9464
Targeted Wage Subsidies	12700	10665
Job Creation Partnership	17283	4832^a
Employment Assistance Services	2485^a

^a The incremental impact on earnings for former claimants who participated in Job Creation Partnership and active claimants who participated in Employment Assistance Services are not statistically significant at the 95% confidence level.

The use of EI is increased for participants compared to the comparison group. For active claimants, this may reflect the inability for some participants to maintain the employment secured in the short-term following participation. In the case of former claimants who participated in Skills Development and Targeted Wage Subsidies, it can be argued that this reflects an increase in labour market attachment since the incremental impacts on employment earnings and incidence of employment are positive, and given the decrease in the use of social assistance. As well, the social benefits of participation exceeded costs of investments for most interventions over time.

As shown in Figure 11, active EI claimants who participated in Skills Development, Targeted Wage Subsidy and Job Creation Partnerships increased their use of EI compared to similar non-participants. It is not surprizing to observe an increase in EI use for claimants who participated in Targeted Wage Subsidies since participants build entitlement to EI while working. Active EI claimants who participated in Employment Assistance Services decreased their use of EI compared to similar non-participants.

**Figure 11. Change in the cumulative use of Employment Insurance relative to non-participants**

	Cumulative Use of EI Benefits
Active Claimants	Former Claimants
Skills Development	467	541
Targeted Wage Subsidies	1281^a	2164
Job Creation Partnership	2605	700
Employment Assistance Services	-618

^a The incremental impact on earnings for active claimants who participated in Targeted Wage Subsidy is not statistically significant at the 95% confidence level.

As shown in Figure 12, program participants decreased their use of social assistance benefits with the exception of active participants in Employment Assistance Services. As previously stated, the relatively modest activities provided under Employment Assistance Services are not expected to lead to substantial effects on participants’ labour market outcomes.

**Figure 12. Change in cumulative use of social assistance relative to non-participants**

	Cumulative Use of Social Assistance Benefits
Active Claimants	Former Claimants
Skills Development	-414	-1743
Targeted Wage Subsidies	260^a	-1714
Job Creation Partnership	-501^a	-858^a
Employment Assistance Services	435

^a The incremental impacts on social assistance use for Targeted Wage Subsidies and Job Creation Partnerships participants are not statistically significant at the 5% level.

Social benefits of participation exceeded costs of investments for most interventions.

As shown in Table 3, social benefits to participation exceeded investment costs in a period ranging between the first year of program participation to about 14 years after participation.

**Table 3. Number of years for the benefits to exceed program costs**
	Skills Development	Targeted Wage Subsidies	Job Creation Partnerships	Employment Assistance Services
Active claimants	5.1	4.3	6.5	5.2
Former claimants	5.5	0.8	14.1	n/a

Overall, incremental impacts demonstrate that LMDA programs and services are improving the labour market attachment of participants in Alberta. As well, social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first eight weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

4. Conclusions

Evaluation evidence presented and discussed in this report demonstrated that programs and services designed and delivered by Alberta under the LMDA are generally helping participants to improve their labour market experience after participation. As such, evaluation evidence suggests that LMDA funded programming contributes to achieve Alberta’s priorities and the following Ministry of Labour outcome: Albertans have the skills required by Alberta’s labour market.

Overall, incremental impacts demonstrate that programs and services are improving the labour market attachment of participants. As well, social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services earlier during an EI claim (first eight weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Key informants interviews with service providers and program managers as well as the documents reviewed and the questionnaires filled by Alberta representatives revealed some lessons learned and specific challenges about program design and delivery.^{Footnote 18}

Skills Development

Undertaking a comprehensive assessment of the participants in order to assist them in making informed choices and to commit to their training.
Having access to disability-related employment and financial supports.

In terms of lessons learned and best practices, key informants indicated the importance of:

Providing long-term assistance and to have an interdisciplinary team of professionals that can provide a continuum of services and supports to individuals facing multiple barriers.
Monitoring participants during training and following up after the training.
Coordination among key labour market stakeholders and service providers.
Offering coordinated foundational learning programs that assess basic skills at the outset of the program, address weaknesses and integrate basic skills development into professional program would be beneficial.
Providing intensive case management for individuals facing multiple barriers to employment.

The main challenges related to Skills Development design and delivery included:

The level and duration of the financial support being too low for individuals facing multiple barriers to employment
Lack of instrument to test literacy level
Identifying learning disability too late in the process
Lack of awareness around available support for those with physical and mental disabilities

Skills Development – Apprentices

Key informants identified the following best practices and lessons learned about the program:

Having close connections with the industry helps to understand where the government can make better investments and offer training closer to the participant’s home. For example, it has been possible to increase the training offered in some locations based on employers input.
Providing accommodations to individuals facing barriers to employment or to completing training. For example, providing disability related supports or help to individuals with learning barriers in order to help them complete their exams.
Existing Canadian labour market literature showed that there is a fairly high non-completion rate among apprentices; however it is not possible with available data to generate a reliable estimation of the completion rate of Skills Development – Apprentices participants in Alberta. According to key informants, the drop-out from the apprenticeship process may be due to external factors such as:

The apprentice realises that he/she is not suited for the job
Employers are not able to let the apprentice go on training

Targeted Wage Subsidies

Key informants identified best practices or the lessons related to Targeted Wage Subsidies’ design and delivery. These included:

Having a client-focused approach to the program application to ensure the participant is ready, willing and able to pursue participation.
Ensuring a good match between employer and employee helps to guarantee participant retention after the end of the subsidy.
Ongoing case management is key for proactively addressing barriers faced by participants (for example, mental health issues, addictions and social barriers).
Maintaining a good relationship with employers and providing them support during the subsidy is important.

Changes in the labour market and the low unemployment rate affect the program
Mismatch between employers needs and participants skills
Lack of capacity among career employment consultants or service providers due to high turnover and lack of resources
Employers may not be inclined to use the program because of:

Time and resource constraints on the part of the employers to apply, receive approval and meet reporting requirements.
A lack of awareness from employers about the program.
The stigma associated with hiring someone with a subsidy.

Challenges related to program design and delivery included:

Some career employment consultants cannot follow-up with participants as often as needed because they lack capacity. In some situations, the caseload of career employment consultants may involve approximately 300 individuals.
High staff turnover results in loss of knowledge and expertise.
It can be difficult to find participants and employers who meet the program guidelines. Eligibility criteria are also seen as being too restrictive by key informants. For example, the program has a limitation where only one employee can be subsidised with one employer.

Employment Assistance Services

Key informants identified some best practices and lessons learned related to the provision of Employment Assistance Services:

It would be beneficial to provide a one-stop-shop for participants instead of relying on referrals made to other organizations.
Long-term interventions and follow-up are required for individuals facing multiple barriers since those barriers are often inter-related and participants need a long time to address them.
Conducting adequate and detailed assessments of participants’ needs to help determine training needs.
Job maintenance services and job placements are helpful. Holding job fairs also help to increase awareness about available services.
Targeting contracts for specialized services for participants with specific needs would be beneficial.
Having adequate labour market information and having well trained staff is important.
Having the capacity to tailor services to community needs is an important consideration.
It is important for service providers to build relationships with employers, specialized service providers and community organizations. This helps with participant referrals and facilitates the integration of participants in the local labour market.
Access to EI Part I data is important.

Challenges related to the provisions of Employment Assistance Services included:

Program guidelines are perceived as being rigid. This can restrict service provider’s ability to address the needs of participants with multiple barriers. As well, the focus on quicker return to employment as the sole indicator of success does not adequately account for the extensive barriers faced by these participants.
Program awareness varies greatly depending on the various participant groups.
Having one year contracts with service providers can limit their capacity to retain qualified staff and to access stable funding.

5. Recommendations

Recommendations that emerged from the evaluation findings presented in this report are as follows:

Recommendation 1: Consideration should be given to providing Alberta with timely access to data on new EI recipients for supporting targeting and increasing awareness.

Key informants reported that lack of essential skills, learning disabilities and literacy issues are barriers to accessing and completing training.

Incremental impact results show that Targeted Wage Subsidies is improving the earnings and employment of participants. However, its use has been falling over the years. According to key informants, the decline may be related to factors such as a lack of awareness about the program, the commitments required from employers in terms of time and resources as well as the stigma associate with hiring someone with a subsidy.

Recommendation 4: Consideration should be given to enhance the capacity of service providers to access, when needed, relevant labour market information.

Recommendation 5: Consideration should be given to examine in more detail the design and delivery of Self-Employment and whether the performance indicators for this program are appropriate.

Having access to complete data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants would be useful to inform policy development and program design.
Having access to complete data on the cost of programming per participant would also provide a refined assessment of how much participation cost for active claimants as compared to former claimants.
The evaluation was not able to provide a comprehensive description of the type of training funded under Skills Development (for example, literacy, essential skills, adult basic education and vocational training). As a result, it is currently not possible to separate the effects of adult basic education from those of vocational training although both types of training may lead to very different labour market outcomes.
In Alberta, the Integrated Training and Immigrant Bridging are separate programs that are recorded as Job Creation Partnerships in the LMDA data. As a result, it is currently not possible to separate the effects of Integrated Training from those of Immigrant Bridging although both programs may lead to very different labour market outcomes for different target groups.
There are various types of Employment Assistance Services provided under the LMDAs. These services can be very different in nature and it possible that some may be more effective than others at helping participants to return to employment. For example, having access to a computer for researching jobs on its own may yield different impacts than receiving counselling and assistance to develop a return-to-work action plan.

Recommendation 7: Improvements in the data collection is recommended to address key program and policy questions of interest to the federal and provincial/territorial governments. Specifically:

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. Alberta, ESDC and other P/Ts should work together to define common categories for both programs.
Examine the added value in reporting separately on participants in the Integrated Training and Immigrant Bridging programs.
Collect detailed data on the cost of interventions.

References

Alberta Ministry of Labour. Business Plan 2016-19 Labour. Ministry of Labour. March 17, 2016.

Alberta Human Services. Alberta Works Policy Manual. Accessed February 29, 2016

Alberta Human Services. Transitioning Supports for Apprenticeship-related Programs from Human Services to Innovation and Advanced Education. July 22, 2015.

Questions and answers Accessed September 23, 2016

Canada Employment Insurance Commission. Employment Insurance Monitoring and Assessment Reports. 2003 to 2004 to 2013 to 2014.

Coe, Patrick. “Apprenticeship programme requirements and apprenticeship completion rates in Canada.” Journal of Vocational Education and Training. 2013. 65(4): 575 to 605.

Employment and Social Development Canada. Study Apprenticeship Technical Training Component of the Occupational Training Program in Alberta: Technical Report. Evaluation Directorate. June 16, 2016

Employment and Social Development Canada. Cost-Benefit Analysis of Employment Benefits and Support Measures in Alberta: Technical Report. Evaluation Directorate. May 30, 2016

Employment and Social Development Canada. Study of Work Foundations / Occupational Training in Alberta: Technical Report. Evaluation Directorate. April 18, 2016

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for the EI Claimant Category: “Long-Tenured Workers” in Alberta: Technical Report. Evaluation Directorate. March 9, 2016

Employment and Social Development Canada. Study on Workplace Training in Alberta. Evaluation Directorate. March 16, 2016.

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for 2006 to 2008 Participants in Alberta: Technical Report. Evaluation Directorate. February 2, 2015

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Medium-Term Incremental Impacts for 2002 to 2005 Participants in Alberta: Technical Report. Evaluation Directorate. May, 2014

Employment and Social Development Canada. Study on Employment Assistance Services: Technical Report for Alberta. Evaluation Directorate. September 22, 2014

Employment and Social Development Canada. Effects of the Timing of Participation in Employment Assistance Services in Alberta: Technical Report. Evaluation Directorate. December 15, 2014

LaRochelle-Côté, Sébastien and Sharanjit Uppal, "The Financial Well-Being of the Self-Employed," Perspectives on Labour and Income, vol. 23, no. 4, Winter 2011

Red Seal. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal, 2014.

Alberta Human Services. Transitioning Supports for Apprenticeship-related Programs from Human Services to Innovation and Advanced Education, July 22, 2015.

Alberta Human Services. Transitioning Supports for Apprenticeship-related Programs from Human Services to Innovation and Advanced Education, July 22, 2015. (Accessed September 23, 2016)

Work Outcomes Reporting Program Survey 2013 to 2016. Summary of Results Submitted by Alberta Ministry of Labour on August 11, 2016.

Acronyms

EBSM Employment Benefits and Support Measures
EI Employment Insurance
ESDC Employment and Social Development Canada
LMDA Labour Market Development Agreements
LMI Labour Market Information
P/T Provinces and Territories

Appendix A - Methodology

Qualitative data

Qualitative data reported in the Skills Development (SD), Skills Development-Apprentices (SD-A), Targeted Wage Subsidy (TWS) and Employment Assistance Services (EAS) studies were collected from key informant interviews with managers and service providers and a document/ literature review. As well, questionnaires were completed by provincial/ territorial government representatives for the SD, SD-A and TWS studies. Table A1 provides the number of key informants interviewed.

Key informant interviews for the EAS study were conducted in 2013 while those for the SD, SD-A and TWS studies were conducted in 2015.

**Table A1. Number of key informants interviewed**
Number of key informant Interviews	Studies
Number of key informant Interviews	SD	SD-A	TWS	EAS
Managers	5	4	3	4
Service providers	5	4	3	4

Quantitative methods

All quantitative analyses were conducted using linked administrative data from EI Part I (EI claim), EI Part II (EBSM participation data) and T1 and T4 taxation files on up to 100% of participants in Alberta.

Incremental impacts

The incremental impact analysis compared the labour market experience of participants before and after their participation with that of a comparison group. The goal was to determine the direct effect of program participation on key labour market indicators (see Figure 1 in the introduction section).

For active claimants, incremental impacts were measured relative to a comparison group of active claimants who could have participated in the EBSMs but did not. Former claimants can be underemployed and unable to requalify for EI; out of the labour force for various reasons; or, on social assistance. Based on previous evaluation methodologies, on expert advice and given the difficulty in generating a suitable comparison for former claimants using administrative data alone, the comparison group for former claimants was created using individuals who participated in EAS only during the reference period. This is a conservative approach given the fact that participation in Employment Assistance Services can lead to limited effects on labour market outcomes.

Participants and non-participants were matched based on a wide array of variables including age, sex, location, skill level required by the last occupation held prior to participation, reason for separation from employment, industry in which they were previously employed as well as employment earnings and use of EI and social assistance for each of the five years before participation.

All analyses were conducted using a unit of analysis called the Action Plan Equivalent, which combines all EBSMs given to an individual within no more than six months of each other. For reporting purposes, incremental impacts were attributed to the longest intervention of the Action Plan Equivalent when SD, TWS, Job Creation Partnerships (JCP) or Self-Employment (SE) was the longest intervention. Impacts for EAS were calculated for Action Plan Equivalent that contained only EAS with no Employment Benefits. These were referred to as EAS-only.

The incremental impact estimates were produced using non-experimental methods, namely propensity score matching, using the Kernel Matching method, along with Difference-in-Differences method to estimate program impacts. Alternative matching techniques (Nearest Neighbour and Inverse Propensity Weighting) were also used for validation purposes.

Incremental impacts were measured for the following indicators:

Employment/self-employment earnings represent the total earnings an individual had from paid employment and/or self-employment. (This information is available by calendar year and is obtained from T1 and T4 tax return records.)
Incidence of employment/self-employment represents the incidence of having earnings from employment and/or self-employment.
Amount of EI benefits received represent the average amount of EI benefits received.
Weeks in receipt of EI benefits represent the average number of weeks during which EI benefits were received.
Social Assistance benefits represent the average amount of social assistance benefits received. (This information is available by calendar year and is obtained from T1 tax return records.)
Dependence on income support represents the ratio of participant’s income that came from EI and social assistance benefits (in other words, EI benefits + social assistance benefits / (EI benefits + social assistance benefits + earnings from employment/self/employment)).

Incremental impacts were estimated for different cohorts of participants:

All active and all former claimants as well as youth (under 30 years old) and older workers (55 years old and over) who started their EBSM participation between April 1, 2002 and March 31, 2005.
All active and all former claimants who stated their EBSM participation between January 1, 2006 and March 31, 2008.
Active and former claimants who were long-tenured workers and who started their EBSM participation between January 1, 2007 and December 31, 2009. Long-tenured workers covered in this study are individuals who have established an EI regular or fishing benefit claimants and who had paid at least 30% of the annual maximum employee EI premiums in seven of the ten years preceding their EI claim and who had collected 35 or fewer weeks of EI regular or fishing benefits in the five years preceding their claim. This definition is similar to the EI claimant category “long-tenured workers” introduced under Connecting Canadians with Available Jobs.

Cost-benefit analysis

The cost-benefit analysis compared how much it cost for individuals to participate in the programs and how much it costs the government to deliver those programs with the benefits both the participants and the government drew from those programs. The analysis was carried out from the society perspective which combines the costs and the benefits for both the participants and the government.

Costs and benefits included in the calculations were as follows:

Program costs included the administration cost and the direct cost of the EBSMs. The cost for each EBSM was calculated at the Action Plan Equivalent level. The costs were determined based on the average composition of the Action Plan Equivalent.
The Marginal Social Cost of Public Funds represented the loss incurred by society when raising additional revenues such as taxes to fund government spending. The value was estimated as 20% of the program cost, sales taxes, income taxes, impacts on EI and impacts on social assistance paid or collected by the government.
Employment earnings consisted of incremental impacts on participants’ earnings during and after participation. The calculation accounts for the participant’s forgone earnings during participation (in other words, opportunity cost). These are based on incremental impacts for the 2002 to 2005 participants.
Fringe benefits included benefits such as employer-paid health and life insurance as well as pension contributions. The rate used to calculate the fringe benefits was 15% of the incremental impact on earnings.

The program effects on EI and social assistance use, and the sale and income tax revenues were not included in the calculations since these costs and benefits cancel each other out from the social perspective by definition. For example, while EI and social assistance are benefits received by participants, they represent a cost for the government. However, as indicated above, these effects are accounted for in the calculation of the Marginal Social Cost of Public Funds.

When producing the results, to bring all costs and benefits to a common base and to account for inflation and interest on foregone government investment, the estimates for the second year of participation and up to the sixth year post-program were discounted by 5% per year. As well, when the benefits were still lower than the costs six years after program end, the payback period was calculated by assuming that the average benefit or cost measured over the fifth and six year post-program would persist over time (discounted at a 5% annual rate).

Strengths and limitations from the studies

Overall, the number of key informants interviewed was relatively small in some studies. The key informants’ responses were representative of their own experience and their own region but it is unclear if they were fully representative of the entire province and territory.

The matching process led to the creation of comparison groups closely matched to the LMDA participants in terms of their background characteristics. Results obtained with Kernel Matching were validated with the use of two other techniques (Inverse Propensity Weighting and Nearest Neighbour), increasing the level of confidence in the results. However, readers should be aware that incremental impacts may be affected by factors not captured by the matching process. For example, the motivation to seek employment was not directly measured except to the extent it was captured in prior income and labour market attachment patterns.

Readers should also keep in mind that it is not possible to compare the results obtained for each claimant type since the results for active claimants represent the effects of the EBSMs relative to non-participation while the results for former claimants represents the Employment Benefits relative to a limited treatment (EAS only).

“Long-tenured workers” are defined as individuals who meet the criteria of the EI claimant category “long-tenured workers” introduced under Connecting Canadians with Available Jobs. These individuals are defined as EI regular or fishing benefit claimants who had paid at least 30% of the annual maximum employee EI premiums in seven of the ten years preceding their EI claim and who had collected 35 or fewer weeks of EI regular or fishing benefits in the five years preceding their claim. This definition differs from the definition used in the literature as it does not consider the number of years the worker remained employed with the same employer.

The cost-benefit analysis was limited in the sense that it only took into account the quantifiable benefits and costs that were directly linked to EBSM delivery and participation and that could be estimated using available administrative data and the EI Monitoring and Assessment Report. The analysis did not capture “intangible”, non-pecuniary and indirect benefits. It did not consider the multiplier effect that improving participant’s income may have on the economy and did not account for the effect of EI Part II investment on sustaining a service delivery infrastructure and creating jobs among the governmental program service providers. As well, this analysis did not consider the displacement effect where participants may take away jobs that would otherwise be filled by other unemployed individuals. Finally, this analysis did not consider the possible effect of EBSMs on increasing skill prices.

Appendix B - Detailed results Skills Development

**Table B1. Socio-demographic and labour market characteristics of Skills Development participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	6,833	2,356	3,189	1,568
Gender
Male	48%	42%	39%	30%
Female	52%	58%	61%	70%
Age
Under 25	14%	15%	15%	18%
25 to 34	32%	31%	38%	40%
35 to 44	32%	33%	28%	26%
45 and over	22%	22%	19%	15%
Marital status
Married or common-law	43%	43%	36%	36%
Widow/ divorced or separated	17%	16%	18%	17%
Single	38%	39%	43%	44%
Missing data / Unknown	2%	2%	3%	3%
Skills level related to National Occupation Code associated with the last EI claim opened before SD participation¹
Managerial	4%	6%	4%	4%
University	4%	3%	4%	2%
College or apprenticeship training	26%	25%	24%	21%
Secondary or occupational training	40%	40%	39%	40%
On-the-job training	26%	25%	29%	32%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$21,589	$22,384³	$9,357	$9,445³
Proportion employed	97%	99%	81%	83%
Proportion on Employment Insurance	35%	36%	64%	63%
Proportion on Social Assistance	6%	7%	17%	22%

Proportions may not add up to 100% due to rounding

¹Skill level corresponds to the type and/or amount of training or education typically required to work in the last occupation participants had before opening the last EI claim they had before participating in EBSMs:

-Managerial: Management occupations
-University: Occupations usually requiring university education (that is, University degree at the bachelor's, master's or doctorate level)
-College or apprenticeship training: Occupations usually requiring college or vocational education or apprenticeship training such as 2 to 3 years of post-secondary education at a community college, institute of technology or CEGEP or 2 to 5 years of apprenticeship training or 3 to 4 years of secondary school and more than 2 years of on-the-job training, specialized training courses or specific work experience and/or occupations with supervisory responsibilities and occupations with significant health and safety responsibilities, such as firefighters, police officers and registered nursing assistants.
-Secondary or occupational training: Occupations usually requiring secondary school and/or occupation-specific training such as one to four years of secondary school education or up to 2 years of on-the-job training specialized training courses or specific work experience.
-On-the-job training: On-the-job training is usually provided for occupations (that is, short work demonstration or on-the-job training or no formal educational requirements).

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

Significance level *** 1%; ** 5%; * 10%

SA: Social Assistance

**Table B2. Incremental impacts for Skills Development – active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
All active claimants
2002 to 2005 participants (n=6,883)
Employment earnings ($)	-5,209***	-3,297***	887***	2,204***	3,400***	4,781***	5,680***	16,952***	8,445***
Incidence of employment (percentage points)	-0.1	4.6***	8.3***	9.7***	9.2***	10.2***	10.8***	n/a	n/a
EI benefits ($)	1,540***	-3	-186***	123*	181***	115	234***	467***	2,004***
EI weeks (weeks)	5.6***	-0.1	-0.5**	0.5**	0.7***	0.5***	0.9***	2.2***	7.7***
SA benefits ($)	60**	136***	31	-83**	-83**	-125***	-154***	-414***	-218
Dependence on income support (percentage points)	16.6***	3.7***	-0.3	0.1	0.6	-0.6	0.4	n/a	n/a
2006 to 2008 participants (n=2,356)
Employment earnings ($)	-6,176***	-4,896***	-$121	1,044	2,359**	-	-	3,324	-7,708**
Incidence of employment (percentage points)	2.0*	3.3**	9.3***	10.0***	10.6***	-	-	n/a	n/a
EI benefits ($)	1,718***	-482***	-277*	137	42	-	-	-98	1,138**
EI weeks (weeks)	4.9***	-1.8***	-1.0**	0.2	0.0	-	-	-0.7	2.3
SA benefits ($)	-147***	94*	93	29	-101*	-	-	22	-31
Dependence on income support (percentage points)	14.7***	1.7	-2.4**	0.5	0.5	-	-	n/a	n/a
Sub-groups of active claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=2,022)
Employment earnings ($)	-4,798***	-2,890***	617	1,020	1,605*	2,514***	2,800***	8,556**	868
Incidence of employment (percentage points)	-1	5.2***	7.2***	7.7***	5.9***	6.5***	7.2***	n/a	n/a
EI benefits ($)	1,185***	-230***	-114	170*	215**	46	153	470	1,425***
EI weeks (weeks)	4.4***	-1.0***	-0.4	0.5	0.6*	0.2	0.5*	1.4	4.8***
SA benefits ($)	-14	89**	-13	-64**	-95*	-93	-31	-296	-222
Dependence on income support (percentage points)	12.9***	-1.1	0.3	0.6	1.0	0.3	1.4*	12	n/a
“Long-Tenured workers”- 2007 to 2009 participants (n=832)
Employment earnings ($)	-8,836***	-8,831***	-2,951***	1,136	3,460***	-	-	1,645	-16,022***
Incidence of employment (percentage points)	-6.5***	-7.7***	3.6***	5.5***	5.2***	-	-	n/a	n/a
EI benefits ($)	3,013***	1,875***	78	-182	-127	-	-	-231	4,656***
EI weeks (weeks)	7.1***	4.1***	0.3	-0.4	-0.3	-	-	-0.4	10.8***
SA benefits ($)	72**	151***	113**	92	62	-	-	267*	489***
Dependence on income support (percentage points)	16.7***	11.1***	-0.2	-2.1**	-2.0**	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table B3. Incremental impacts for Skills Development – former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
All former claimants
2002 to 2005 Participants (n=3,189)
Employment earnings ($)	-2,978***	-1,509***	1,382***	1,536***	1,871***	2,114***	2,561***	9,464***	4,978**
Incidence of employment (percentage points)	-9.8***	2.9***	5.2***	5.6***	5.9***	6.1***	6.3***	n/a	n/a
EI benefits ($)	373***	-63	40	186***	100*	137**	78	541***	851***
EI weeks (weeks)	1.3***	-0.4**	0.1	0.7***	0.3	0.4*	0.2	1.6**	2.6***
SA benefits ($)	-502***	-491***	-322***	-306***	-356***	-387***	-371***	-1,743***	-2,736***
Dependence on income support (percentage points)	5.3***	-5.0***	-4.2***	-2.2***	-3.9***	-3.5***	-4.0***	n/a	n/a
2006 to 2008 Participants (n=1,568)
Employment earnings ($)	-4,830***	-4,514***	66	1,987***	2,130***	-	-	4,136***	-5,219**
Incidence of employment (percentage points)	-9.1***	-3.7***	7.2***	9.1***	8.7***	-	-	n/a	n/a
EI benefits ($)	524***	-190***	-318***	162	136	-	-	-20	314
EI weeks (weeks)	1.7***	-0.6**	-1.1***	0.3	0.2	-	-	-0.6	0.6
SA benefits ($)	-470***	-875***	-618***	-524***	-523***	-	-	-1,666***	-3,016***
Dependence on income support (percentage points)	7.7***	-5.4***	-7.5***	-5.7***	-5.2***	-	-	n/a	n/a
Sub-groups of former claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=1,115)
Employment earnings ($)	-2,947***	-1,079***	2,328***	1,200**	816	1,106	1,780**	7,229**	3,203
Incidence of employment (percentage points)	-8.7***	3.9***	5.6***	4.1***	5.1***	5.8***	6.6***	n/a	n/a
EI benefits ($)	407***	-92	62	390***	233**	78	138	901***	1,216***
EI weeks (weeks)	1.6***	-0.5*	0.1	1.4***	0.7**	0.2	0.3	2.7**	3.9***
SA benefits ($)	-484***	-534***	-350***	-304***	-207***	-362***	-251***	-1,475***	-2,493***
Dependence on income support (percentage points)	7.2***	-5.6***	-4.2***	-0.2	-1.2	-3.3***	-3.1***	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table B4. Cost-benefit results from the social perspective for Skills Development**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=6,883)	Former claimants (n=3,189)
Program cost	-$5,081	-$4,182
Marginal social costs of public funds	-$1,056	-$454
Employment earnings (including participant’s forgone earnings)	$9,372	$4,742
Fringe benefit	$1,406	$711
Net present value (By how much do the benefits exceed the costs 6 years after participation?)	$4,640	$818
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.50	$0.80
Payback period (How many years after participation would it take for the benefits to recover the costs?)	5.1 years after participation	5.5 years after participation

Appendix C - Detailed results Skills Development-Apprentices

**Table C1. Socio-demographic and labour market characteristics of Skills Development-Apprentices participants**
	Active claimants		Former claimants
	2003 to 2005	2013 to 2014	2003 to 2005	2013 to 2014
Number of observations	7,555	915	160	146
Gender
Male	94%	95%	89%	89%
Female	6%	5%	10%	10%
Age
Under 25	49%	14%	36%	19%
25 to 34	37%	57%	46%	58%
35 to 44	11%	21%	14%	18%
45 and over	3%	8%	5%	5%
Skills level related to National Occupation Code associated with the last EI claim opened before Skills Development-Apprentices participation¹
Managerial	0%	0%	1%	1%
University	0%	0%	1%	0%
College or apprenticeship training	91%	92%	58%	89%
Secondary or occupational training	5%	2%	16%	3%
On-the-job training	4%	4%	24%	7%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$28,554	$39,761³	$23,468	$38,470³
Proportion employed	100%	100%	98%	98%
Proportion on Employment Insurance	15%	34%	48%	44%
Proportion on Social Assistance	1%	1%	4%	0%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2013 to 2014 participants have been adjusted according to the Consumer Price Index (CPI), published by Statistics Canada, to the 2002 base year.

**Table C2. Labour market outcomes for active claimants who started Skills Development-Apprentices in 2003 to 2005**
Average outcomes	Pre-program period					After the program start year
Average outcomes	5 year pre	4 year pre	3 year pre	2 year pre	1 year pre	Program start year	1 year	2 year	3 year	4year	5 year	6 year	7 year
Earnings including $0	$15,190	$18,590	$22,367	$25,974	$31,105	$30,895	$39,945	$47,597	$55,268	$62,487	$64,277	$66,510	$71,012
Earnings excluding $0¹	$16,518	$19,365	$23,019	$26,393	$31,179	$30,958	$40,351	$48,455	$56,810	$65,169	$67,628	$70,661	$75,507
Proportion employed	92%	96%	97%	98%	100%	100%	99%	98%	97%	96%	95%	94%	94%
Proportion on EI	18%	20%	21%	19%	17%	99%	63%	54%	39%	23%	21%	20%	16%
EI benefits	$666	$722	$750	$712	$625	$3,177	$2,190	$2,005	$1,526	$968	$1,059	$1,058	$930
Number of weeks on EI	2.32	2.38	2.37	2.14	1.79	9.18	5.94	5.1	3.73	2.34	2.63	2.51	2.15
Proportion on SA	3%	2%	2%	2%	1%	0%	1%	1%	1%	1%	1%	1%	1%
SA benefits	$94	$85	$74	$58	$27	$9	$16	$17	$20	$27	$39	$43	$49
Dependence on income support	5%	5%	4%	3%	2%	11%	7%	5%	4%	3%	3%	3%	3%
Proportion self employed	8%	9%	10%	10%	11%	15%	17%	18%	18%	19%	20%	17%	17%

N= 5,970. Data exclude individuals with no Canada Revenue Agency data for the 5 years before participation.

¹Earnings outcomes excluding individuals who reported no earnings in a given year.

SA: Social Assistance

**Table C3. Labour market outcomes for former claimants who started Skills Development-Apprentices in 2003 to 2005**
Average outcomes	Pre-program period					After the program start year
Average outcomes	5 year pre	4 year pre	3 year pre	2 year pre	1 year pre	Program start year	1 year	2 year	3 year	4year	5 year	6 year	7 year
Earnings including $0	$15,804	$19,281	$23,059	$25,033	$24,416	$26,920	$33,744	$42,781	$46,336	$51,764	$52,234	$60,904	$58,193
Earnings excluding $0¹	$16,642	$20,138	$23,388	$25,759	$24,947	$27,310	$34,985	$44,711	$48,794	$56,617	$58,180	$65,360	$63,898
Proportion employed	95%	96%	99%	97%	98%	99%	97%	96%	95%	91%	90%	93%	91%
Proportion on EI	24%	26%	38%	48%	49%	42%	47%	42%	32%	29%	31%	36%	27%
EI benefits	$752	$1,015	$1,378	$1,665	$2,430	$1,361	$1,570	$1,332	$1,261	$1,279	$2,219	$2,139	$1,780
Number of weeks on EI	2.78	3.5	4.85	5.4	7.29	4.05	4.66	4.02	3.32	3.04	5.2	4.92	3.57
Proportion on SA	10%	6%	5%	2%	5%	6%	2%	2%	3%	4%	4%	5%	4%
SA benefits	$337	$219	$153	$85	$185	$181	$94	$40	$130	$121	$256	$261	$344
Dependence on income support	8%	8%	9%	8%	13%	8%	6%	5%	4%	5%	8%	8%	9%
Proportion self employed	9%	15%	16%	15%	18%	22%	19%	24%	25%	21%	23%	19%	24%

N= 143 Data exclude individuals with no Canada Revenue Agency data for the 5 years before participation

¹Earnings outcomes excluding individuals who reported no earnings in a given year.

SA: Social Assistance

Appendix D - Detailed results Targeted Wage Subsidies

**Table D1. Socio-demographic and labour market characteristics of Targeted Wage Subsidies participants**

	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	315	176	553	202
Gender
Male	50%	44%	47%	44%
Female	50%	55%	53%	56%
Age
Under 25	14%	3%	17%	8%
25 to 34	31%	20%	37%	38%
35 to 44	29%	27%	27%	25%
45 and over	25%	50%	18%	29%
Marital status
Married or common-law	44%	59%	41%	46%
Widow/ divorced or separated	17%	14%	15%	10%
Single	38%	24%	41%	41%
Missing data / Unknown	2%	4%	3%	3%
Skills level related to National Occupation Code associated with the last EI claim opened before Targeted Wage Subsidies participation¹
Managerial	8%	14%	3%	8%
University	6%	1%	6%	8%
College or apprenticeship training	26%	22%	31%	27%
Secondary or occupational training	41%	49%	39%	36%
On-the-job training	18%	15%	21%	21%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$22,254	$29,075³	$12,523	$15,635³
Proportion Employed	97%	99%	87%	86%
Proportion on Employment Insurance	38%	19%	55%	58%
Proportion on Social Assistance	6%	3%	12%	10%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table D2. Incremental impacts for Targeted Wage Subsidies – active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=315)
Employment earnings ($)	-1,304*	1,745*	976	554	2,690	3,654**	4,827***	12,700**	13,141*
Incidence of employment (percentage points)	5.1***	9.6***	8.4***	7.4***	5.4***	8.0***	7.4***	n/a	n/a
EI benefits ($)	-444**	-468**	40	376*	235	131	499**	1,281**	369
EI weeks (weeks)	-0.5	-1.2*	1.0*	1.6***	1.5***	0.8	1.9***	6.8***	5.2**
SA benefits ($)	-5	32	21	60	72	84	24	260	287
Dependence on income support (percentage points)	-2.3	-6.9***	0.0	1.5	1.4	0.6	2.2	n/a	n/a

Significance level *** 1%; ** 5%; * 10%

SA: Social Assistance

**Table D3. Incremental impacts for Targeted Wage Subsidies – former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 Participants (n=553)
Employment earnings ($)	1,850***	3,812***	2,114***	1,855**	2,387**	2,057*	2,252*	10,665***	16,326***
Incidence of employment (percentage points)	9.0***	10.7***	5.5***	4.6***	5.0***	1.0	3.5*	n/a	n/a
EI benefits ($)	112	467***	659***	510***	494***	278**	223	2,164***	2,743***
EI weeks (weeks)	0.4	2.1***	2.2***	1.6***	1.3***	0.7*	0.3	6.1***	8.5***
SA benefits ($)	-406***	-556***	-397***	-320***	-337***	-386***	-275**	-1,714***	-2,677***
Dependence on income support (percentage points)	-6.7***	-2.9***	-0.5	-0.9	-1.7	-2.8**	-2.7**	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table D4. Cost-benefit results from the social perspective for Targeted Wage Subsidies**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=315)	Former claimants (n=553)
Program cost	-$7,440	-$6,912
Marginal social costs of public funds	-$1,404	-$905
Employment earnings (including participant’s forgone earnings)	$12,260	$15,995
Fringe benefit	$1,839	$2,399
Net present value (By how much do the benefits exceed the costs 6 years after participation?)	$5,256	$10,578
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.60	$0.40
Payback period (How many years after participation would it take for the benefits to recover the costs?)	4.3 years after participation	0.8 years after participation

Appendix E - Detailed results Self-Employment

**Table E1. Socio-demographic and labour market characteristics of Self-Employment participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	1,390	658	404	194
Gender
Male	55%	48%	46%	35%
Female	45%	52%	54%	65%
Age
Under 25	3%	3%	5%	3%
25 to 34	30%	26%	31%	38%
35 to 44	35%	35%	33%	36%
45 and over	32%	36%	31%	24%
Marital status
Married or common-law	56%	51%	50%	51%
Widow/ divorced or separated	16%	15%	22%	13%
Single	25%	28%	25%	29%
Missing data / Unknown	3%	6%	3%	7%
Skills level related to National Occupation Code associated with the last EI claim opened before Self-Employment participation¹
Managerial	13%	14%	8%	7%
University	15%	13%	16%	15%
College or apprenticeship training	38%	37%	32%	29%
Secondary or occupational training	25%	29%	30%	37%
On-the-job training	9%	7%	14%	12%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$31,358	$31,370³	$12,750	$12,222³
Proportion Employed	98%	100%	78%	85%
Proportion on Employment Insurance	34%	35%	63%	58%
Proportion on Social Assistance	2%	3%	12%	10%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table E2. Incremental impacts for Self-Employment – active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=1,390)
Employment earnings ($)	-9,470***	-13,904***	-10,951***	-10,353***	-10,365***	-9,647***	-8,865***	-50,181***	-73,555***
Incidence of employment (percentage points)	-7.9***	-21.5*	-15.4***	-12.8***	-11.3***	-10.0***	-9.0***	n/a	n/a
EI benefits ($)	2,766***	-159***	-723***	-446***	-259***	-298***	-224***	-1,950***	657**
EI weeks (weeks)	7.6***	-1.2***	-1.8***	-1.0***	-0.2	-0.2	0.1	-3.1***	3.4***
SA benefits ($)	19	20	-13	-21	-61**	-76**	-93***	-263**	-224
Dependence on income support (percentage points)	21.5***	11.7***	-2.7***	-2.1***	-0.8	-1.7***	-1.5**	n/a	n/a
2006 to 2008 Participants (n=658)
Employment earnings ($)	-10,558***	-15,267***	-13,493***	-12,430***	-13,046***	-	-	-38,970***	-64,794***
Incidence of employment (percentage points)	-6.1***	-19.4***	-14.7***	-14.4***	-13.9***	-	-	n/a	n/a
EI benefits ($)	2,457***	-614***	-971***	-658***	-308*	-	-	-1,938***	-94
EI weeks (weeks)	5.8***	-1.9***	-2.4***	-1.6***	-0.8*	-	-	-4.9***	-1.0
SA benefits ($)	13	10	-31	-69	-117*	-	-	-217	-195
Dependence on income support (percentage points)	18.0***	6.2***	-3.3***	-2.0*	-0.3	-	-	n/a	n/a
Sub-groups of active claimants
“Long-Tenured workers”- 2007 to 2009 participants (n=337)
Employment earnings ($)	-10,627***	-17,427***	-14,851***	-14,529***	-12,146***	-	-	-41,525***	-69,579***
Incidence of employment (percentage points)	-5.3***	-22.4***	-15.5***	-14.7***	-8.4***	-	-	n/a	n/a
EI benefits ($)	2,821***	-118	-963***	-604***	-406**	-	-	-1,973***	729
EI weeks (weeks)	6.4***	-0.6	-2.4***	-1.3**	-0.9**	-	-	-4.6***	1.2
SA benefits ($)	32	11	-19	-66	-99**	-	-	-184**	-141
Dependence on income support (percentage points)	16.9***	13.1***	-6.0***	-3.0**	-4.0***	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table E3. Incremental impacts for Self-Employment – former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 Participants (n=403)
Employment earnings ($)	-6,188***	-9,423***	-7,815***	-6,515***	-7,510***	-5,655***	-5,694***	-33,189***	-48,801***
Incidence of employment (percentage points)	-18.7***	-18.3***	-14.4***	-10.9***	-7.9***	-5.1**	-4.2*	n/a	n/a
EI benefits ($)	727***	-394***	-387***	-63	62	0	-185	-573	-241
EI weeks (weeks)	2.7***	-1.6***	-1.4***	-0.5	-0.1	-0.6	-0.8	-3.4*	-2.2
SA benefits ($)	-439***	-349***	-253***	-261***	-243**	-264**	-178	-1,200**	-1,988***
Dependence on income support (percentage points)	7.4***	-5.5***	-5.1***	-3.8***	-0.8	-2.6*	-2.5	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

Appendix F - Detailed results Job Creation Partnerships

**Table F1. Socio-demographic and labour market characteristics of Job Creation Partnerships participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	624	521	1,035	396
Gender
Male	60%	59%	52%	48%
Female	40%	41%	48%	52%
Age
Under 25	12%	10%	13%	11%
25 to 34	28%	28%	29%	29%
35 to 44	33%	31%	31%	29%
45 and over	27%	30%	27%	31%
Marital status
Married or common-law	45%	43%	29%	35%
Widow/ divorced or separated	15%	14%	20%	21%
Single	37%	39%	47%	40%
Missing data / Unknown	2%	4%	3%	4%
Skills level related to National Occupation Code associated with the last EI claim opened before Job Creation Partnerships participation¹
Managerial	4%	3%	4%	4%
University	3%	3%	4%	2%
College or apprenticeship training	23%	26%	22%	23%
Secondary or occupational training	38%	40%	38%	41%
On-the-job training	32%	27%	33%	30%
Key Labour Market Indicators in the year preceding the start of participation
Earnings²	$19,084	$23,071³	$8,712	$11,533³
Proportion Employed	97%	99%	81%	86%
Proportion on Employment Insurance	41%	34%	61%	52%
Proportion on Social Assistance	10%	6%	28%	21%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table F2. Incremental impacts for Job Creation Partnerships – active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=624)
Employment earnings ($)	-6,017***	-1,958**	2,429**	2,126	3,584**	4,424***	4,720**	17,283**	9,308
Incidence of employment (percentage points)	n/a	n/a	10.5***	8.4***	12.6***	10.5***	12.1***	n/a	n/a
EI benefits ($)	1,533***	-348**	165	440***	685***	760***	556***	2,605***	3,790***
EI weeks (weeks)	6.0***	-1.1**	0.7***	1.5***	2.0***	2.4***	1.9***	8.5***	13.5***
SA benefits ($)	252***	113	-46	-109	-162	-173	-11	-501	-137
Dependence on income support (percentage points)	16.9***	-1.0	-0.5	-0.8	-0.3	1.2	1.8	n/a	n/a
2006 to 2008 Participants (n=521)
Employment earnings ($)	-6,460***	9	144	-950	-1,144	-	-	-1,950	-8,402**
Incidence of employment (percentage points)	n/a	n/a	7.0***	9.8***	3.7	-	-	n/a	n/a
EI benefits ($)	2,046***	-438**	314	616***	503**	-	-	1,433***	3,040***
EI weeks (weeks)	6.1***	-1.3**	0.9	1.6***	1.4**	-	-	4.0***	8.8***
SA benefits ($)	-11	46	48	57	14	-	-	119	154
Dependence on income support (percentage points)	12.3***	-3.7**	1.6	0.7	2.7	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table F3. Incremental impacts for Job Creation Partnerships – former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 Participants (n=1,035)
Employment earnings ($)	-2,413***	265	1,159**	727	1,064	872	1,009	4,832	2,683
Incidence of employment (percentage points)	n/a	n/a	5.6***	3.8***	5.0***	5.9***	5.8***	n/a	n/a
EI benefits ($)	187**	-87	111	163*	167	105	155	700**	801**
EI weeks (weeks)	0.4	-0.6**	0.1	0.3	0.3	0.2	0.2	1.0	0.9
SA benefits ($)	260***	146	-158*	-243**	-232**	-198*	-26	-858*	-452
Dependence on income support (percentage points)	5.6***	-0.5	-2.6**	-2.0*	-1.9*	-1.7	-1.1	n/a	n/a
2006 to 2008 Participants (n=396)
Employment earnings ($)	-2,732***	1,041	1,240	454	1,727	-	-	3,431	1,747
Incidence of employment (percentage points)	n/a	n/a	8.9***	6.1**	6.9***	-	-	n/a	n/a
EI benefits ($)	146	-1	256	655***	83	-	-	994**	1,139**
EI weeks (weeks)	0.8*	0.3	1.3**	2.0***	0.3	-	-	3.6***	4.7***
SA benefits ($)	-113	-290	-327	-97	-92	-	-	-516	-917
Dependence on income support (percentage points)	1.3	-4.6***	-2.4	1.7	0.4	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table F4. Cost-benefit results from the social perspective for Job Creation Partnerships**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=624)	Former claimants (n=1,035)
Program cost	-$8,790	-$6,636
Marginal social costs of public funds	-$2,272	-$1,270
Employment earnings (including participant’s forgone earnings)	$8,367	$2,445
Fringe benefit	$1,255	$367
Net present value (By how much do the benefits exceed the costs within 6 years after participation?)	-$1,440	-$5,094
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$1.20	$4.30
Payback period (How many years after participation would it take for the benefits to recover the costs?)	6.5 years after participation	14.1 years after participation

Appendix G - Detailed results Employment Assistance Services only

**Table G1. Socio-demographic and labour market characteristics of Employment Assistance Services participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	38,577	20,997	34,518	26,557
Gender
Male	54%	49%	53%	47%
Female	46%	51%	47%	53%
Age
Under 25	11%	11%	13%	11%
25 to 34	28%	25%	31%	31%
35 to 44	30%	28%	30%	28%
45 and over	30%	35%	25%	30%
Marital status
Married or common-law	41%	36%	31%	27%
Widow/ divorced or separated	17%	18%	19%	20%
Single	39%	41%	46%	46%
Missing data / Unknown	3%	5%	5%	7%
Skills level related to National Occupation Code associated with the last EI claim opened before Employment Assistance Services participation1
Managerial	6%	6%	4%	4%
University	7%	5%	5%	4%
College or apprenticeship training	29%	29%	26%	27%
Secondary or occupational training	37%	37%	37%	37%
On-the-job training	21%	23%	29%	28%
Key Labour Market Indicators in the year preceding the start of participation
Earnings2	$24,638	$24,3213	$12,863	$15,2293
Proportion Employed	97%	99%	85%	89%
Proportion on Employment Insurance	37%	33%	52%	48%
Proportion on Social Assistance	7%	10%	21%	20%

Proportions may not add up to 100% due to rounding

²Average earnings for all individuals included in the study. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table G2. Incremental impacts for Employment Assistance Services**
Indicators	In-program period	Post-program period						Total in- and post-program
Indicators	In-program period	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=23,119 or a random sample of 60% of participants)
Employment earnings ($)	-1,831***	-189	431	554*	627	1,062***	2,485*	654
Incidence of employment (percentage points)	1.1***	3.0***	2.3***	2.2***	1.5***	2.0***	n/a	n/a
EI benefits ($)	70	-417***	-124***	-109**	7	25	-618***	-548***
EI weeks (weeks)	0.2	-1.3***	-0.3**	-0.3**	0.0	0.1	-1.8***	-1.6***
SA benefits ($)	141***	150***	84***	63***	43**	95***	435***	576***
Dependence on income support (percentage points)	1.6***	-1.9***	0.2	0.2	0.6*	0.8**	n/a	n/a
2006 to 2008 Participants (n=20,997)
Employment earnings ($)	-2,040***	-969***	-627**	-602**	-	-	-2,198***	-4,238***
Incidence of employment (percentage points)	1.3***	1.7***	1.6***	0.8*	-	-	n/a	n/a
EI benefits ($)	-110**	-530***	-253***	-164***	-	-	-946***	-1,056***
EI weeks (weeks)	-0.2	-1.5***	-0.6***	-0.4***	-	-	-2.5***	-2.7***
SA benefits ($)	242***	494***	472***	368***	-	-	1,334***	1,576***
Dependence on income support (percentage points)	1.6***	0.3	2.6***	2.7***	-	-	n/a	n/a
Sub-groups of active claimants
Youth (below 30 years old) – 2002 to 2005 participants (n=9,591)
Employment earnings ($)	-1,347***	-481*	-493	-161	267	85	-784	-2,131
Incidence of employment (percentage points)	0.6*	3.2***	3.5***	3.7***	3.6***	2.4***	n/a	n/a
EI benefits ($)	-135***	-483***	-8	-67	105**	121**	-332**	-468***
EI weeks (weeks)	-0.4*	-1.6***	0.1	-0.2	0.3**	0.4**	-1**	-1.4**
SA benefits ($)	151***	190***	107***	54**	14	70***	435***	585***
Dependence on income support (percentage points)	0.3	-2.7***	1.6***	0.7*	0.8**	1.5***	n/a	n/a
Older workers (55 years old and over) – 2002 to 2005 participants (n=3,035)
Employment earnings ($)	-1,465***	1,330***	2,549***	3,183***	3,749***	4,283***	15,094***	13,629***
Incidence of employment (percentage points)	2.4***	8.1***	8.5***	7.8***	9***	9.6***	n/a	n/a
EI benefits ($)	186*	-298***	-211**	-5	119	147	-248	-62
EI weeks (weeks)	0.7*	-0.8**	-0.5	0.1	0.3	0.4	-0.6	0.1
SA benefits ($)	26	68**	54*	56	66*	82**	325**	351**
Dependence on income support (percentage points)	0.8	-3.7***	-1.6*	-1.6*	-1.9*	-2.6**	n/a	n/a
“Long-Tenured workers”- 2007 to 2009 participants (n=9,429)
Employment earnings ($)	-2,671***	-375	504	793*	-	-	921	-1,749
Incidence of employment (percentage points)	0.3	2.3***	1.2**	1.2**	-	-	n/a	n/a
EI benefits ($)	391***	-411***	-423***	-251***	-	-	-1,086***	-695***
EI weeks (weeks)	0.8***	-1.1***	-1.0***	-0.6***	-	-	-2.7***	-1.9***
SA benefits ($)	234***	335***	257***	204***	-	-	796***	1,030***
Dependence on income support (percentage points)	3.1***	-0.6	0.1	0.3	-	-	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table G3. Incremental impacts related to the timing of participation in Employment Assistance Services**
Cohorts (start of EAS-only after start of an EI claim)	n=	In-program	Post-program period					Total impact post-program	Total impact in- and post- program
Cohorts (start of EAS-only after start of an EI claim)	n=	In-program	1 year	2 years	3 years	4 years	5 years	Total impact post-program	Total impact in- and post- program
Employment earnings ($)
1 to 4 weeks	13,319	152	1,024***	2,729***	3,561***	4,366***	4,484***	16,164***	16,316***
5 to 8 weeks	4,714	-1,487***	815***	2,222***	2,732***	3,412***	3,238***	12,418***	10,931***
9 to 12 weeks	3,721	-2,754***	288	1,871***	2,729***	3,443***	3,279***	11,609***	8,855***
2nd quarter	6,549	-4,522***	-311	562*	1,466***	1,703***	1,979***	5,399***	877
3rd quarter	3,236	-6,536***	663*	1,154***	1,805***	2,277***	2,442***	8,342***	1,806
4th quarter	2,151	-5,586***	825*	-194	87	899	852	2,469	-3,117
Incidence of employment (percentage points)
1 to 4 weeks	13,319	2.6***	2.2***	1.8***	1.3***	0.7	-0.4	n/a	n/a
5 to 8 weeks	4,714	2.1***	2.1***	2.2***	1.1*	0.2	-0.6	n/a	n/a
9 to 12 weeks	3,721	0.7	1.0*	1.0	0.8	0.0	-0.8	n/a	n/a
2nd quarter	6,549	-1.4***	0.9*	0.2	0.3	-0.5	-0.7	n/a	n/a
3rd quarter	3,236	-3.9***	1.4**	1.6**	1.4*	-0.3	-0.3	n/a	n/a
4th quarter	2,151	-6.3***	1.6***	0.9	0.9	1.1	0.4	n/a	n/a
EI Benefits ($)
1 to 4 weeks	13,319	-20	-64*	-154***	-93***	6	52	-253**	-274**
5 to 8 weeks	4,714	943***	-229***	-138***	-65	17	85*	-330**	613***
9 to 12 weeks	3,721	1,251***	-489***	-244***	-134***	-48	123**	-791***	459**
2nd quarter	6,549	1,669***	-807***	-255***	-123***	-30	50	-1,164***	504***
3rd quarter	3,236	1,803***	-1,646***	-185***	-93	-12	81	-1,855***	-51
4th quarter	2,151	639***	-2,040***	-229***	-249***	-133	-55	-2,706***	-2,067***

* Significant at 10%; ** significant at 5%; *** significant at 1%

**Table G4. Incremental time of return to employment for Employment Assistance Services participants based on timing of participation**
Cohorts (start of EAS-only after start of an EI claim)	1 to 4 weeks	5 to 8 weeks	9 to 12 week	2nd quarter	3rd quarter	4th quarter
Cohorts (start of EAS-only after start of an EI claim)	(N=13,319)	(N=4,714)	(N=3,721)	(N=6,549)	(N=3,236)	(N=2,151)
Time of Return to Employment (in weeks)	4.1***	1.0***	-0.4**	-1.7***	-1.4***	-0.1

* Significant at 10%; ** significant at 5%; *** significant at 1%

**Table G5. Cost-benefit results from the social perspective for Employment Assistance Services**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=23,119)
Program cost	-$252
Marginal social costs of public funds	-$183
Employment earnings (including participant’s forgone earnings)	$1,200
Fringe benefit	$180
Net present value (By how much do the benefits exceed the costs within 6 years after participation?)	$944
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.20
Payback period (How many years after participation would it take for the benefits to recover the costs?)	5.2 years after participation

Appendix H. List of nine studies included in the synthesis report

**Table H1. Overview of studies included in this summary report**
Study	Evidence included in this summary report	Methods	Reference period	Observation period
Analysis of EBSM Profile, Outcomes and Medium-Term Incremental Impacts for 2002 to 2005 Participants in Alberta (Completed in 2014)	- Incremental impacts for participants including youth and older workers - Profile and socio-demographic characteristics of participants	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Statistical profiling	2002 to 2005 participants	7 years between 2002 and 2011 (that is, 2 years in program and 5 years post-program)
Effects of the Timing of Participation in Employment Assistance Services in Alberta (Completed in 2014)	- Incremental impacts	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Statistical profiling
Cost-Benefit Analysis of Employment Benefits and Support Measures (Completed in 2016)	- Cost-benefit analysis	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Cost analysis		8 years between 2002 and 2013 (that is, 2 years in-program and 6 years post-program)
Analysis of EBSMs Profile, Outcomes and Incremental Impacts for 2006 to 2008 Participants in Alberta (Completed in 2015)	- Incremental impacts - Profile and socio-demographic characteristics of participants	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Statistical profiling	2006 to 2008 participants	5 years between 2006 and 2012 (that is, 2 years in-program and 3 years post-program)
Analysis of EBSMs Profile, Outcomes, and Incremental Impacts for EI Claimants Category “Long-Tenured Workers” in Alberta (Completed in 2016)	- Incremental impacts - Statistical profile of socio-demographic characteristics	- Propensity score matching and Difference-in-Differences - Statistical profiling	2007 to 2009 participants	5 years between 2007 and 2013 (that is, 2 years in-program and 3 years post-program)
Study on Employment Assistance Services (Completed in 2014)	- Program design and delivery - Challenges and lessons learned	- 8 key informants interviews - Literature and document review	Design and delivery at the time of the data collection (that is., 2013)
Study on Workplace Training in Alberta (Completed in 2016)	- Program design and delivery - Challenges and lessons learned	- 3 key informants interviews - Literature and document review -Questionnaire filled by Alberta officials s	Design and delivery at the time of the data collection (that is, 2015)
Study on Study on Work Foundations / Occupational Training in Alberta (Completed in 2016)	- Program design and delivery - Challenges and lessons learned	- 5 key informants interviews - Literature and document review - Questionnaire filled by Alberta officials
Study of Apprenticeship Technical Training Component of the Occupational Training Program in Alberta (Completed in 2016)	- Program design and delivery - Challenges and lessons learned	- 4 key informants interviews - Literature and document review - Questionnaire filled by Alberta officials

Footnotes

Footnote 1

Long-tenured workers covered in the evaluation are individuals who had long-term attachment to the labour market but not necessarily a long tenure with the same employer.

Return to footnote 1 referrer

Footnote 2

The estimates of Figure i represent an arithmetic average of the annual incidence of employment estimates reported in the annexes.

Return to footnote 2 referrer

Footnote 3

The incremental impacts on earnings are exclusively derived from Canada Revenue Agency’s income tax records.

Return to footnote 3 referrer

Footnote 4

When interpreting qualitative findings, readers should keep in mind that those are based on the perception of a small number of key informants (ranging between 3 and 8) who are directly involved in the design or delivery of the program. Their perception may be representative of their own region or community but not necessarily of the entire province. Since the number of key informants interviewed in each study is small, the number of informants who reported a specific finding is not indicated in the report.

Return to footnote 4 referrer

Footnote 5

Former claimants who received maternity or parental benefits were not covered by the evaluation given the difficulty in finding a suitable comparison group.

Return to footnote 5 referrer

Footnote 6

Full details about the incremental impact methodology can be found in the following report: Stream 1 study for 2013-2014: national level analysis of EBSM incremental impacts. Methodology Report, Evaluation Directorate, ESDC. September 16, 2013.

Return to footnote 6 referrer

Footnote 7

Incremental impacts on earnings are estimated relative to pre-participation levels and to the comparison group. They are estimated using current dollars.

Return to footnote 7 referrer

Footnote 8

Return to footnote 8 referrer

Footnote 9

Alberta Community and Social Services (formerly Human Services). Transitioning Supports for Apprenticeship-related Programs from Human Services to Innovation and Advanced Education. July 22, 2015. Questions and answers Accessed September 23, 2016

Return to footnote 9 referrer

Footnote 10

Return to footnote 10 referrer

Footnote 11

Red Seal. 2014. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal.

Return to footnote 11 referrer

Footnote 12

Patrick Coe. 2013. “Apprenticeship programme requirements and apprenticeship completion rates in Canada.” Journal of Vocational Education and Training. 65(4): 575−605.

Return to footnote 12 referrer

Footnote 13

Employment & Training Programs and Services Accessed February 29, 2016

Return to footnote 13 referrer

Footnote 14

Return to footnote 14 referrer

Footnote 15

Sébastien LaRochelle-Côté and Sharanjit Uppal, "The Financial Well-Being of the Self-Employed," Perspectives on Labour and Income, vol. 23, no. 4, Winter 2011.

Return to footnote 15 referrer

Footnote 16

Return to footnote 16 referrer

Footnote 17

The estimates of Figure 9 represent an arithmetic average of the annual incidence of employment estimates reported in the annexes.

Return to footnote 17 referrer

Footnote 18

Return to footnote 18 referrer

Page details

Date modified:: 2022-04-14

Evaluation of the Canada – Alberta Labour Market Development Agreement

On this page

Alternate formats

Executive summary

Introduction

Key findings

Effectiveness and efficiency of EBSMs

Lessons learned about program design and delivery

Skills Development

Skills Development – for Apprentices

Targeted Wage Subsidies

Employment Assistance Services

Recommendations

Management response

Introduction

Alberta management response

1. Introduction

1.1 Labour Market Development Agreement background

Employment Benefits

Support Measures

1.2 Methodology

Incremental impacts analysis

Factors accounted for in the cost-benefit analysis

Strengths and limitations of the studies

1.3 Overview of the studies summarized in this report

2. Evaluation findings

2.1 Rationale and labour market context

LMDA investments align with provincial government priorities

2.2 Skills Development

2.2.1 Program description

2.2.2 Program delivery

2.2.3 Targeting to labour market demand

2.2.4 Profile of Skills Development participants

Socio-demographic characteristics

Barriers faced by participants

2.2.5 Incremental impacts

Active claimants

Former claimants

2.2.6 Cost-benefit results

2.2.7 Challenges and lessons learned about Skills Development design and delivery

2.3 Skills Development-Apprentices

2.3.1 Program description

2.3.2 Profile of Skills Development-Apprentices participants

2.3.3 Labour market outcomes

Active claimants

Former claimants

2.3.4 Challenges and lessons learned about Skills Development-Apprentices design and delivery

2.4 Targeted Wage Subsidies

2.4.1 Program description

2.4.2 Program delivery

2.4.3 Profile of Targeted Wage Subsidies participants

2.4.4 Incremental impacts

Active claimants

2.4.5 Cost-benefit results

2.4.6 Challenges and lessons learned about Targeted Wage Subsidies design and delivery

2.5 Self-Employment

2.5.1 Program description

2.5.2 Profile of Self-Employment participants

2.5.3 Incremental impacts

2.6 Job Creation Partnerships

2.6.1 Program description

2.6.2 Program delivery

2.6.3 Profile of Job Creation Partnerships participants

2.6.4 Incremental impacts

Active claimants

Former claimants

2.6.5 Cost-benefit results

2.7 Employment Assistance Services

2.7.1 Program description and delivery

2.7.2 Profile of Employment Assistance services participants

Socio-demographic characteristics

Labour market barriers faced by Employment Assistance Services participants

2.7.3 Incremental impacts

2.7.4 Cost-benefit results

2.7.5 Challenges and lessons learned about Employment Assistance Services design and delivery

3. Comparison of key findings by program type

4. Conclusions

Skills Development

Skills Development – Apprentices

Targeted Wage Subsidies