Evaluation of the Canada – Manitoba Labour Market Development Agreement

From: Employment and Social Development Canada

3. Comparison of key findings by program type
4. Conclusions
5. Recommendations
References
Acronyms
Appendix A - Detailed Results Skills Development
Appendix B - Detailed Results Skills Development - Apprenticeship
Appendix C - Detailed Results Wage Subsidies
Appendix D – Detailed Results Self-Employment
Appendix E - Detailed Results Employment Partnerships
Appendix F - Detailed Results Employment Assistance Services
Appendix G - List of Nine Studies Included in the Synthesis Report

Alternate formats

Evaluation of the Canada – Manitoba Labour Market Development Agreement Synthesis Report [PDF - 1.15 KB]

Large print, braille, MP3 (audio), e-text and DAISY formats are available on demand by ordering online or calling 1 800 O-Canada (1-800-622-6232). If you use a teletypewriter (TTY), call 1-800-926-9105.

Executive summary

Introduction

Employment and Social Development Canada (ESDC) worked jointly with Manitoba and eleven other Provinces and Territories (P/Ts) between 2012 and 2017 to undertake the second cycle for the evaluation of the Labour Market Development Agreements (LMDAs). LMDAs are bilateral agreements between Canada and each P/T that are established under Part II of the 1996 Employment Insurance (EI) Act. Under the LMDAs, Canada transfers $2.14B in EI funds to P/Ts for the design and delivery of programs and services to help unemployed individuals (mainly those eligible under EI) to find and maintain employment.

Programs and services delivered by Manitoba correspond to the Employment Benefits and Support Measures (EBSMs) categories defined under the EI Act. The five categories of programs and services examined in the evaluation are as follows:

Skills Development (including Apprenticeship) helps participants obtain employment skills by giving them financial assistance in order to attend classroom training.
Targeted Wage Subsidies (wage subsidies) help participants obtain on-the-job work experience by providing employers with a wage subsidy.
Self-Employment provides financial assistance and business planning advice to participants to help them start their own business.
Job Creation Partnerships (Employment Partnerships) provide participants with opportunities to gain work experience that will lead to ongoing employment. Employment opportunities are provided by projects that contribute to developing the community and the local economy.
Employment Assistance Services such as counselling, job search skills, job placement services, the provision of labour market information and case management.

Table i provides an overview of the share of funding allocated to the five EBSMs examined under the evaluation and the average cost per intervention.

**Table i. Share of LMDA funding and average cost per intervention in Manitoba**
Program and Service	Share of funding 2014 to 2015	Average cost per intervention 2002 to 2005
Skills Development (including Apprenticeship)	59%	$6,203
Employment Assistance Services	18%	$221
Wage Subsidies	1%	$4,368
Self-Employment	2%	$10,603
Employment Partnerships	2%	$6,382
Total	82%*	–

Sources: EI Monitoring and Assessment Reports 2002 to 2003 to 2014 to 2015.

* Note: Labour Market Partnerships and Research and Innovation represented about 18% of EBSM expenditures in 2014 to 2015.

The evaluation work consisted of conducting two to three studies per year on the EBSM similar programming delivered under the LMDAs for Canada overall and for each participating P/T. This report presents a summary of the findings from nine studies produced for Manitoba, which used a mix of qualitative and quantitative methods to examine issues related to program effectiveness, efficiency and design/delivery. Evaluation methods included incremental impact analysis (using a comparison group), cost-benefit analysis, outcome analysis, key informant interviews as well as document review (for example, literature reviews and written questionnaires completed by Manitoba representatives). The unit of analysis adopted in the evaluation is the Action Plan Equivalent which combines all EBSMs given to an individual within no more than six months of each other.

Results are presented by type of EBSM for active and former EI claimants as well as for long-tenured workers^{Footnote 1}, youth (under 30 years old) and older workers (55 years old and over) when the number of participants was sufficient to conduct quantitative analyses. Active EI claimants are classified as those who were receiving EI benefits at the time of their EBSM participation. Former EI claimants received EI up to three years before starting their EBSM participation.

Key findings on the effectiveness and efficiency of EBSMs

The evaluation evidence presented in this report demonstrates that programs and services delivered by Manitoba under the LMDA are generally improving participants’ labour market attachment following participation and contributing to helping Manitobans access the education and training they need to prepare them for their future economic success.

As well, social benefits of participation exceeded the cost of investments for all examined interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first four weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Detailed quantitative findings

Figure i presents the incremental impacts on the incidence of employment for active and former claimants by type of program. The estimates can be interpreted as a change in the probability of being employed following participation. For example, participation in Skills Development increases the probability of being employed by 6.4 percentage points for active EI claimants relative to unemployed non- participants.

**Figure i. Change in probability of being employed in participants relative to non-participants**

**Figure i. Change in probability of being employed in participants relative to non-participants**
	Percentage Points
Active Claimants	Former Claimants
Skills Development	6.4	4.9
Wage Subsidies	6.8	6.3
Employment Partnerships	3.9	4.8
Employment Assistance Services	1	N/A

* The estimates in Figure i are arithmetic estimates. The estimate for active claimants who participated in Employment Partnerships is based on five estimates with different statistical significance levels: one estimate is statistically significant at the 95% confidence level, three estimates are statistically significant at the 90% confidence level and one estimate is not statistically significant.

Figure ii presents the cumulative increase in employment earnings for active and former claimants over the 5 years post-participation. It is noted that Employment Assistance Services are relatively modest activities and, by themselves, are not expected to lead to substantial effects on labour market outcomes. In other words, these services aim to support the return to work of unemployed participants and not necessarily to secure a better paying job than pre-participation. However, as demonstrated later in the report, providing Employment Assistance Services earlier during the EI claim (first 4 weeks) produced larger impacts on earnings and employment and facilitated earlier returns to work.

**Figure ii. Increased cumulative earnings of participants relative to non-participants**

**Figure ii. Increased cumulative earnings of participants relative to non-participants**
	Cumulative gains in earnings
Active Claimants	Former Claimants
Skills Development	$ 23,827	$ 9,586
Wage Subsidies	$ 21,972	$ 11,355
Employment Partnerships	$ 19,211	$ 13,961
Employment Assistance Services	-$ 556	N/A

* The estimate is not statistically significant at the 95% confidence level.

Table ii presents the number of years required for the social benefits to exceed program cost. Social benefits to participation exceeded investment costs in a period ranging from less than a year after program participation to about 11 years after participation for all interventions.

**Table ii. Number of years for the benefits to exceed program costs**
	Skills Development	Wage Subsidies	Employment Partnerships	Employment Assistance Services
Active claimants	4.4	1.8	3.9	11.4
Former claimants	7.5	0.1	1.2	n/a

Challenges and lessons learned about program design and delivery

Key informants interviews with service providers and program managers, as well as the documents reviewed and the questionnaires filled by Manitoba representatives, revealed specific challenges and lessons learned about program design and delivery:

Skills Development

Skills Development funding is directed toward training for occupations in demand in the labour market. This is mainly done through the application process where participants demonstrate that sufficient demand exists in their chosen occupation. Certain high-demand occupations are given priority and career development counsellors may turn down applications for occupations that are not in-demand.

According to service providers and managers interviewed as part of the evaluation, the main challenges related to Skills Development include:

Lack of awareness among potential participants, including new EI claimants
More in depth and up front skills assessment to identify barriers and required supports
Improve local labour market information to inform training and investment decisions
Capacity issues associated with large caseloads and serving individuals facing personal or mental health issues
The insufficiency of living allowance for Manitobans in Northern or remote areas
EI eligibility criteria limit access for those who quit or have breaks in work history

Skills Development - Apprenticeship

Canadian literature shows a fairly high non-completion rate among apprentices (40-50%) and that despite the growth in apprenticeships registrations in Canada, there has not been a corresponding increase in completion rates. Using available data, the evaluation could not generate a reliable estimation for completion rates among apprentices in Manitoba. Key informants identified the following factors:

Availability of employers within an apprentice’s community and willingness of employers to let the apprentices go on training
Delay in the first EI benefits payment and financial pressures during training. Apprentices may interrupt their training to take other employment
Time out of school may result in skill atrophy for some apprentices
Importance of assessment to identify good fit for the training or trade

Wage Subsidies

The number of participants in the program has been falling in recent years from 505 in 2003 to 2004 to 53 in 2014 to 2015 despite its demonstrated effectiveness. Key informants identified the following potential reasons to explain this decline:

The prioritization of funding and staff roles to support other special projects and programs like the Canada-Manitoba Job Grant
Lack of awareness among employers and perceived administrative burden
A decrease in awareness of the program and in the number of people seeking EI benefits
Increased targeting of subsidies based on suitability and best fit with the employer

Employment Assistance Services

Challenges with the design and delivery of Employment Assistance Services include:

The lack of labour market information that is readily available at the local level and in northern and remote areas of the province
Funding levels have not increased yet the cost of serving participants has
Developing the skill and knowledge of service providers who receive a limited allowance for training and upgrading
One-year contracts with service providers can create a level of uncertainty in job security and impede organizations’ ability to retain staff

Recommendations

A total of seven recommendations emerge from the evaluation findings. They are as follows:

The study on the timing of Employment Assistance Services participation showed that receiving assistance early after starting an EI claim leads to better labour market impacts. However, key informants repeatedly reported a lack of awareness about the program.

Recommendation 1: Consideration should be given to providing Manitoba with timely access to data on new EI recipients for supporting targeted outreach and, if necessary, increasing awareness.

Key informants spoke about the importance of providing more in depth and up front skills assessment to identify barriers and required supports. It is recognized that the lack of essential skills, learning disabilities and literacy issues are common barriers to accessing and completing training.

Recommendation 2: Consideration should be given to remove barriers to completing training such as literacy/essential skills training and learning disability assessments. These measures would help individuals with multiple barriers to prepare for vocational training and reintegrating into the labour market. These measures should be reported separately from other Skills Development interventions given their unique objectives.

Incremental impact results show that the Wage Subsidies program is improving the earnings and employment of participants. However, program use has declined over the years. According to key informants, the decline may be related to factors such as a lack of awareness about the program and the administrative burden on employers.

Recommendation 3: Depending on labour market priorities and budget allocation across the various EBSMs, Manitoba may wish to explore opportunities to increase the use of the Wage Subsidies program.

Key informants interviewed in the evaluation confirmed the necessity of having labour market information to support the delivery of Employment Assistance Services and other programs such as Skills Development. They, however, pointed to the potential difficulty of accessing and using labour market information at the regional and local levels.

Recommendation 4: Consideration should be given about enhancing the capacity of service providers to access and produce, when needed, relevant labour market information.

The evaluation was not able to produce a conclusive assessment of Self-Employment effectiveness and efficiency since the data used to assess impacts on earnings may not be the best source of information available to reflect the financial wellbeing of the participants. Overall, it is not clear whether participant’s success in improving their labour market attachment through self-employment is more closely associated with their business idea and their entrepreneurship skills or the assistance provided under Self-Employment.

Recommendation 5: Consideration should be given to examine in more detail the design and delivery of Self-Employment and whether the performance indicators for this program are appropriate.

Employment Partnerships were found to be effective at improving earnings and incidence of employment of active claimants. However, the evaluation has not yet examined the design and delivery of this program. Therefore, a lot remains unknown about how this program operates and the factors that contribute to its effectiveness.

Recommendation 6: Consideration should be given to examine the design and delivery of Employment Partnerships in order to better understand how this program operates.\

Overall, the LMDA evaluation was able to produce a sound assessment of EBSM effectiveness and efficiency because the team had access to rich data on EI claimants, EBSM participation data and Canada Revenue Agency taxation files. However, some data gaps limited the evaluation’s ability to assess how EBSMs operate.

Recommendation 7: Improvements in the data collection process are recommended to address key program and policy questions of interest to the federal and provincial/territorial governments. Specifically:

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. Manitoba, ESDC, and other provinces and territories should work together to define common categories for both EBSMs.

Management Response

Introduction

The purpose of the evaluation of the Canada-Manitoba LMDA was to examine the incremental impacts of participation in Manitoba’s Provincial Benefits and Provincial Measures. The evaluation was jointly undertaken with Manitoba Education and Training, Employment and ESDC and eleven other P/Ts between 2012 and 2017.

Background

LMDAs are bilateral agreements between Canada and P/Ts that are established under Part II of the 1996 EI Act. The main objective of EI Part II is to maintain a sustainable EI system by getting clients back to work quickly. Under the LMDAs, Canada transfers $2.14B in EI funds ($1.95 billion in program funding and $190 million in administrative funding) to P/Ts for the design and management of programs targeting unemployed Canadians.

Manitoba has delivered the Provincial Benefits and Provincial Measures since 1997 when responsibility for Employment Insurance Part II programming was devolved to the provinces and territories. The LMDA assists approximately 30,000 unemployed Manitobans annually to prepare for, find, and/or obtain employment. This evaluation provides strong evidence of the positive impacts of this programming for individuals and Manitoba’s labour market. Manitoba would like to acknowledge the contributions of ESDC, provinces and territories, and program staff who participated in the conduct and oversight of this evaluation including those who participated in key informant interviews and questionnaires.

Response from Manitoba

The evaluation demonstrates that Manitoba’s programs and services are improving the labour market attachment of participants and their labour market success. The evaluation also identified areas for further investigation in the seven recommendations emerging from the evaluation findings. Manitoba has the following observations and actions in relation to the seven recommendations.

Recommendation 1: Consideration should be given to providing Manitoba with timely access to data on new EI recipients for supporting targeted outreach and, if necessary, increasing awareness.

Response: The evaluation found that receiving Employment Assistance Services during the first four weeks of an individual’s EI claim had greater impacts on employment and earnings and earlier returns to work. This evidence demonstrates the importance of providing services to EI claimants as early as possible in their EI claim.

Manitoba recognizes the importance of reaching out to EI claimants as soon as possible following their loss of employment. Manitoba is committed to working in partnership with ESDC to increase the awareness of EI claimants to opportunities to develop their skills and successfully transition back into employment. Manitoba also recognizes the important role its network of Employment Assistance Service providers plays in assisting Manitobans to prepare for new labour market opportunities. Manitoba recently undertook a comprehensive review of its Employment Assistance Services program; the results of which are informing the modernization of services to better respond to client and labour market needs.

Response: Ensuring individuals are successful in their employment and training plans is a key priority for Manitoba. Manitoba has observed an increase in individuals seeking assistance who are facing complex barriers to employment that require new, innovative, intensive and longer-term interventions. Manitoba has developed a new Employment and Training Assessment tool to support employment plans that respond to individual’s specific needs and barriers. Future iterations of this tool will incorporate a distance to the labour market lens to assist individuals as they progress along the pathway to sustainable employment. Manitoba recognizes the importance of literacy and essential skills as well as learning disability assessments as foundational for success in education, training and the workplace and will be a key focus for the province in its programming.

Response: Manitoba recognizes wage subsidies as a useful tool for connecting individuals to work opportunities and assisting employers to access new workers who might not otherwise be considered for a job. The evaluation points to the decline of the use in the Wage Subsidies program over time and, through qualitative studies, identified possible contributing factors. Manitoba is interested in further examination of the factors relating to the decline of Wage Subsidies and exploring opportunities to expand its use including work-integrated learning, to assist individuals along the pathway to employment.

Recommendation 4: Consideration should be given about enhancing the capacity of service providers to access and produce, when needed, relevant labour market information.

Response: During consultations in 2016, Manitoba heard from stakeholders from across sectors and regions of the province on the importance of labour market information to improving the relevance, responsiveness, and effectiveness of employment and training programming. Advancements are being made provincially and on a pan-Canadian level to strengthen the production, dissemination, and use of labour market information by government, education and training providers, service providers, employers, and individuals. Manitoba will continue to explore new labour market information products and partnerships that support informed decisions and alignment of investments to high demand opportunities.

Recommendation 5: Consideration should be given to examine in more detail the design and delivery of Self-Employment and whether the performance indicators for this program are appropriate.

Response: The Self-Employment program in Manitoba assists eligible individuals to create jobs for themselves by starting their own business. The program is administered by third party service providers who evaluate the individual’s business idea and suitability, provide coaching in business plan development and implementation, offer advice and support, and direct the participant to other supports as needed. The evaluation was unable to produce conclusive evidence of the impact of participation in Manitoba’s Self-Employment program. Manitoba will give consideration to exploring the design and delivery as well as the longer-term outcomes to understand its effectiveness in achieving its objectives.

Recommendation 6: Consideration should be given to examine the design and delivery of Employment Partnerships in order to better understand how this program operates.

Response: Manitoba is pleased with the findings related to its Employment Partnerships program including its effectiveness at improving employment and earnings of active EI claimants. The program assists unemployed individuals to gain sustainable employment while assisting communities, industry sectors and employers to address their labour force development needs. Manitoba will give consideration to exploring its design and delivery as well as the longer-term outcomes to understand its effectiveness in achieving its objectives.

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. Manitoba, ESDC, and other provinces and territories should work together to define common categories for both EBSMs.

Response: Manitoba is strongly committed to continuous improvement of its data collection and its ability to monitor, assess, and evaluate the effectiveness of its programs and support evidence informed decisions. A new outcomes framework, as part of Manitoba’s service modernization, will measure gains in employability, including readiness to learn and readiness to work, and employment outcomes over time for a variety of client groups, including Indigenous peoples, persons with disabilities, and newcomers. Manitoba is also collaborating with ESDC and other provinces and territories to improve the measurement of performance along the pathway to employment and labour market outcomes. Manitoba is also committed to working with ESDC to improve data collection and two-way data sharing to improve its programs.

1. Introduction

Employment and Social Development Canada (ESDC) worked jointly with Manitoba and eleven other Provinces and Territories (P/Ts) to undertake the 2012 to 2017 second cycle for the Labour Market Development Agreement (LMDA) evaluations. The first cycle of LMDA evaluation was carried out between 1998 and 2012 and involved the conduct of bilateral formative and summative evaluations in all P/Ts. Under the second cycle, the evaluation work consisted of conducting two to three studies per year on the Employment Benefits and Support Measures (EBSMs) similar programming delivered under these agreements. The various studies generated evaluation evidence on the effectiveness, efficiency and design/delivery of LMDA programs and services for Canada overall and for each participating P/T, including Manitoba.

This report presents a summary of the findings from nine separate studies produced on Manitoba LMDA interventions and participants between 2012 and 2017. These studies examined issues and questions related to program effectiveness, efficiency and design/delivery using a mix of qualitative and quantitative methods. Each study examined evaluation issues in relation to active and former EI claimants.

Table G1 in Appendix G presents an overview of these studies, including the type of evidence generated, the methods used, the reference period and the length of the post-program period over which program effects were observed.

The report is organized as follows:

Introduction with an overview of the studies summarized in this report including their scope and methodology, and contextual information on the LMDAs
Findings section with a discussion around the rationale for investing in labour market programming and summary of evaluation evidence
Comparison of key findings by program type
Conclusions and lessons learned
Recommendations that emerged from the evaluation findings and areas for future investigation

1.1 Labour Market Development Agreement background

LMDAs are bilateral agreements between Canada and each P/T established under Part II of the1996 Employment Insurance (EI) Act. As part of these agreements, Canada transfers $2.14B annually to P/Ts (including $190M in administration funds) for the design and delivery of programs and services to assist individuals to prepare for, obtain and maintain employment. Specifically, Manitoba receives approximately $43.1M in EBSM funding each year.

The Canada-Manitoba LMDA was signed on April 17, 1997. The agreement transferred responsibility for the design and delivery of EI funding employment programs and services to Manitoba. Programs and services are classified under two categories: 1) Employment Benefits and 2) Support Measures.

Employment Benefits

Employment Benefits are offered to unemployed individuals who 1) are actively on EI (active claimants); 2) ended their benefit period within three years before participating (former claimants); or 3) established a claim for maternity or parental benefits within the past five years and are returning to the labour force for the first time (former claimants)^{Footnote 2}. Employment Benefits include the following categories:

Skills Development (including Apprenticeship) helps participants obtain employment skills by giving them financial assistance to enable them to select, arrange and pay for classroom training.
Targeted Wage Subsidies (Wage Subsidies) helps participants obtain on-the-job work experience by providing employers with financial assistance to help with the wages of participants.
Self-Employment provides financial assistance and business planning advice to EI-eligible participants to help them start their own business.
Job Creation Partnerships (Employment Partnerships) provides participants with opportunities to gain work experience that will lead to ongoing employment.
Targeted Earnings Supplements encourages unemployed persons to accept employment by offering them financial incentives. This program was not offered in Manitoba and therefore not covered by the evaluation.

Support Measures

Support Measures are available to all unemployed individuals including those not eligible to receive EI and include:

Employment Assistance Services such as individual counselling, action planning, help with job search skills, job-finding clubs, job placement services, the provision of labour market information, case management and follow-up.
Labour Market Partnerships provide funding to help employers, employee and employer associations, and communities improve their capacity to deal with human resource requirements and implement labour force adjustments. These partnerships involve developing plans and strategies, and implementing labour force adjustment measures.
Research and Innovation supports activities that identify better ways of helping people prepare for or keep employment and be productive participants in the labour force. Funds are provided to eligible recipients to enable them to carry out demonstration projects and research for this purpose.

Table 1 provides an overview of the share of funding allocated to the five programs and services examined under the second cycle for LMDA evaluation and the average cost per intervention. It is noted that the average cost per intervention was calculated based on the 2002 to 2005 data from the EI Monitoring and Assessment Reports. The 2002 to 2005 period corresponds to the cohort of participants selected for incremental impacts and cost-benefit analysis in the LMDA evaluation.

**Table 1. Share of LMDA Funding and Average Cost per Intervention in Manitoba**
Program and Service	Share of funding 2014 to 2015	Average cost per intervention 2002 to 2005
Skills Development (including Apprenticeship)	59%	$6,203
Employment Assistance Services	18%	$221
Wage Subsidies	1%	$4,368
Self-Employment	2%	$10,603
Employment Partnerships	2%	$6,382
Total	82%*	–

Sources: EI Monitoring and Assessment Reports 2002 to 2003 to 2014 to 2015.

* Note: Labour Market Partnerships and Research and Innovation represented about 18% of EBSM expenditures in 2014 to 2015.

1.2 Methodology

This section presents key aspects of the qualitative and quantitative analyses carried out as part of the LMDA studies.

1.2.1 Qualitative Data

Qualitative data reported in the Skills Development, Skills Development – Apprenticeship, Wage Subsidies, and Employment Assistance Services studies were collected from key informant interviews with managers and service providers and a document/literature review. As well, questionnaires were completed by provincial government representatives for the Skills Development, Skills Development – Apprentices and Wage Subsidies studies. Table 2 provides the number of key informants interviewed.

Key informant interviews for the Employment Assistance Services study were conducted in summer 2013 while those for the Skills Development, Skills Development – Apprenticeship and Wage Subsidies studies were conducted in summer 2015.

**Table 2. Number of Key Informant Interviews**
LMDA Studies	Skills Development	Skills Development – Apprenticeship	Wage Subsidies	Employment Assistance Services
Number of Key Informant Interviews
Managers	6	7	6	5
Service Providers	6	7	6	5

Studies aimed to examine issues and questions related to design and delivery of Self-employment, Employment Partnerships, Labour Market Partnership, and Research and Innovations were not carried out during the 2012 to 2017 second evaluation cycle. Studies were selected based on established priorities by the LMDA Evaluation Steering Committee.

1.2.2 Quantitative Methods

All quantitative analyses were based on administrative data from the EI Part I (EI claim data) and Part II (EBSM participation data collected by Manitoba and transferred to ESDC) linked to T1 and T4 taxation files from the Canada Revenue Agency.

Incremental Impacts

Five studies assessed program effectiveness by estimating incremental impacts from EBSM participation on participants’ labour market experience (for example earnings from employment/self-employment, incidence of employment, use of EI or social assistance and dependence on income support) after participation. The role of the incremental impact analysis is to isolate the effects of participation from other factors such as the economic cycle. In order to achieve this, the incremental impact analyses compared the labour market experience of participants before and after their participation with that of non-participants (see the example of incremental impact calculation in Figure 1).

**Figure 1. Example of Incremental Impact Calculation**

Incremental impacts were measured for the following indicators:

Employment/self-employment earnings represent the total earnings an individual had from paid employment and/or self-employment. (This information is available by calendar year and is obtained from T1 and T4 tax return records.)
Incidence of employment/self-employment represents the incidence of having earnings from employment and/or self-employment.
Amount of EI benefits received represent the average amount of EI benefits received.
Weeks in receipt of EI benefits represent the average number of weeks during which EI benefits were received.
Social Assistance benefits represent the average amount of social assistance benefits received. (This information is available by calendar year and is obtained from T1 tax return records.)
Dependence on income support represents the ratio of participant’s income that came from EI and SA benefits (i.e. EI benefits + social assistance benefits) / (EI benefits + social assistance benefits + earnings from employment/self-employment).

Incremental impacts were estimated for different cohorts of participants:

Active and former claimants as well as youth (under 30 years old) and older workers (55 years old and over) who started their EBSM participation between April 1, 2002 and March 31, 2005. The incremental impacts were produced for five years post-participation and covered between 90 to 100% of participants.
All active and all former claimants who stated their EBSM participation between January 1, 2006 and March 31, 2008. The incremental impacts were produced for three years post-participation and covered 100% of participants.
Active and former claimants who were long-tenured workers and started their EBSM participation between January 1, 2007 and December 31, 2009. The incremental impacts were produced for three years post-participation and covered 100% of participants. Long-tenured workers covered in this study are individuals who have established an EI regular or fishing benefit claimants and who had paid at least 30% of the annual maximum employee EI premiums in seven of the ten years preceding their EI claim and who had collected 35 or fewer weeks of EI regular or fishing benefits in the five years preceding their claim. This definition is similar to the EI claimant category long-tenured workers introduced under Connecting Canadians with Available Jobs.

The matching of participants and comparison group members used up to 75 socio-demographic and labour market variables observed over five years before participation. Two different comparison groups were used to measure impacts for active and former EI claimants. For active claimants, the incremental impacts were measured relative to a comparison group of active claimants who were eligible to, but did not, participate in EBSMs during the reference period.

Former claimants can be underemployed and unable to requalify for EI, out of the labour force for various reasons or on social assistance. Based on previous evaluation methodologies, on expert advice and given the difficulty in generating a suitable comparison for former claimants using administrative data alone, the comparison group for former claimants was created using individuals who participated in Employment Assistance Services only during the reference period. This is a conservative approach given the fact that participation in Employment Assistance Services can lead to limited effects on labour market outcomes. In other words, the experience of former claimants who received Employment Benefits (that is, Skills Development, Wage Subsidies, Self-Employment and Employment Partnerships) was compared to the experience of former claimants who received low intensity employment services (that is, Employment Assistance Services only). Due to this difference in measurement, incremental impacts estimated for active claimant participants should not be directly compared to those of former claimant participants^{Footnote 3}.

All analyses were conducted using a unit of analysis called the Action Plan Equivalent, which combines all EBSMs given to an individual within no more than six months of each other. For reporting purposes, incremental impacts were attributed to the longest intervention of the Action Plan Equivalent when Skills Development, Wage Subsidies, Employment Partnerships or Self-Employment was the longest intervention. Impacts for Employment Assistance Services were calculated for Action Plan Equivalent that contained only Employment Assistance Services with no other Employment Benefits.

The incremental impact estimates were produced using non-experimental methods, namely propensity score matching, using the Kernel Matching method, along with Difference-in-Differences method to estimate program impacts. Alternative matching techniques (that is, Nearest Neighbour and Inverse Propensity Weighting) were also used for validation purposes.

Cost-Benefit Analysis

The cost-benefit analysis compared how much it cost for individuals to participate in the programs and how much it costs the government to deliver those programs with the benefits both the participants and the government drew from those programs. The analysis was carried out from the society perspective, which combines the costs and the benefits for both the participants and the government.

Costs and benefits included in the calculations were as follows:

Program costs included the administration cost and the direct cost of the EBSMs. The cost for each EBSM was calculated at the Action Plan Equivalent level. The costs were determined based on the average composition of the Action Plan Equivalent.
The Marginal Social Costs of Public Funds represented the loss incurred by society when raising additional revenues such as taxes to fund government spending. The value was estimated as 20% of the program cost, sales taxes, income taxes, impacts on EI and impacts on social assistance paid or collected by the government.
Employment earnings consisted of incremental impacts on participants’ earnings during and after participation. The calculation accounts for the participant’s forgone earnings during participation (i.e. opportunity cost). These are based on incremental impacts for the 2002 to 2005 participants.
Fringe benefits included benefits such as employer-paid health and life insurance as well as pension contributions. The rate used to calculate the fringe benefits was 15% of the incremental impact on earnings.

The program effects on EI and social assistance use, as well as sale and income tax revenues, were not included in the calculations since these costs and benefits cancel each other out from the social perspective by definition. For example, while EI and social assistance are benefits received by participants, they represent a cost for the government. However, as indicated above, these effects are accounted for in the calculation of the Marginal Social Costs of Public Funds.

When producing the results, to bring all costs and benefits to a common base and to account for inflation and interest on foregone government investment, the estimates for the second year of participation and up to the sixth year post-program were discounted by 5% per year. As well, when the benefits were still lower than the costs six years after program end, the payback period was calculated by assuming that the average benefit or cost measured over the fifth and six year post-program would persist over time (discounted at a 5% annual rate).

1.2.3 Strengths and Limitations from the Studies

One of the key strengths of the studies is that all quantitative analyses were based on administrative data rather than survey responses. Compared to survey data, administrative data are not subject to recall errors or response bias.

When interpreting qualitative findings, readers should keep in mind that these are based on the perception of a small number of key informants who are directly involved in the design or delivery of the program. Their perception may be representative of their own region or community but not necessarily of the entire province. Since the number of key informants interviewed in each study is small (that is, ranging between 6 and 10), the number of informants who reported a specific finding is not indicated in the report. However, the report does note when there was a clear disagreement between key informants.

The matching process led to the creation of comparison groups closely matched to the EBSM participants in terms of their background characteristics. Results obtained with Kernel Matching were validated with the use of two other techniques (that is, Inverse Propensity Weighting and Nearest Neighbour), increasing the level of confidence in the results. However, readers should be aware that incremental impacts may be affected by factors not captured by the matching process. For example, themotivation to seek employment was not directly measured except to the extent it was captured in prior income and labour market attachment patterns.

Readers should also keep in mind that it is not possible to compare the results obtained for each claimant type since the results for active claimants represent the effects of the EBSMs relative to non-participation while the results for former claimants represents the Employment Benefits relative to a limited treatment (that is, those who participated exclusively in Employment Assistance Services).

The definition of long-tenured workers used in this report differs from the definition used in the literature as it does not consider the number of years the worker remained employed with the same employer.

The cost-benefit analysis was limited in the sense that it only took into account the quantifiable benefits and costs that were directly linked to EBSM delivery and participation and that could be estimated using available administrative data and the EI Monitoring and Assessment Report. The analysis did not capture “intangible”, non-pecuniary and indirect benefits. It did not consider the multiplier effect that improving participant’s income may have on the economy and did not account for the effect of EI Part II investment on sustaining a service delivery infrastructure and creating jobs among the governmental program service providers. As well, this analysis did not consider the displacement effect where participants may take away jobs that would otherwise be filled by other unemployed individuals. Finally, this analysis did not consider the possible effect of EBSMs on increasing skill prices.

It should be noted that it is not possible to produce an analysis of the incremental impacts for participants in Skills Development- Apprenticeship. Assessing these impacts poses a methodological challenge because program participants are already employed and are expected to return to their employment after completing their training. Therefore, expected labour market impacts cannot be examined using a similar approach as for other EBSMs, which are expected to help participants return to employment. As well, the data available does not permit the identification of a proper comparison group since program participants alternate between work and training and no other potential counterfactuals have similar employment and training patterns. In this context, it is possible to provide an analysis of labour market outcomes associated with program participants but not possible to produce estimates of program effectiveness. While outcomes can provide some insights about the labour market experience of participants before and after participation, it is not possible to attribute the change observed in the outcomes to program participation. For example, a change in average annual earnings from before to after participation could be due to program participation or to other factors such as the maturation effect of youth, the economic cycle, lay off, etc.

2. Evaluation Findings

2.1 Rationale and labour market context

Active labour market programs aim to help unemployed or underemployed individuals find and maintain employment. These programs are fairly similar across Organization for Economic Co-operation and Development countries and consist of skills training in a classroom setting, work experience with employers (often subsidized) or in the public/non-profit sector, return-to-employment assistance and self-employment assistance.

In Manitoba, the Department of Education and Training administers a range of active labour market programming targeted at various groups of individuals through its Post-Secondary Education and Workforce Developmentdivision. These programs and services are funded under the LMDA, the Canada-Manitoba Job Fund Agreement, the Targeted Initiative for Older Workers, and provincial training dollars. Programs and services offered to active and former EI claimants are mainly funded under the LMDA.

As part of stakeholder consultations undertaken by the Forum of Labour Market Minsters during the summer of 2016, Manitoba consulted a range of stakeholders involved in labour market training across the province and synthesized the feedback into the What We Heard Summary Report^{Footnote 4}. The consultations sought to ensure that Manitoba’s employment and training programs are relevant and responsive to current and future labour market needs and priorities, and the corresponding findings have helped identify ways to improve labour market programming in Manitoba and inform the investments necessary to realize these improvements.

The report highlighted that several segments of the Manitoba labour force, including northern communities, Indigenous peoples, and other sub-groups (such as women and caregivers, youth, older workers and persons with disabilities), face additional barriers to labour market integration and require intentional accommodation in order to succeed. The report further pointed to the need for flexible, innovative, culturally-sensitive approaches to employability training and labour market partnerships, as well as the value of a holistic approach to long term success. Work readiness, which includes essential and employability skills, was identified as an imperative service. Ultimately, the report’s findings emphasized the importance of employment and skills training programs in meeting the needs of Manitobans and for improving the province’s labour market performance. It also underscored the contributory role of federal and provincial investments in providing these training programs.

The incremental impacts reported in the LMDA evaluation and discussed in this report demonstrate that, in general, LMDA-funded programs and services delivered in Manitoba are improving participants’ labour market experience following participation and contributing to helping Manitobans access “the education and training they need to prepare them for their future economic success.”^{Footnote 5}

2.2 Skills Development

2.2.1 Program Description

Based on a document review and six key informant interviews completed in summer 2015

In Manitoba, the Skills Development program aims to:

Increase participants’ access to labour market training;
Support unemployed individuals to return to work;
Increase earnings through higher-skilled employment;
Decrease skill shortages;
Reduce costs to provincial Employment and Income Assistance; and
Generate savings to the EI Account.

The program provides financial support to eligible individuals to cover expenses related to training, including tuition and course-related costs, living, transportation, daycare, relocation, work/training supplies, certification/licensing and special educational equipment to accommodate persons with disabilities. The nature and level of financial assistance provided is determined through a negotiated process between the participant and the career development counsellor and is based on individual need within established funding guidelines, suitability, the likelihood of success and the nature of/demand for the skill training being pursued. The maximum level of financial support was $15,000 per individual in summer 2015.

Participants are expected to make a financial contribution to their training. To do so, participants are encouraged to apply for student grants and loans through Manitoba Student Aid. Career development counsellors have the authority to approve a discretionary student loan once the individual has applied and met eligibility criteria for a regular student loan through Manitoba Student Aid. The discretionary loan is used to contribute to the difference between the student loan and what the individual needs in order to meet their financial contribution according to the employment plan.

A range of training is supported by the program, including academic upgrading, essential skills and literacy training, English as an additional language, and occupational-specific training programs. Training programs are usually up to two years in duration; however, participants may also be eligible for up to one additional year of upgrading (i.e. academic, essential skills, and/or English as-an-additional-language training) provided it is a requirement for the participant to proceed with training. Although university degree programs are generally not supported, it has been provided in specific cases involving short-duration programs such as teaching certificates, high demand occupations such as nursing, or where it is part of an achievable and employment oriented plan.

2.2.2 Program delivery

Based on a document review and six key informant interviews completed in summer 2015

The Skills Development program is delivered by the Manitoba government through 13 Manitoba Jobs and Skills Development Centres located across the province. Prospective participants undergo an assessment of their needs, EI eligibility, education and skills levels, employment history, goals and health/medical conditions. Based on that information, a return-to-work action plan is developed. Career development counsellors may conduct an employment training assessment to determine barriers that the individual may face. Depending on the outcome, individuals may be referred for a three-hour assessment with Workplace Education Manitoba or a five-day “career launch” class.

If Skills Development is identified as an appropriate intervention, the individual must research training options and the labour market to demonstrate that there is demand for their chosen field of training. In addition, career development counsellors assess the participant in order to determine the appropriate financial supports and amount of each support that is required.

Generally, career development counsellors approve applications. However, managers must approve applications that are above the financial support limit, or applications that are from individuals who have already participated in training in the last five years. The length of the application and approval process varies depending on participant readiness. The application process usually takes 1-3 weeks, on average, and in some cases up to 4 to 6 months.

2.2.3 Targeting to Labour Market Demand

Based on a document review and six key informant interviews completed in summer 2015

Most key informants confirmed that the program is directed toward occupations in demand. This is mainly accomplished during the application phase of the process. In order to receive funding, participants must conduct labour market research and may conduct informational interviews with potential employers to demonstrate that sufficient demand exists in their chosen occupation. Participants are provided with an Occupational Research Form, which provides guidance on the type of information to consider when making occupational decisions, as well as resources to assist in the research process. Certain high demand occupations are given priority, and career development counsellors may turn down applications for occupations that are not in-demand.

2.2.4 Profile of Skills Development participants

Socio-demographic characteristics

As shown in Table A1 in Appendix A, active claimants who started their Skills Development intervention in 2002 to 2005 or 2006 to 2008 were almost evenly split between male and female (51% and 48% respectively for the 2002 to 2005 cohort and 50% each for the 2006 to 2008 cohort). The majority of participants in both cohorts were between 25 and 44 years of age (61% and 57% respectively) and most frequently occupied jobs in the year before participation that required secondary school or occupational training (43% and 42% respectively). Additionally, 12% of the 2005 to 2005 participants and 14% of the 2006 to 2008 participants self-identified as Indigenous individuals.

Former claimants who started their intervention in 2002 to 2005 or 2006 to 2008 were mainly female (58% and 61% respectively) and between 25 and 44 years of age (63% and 61% respectively). As well, 11% of 2002 to 2005 participants and 20% of 2006 to 2008 participants self-identified as Indigenous individuals. These participants most frequently occupied jobs requiring secondary school or occupational training in the year before participation (46% in each cohort).

Barriers faced by participants

Based on a document review and six key informant interviews completed in summer 2015

Key informants were asked to identify the main barriers to employment or to accessing and completing Skills Development training faced by participants. The most frequently identified barriers were the following:

Lack of education and skills or outdated skills
Financial hardships (that is, the existing funding cap may be too low to support participants on training for longer durations)
Difficult to relocate to access training. This is particularly a challenge for Indigenous youth who face social challenges relocating outside of their community for training.
Lack of transportation and lack of driver’s license
Lack of available child care
Family/personal issues as well as mental health and addictions issues
Low essential skills and literacy levels
Lack of work experience
New Canadians may hold credentials that are not formally recognized
Criminal records and legal issues

2.2.5 Incremental Impacts

Active claimants

Overall, active claimants increased their labour market attachment through increases in earnings, incidence of employment and decreases in the level of dependence on government income support (use of EI and social assistance).

As shown in Table A2 in Appendix A, active claimants who started their Skills Development participation between 2002 and 2005 had incremental gains in employment earnings and incidence of employment in each of the five years after participation. Gains in earnings grew steadily over time and ranged from $931 in the first year after participation to $7,268 in the fifth year (see Figure 2). Similarly, increases in annual incidence of employment ranged between 5.0 and 7.1 percentage points in the five years after participation.

**Figure 2. Increased earnings of active and former Skills Development participants relative to non-participants** ^{Footnote 6}

**Figure 2. Increased earnings of active and former Skills Development participants relative to non-participants**
	Gains in earnings
Active Claimants	Former Claimants
Year 1 Post-Program	$ 931	$ 419
Year 2 Post-Program	$ 3,717	$ 1,976
Year 3 Post-Program	$ 5,353	$ 2,369
Year 4 Post-Program	$ 6,559	$ 2,163
Year 5 Post-Program	$ 7,268	$ 2,659

Gains in employment earnings and incidence of employment were accompanied by decreased EI use in the first two years following program participation ($439 and $143 respectively), as well as decreased social assistance use and dependence on income support in all post-program years.

Active claimants who started their Skills Development intervention between 2006 and 2008 also had incremental gains in earnings and incidence of employment in each of the three years after participation. As well, their use of EI and social assistance and their dependence on income support decreased over the three years post-program period.

Results for sub-groups of active claimants were as follows:

Long-tenured workers who started their Skills Development participation between 2007 and 2009 had incremental gains in earnings in the second and third years after participation and gains in incidence of employment in all years post-program. They also decreased their level of dependence on income support in their second and third years post-program. Results for EI and social assistance use were generally non-statistically significant.

Impacts for youth (under 30 years old) were not presented because the conditions associated with balancing tests were not satisfied. As well the number of older worker participants (55 years old and over) was too small.

Former claimants

Overall, former claimants increased their labour market attachment through increases in earnings, incidence of employment and decreases in the use of social assistance and the level of dependence on government income support.

Former claimants who started their Skills Development participation between 2002 and 2005 had incremental gains in earnings and incidence of employment in all five years after participation. Gains in earnings ranged between $419 in the first year after participation to $2,659 in the fifth year (Figure 2). As well, participants had gains in their incidence of employment that ranged between 2.6 and 5.8 percentage points over the five-year post-program period (see Table A3 in Appendix A). Their use of social assistance decreased in the second ($155) and third ($166) years after participation and their dependence on income support decreased in the second and fifth years after participation (other results were not statistically significant at the 95% level).

EI use by these former claimants generally increased over the post-program period, as well as the amount of EI benefits collected (in the second ($185), third ($227) and fourth years ($235) after participation). This indicates the inability of some former claimants to maintain the employment secured following participation. It can also be argued that the increase in EI use is an indication of increased labour market attachment for this client group since they did experience increases in employment earnings and incidence of employment as well as a decrease in the use of social assistance. As a reminder, former claimants are participants for whom the EI benefit period ended up to three years pre-participation.

Former claimants who started their Skills Development intervention between 2006 and 2008 also had incremental gains in earnings and incidence of employment, as well as decreased social assistance use and dependence on income support, over the three year post-program period. Their results for EI use were generally not statistically significant.

Results for sub-groups of former claimants were as follows:

Youth who started their Skills Development participation between 2002 and 2005 had gains in their incidence of employment in all five post-program years. Most results for earnings, social assistance use and dependence on income support were not statistically significant, while EI benefits receipt generally increased after participation

Incremental impacts for older workers and long-tenured workers were not produced as the number of participants was too small.

2.2.6 Cost-benefit results

From the social perspective, the benefits of Skills Development for active claimants recovered the costs of the program 4.4 years after participation. Six years after participation, the benefits were $9,460 higher than the costs (see Table A4 in Appendix A). For former claimants, it takes 7.5 years for the benefits to recover the program costs.

2.2.7 Challenges and lessons learned about Skills Development design and delivery

Based on a document review and six key informant interviews completed in summer 2015

Key informants identified a range of challenges related to the design and delivery of the Skills Development program (each mentioned by less than three key informants), including:

Capacity issues associated with large caseloads and serving individuals facing personal or mental health issues.
Improve local labour market information to inform training and investment decisions.
The insufficiency of living allowance for Manitobans in northern or remote areas.
Staff may face challenges in keeping up to date with changes to program policies.
More in depth and up front skills assessment to identify barriers and required supports.
Attendance sheets are no longer requested from training institutions. This limit the ability to address attendance issues and to ensure the progress along the return to work action plans.
Lack of awareness among potential participants, including new EI claimants.
EI eligibility criteria limit access for those who quit or have breaks in work history.

Key informants also identified a number of lessons learned:

Having a job connection worker (or similar position) to assist participants with personal and family issues would benefit participants with multiple barriers.
Encouraging participants to conduct informational interviews helps them make informed career choices.
Providing support for up to eight weeks of job search activities after the completion of training has been positive.
Monthly monitoring of progress helps to identify challenges experienced by participants.
Investment in preparatory training such as upgrading, second language training and soft skills development (for example, working with others) helps to improve participant readiness prior to entering occupation-specific training.
Fee-payers^{Footnote 7} should not have to go through all of the case management steps. Approximately 10% of participants are fee-payers.
Having a close relationship with the student financial assistance office allows career development counsellors to assist participants in navigating this source of support.

2.3 Skills Development - Apprenticeship

2.3.1 Program description

Based on a document review and seven key informant interviews completed in summer 2015

The objective of the Skills Development – Apprenticeship program is to facilitate apprenticeship training, equipping participants to improve their labour market attachment and reducing their likelihood of needing unemployment benefits in the future. This program provides financial assistance to apprentices to help them offset the costs incurred while they are released from employment to attend in-class technical training.

Eligibility for funding is tied to EI eligibility, having an employer and being registered with Apprenticeship Manitoba. Generally, an apprentice must have a grade 12 education and be at least 18 years of age.

Manitoba’s contribution towards technical training for EI-eligible apprentices ranges from $1,700 to $12,700. Apprentices are required to contribute $200 to $300 toward the cost of their training. Manitoba’s contribution aims to cover the following expenses:

Living allowance
Tuition
Travel and transportation
Childcare
Living away from home allowance
Other supports may be provided based on a participant’s circumstances:

Assistance for persons with disabilities if adequate support is not provided by the training institution or other sources
Participants in exceptional circumstances may receive additional funding related to training
Pre-apprenticeship and essential skills training may be provided as needed

2.3.2 Program Delivery

Based on a document review and seven key informant interviews completed in summer 2015

The Skills Development – Apprenticeship program is administered by the Post-Secondary Education and Workforce Development Division of the Department of Education and Training. Apprenticeship Manitoba is responsible for planning apprenticeship training, developing training standards, and the assessment and approval of apprentices, including case management and monitoring of registered apprentices. As well, Apprenticeship Manitoba collaborates with industry to develop occupational training, which is used by colleges to design training courses. On the first day of technical training, staff meet with apprentices and collect the information required for tuition purposes.

2.3.3 Profile of Skills Development - Apprenticeship participants

Labour market outcomes were produced for participants in the Skills Development – Apprenticeship program who started participation in the 2003 to 2005 period. This selection was based on the average number of years required to complete the apprenticeship training in Canada as suggested in the literature. The profile of the 2003 to 2005 participants was compared to a cohort of more recent participants (that is 2013 to 2014) in order to account for key changes in participants’ socio-demographic characteristics.

Socio-demographic characteristics

As shown in Table B1 in Appendix B, the vast majority of active claimants who started their Skills Development - Apprenticeship participation in 2003 to 2005 or 2013 to 2014 were male (90% and 97% respectively). A larger majority of 2003 to 2005 participants were below 35 years of age (80%) compared to the 2013 to 2014 participants (66%). Additionally, 11% of the 2003 to 2005 cohort and 22% of the 2013 to 2014 cohort self-identified as Indigenous individuals. Before participating in the program, most participants had occupations that required college or apprenticeship training, though the share was lower for 2003 to 2005 participants (77%) than 2013 to 2014 participants (94%).

For former claimants, the statistical profile was only produced for those participants who started their Skills Development – Apprenticeship intervention between 2003 and 2005, as the number of 2013 to 2014 participants was too small to produce results. The majority of former claimants who started the program in 2003 to 2005 were male (75%) and between 25 and 44 years old (73%). As well, nearly one-third (32%) of these participants self-identified as Indigenous individuals. Before participating in the program, former claimants most frequently had occupations that required college or apprenticeship training (41%), followed by secondary or occupational training (32%).

2.3.4 Labour Market outcomes

As indicated in the methodology section, incremental impacts could not be estimated for apprentices due to the difficulty in finding a suitable comparison group. Table B2 and Table B3 in Appendix C present the labour market outcomes for active and former EI claimants who started a Skills Development - Apprenticeship intervention during the 2003 to 2005 period.

Active claimants

As shown in Figure 3, average earnings of active claimant participants grew from $15,267 in the fifth year pre-program to $52,996 in the seventh year after the program start year. The proportion of employed participants declined (from 98% to 93%) in the years following program participation but remained at or above 93% employed in all seven years. The proportion of participants on EI declined in the years after participation to 17% by the seventh year after starting the program. On average, participants’ reliance on social assistance or dependence on income support did not vary greatly between the pre- and post-program periods.

**Figure 3. Average earnings for active claimant participants in Skills Development - Apprenticeship (in current dollars)**

**Figure 3. Average earnings for active claimant participants in Skills Development - Apprenticeship (in current dollars)**
	Employment earnings
5 yrs pre-program	$ 15,267
4 yrs pre-program	$ 17,529
3 yrs pre-program	$ 19,223
2 yrs pre-program	$ 21,895
1 yr pre-program	$ 26,352
Program start year	$ 22,987
1 yr post-program	$ 27,657
2 yrs post-program	$ 34,390
3 yrs post-program	$ 41,166
4 yrs post-program	$ 46,149
5 yrs post-program	$ 47,875
6 yrs post-program	$ 50,097
7 yrs post-program	$ 52,996

Former claimants

Average annual employment earnings of former claimant program participants (shown in Figure 4) increased from $15,408 in the fifth year pre-program to $41,390 in the seventh year following the program start year. The proportion of employed participants fluctuated in the years pre- and post-participation and eventually returned to 89% by the seventh year post-program (the same proportion employed in the fifth year pre-program). The proportion of participants on EI ranged between 41% and 53% in the five years pre-program but declined to a low of 28% in the seventh year following program start. The level of dependence on income support for former claimants ranged between 8% and 20% in the seven years following the program start year.

**Figure 4. Average earnings for former claimant participants in Skills Development - Apprenticeship (in current dollars)**

**Figure 4. Average earnings for former claimant participants in Skills Development - Apprenticeship (in current dollars)**
	Employment earnings
5 yrs pre-program	$ 15,408
4 yrs pre-program	$ 16,590
3 yrs pre-program	$ 17,718
2 yrs pre-program	$ 18,362
1 yr pre-program	$ 21,262
Program start year	$ 21,146
1 yr post-program	$ 21,419
2 yrs post-program	$ 25,180
3 yrs post-program	$ 28,950
4 yrs post-program	$ 33,253
5 yrs post-program	$ 37,331
6 yrs post-program	$ 42,013
7 yrs post-program	$ 41,390

2.3.5 Challenges and lessons learned about Skills Development - Apprenticeship design and delivery

Based on a document review and seven key informant interviews completed in summer 2015

Existing literature shows a fairly high non-completion rate among apprentices in Canada (40 to 50%)^{Footnote 8} and that despite the growth in apprenticeship registrations, there has not been a corresponding increase in completions^{Footnote 9}. Using available data, the evaluation could not generate a reliable estimation of completions rates among apprentices in Manitoba. Key informants confirmed a high non-completion rate among apprentices in Manitoba and identified reasons that could lead them to drop-out from the apprenticeship process. These include:

Delay in the first EI benefits payment and financial pressures during training. Apprentices may interrupt their training to take other employment
Availability of employers within an apprentice's community and willingness of employers to let the apprentices go on training
Time out of school may result in skill atrophy for some apprentices
Some apprentices are not assessed correctly ahead of the training.

As well, according to key informants, the main challenges associated with program design and delivery, and apprenticeships in general include:

Difficulty in forecasting how many registered apprentices will go on training in a given year.
Language and literacy issues may prevent individuals (such as immigrants) from entering or continuing with their apprenticeships.
There is a lack of incentive for employers to hire someone as an apprentice, and apprentices are responsible to find an employer on their own.
Not all personal issues (for example, having a criminal record) are identified in the application process, yet these issues may make the apprentice unemployable.
There is a lack of financial resources available to help apprentices with multiple barriers address their personal issues at the beginning of the apprenticeship process.

Finally, they also identified some best practices and lessons learned in terms of program design and delivery, and apprenticeship in general. These include (each mentioned by one to three key informants):

Continually monitor the apprentice’s needs and adjust the program to meet these needs.
Continue offering essential skill training.
Provide guidance and counselling to apprentices.
Having the Apprenticeship Manitoba staff and Industry, Training and Employment Services staff integrated in the same office works very well.
Having knowledgeable staff.
Provide tutoring services.
Include mentorship as part of the program.
Increase financial support for apprentices with a family.
Provide more consistent and clear labour market information at the national and provincial levels.
Reduce the EI benefit waiting period.
Work with apprentices facing multiple barriers to employment on a case by case basis and working closely with the training institution to support these individuals.

2.4 Wage Subsidies

2.4.1 Program description

Based on a document review and six key informant interviews completed in summer 2015

The Wage Subsidies program is a self-marketing tool to assist eligible, unemployed Manitobans to enter/re-enter the labour market. It provides opportunities for direct work experience and/or job-specific skills development to gain sustainable employment. In Manitoba, the program has two separate governance and management structures – one for Winnipeg and one for the remainder of the province.

Wage Subsidies objectives include:

Encouraging Manitoba employers to hire selected individuals who might not otherwise be considered by helping them offset the costs of hiring and training new workers
Providing eligible Manitobans with a self-marketing tool to assist their entry/re-entry into the labour market
Encouraging employers to tap into the vast pool of potential workers in the current era of skill shortages

Objectives specific to Winnipeg integrated services are:

Providing temporary support to employers to assist in on-boarding unemployed job-seeker(s) who require one or more of the following: skills training; work experience; longer on-boarding time; and/or other supports
Increasing opportunities for unemployed job-seekers to obtain long-term, sustainable employment through one or more of the following: skills training; work experience; longer on-boarding time; and/or other supports

Although the typical program participant may have some entry-level skills or particular career that they wish to explore, Wage Subsidies is focused on job-seekers whose main barrier to employment is a lack of work experience. Key informants reported that the program is intended to lead to permanent full-time employment.

In this regard, according to three key informants, between 60% and 90% of program participants remain with the employer after the subsidy has ended. The questionnaire completed by provincial representatives reported that employers must make every effort to provide participants with on-going, full-time, permanent employment after the wage subsidy period ends. This commitment is gauged during the suitability assessment as part of the application process in Winnipeg and is confirmed in the employer agreement.

According to the questionnaire, the duration of the subsidy varies depending on the job requirements, the skills and the knowledge of the participant. The duration is negotiated directly with the employer. The total contract length is typically up to 30 weeks and managerial approval is required for an extension. One key informant reported that employers can receive wage subsidies for up to 52 weeks for EI claimants (and up to 26 weeks for Income Assistance recipients). Another key informant noted that participants who are job ready may only receive the subsidy for eight weeks.

Employers may receive up to 50% of the participant’s wages with no maximum dollar-amount limit. Employers must demonstrate the typical wage for the position by providing job ads for similar positions. Generally, the proportion of the subsidized wage is adjusted based on the participant’s job readiness.

Related contributions to certain costs can be negotiated with employers to support and enhance the likelihood of ongoing job maintenance. For example, additional employment-related costs such as adaptive tools for those with disabilities or a bus pass may be funded based on the participant’s needs. These costs are negotiated on a case-by-case basis. They may also include short-term job-specific training as long as it provides each employee with specific skills related to the job as described in the application.

The Wage Subsidies program is not targeted towards any specific occupations in Manitoba. The subsidy is generally provided to all employers who are filling vacancies.

2.4.2 Program delivery

Based on a document review and six key informant interviews completed in summer 2015

In Winnipeg, the Wage Subsidies program is delivered by Industry, Training and Employment Services through Manitoba Jobs and Skills Development Centers and Industry Services. Labour Market Services Advisors and Career Development Consultants provide services to participants while Industry Services provide services directly to the employer. The Labour Market Services Advisors, Career Development Consultants and Industry Services staff work together to coordinate program delivery and help employers connect with job seekers.

In the remainder of the province, the program is delivered by Industry, Training and Employment Services through Manitoba Jobs and Skills Development Centres and Training and Employment Services. Labour Market Services Advisors and Career Development Consultants provide services to both the participant and employer.

2.4.3 Profile of Wage Subsidies participants

Socio-demographic characteristics

As indicated in Table C1 in Appendix C, active claimants who started participating in the Wage Subsidies program in 2002 to 2005 or 2006 to 2008 were more often male (64% and 62% respectively). Those who started between 2002 and 2005 were younger, on average, with 64% aged 25 to 44 years old (compared to 51% of 2006 to 2008 participants). The majority of the 2006 to 2008 cohort was above 34 years of age (72%), with notable proportions of participants who were aged 45 to 54 (26%) and older workers aged 55 and over (15%). Moreover, 11% of 2002 to 2005 participants and 9% of 2006-2008 participants self-identified as Indigenous individuals. Participants in both cohorts most frequently had a job requiring secondary or occupational training before participation.

Former claimants in both the 2002 to 2005 and 2006 to 2008 cohorts were mainly male (62% and 63% respectively) and between 25 and 44 years of age (61% and 55% respectively), though both cohorts still had sizeable proportions of participants 45 years and older (26% and 29% respectively). As well, 8% of 2002 to 2005 cohort self-identified as Indigenous individuals compared to 17% for the 2006-2008 cohort. Before participation, former claimants in both cohorts most frequently had a job requiring secondary or occupational training (38% and 41% respectively) followed by college or apprenticeship training (29% and 27% respectively).

2.4.4 Incremental impacts

Detailed incremental impact results for active and former claimants are presented in Table C2 and Table C3 in Appendix C.

Active claimants

Overall, active claimants who started participation in 2002 to 2005 increased their labour market attachment through increases in earnings, incidence of employment and decreases in the level of dependence on government income support.

Active claimants who started participation in Wage Subsidies between 2002 and 2005 had incremental gains in employment earnings and incidence of employment in each of the five years after participation. Gains in earnings ranged from $2,611 in the first year to $5,378 in the fifth year (see Figure 5), for a cumulative incremental impact of $21,972 over the five post-program years. Increases in average annual incidence of employment ranged between 5.1 and 8 percentage points for program participants. Results for EI and social assistance use were not statistically significant; however, participants decreased their dependence on government income support in three of the five post-program years.

Most results for active claimants who started their Wage Subsidies intervention between 2006 and 2008 were not statistically significant. Therefore, it is not possible to draw any clear conclusion about program effectiveness for active claimants who started during this period. As well, impacts for sub-groups were not produced as the number of participants was too small.

**Figure 5. Increased earnings of active and former Wage Subsidies participants relative to non-participants**

**Figure 5. Increased earnings of active and former Wage Subsidies participants relative to non-participants**
	Gains in earnings
Active Claimants	Former Claimants
Year 1 Post-Program	$ 2,611	$ 2,204
Year 2 Post-Program	$ 4,713	$ 1,876
Year 3 Post-Program	$ 4,842	$ 2,288
Year 4 Post-Program	$ 4,428	$ 2,661
Year 5 Post-Program	$ 5,378	$ 2,326

Former claimants

As shown in Figure 5, former claimants who started the Wage Subsidies program between 2002 and 2005 had incremental gains in earnings and incidence of employment in all years after program participation. Annual gains in earnings ranged from $1,876 to $2,661, for a cumulative gain of $11,355 over the five post-program years. These gains were accompanied by incremental increases in incidence of employment ranging between 5.2 and 7.1 percentage points during the five post-program years. Except for the first year post-program, most results concerning the use of EI were statistically non-significant. Since participants build entitlements to EI while working during the program, it is not uncommon to observe increased EI use post-program if participants are unable to maintain the subsidized job after the subsidy ends. Both social assistance use and dependence on income support decreased for program participants in all post-program years.

While former claimants experience short-term increase in EI, this indicates the inability of some former claimants to maintain the employment secured in the short-term. It can also be argued that the increase in EI use is an indication of increased labour market attachment for this client group since they did experience increases in employment earnings and incidence of employment as well as a decrease in the use of social assistance. As a reminder, former claimants are participants for whom the EI benefit period ended up to three years pre-participation.

Former claimants who started their Wage Subsidies intervention between 2006 and 2008 realized similar program impacts. They experienced gains in earnings and incidence of employment and reduced social assistance use. Estimated program impacts on EI use and dependence on income support were not statistically significant.

Results for sub-groups of former claimants were as follows:

Youth (under 30 years old) former claimants who participated in the program between 2002 and 2005 decreased their use of social assistance benefits in three out of five years after participation. They had incremental gains in earnings in year 1 and 3 post-participation only. Aside from increased EI use in the first year post-program (possibly due to inability to maintain the subsidized job, as explained above), most of the results for other indicators were not statistically significant at the 95% level or above.

Impacts for older and long-tenured workers were not produced as the number of participants was too small.

2.4.5 Cost-benefit results

From the social perspective, the benefits of the Wage Subsidies program for active claimants matched the costs of the program 1.8 years after participation (Table C4). Six years after participation, the benefits to society exceeded the costs by $17,853.

The benefits of Wage Subsidies for former claimants matched the costs almost immediately after the participation period (0.1 year). Six years after participation, the benefits to society exceeded the costs by $12,226.

2.4.6 Challenges and lessons learned about Wage Subsidies design and delivery

Based on a document review and six key informant interviews completed in summer 2015

Despite being effective at improving the labour market experience of participants, there has been a steady decline in use of the Wage Subsidies program in recent years. In Manitoba, the number of new interventions decreased from 505 in 2003 to 2004 to 53 in 2014 to 2015.

All key informants observed a decrease in the number of new participants, and suggested the following reasons (listed by most-frequently mentioned to least-frequently mentioned):

Funding has been prioritized towards other special projects and programs such as the Canada Job Grant or Labour Market Partnerships.
The program is not well advertised or marketed. There is minimal information on the program available at the provincial government website, and case workers are not available to conduct outreach with employers.
The number of people accessing EI benefits has declined.
The “employment liaison officer” position was eliminated. This officer used to match employers and participants based on their needs.
In the past, the program focused on getting as many individuals hired as possible without considering if the job was the best fit for the employer and the participant. Now, the program is targeted to those who are most suited and fit well with the employer.

Key informants were asked to discuss why employers may not be inclined to use Wage Subsidies, and their responses mainly focused on lack of awareness and administrative burden for employers (for example, application form, the monthly claims and participant’s daily activities list). They were also asked to identify the main challenges with the design and delivery of the program. Most key informants stated that the primary challenge is a lack of awareness and outreach to employers. Other challenges to program design and delivery include (one key informant each):

Providing a self-marketing letter to participants in search of an employer negatively impacts hiring decisions by influencing the employer’s perception of the participant.
The follow-up with participants occurs too quickly after participation (within 90 days) and should be increased to at least one year after participation.
Organizational changes, including a move to integrate service delivery and change to program delivery in Winnipeg, led to lack of clarity in roles between employer and individual focused caseworkers.
Government budget restraints limit the funding available for Wage Subsidies. Some applications may be denied because there are not enough financial resources available to meet the demand for the program. As well, resources may not be sufficient enough to mediate conflicts between the employers and participants

Further, key informants provided a number of lessons learned related to the design and delivery of the program, including (each mentioned by three or less key informants):

Employers want a list of potential candidates, and it would be useful if there was an integrated database that could be used to match the needs of employers to the participant’s skills. Another suggestion is to create an ‘Employment Liaison Officer’ position directly responsible for managing the database and the matching process.
Increasing flexibility in the program guidelines (for example, increasing participant’s wages) could help to expand the number of participants.
Increasing the length of the subsidy to at least 30 weeks in duration.
Streamlining the application process (that is, reduce approval time and reduce paperwork) for employers that are ready and able to hire an individual immediately.
Offering Wage Subsidies to small businesses works best.
Modifying the self-marketing letter to ensure employers do not perceive the participant as being unemployable. Participants should be coached on how to use the letter when meeting with potential employers.
Monitoring participants regularly and based on employer information.
Assessing the suitability of the employer to ensure a successful match with participants is essential.

2.5 Self-Employment

2.5.1 Program description and delivery

Self-Employment helps individuals create jobs for themselves by starting a business or otherwise becoming self-employed. Itprovides financial assistance and business planning advice to EI eligible participants to help them start their own business. This financial assistance is intended to cover personal living expenses and other expenses during the initial stages of the business.

2.5.2 Profile of Self-Employment participants

Socio-demographic characteristics

As shown in Table D1 in Appendix D, the majority of active claimants who started their program participation either in 2002 to 2005 or 2006 to 2008 were male (63% and 57% respectively). Around one-third of participants in both cohorts were between 35 and 44 years old (36% and 32% respectively), while another 31% of 2002 to 2005 participants and 34% of 2006 to 2008 participants were aged 45 years or older. As well, nearly 10% of participants self-identified as Indigenous individuals (9% of the 2002 to 2005 cohort and 11% of the 2006 to 2008 cohort). Participants from both cohorts most frequently had an occupation requiring college or apprenticeship training prior to program participation (38% and 41% respectively) and, compared to participants in other EBSMs, a notable proportion also held occupations requiring managerial skills (17% and 15%) before starting their Self-Employment program.

Former claimants who participated in Self-Employment were more closely split between male and female in both the 2002 to 2005 cohort (43% and 57% respectively) and the 2006 to 2008 cohort (51% and 49%). Those who participated in the 2002 to 2005 period were slightly younger, with 36% below 35 and 28% above 44 years old (compared to 32% and 38% respectively for 2006 to 2008 participants). Additionally, 6% of 2002 to 2005 participants and 11% of 2006 to 2008 participants self-identified as Indigenous individuals. Prior to participation, 2002 to 2005 participants most frequently had a job requiring either secondary school or occupational training (34%) or college/apprenticeship training (33%). Those who participated in the 2006 to 2008 period more frequently held occupations requiring college or apprenticeship skills level (40%) prior to participation.

2.5.3 Challenges in measuring Self-Employment incremental impacts

Like other EBSMs, incremental impacts were estimated for active claimants who participated in Self-Employment in 2002 to 2005 and in 2006 to 2008. Results showed large decreases in employment/self-employment earnings and decreases in the incidence of employment. As well, compared to similar non-participants, program participants decreased their use of EI and reduced their dependence on government income support.

Detailed estimates are presented in Table D2 in Appendix D. However, they are not discussed in the report as they may not provide an accurate depiction of the financial well-being of participants in the post-program period. Impacts were examined using individual earnings reported in the T1 and T4 taxation files from Canada Revenue Agency, and measured relative to active claimants who did not participate in other programs and services.

According to a study from Statistics Canada, self-employed individuals in Canada have a lower average annual income than paid employees ($46,200 versus $52,400 in 2009), but the average net worth of their households is 2.7 times greater than that of the paid employee households, which indicates that some self-employed individuals may leave funds within their business for reinvestment purposes^{Footnote 10}. Overall, this suggests that looking at individual earnings alone, without taking the net worth into consideration, may not provide a fair assessment of how well Self-Employment participants are doing financially after participation.

Finally, readers should also be aware that little is currently known about the design and delivery of this program. In particular, there is a lack of understanding around the role played by this program in helping future entrepreneurs to implement viable business plans and to develop their entrepreneurship skills. Overall, it is not clear whether a participant’s success in improving their labour market attachment through self-employment is more closely associated with their business idea and their entrepreneurship skills or the assistance provided under the program.

2.6 Employment Partnerships

2.6.1 Program description

Employment Partnerships projects provide participants with opportunities to gain work experience. Participants continue to receive their EI Part I benefits or receive an allocation while they are employed by a project funded under the program. Activities of the project help develop the community and the local economy.

2.6.2 Profile of Employment Partnerships Participants

Socio-demographic characteristics

Active claimants who participated in Employment Partnerships between 2002 and 2005 or 2006 and 2008 were mostly male (70% and 75% respectively). As shown in Table E1 in Appendix E, 60% of those who participated in 2002 to 2005 and 58% of those in the 2006 to 2008 cohort were aged 25 to 44 years old, and nearly one-fifth of participants (19% in each cohort) self-identified as Indigenous individuals. Participants in both cohorts most frequently had an occupation requiring secondary or occupational training before participation (37% and 39% respectively).

Like active claimants, former claimants who participated in the program either between 2002-2005 or 2006 to 2008 were mainly male (57% and 65% respectively) and between 25 and 44 years old (67% and 60%). While 19% of the 2002 to 2005 participants self-identified as Indigenous individuals, the proportion reached 39% among the 2006 and 2008 participants. Prior to participation, former claimants in both cohorts most frequently had occupations requiring secondary or occupational training before participation (41% and 37% respectively).

2.6.3 Incremental Impacts

Active claimants

Overall, active claimants increased their labour market attachment through increases in earnings and incidence of employment, and decreases in the use of EI and social assistance.

Shown in Table E2 in Appendix E, active claimants who participated in Employment Partnerships between 2002 and 2005 had incremental gains in earnings in all years after program end. Figure 6 depicts earnings gains that ranged between $2,515 and $4,571 annually. Gains in incidence of employment were found in four out of five post-program years, but most results were only statistically significant at the 90% confidence level. As well, participants experienced a cumulative decrease of $1,376 in EI use for the entire five years post-program. Finally, participants had incremental decreases in the amount of social assistance benefits collected in the third, fourth and fifth post-program years.

**Figure 6. Increased earnings of active and former Employment Partnerships participants relative to non-participants**

**Figure 6. Increased earnings of active and former Employment Partnerships participants relative to non-participants**
	Gains in earnings
Active Claimants	Former Claimants
Year 1 Post-Program	$ 2,515	$ 2,702
Year 2 Post-Program	$ 3,284	$ 3,079
Year 3 Post-Program	$ 4,571	$ 2,745
Year 4 Post-Program	$ 4,336	$ 2,688
Year 5 Post-Program	$ 4,505	$ 2,747

Like the 2002 to 2005 participants, active claimants who participated in Employment Partnerships between 2006 and 2008 realized incremental gains in earnings and incidence of employment over all observed years following their program participation. They had short-term decreases in EI use as well as consistently decreased dependence on income support after program participation. Their results for social assistance use were not statistically significant.

Impacts for sub-groups were not produced as the number of participants was too small.

Former claimants

Overall, former claimants increased their labour market attachment through increases in earnings and incidence of employment and a decrease in EI use.

As depicted in Figure 6 (and contained in Table E3 in Appendix E), former claimants who participated in Employment Partnerships between 2002 and 2005 had incremental gains in earnings over all five post-program years that ranged from $2,688 to $3,079 annually. These program participants also had annual gains in incidence of employment over the 5 years post-participation that ranged between 4.1 and 5.7 percentage points. These participants also decreased their use of EI benefits by a cumulative of $823 over the five post-participation years. All of their results for social assistance use and most of their results for dependence on income support were not statistically significant.

Impacts for the 2006 to 2008 cohort and all sub-groups were not produced as the number of participants was too small.

2.6.4 Cost-benefit results

From the social perspective, the benefits of Employment Partnerships for active claimants matched the costs of the program 3.9 years after participation (Table E4). Six years after participation, the benefits to society exceeded the costs by $7,373. For former claimants, the benefits matched the costs 1.2 years after participation, and the benefits exceeded the costs by $12,661 by the sixth year after participation.

Overall, when interpreting cost-benefit results for Employment Partnerships, it should also be acknowledged that program funding helps to develop the community and the local economy and that none of those benefits were accounted for in the calculations as they are difficult to quantify.

2.7 Employment Assistances Services

2.7.1 Program description and delivery

Based on a document review and ten key informant interviews completed in summer 2013

In Manitoba, Employment Assistance Services assist unemployed individuals to prepare for, find and keep jobs by providing a variety of short-term, employment-related services. The goal of the program is to provide participants with the highest quality, most responsive, and cost-effective way of preparing for, finding, and keeping a job by providing a variety of employment related services.

The program is delivered through a combination of direct service delivery by provincial staff in 13 Manitoba Jobs and Skills Development Centres and through a network of 60 third-party service providers located in communities across the province. Services are accessible to all unemployed persons seeking employment-related supports. Services can be tailored to meet the specialized needs of individual participants and/or the needs of unemployed residents of local communities. For example, service providers may be contracted to deliver specialized services for persons living with disabilities, women, and/or immigrants. Services include:

Employment counselling and action plan development
Assessment services
Short-term employment preparation activities that increase the employability of project participants
Job search assistance, job referral and/or placement
Employment maintenance services
Information on the labour market and on education or training opportunities

Service providers noted that labour market information is essential and is used to support the delivery of services by having up-to-date information on various services offered in the community. Most service providers reported that they dedicate time and resources for labour market information collection. However it was also noted that sometimes there is limited capacity to seek local-level information from the community and employers. Labour market information is not readily available in the northern remote areas of Manitoba.

2.7.2 Profile of Employment Assistance Services participants

Socio-demographic characteristics

The following profile presents the main socio-demographic characteristics of active and former claimants who participated exclusively in Employment Assistance Services without participating in any other Employment Benefits programs.

As shown in Table F1 in Appendix F, active claimants who participated exclusively in Employment Assistance Services between 2002 to 2005 or 2006 to 2008 were more often male (60% and 57% respectively). Participants in both cohorts were fairly evenly distributed across the different adult age groups, with relatively high concentrations of participants aged 45 and older (32% of 2002 to 2005 participants and 38% of 2006 to 2008 participants). Notably, 11% of 2002 to 2005 participants and 14% of 2006 to 2008 participants were older workers (age 55 and over). Active claimants who participated in either 2002 to 2005 or 2006 to 2008 self-identified as Indigenous individuals in equal proportions (13%). Participants in both cohorts most frequently had a job requiring secondary or occupational training before participation (41% each).

Like active claimants, slightly more than half of former claimants who only participated in Employment Assistance Services during the 2002 to 2005 and 2006 to 2008 periods were male (54% and 52% respectively). About a third of the participants in both cohorts were between 25 and 34 years old (33% and 31% respectively) while 25% of 2002 to 2005 participants and 29% of 2006 to 2008 participants were 45 years old and over. Former claimants in both cohorts self-identified as Indigenous individuals in greater proportions (17% and 23% respectively), and 43% of each cohort was employed in occupations requiring secondary or occupational training before participation.

Labour Market barriers faced by Employment Assistance Services participants

Based on a document review and ten key informant interviews completed in summer 2013

According to key informants, the main labour market barriers experienced by individuals who access Employment Assistance Services in Manitoba are:

Lack of foundational and essential skills
Lack of job searching and interview skills
Low self-esteem and negative attitudes
Lack of transportation in some areas
Lack of child care spaces
Homelessness
Addiction and mental health issues
Seasonal employment
Lack of job opportunities in rural areas
Lack of awareness of available resources in rural areas
Lack of knowledge regarding labour market information
Youth with improper work ethic
Immigrants lack English skills, work experience and knowledge of the Canadian workplace culture. They also face issues with foreign credential recognition.
Participants may fear that disclosing their disability will prevent them from securing a position. Employers may believe that hiring a person with a disability is costly due to accommodations.
Indigenous individuals face discrimination

2.7.3 Incremental impacts

Incremental impacts were produced only for active claimants, since former claimants who participated only in Employment Assistance Services were used as a comparison group for former claimants who participated in other EBSMs.

Overall, active claimants who participated in Employment Assistance Services increased their labour market attachment through an increase in their incidence of employment and decrease in their dependence on government income support.

Results presented in Table F2 in Appendix F indicate that active claimants who participated exclusively in Employment Assistance Services between 2002 and 2005 had incremental gains in their annual incidence of employment ranging from 0.9 to 1.6 percentage points between the second and fifth post-program years (see Figure 7). As well, they had average annual reductions in EI use ranging between $100 and $273 over the post-program period. Their results for social assistance use were not statistically significant, while their dependence on income support decreased in all post-program years.

**Figure 7. Incidence of employment for active claimant participants in Employment Assistance Services**

**Figure 7. Incidence of employment for active claimant participants in Employment Assistance Services**
	Percentage points
Year 1 Post-Program	0.3
Year 2 Post-Program	0.9
Year 3 Post-Program	0.9
Year 4 Post-Program	1.3
Year 5 Post-Program	1.6

* The estimate is not statistically significant at the 95% confidence level.

Active claimants also had a reduction in their employment earnings in the first year after participation (-$950), but increases in the fourth ($360) and fifth years ($492). To contextualise these results, it should be noted that Employment Assistance Services are short-term, low intensity measures that are not focused on human capital development. It mostly includes services such as counselling, help with job search, development of return-to-work action plans and - in some cases - very short training such as first aid. In this context, it may not be reasonable to expect that participating only in Employment Assistance Services would result in improved participant earnings.

Active claimants who participated exclusively in Employment Assistance Services between 2006 and 2008 had similar results for incidence of employment, EI use and dependence on income support. Additionally, they also reduced their use of social assistance in the second and third years post-program. Their results for earnings were not statistically significant at the 95% confidence level.

Results for sub-groups of active claimants who participated exclusively in Employment Assistance Services were as follows:

Youth (under 30 years old) who participated between 2002 and 2005 had incremental gains in their incidence of employment and earnings in the fourth and fifth year after participation. As well, their use of EI and social assistance and their dependence on income support decreased over most of the post-program period.
Older workers (55 years old and over) who participated in 2002 to 2005 had gains in incidence of employment in the fourth and fifth years after participation. They experienced a decrease in earnings in the first year after participation, but earnings results for the remaining post-program period were not statistically significant. In general, impacts for EI and social assistance use and dependence on income support were not statistically significant.
Long-tenured workers who participated between 2007 and 2009 decreased their use of EI over most post-program years. While their use of social assistance generally increased post-program, their overall level of dependence on income support decreased over most of the post-program period. Results for earnings and incidence of employment were generally not statistically significant.

Earlier Participation in Employment Assistance Services Improves Participants’ Labour Market Outcomes (Gains in Earnings and Earlier Return to Work)

A study on the effects related to the timing of participation showed that the incremental impacts on earnings and employment were larger for individuals who participated in Employment Assistance Services early during their EI claim. Participants were compared to non-participants and to individuals who remained on EI for a longer period before receiving similar services (see Figure 8 below and Table F3 in Appendix F). Specifically, individuals who started their participation within four weeks after the start of their EI benefit period had total gains of $11,097 in their earnings over the five post-program years. These early starters were also the only group of participants to experience some gains in incidence of employment following participation (1.3 and 1.5 percentage points during the first two years post-program).

Participants who received Employment Assistance Services between five and eight weeks, after the start of their EI claim, also had gains in earnings totalling $7,585 over the post-program period. Participants who started between 9 and 12 weeks after the start of their claim, as well as those who started in the fourth quarter, had non-statistically significant cumulative gains in employment earnings. The participants who started in the second and third quarters generally had decreases in their employment earnings.

**Figure 8. Cumulative Incremental Impacts on Earnings Related to the Timing of Participation in Employment Assistance Services**

**Figure 8. Cumulative incremental impacts on earnings related to the timing of participation in Employment Assistance Services**
	Cumulative gains in earnings
1 to 4 weeks	$ 11,097
5 to 8 weeks	$ 7,585
9 to 12 weeks	$ 1,950
2nd quarter	-$ 2,147
3rd quarter	-$ 4,639
4th quarter	$ 4,101

* The estimates are not statistically significant at the 95% confidence level.

The study also looked at the difference between the number of EI weeks unused by participants and the number of EI weeks unused by their comparison groups in order to determine the effect of the timing of participation in Employment Assistance Services on the return to employment (see Table F4 in Appendix F). It was found that those who received assistance within the first four weeks of their claim returned to employment earlier than the comparison group (all other participants returned to employment later than their comparison group). Specifically, those who started within 4 weeks returned to employment 3.6 weeks earlier than the comparison group.

Among all participants during the 2002 to 2005 period, 5,470 received assistance within the first 4 weeks of establishing an EI claim. With an average weekly EI benefit of $296 during this period, the 5,470 participants did not use $5,828,832 in EI benefits (5,470 * $296 * 3.6 weeks). The average cost of delivering Employment Assistance Services during the 2002 to 2005 period was $221. This represents a cost of $1,208,870 (5,470 * $221) for a net saving of $4,619,962 in EI benefits.

2.7.4 Cost-Benefit results

From the social perspective, the benefits of Employment Assistance Services for active claimants would need to persist over 11.4 years in order to recover the costs of the program (as shown in Table F5 in Appendix F). This result is partly due to the decreases in earnings experienced by active claimants during and immediately after participation. The main expected benefit from Employment Assistance Services is the return to employment by itself and not necessarily improvements in earnings that may follow, as Employment Assistance Services alone do not focus on skills acquisition.

Incremental impact analyses showed that Employment Assistance Services are achieving this objective since it increased participants’ incidence of employment and decreased their EI use after participation. The value of the return to employment was not accounted for in this analysis since it is difficult to attribute a dollar figure to this impact. The decrease in EI use was also not considered from the social perspective because it represents a benefit for the government and a cost to the individual – they cancel each other out.

2.7.5 Challenges and lessons learned about Employment Assistance Services design and delivery

Based on a document review and ten key informant interviews completed in summer 2013

Key informants identified a number of challenges related to the design and delivery of Employment Assistance Services:

The lack of labour market information that is readily available at the local level and in northern and remote areas of the province.
The perceived insufficient level of funding and lack of flexibility with the budget allocation.
Difficulty in hiring skilled and knowledgeable service providers. As well, training opportunities for service providers are limited given the limited allowance for training and upgrading.
At the time of the study, contracts with services providers were currently of one year duration. This created a level of uncertainty in job security and impedes organizations’ ability to retain staff. However, the managers interviewed indicated that Manitoba is engaging in a multi-year funding pilot to review the strategic planning and operational reporting.
It is difficult to reach and to conduct follow-ups with participants facing multiple barriers as they often do not have a phone and tend to move around a lot.
Reliability of internet service in the north and remote areas can also pose a challenge for service providers in their capacity to assist and reach participants.
Referrals to services are usually made through word of mouth.

Key informants also identified some best practices and lessons learned in relation to the general design and delivery of the program:

Conducting comprehensive assessments up-front and understanding the needs of participants.
Adopting a holistic approach and a one-on-one approach.
Having one-stop-shop for services or co-locating federal and provincial services (that is, Service Canada, child and family services, social assistance services, etc.)
Frequent visits of government representatives to service providers and ongoing sharing of information between government representatives and service providers.
Maintain a network between service providers, which can help to make good referrals and to be resourceful. Multi-year contracts would help with staff retention.
In rural locations, it is essential to be flexible when delivering services, including becoming knowledgeable in areas outside of the funding criteria. As well, when covering large rural areas, having a board of directors from different communities to help service providers understand the various needs of participants.
A Career Development Practitioner certificate has been developed by Manitoba with a university and has become a screening tool for hiring staff.

3. Comparison of key findings by program type

This section provides an overview of the key findings from the incremental impact analysis for Skills Development, Wage Subsidies, Employment Partnerships and Employment Assistance Services for both active and former EI claimant participants who started participation in the 2002 to 2005 period.

Overall, incremental impacts demonstrate that LMDA programs and services are improving the labour market attachment of participants. As well, social benefits of participation exceeded the cost of investments for most interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first four weeks) produced larger impacts on earnings and employment, and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Program participants have a higher probability of being employed than comparison group members

As shown in Figure 9, active EI claimants who participated in Skills Development, Wage Subsidies, Employment Partnerships and Employment Assistance Services had higher probability of being employed (that is,increased their incidence of employment) compared to similar non-participants.

As well, former EI claimants who participated in Skills Development, Wage Subsidies and Employment Partnerships had a higher probability of being employed compared to former EI claimants who received low intensity interventions under Employment Assistance Services.

**Figure 9. Change in probability of being employed in participants relative to non-participants**

**Figure 9. Change in probability of being employed in participants relative to non-participants**
	Percentage Points
Active Claimants	Former Claimants
Skills Development	6.4	4.9
Wage Subsidies	6.8	6.3
Employment Partnerships	3.9	4.8
Employment Assistance Services	1	N/A

* The estimates in Figure 9 are arithmetic averages. The estimate for active claimants who participated in Employment Partnerships is based on five estimates with different statistical significance levels: one estimate is statistically significant at the 95% confidence level, three estimates are statistically significant at the 90% confidence level and one estimate is not statistically significant.

Increased earnings for participants compared to comparison group members

As shown in Figure 10, active EI claimants who participated in Skills Development, Wage Subsidies and Employment Partnerships increased their employment earnings compared to similar non-participants. Former EI claimants who participated in Skills Development, Wage Subsidies and Employment Partnerships also increased their employment earnings compared to former EI claimants who received Employment Assistance Services exclusively.

It is noted that Employment Assistance Services are relatively modest activities such as counselling, job search assistance and case management. By themselves, they are not expected to lead to substantial effects on participants’ labour market outcomes. These services aim to support the return to work of unemployed participants and not necessarily to secure a better paying job than pre-participation. However, as demonstrated in the report, providing Employment Assistance Services earlier during the EI claim (first 4 weeks) generates significantly greater returns (see Figure 8 in sub-section 2.7.3).

**Figure 10. Increased cumulative earnings of participants relative to non-participants**

**Figure 10. Increased cumulative earnings of participants relative to non-participants**
	Cumulative gains in earnings
Active Claimants	Former Claimants
Skills Development	$ 23,827	$ 9,586
Wage Subsidies	$ 21,972	$ 11,355
Employment Partnerships	$ 19,211	$ 13,961
Employment Assistance Services	-$ 556	N/A

* The estimate is not statistically significant at the 95% confidence level.

The incremental impacts on the use of EI and social assistance demonstrate that LMDA participants increased their labour market attachment overall. Active claimants decreased their use of EI and former claimants decreased their use of social assistance.

As shown in Figure 11, active EI claimants who participated in Skills Development, Employment Partnerships and Employment Assistance Services decreased their use of EI compared to similar non-participants. As well, active claimants who participated in Skills Development decreased their use of social assistance benefits (the decrease in the use of social assistance benefits was not statistically significant for other programs).

**Figure 11. Change in the cumulative use of Employment Insurance and social assistance for active claimants relative to non-participants**

**Figure 11. Change in the cumulative use of Employment Insurance and social assistance for active claimants relative to non-participants**
	Cumulative use of EI and social assistance benefits
Employment Insurance	Social Assistance
Skills Development	-$ 722	-$ 564
Wage Subsidies	-$ 790	-$ 676
Employment Partnerships	-$ 1,376	-$ 625
Employment Assistance Services	-$ 887	-$ 24

* The estimates are not statistically significant at the 95% confidence level.

As shown in Figure 12, former claimants who participated in Skills Development and Wage Subsidies decreased their use of social assistance. As well, former claimants who participated in Employment Partnerships between 2002 and 2005 decreased their use of EI following participation while those who participated in Skills Development increased their EI use. This may indicate the inability of some Skills Development participants to maintain the employment that they secured in the short-term. It can also be argued that the increase in EI use is an indication of increase labour market attachment for this client group since they did experience increases in employment earnings and incidence of employment as well as a decrease in the use of social assistance. As a reminder, former claimants are participants for whom the EI benefit period ended up to three years pre-participation.

**Figure 12. Change in the cumulative use of Employment Insurance and Social Assistance for former claimants relative to non-participants**

**Figure 12. Change in the cumulative use of Employment Insurance and social assistance for former claimants relative to non-participants**
	Cumulative use of EI and social assistance benefits
Employment Insurance	Social Assistance
Skills Development	$ 776	-$ 658
Wage Subsidies	$ 328	-$ 1,779
Employment Partnerships	-$ 823	-$ 416

* The estimates are not statistically significant at the 95% confidence level.

Social benefits of participation exceeded costs of investments for most interventions

As shown in Table 3, social benefits to participation exceeded investment costs in a period ranging from less than a year after program participation to about 11 years after participation for all interventions.

**Table 3. Number of years for the benefits to exceed program costs**
	Skills Development	Wage Subsidies	Employment Partnerships	Employment Assistance Services
Active claimants	4.4	1.8	3.9	11.4
Former claimants	7.5	0.1	1.2	n/a

4. Conclusion

The evaluation evidence demonstrates that programs and services delivered by Manitoba under the LMDA are generally improving participants’ labour market attachment following participation and contributing to helping Manitobans access the education and training they need to prepare them for their future economic success.^{Footnote 11}

Overall, incremental impacts demonstrate that programs and services are improving the labour market attachment of participants. As well, social benefits of participation exceeded the cost of investments for all examined interventions over time. Finally, providing Employment Assistance Services interventions earlier during an EI claim (first four weeks) produced larger impacts on earnings and employment and facilitated earlier return to work. This demonstrates the importance of targeting early participation of EI active claimants.

Key informants interviews with service providers and program managers, as well as the documents reviewed and the questionnaires filled by Manitoba representatives, also revealed specific challenges and lessons learned about program design and delivery, which may identify areas for future investigation.

5. Recommendations

A total of seven recommendations emerge from the evaluation findings. They are as follows:

The study on the timing of Employment Assistance Services participation showed that receiving assistance early after starting an EI claim can lead to better labour market impacts. However, key informants repeatedly reported a lack of awareness about the program.

Recommendation 1: Consideration should be given to providing Manitoba with timely access to data on new EI recipients for supporting targeted outreach and, if necessary, increasing awareness.

Recommendation 4: Consideration should be given about enhancing the capacity of service providers to access and produce, when needed, relevant labour market information.

Recommendation 5: Consideration should be given to examine in more detail the design and delivery of Self-Employment and whether the performance indicators for this program are appropriate.

Recommendation 6: Consideration should be given to examine the design and delivery of Employment Partnerships in order to better understand how this program operates.

Collect data on whether participants are members of designated groups including Indigenous peoples, persons with disabilities and recent immigrants.
Collect data on the type of training funded under Skills Development and the type of assistance provided under Employment Assistance Services. Manitoba, ESDC, and other provinces and territories should work together to define common categories for both EBSMs.

References

Canada Employment Insurance Commission. Employment Insurance Monitoring and Assessment Reports. 2003 to 2004 to 2013 to 2014.

Coe, Patrick. “Apprenticeship programme requirements and apprenticeship completion rates in Canada.” Journal of Vocational Education and Training. 2013. 65(4): 575 to 605.

Employment and Social Development Canada. Study of Skills Development-Apprenticeship

In Manitoba: Technical Report. Evaluation Directorate. June 22, 2016

Employment and Social Development Canada. Cost-Benefit Analysis of Employment Benefits and Support Measures in Manitoba: Technical Report. Evaluation Directorate. May 25, 2016

Employment and Social Development Canada. Study on the Skills Development Program in Manitoba: Technical Report. Evaluation Directorate. April 27, 2016

Employment and Social Development Canada. Study on the Wage Subsidy Program in Manitoba. April 7, 2016.

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for the EI Claimant Category: “Long-Tenured Workers” in Manitoba: Technical Report. Evaluation Directorate. March 9, 2016

Employment and Social Development Canada. Analysis of Employment Benefits and Support Measures (EBSM) Profile, Outcomes and Incremental Impacts for 2006-2008 Participants in Manitoba: Technical Report. Evaluation Directorate. March 3, 2015

Employment and Social Development Canada. Profile, Outcomes and Incremental Impacts of Employment Benefits and Support Measures (EBSM) Participants in Manitoba: Technical Report. Evaluation Directorate. June, 2015

Employment and Social Development Canada. Study on Employment Assistance Services: Technical Report for Manitoba. Evaluation Directorate. October 29, 2014

Employment and Social Development Canada. Effects of the Timing of Participation in Employment Assistance Services in Manitoba: Technical Report. Evaluation Directorate. December 19, 2014

Government of Manitoba. Workforce Development webpage Accessed November 29, 2016.

Government of Manitoba. Manitoba Consultations on Labour Market Training | What We Heard Summary Report.

Manitoba What We Heard Summary Report. October 2016.

Government of Manitoba. Speech From the Throne November 2016.

LaRochelle-Côté, Sébastien and Sharanjit Uppal, "The Financial Well-Being of the Self-Employed," Perspectives on Labour and Income, vol. 23, no. 4, Winter 2011

Red Seal. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal, 2014

Acronyms

EBSM	Employment Benefits and Support Measures
EI	Employment Insurance
ESDC	Employment and Social Development Canada
LMDA	LMDA Labour Market Development Agreements
P/Ts	Provinces and Territories

Appendix A - Detailed results Skills Development

**Table A1. Socio-Demographic and Labour Market characteristics of Skills Development participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	4,612	3,750	2,115	1,209
Gender
Male	51%	50%	41%	39%
Female	48%	50%	58%	61%
Age
Under 25	21%	18%	22%	21%
25 to 34	34%	30%	38%	37%
35 to 44	27%	27%	25%	24%
45 to 54	15%	20%	12%	15%
55 and over	3%	5%	3%	4%
Socio-demographic group
Indigenous individual*	12%	14%	11%	20%
Person with disability*	4%	4%	5%	5%
Visible minority*	14%	18%	7%	15%
Immigrant	12%	20%	6%	14%
Marital status
Married or common-law	44%	48%	34%	42%
Widow /divorced or separated	12%	11%	15%	12%
Single	43%	39%	49%	43%
Missing data / unknown	1%	1%	3%	3%
Skills level related to National Occupation Code associated with the last EI claim opened before Skills Development participation¹
Managerial	4%	5%	3%	3%
University	4%	4%	4%	2%
College or apprenticeship training	27%	28%	23%	27%
Secondary or occupational training	43%	42%	46%	46%
On-the-job training	21%	21%	25%	22%
Key labour market indicators in the year preceding the start of participation
Earnings²	$19,929	$19,539³	$7,172	$10,810³
Proportion employed	98%	99%	70%	86%
Proportion on EI	36%	37%	39%	66%
Proportion on Social Assistance	5%	3%	32%	13%

Proportions may not add up to 100% due to rounding.

*Status self-reported by participant.

¹Skill level corresponds to the type and/or amount of training or education typically required to work in the last occupation participants had before opening the last EI claim they had before participating in EBSMs:

- Managerial: Management occupations.
- University: Occupations usually requiring university education (University degree at the bachelor's, master's or doctorate level).
- College or apprenticeship training: Occupations usually requiring college or vocational education or apprenticeship training such as 2 to 3 years of post-secondary education at a community college, institute of technology or CEGEP or 2 to 5 years of apprenticeship training or 3 to 4 years of secondary school and more than 2 years of on-the-job training, specialized training courses or specific work experience and/or occupations with supervisory responsibilities and occupations with significant health and safety responsibilities, such as firefighters, police officers and registered nursing assistants.
- Secondary or occupational training: Occupations usually requiring secondary school and/or occupation-specific training such as one to four years of secondary school education or up to 2 years of on-the-job training specialized training courses or specific work experience.
- On-the-job training: On-the-job training is usually provided for occupations (short work demonstration or on-the-job training or no formal educational requirements).

²Average earnings for all individuals included in the studies. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table A2. Incremental impacts for Skills Development – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
All active claimants
2002 to 2005 participants (n=4,610)
Employment earnings ($)	-4,021***	-4,769***	931***	3,717***	5,353***	6,559***	7,268***	23,827***	15,038***
Incidence of employment (percentage points)	-0.9	0.3	5***	6.5***	7.1***	6.8***	6.6***	n/a	n/a
EI benefits ($)	1,668***	341***	-439***	-143**	-57	-20	-62	-722***	1,287***
EI weeks	5.5***	0.9***	-1.4***	-0.4***	-0.3	-0.1	-0.2	-2.4***	4.0***
SA benefits ($)	-12	-79***	-90***	-108***	-141***	-100***	-125***	-564***	-656***
Dependence on income support (percentage points)	13***	5***	-4***	-2***	-2***	-1***	-2***	n/a	n/a
2006 to 2008 participants (n=3,750)
Employment earnings ($)	-2,986	-3,198***	3,387***	5,971**	7,032**	n/a	n/a	16,384**	10,201**
Incidence of employment (percentage points)	5.4***	4.0***	10.6**	10.1**	11.8**	n/a	n/a	n/a	n/a
EI benefits ($)	1,576***	511***	-546***	-458***	-212**	n/a	n/a	-1,215***	872***
EI weeks	4.7***	1.2***	-1.8***	-1.3***	-0.6**	n/a	n/a	-3.7***	2.2**
SA benefits ($)	-14	-123***	-208***	-201***	-264***	n/a	n/a	-673***	-811***
Dependence on income support (percentage points)	9.8***	6.3***	-4.8***	-3.8***	-3.1***	n/a	n/a	n/a	n/a
Sub-groups of active claimants
Long-tenured workers – 2007 to 2009 participants (n=1,372)
Employment earnings ($)	-5,754***	-6,007***	-313	2,955***	3,561***	n/a	n/a	6,204***	-5,557*
Incidence of employment (percentage points)	-1.6*	-3.3**	2.5**	3.3**	3.8**	n/a	n/a	n/a	n/a
EI benefits ($)	2,252***	2,270***	-61	-178	-247**	n/a	n/a	-486	4,036***
EI weeks	6.0***	6.0***	-0.3	-0.4	-0.6	n/a	n/a	-1.3	10.7***
SA benefits ($)	-21**	-18	-17	-36	-18	n/a	n/a	-71	-111
Dependence on income support (percentage points)	13.4***	13.1***	-0.4	-2.0**	-2.1**	n/a	n/a	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table A3. Incremental impacts for Skills Development – Former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
All former claimants
2002 to 2005 Participants (n=2,115)
Employment earnings ($)	-2,068***	-2,411***	419***	1,976***	2,369***	2,163***	2,659***	9,586***	5,107***
Incidence of employment (percentage points)	-8.9***	-2.5**	2.6**	5.6***	5.2***	5.2***	5.8***	n/a	n/a
EI benefits ($)	492***	303***	9	185**	227***	235***	120	776***	1,571***
EI weeks	1.7***	0.9	-0.2***	0.5*	0.8***	0.5**	0.2	1.7*	4.2***
SA benefits ($)	49	-149**	-117*	-155**	-161**	-113*	-112*	-658**	-758**
Dependence on income support (percentage points)	9.4***	4.6***	-1.0	-2.0**	-0.9	-0.8	-1.7**	n/a	n/a
2006 to 2008 Participants (n=1,209)
Employment earnings ($)	-3,575***	-3,912***	2,668***	5,533***	6,373***	n/a	n/a	14,568***	7,075***
Incidence of employment (percentage points)	-6.7***	-0.7	5.0***	7.6***	6.1***	n/a	n/a	n/a	n/a
EI benefits ($)	1,251***	810***	44	140	272**	n/a	n/a	456*	2,517***
EI weeks	4.5***	2.5***	-0.2	0.0	0.6*	n/a	n/a	0.4	7.4***
SA benefits ($)	82	-61	-266***	-267***	-234***	n/a	n/a	-767***	-746*
Dependence on income support (percentage points)	13.6***	5.9***	-4.0***	-3.7***	-2.8***	n/a	n/a	n/a	n/a
Sub-groups of former claimants
Youth (under 30 years old) – 2002 to 2005 participants (n=933)
Employment earnings ($)	-2,158***	-2,686***	-366	554	570	453	1,017	2,228	-2,616
Incidence of employment (percentage points)	-9.1***	-4.3***	0.9***	4.9**	3.5**	3.1**	3.5**	n/a	n/a
EI benefits ($)	436***	345***	53	375***	307***	248**	1	984***	1,765***
EI weeks	1.7***	1.2***	-0.2	1.1***	0.9**	0.4	-0.3	1.9	4.7**
SA benefits ($)	17	-269***	-100	-131	-171*	-181*	-191**	-774*	-1,027*
Dependence on income support (percentage points)	9.8***	4.6***	-0.2	-0.1	0.7	-1.4	-2.9**	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table A4. Cost-benefit results from the social perspective for Skills Development**
Total Costs and Benefits Over Participation (1 to 2 years) and 6 Years Post-Program	Active claimants (n=4,610)	Former claimants (n=2,115)
Program cost	-$7,327	-$7,256
Marginal social costs of public funds	-$785	-$1,418
Employment earnings (including participant’s forgone earnings)	$15,280	$4,989
Fringe benefit	$2,292	$748
Net present value (By how much do the benefits exceed the costs 6 years after participation?)	$9,460	-$2,937
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.40	$1.70
Payback period (How many years after participation would it take for the benefits to recover the costs?)	4.4 years after participation	7.5 years after participation

Appendix B - Detailed results Skills Development - Apprenticeship

**Table B1. Socio-demographic and Labour Market characteristics of Skills Development - Apprenticeship participants**
	Active claimants		Former claimants
	2003 to 2005	2013 to 2014	2003 to 2005
Number of observations	1,339	318	73
Gender
Male	90%	97%	75%
Female	10%	3%	25%
Age
Under 25	44%	11%	21%
25 to 34	36%	55%	41%
35 to 44	13%	28%	32%
45 to 54	5%	5%	5%
55 and over	1%	1%	1%
Socio-demographic group
Indigenous individual*	11%	22%	32%
Person with disability*	1%	2%	1%
Visible minority*	6%	12%	21%
Immigrant	3%	3%	4%
Skills level related to National Occupation Code associated with the last EI claim opened before Apprenticeship participation¹
Managerial	2%	0%	1%
University	1%	0%	4%
College or apprenticeship training	77%	94%	41%
Secondary or occupational training	10%	1%	32%
On-the-job training	9%	4%	22%
Key labour market indicators in the year preceding the start of participation
Earnings²	$24,516	$31,732³	$20,343
Proportion employed	100%	100%	97%
Proportion on EI	17%	18%	52%
Proportion on Social Assistance	1%	0%	13%

Proportions may not add up to 100% due to rounding.

*Status self-reported by participant.

²Average earnings for all individuals included in the studies. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table B2. Labour Market outcomes for active claimants who started Skills Development - Apprenticeship in 2003 to 2005**
Average outcomes	Pre-program period					Program start year	After the program start year
Average outcomes	5 years pre	4 years pre	3 years pre	2 years pre	1 year pre	Program start year	1 year	2 years	3 years	4 years	5 years	6 years	7 years
Earnings including $0	$15,267	$17,529	$19,223	$21,895	$26,352	$22,987	$27,657	$34,390	$41,166	$46,149	$47,875	$50,097	$52,996
Earnings excluding $0¹	$16,069	$18,120	$19,752	$22,253	$26,428	$23,296	$28,140	$35,268	$42,899	$48,847	$51,013	$53,804	$57,320
Proportion employed	95%	97%	97%	98%	100%	99%	98%	98%	96%	95%	94%	93%	93%
Proportion on EI	18%	23%	27%	21%	19%	99%	74%	56%	40%	27%	21%	20%	17%
EI benefits	$566	$814	$936	$721	$586	$3,803	$2,769	$2,044	$1,495	$1,033	$959	$1,104	$873
Number of weeks on EI	2.16	2.86	3.3	2.47	1.92	12.63	8.49	5.68	3.97	2.68	2.46	2.72	2.1
Proportion on SA	3%	1%	2%	2%	1%	1%	1%	1%	1%	1%	1%	1%	0%
SA benefits	$96	$43	$58	$56	$22	$36	$43	$64	$57	$60	$52	$41	$27
Dependence on income support	5%	5%	5%	4%	3%	18%	12%	8%	6%	4%	4%	4%	3%
Proportion self employed	11%	10%	10%	8%	9%	10%	12%	12%	11%	12%	15%	12%	12%

n= 1,063. Data exclude individuals with no Canada Revenue Agency data for the 5 years before participation.
¹Earnings outcomes excluding individuals who reported no earnings in a given year.
SA: Social Assistance

**Table B3. Labour Market outcomes for former claimants who started Skills Development - Apprenticeship in 2003 to 2005**
Average outcomes	Pre-program period					Program start year	After the program start year
Average outcomes	5 years pre	4 years pre	3 years pre	2 years pre	1 year pre	Program start year	1 year	2 years	3 years	4 years	5 years	6 years	7 years
Earnings including $0	$15,408	$16,590	$17,718	$18,362	$21,262	$21,146	$21,419	$25,180	$28,950	$33,253	$37,331	$42,013	$41,390
Earnings excluding $0¹	$17,334	$17,380	$17,990	$19,250	$21,948	$21,146	$22,847	$26,398	$30,350	$38,899	$40,550	$46,596	$46,674
Proportion employed	89%	96%	99%	95%	97%	100%	94%	95%	95%	86%	92%	90%	89%
Proportion on EI	41%	46%	53%	45%	53%	57%	64%	49%	42%	44%	35%	33%	28%
EI benefits	$1,386	$1,509	$2,286	$2,661	$1,993	$1,965	$2,904	$2,667	$1,660	$1,391	$2,251	$1,283	$1,533
Number of weeks on EI	5.24	5.95	8.52	10.21	7.18	7.7	10.85	8.83	5.3	4.58	5.74	3.62	3.76
Proportion on SA	5%	5%	6%	8%	13%	3%	6%	0%	3%	5%	6%	7%	6%
SA benefits	$146	$88	$153	$191	$195	$66	$71	$0	$91	$219	$227	$267	$247
Dependence on income support	15%	13%	15%	18%	14%	13%	20%	16%	8%	12%	11%	10%	12%
Proportion self employed	10%	5%	6%	3%	8%	15%	6%	9%	11%	15%	11%	13%	15%

n= 66 Data exclude individuals with no Canada Revenue Agency data for the 5 years before participation.
¹Earnings outcomes excluding individuals who reported no earnings in a given year.
SA: Social Assistance

Appendix C - Detailed results Wage Subsidies

**Table C1. Socio-demographic and Labour Market characteristics of Wage Subsidies participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	334	339	880	311
Gender
Male	64%	62%	62%	63%
Female	36%	37%	38%	36%
Age
Under 25	10%	8%	13%	18%
25 to 34	33%	20%	33%	32%
35 to 44	31%	31%	28%	22%
45 to 54	21%	26%	20%	22%
55 and over	5%	15%	6%	7%
Socio-demographic group
Indigenous individual*	11%	9%	8%	17%
Person with disability*	3%	6%	4%	5%
Visible minority*	9%	15%	5%	6%
Immigrant	7%	15%	5%	8%
Marital status
Married or common-law	49%	57%	39%	44%
Widow / divorced or separated	13%	8%	15%	10%
Single	36%	34%	43%	44%
Missing data / unknown	2%	2%	3%	2%
Skills level related to National Occupation Code associated with the last EI claim opened before Wage Subsidies participation¹
Managerial	7%	7%	4%	6%
University	6%	3%	6%	4%
College or apprenticeship training	31%	24%	29%	27%
Secondary or occupational training	40%	52%	38%	41%
On-the-job training	16%	14%	22%	23%
Key labour market indicators in the year preceding the start of participation
Earnings²	$21,606	$22,679³	$10,439	$14,7393
Proportion employed	97%	99%	79%	93%
Proportion on EI	41%	27%	41%	56%
Proportion on Social Assistance	5%	3%	23%	8%

Proportions may not add up to 100% due to rounding.

*Status self-reported by participant.

²Average earnings for all individuals included in the studies. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table C2. Incremental impacts for Wage Subsidies – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=334)
Employment earnings ($)	-1,293**	2,231***	2,611***	4,713***	4,842***	4,428***	5,378***	21,972***	22,910***
Incidence of employment (percentage points)	4.7***	8.1***	8.0***	7.7***	7.8***	5.1***	5.4***	n/a	n/a
EI benefits ($)	-190	-410**	-38	-273*	-168	-129	-182	-790	-1,391*
EI weeks	-0.9	-1.1	0	-0.6	-0.5	-0.2	-0.1	-1.5	-3.4
SA benefits ($)	-57	-91	-137*	-131*	-136*	-151*	-121	-676*	-824*
Dependence on income support (percentage points)	-5.3***	-5.8***	-2.4**	-2.1*	-3.3***	-3.2**	-2.5*	n/a	n/a
2006 to 2008 Participants (n=339)
Employment earnings ($)	-768	-707	-1,345	-768	-86	n/a	n/a	-2,166	-3,531
Incidence of employment (percentage points)	3.7	10.7***	6.9*	2.6	2.7	n/a	n/a	n/a	n/a
EI benefits ($)	-407	133	-407	-751**	-251	n/a	n/a	-1,409	-1,683
EI weeks	-1.8	-0.5	-1.4	-1.8	-0.5	n/a	n/a	-3.8	-6.0
SA benefits ($)	103	61	25	123	77	n/a	n/a	225	390
Dependence on income support (percentage points)	-4.9*	-0.3	-3.0	-3.7	-0.6	n/a	n/a	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table C3. Incremental impacts for Wage Subsidies – Former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 Participants (n=880)
Employment earnings ($)	2,021***	3,350***	2,204***	1,876***	2,288***	2,661***	2,326***	11,355***	16,726***
Incidence of employment (percentage points)	14.3***	11.9***	7.1***	5.2***	6.7***	6.0***	6.7***	n/a	n/a
EI benefits ($)	243**	354***	214**	87	90	-123	60	328	925**
EI weeks	0.8**	2.2***	1.4***	0.4	0.3	-0.4	0.2	1.9	4.9***
SA benefits ($)	-495***	-667***	-426***	-337***	-336***	-333***	-347***	-1,779***	-2,941***
Dependence on income support (percentage points)	-8.2***	-5.0***	-3.0***	-3.8***	-3.9***	-4***	-3.6***	n/a	n/a
2006 to 2008 Participants (n=311)
Employment earnings ($)	2,208***	2,946***	2,003**	2,934***	3,113***	n/a	n/a	8,053***	13,205***
Incidence of employment (percentage points)	8.1***	7.3***	5.0**	5.8***	5.9**	n/a	n/a	n/a	n/a
EI benefits ($)	600***	444**	255	-50	141	n/a	n/a	346	1,390**
EI weeks	2.2***	1.6**	0.8	0.0	0.7	n/a	n/a	1.5	5.3**
SA benefits ($)	-494***	-525***	-381***	-344***	-468***	n/a	n/a	-1,193***	-2,212***
Dependence on income support (percentage points)	-3.9*	-1.9	-2.6*	-2.4	-3.1*	n/a	n/a	n/a	n/a
Sub-groups of former claimants
Youth (under 30 years old) – 2002 to 2005 participants (n=270)
Employment earnings ($)	1,409***	3,238***	1,857**	1,508	2,493**	1,847	2,506*	10,210*	14,856**
Incidence of employment (percentage points)	11.4***	8.3***	3.7	1.5	2.4	2.8	3.1	n/a	n/a
EI benefits ($)	331**	252*	439**	74	-179	-43	196	487	1,070
EI weeks	2.1***	1.7***	2.4***	0.5	-0.4	-0.1	0.7	3.1	6.9**
SA benefits ($)	-503***	-511***	-268**	-237	-187	-279**	-295**	-1,266**	-2,281***
Dependence on income support (percentage points)	-6.5***	-3.3*	0.7	-2.0	-2.8	-3.2*	-2.1	n/a	n/a

Significance level *** 1%; ** 5%; * 10%

SA: Social Assistance

**Table C4. Cost-benefit results from the social perspective for Wage Subsidies**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=334)	Former claimants (n=880)
Program cost	-$6,455	-$5,367
Marginal social costs of public funds	-$479	-$352
Employment earnings (including participant’s forgone earnings)	$21,554	$15,604
Fringe benefit	$3,233	$2,341
Net present value (By how much do the benefits exceed the costs 6 years after participation?)	$17,853	$12,226
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.30	$0.30
Payback period (How many years after participation would it take for the benefits to recover the costs?)	1.8 years after participation	0.1 year after participation

Appendix D - Detailed results Self-Employment

**Table D1. Socio-demographic and Labour Market characteristics of Self-Employment participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	572	517	162	91
Gender
Male	63%	57%	43%	51%
Female	37%	43%	57%	49%
Age
Under 25	4%	4%	6%	0%
25 to 34	29%	30%	30%	32%
35 to 44	36%	32%	35%	30%
45 to 54	23%	24%	22%	26%
55 and over	8%	10%	6%	12%
Socio-demographic group
Indigenous individual*	9%	11%	6%	11%
Person with disability*	3%	3%	2%	12%
Visible minority*	4%	5%	3%	9%
Immigrant	3%	5%	7%	0%
Marital status
Married or common-law	63%	56%	50%	63%
Widow / divorced or separated	13%	13%	17%	14%
Single	22%	27%	29%	20%
Missing data / unknown	2%	3%	4%	3%
Skills level related to National Occupation Code associated with the last EI claim opened before Self-Employment participation¹
Managerial	17%	15%	7%	11%
University	13%	12%	14%	22%
College or apprenticeship training	38%	41%	33%	40%
Secondary or occupational training	23%	25%	34%	24%
On-the-job training	9%	7%	12%	3%
Key labour market indicators in the year preceding the start of participation
Earnings²	$31,495	$29,666³	$9,413	$17,060³
Proportion employed	99%	99%	69%	88%
Proportion on EI	35%	36%	37%	70%
Proportion on Social Assistance	1%	1%	18%	6%

Proportions may not add up to 100% due to rounding.

*Status self-reported by participant.

²Average earnings for all individuals included in the studies. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table D2. Incremental impacts for Self-Employment – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=572)
Employment earnings ($)	-8,509***	-15,497***	-12,793***	-10,802***	-9,418***	-8,166***	-8,375***	-49,554***	-73,559***
Incidence of employment (percentage points)	-9.5***	-33.4***	-26.6***	-20.7***	-17.9***	-16.3***	-11.9***	n/a	n/a
EI benefits ($)	2,681***	587***	-1,053***	-864***	-567***	-654***	-453***	-3,591***	-323
EI weeks	8.7***	1.1**	-3.1***	-2.2***	-1.0***	-0.8**	-0.1	-7.2***	2.6
SA benefits ($)	28	46	15	-20	-47	-40	-52	-145	-71
Dependence on income support (percentage points)	20***	19.9***	-4.5***	-3.4***	-2.5***	-2.8***	-2.3***	n/a	n/a
2006 to 2008 Participants (n=517)
Employment earnings ($)	-9,588***	-15,775***	-12,450***	-9,940***	-9,231***	n/a	n/a	-31,621***	-56,984***
Incidence of employment (percentage points)	-10.2***	-34.0***	-26.6***	-21.3***	-19.1***	n/a	n/a	n/a	n/a
EI benefits ($)	3,210***	189	-1,364***	-1,177***	-999***	n/a	n/a	-3,541***	-141
EI weeks	7.5***	-0.2	-3.8***	-3.0***	-2.5***	n/a	n/a	-9.3***	-2.0
SA benefits ($)	57	-50*	-69**	-70*	-93**	n/a	n/a	-232**	-226
Dependence on income support (percentage points)	21.9***	14.3***	-7.7***	-6.0***	-5.2***	n/a	n/a	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

Appendix E - Detailed results Employment Partnerships

**Table E1. Socio-Demographic and Labour Market characteristics of Employment Partnerships participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	278	286	655	158
Gender
Male	70%	75%	57%	65%
Female	30%	24%	43%	35%
Age
Under 25	9%	14%	13%	15%
25 to 34	35%	33%	38%	32%
35 to 44	25%	25%	29%	28%
45 to 54	22%	18%	16%	20%
55 and over	8%	10%	5%	6%
Socio-demographic group
Indigenous individual*	19%	19%	19%	39%
Person with disability*	3%	2%	4%	4%
Visible minority*	16%	18%	16%	9%
Immigrant	9%	14%	9%	3%
Marital status
Married or common-law	49%	45%	43%	35%
Widow / divorced or separated	10%	6%	13%	9%
Single	40%	47%	42%	50%
Missing data / unknown	1%	3%	2%	5%
Skills level related to National Occupation Code associated with last EI claim opened before Employment Partnerships’ participation¹
Managerial	4%	3%	5%	4%
University	9%	1%	6%	3%
College or apprenticeship training	29%	31%	23%	30%
Secondary or occupational training	37%	39%	41%	37%
On-the-job training	20%	26%	24%	25%
Key labour market indicators in the year preceding the start of participation
Earnings²	$22,267	$21,610³	$13,778	$10,224³
Proportion employed	98%	99%	91%	88%
Proportion on EI	47%	39%	59%	56%
Proportion on Social Assistance	4%	4%	10%	13%

Proportions may not add up to 100% due to rounding.

*Status self-reported by participant.

²Average earnings for all individuals included in the studies. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table E2. Incremental impacts for Employment Partnerships – Active claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=278)
Employment earnings ($)	-3,558***	-143	2,515***	3,284***	4,571***	4,336***	4,505***	19,211***	15,509***
Incidence of employment (percentage points)	N/a	N/a	2.2	3.5*	5.4**	4.4*	4.1*	n/a	n/a
EI benefits ($)	1,807***	-419**	-297	-363**	-346	-157	-214	-1,376**	11
EI weeks	3.4***	-0.8	-0.7	-0.7	-1.2**	-1.0	-1.2**	-4.8**	-2.3
SA benefits ($)	-29	-17	-69	-64	-170**	-156**	-166**	-625*	-671
Dependence on income support (percentage points)	11.4***	-3.4**	-1.7	-1.2	-3.6***	-2.2	-2.7*	n/a	n/a
2006 to 2008 Participants (n=286)
Employment earnings ($)	-4,119***	1,531	4,126***	4,655***	5,029***	n/a	n/a	13,409***	10,683**
Incidence of employment (percentage points)	n/a	n/a	8.1***	7.5***	9.9***	n/a	n/a	n/a	n/a
EI benefits ($)	1,373***	-395	-549**	-427	-$130	n/a	n/a	-1,106*	-129
EI weeks	4.2***	-1.2	-1.8**	-1.7**	-0.6	n/a	n/a	-4.1**	-1.2
SA benefits ($)	137*	79	2	24	-$32	n/a	n/a	-7	211
Dependence on income support (percentage points)	7.4***	-4.3**	-3.6*	-3.9**	-3.7**	n/a	n/a	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table E3. Incremental impacts for Employment Partnerships – Former claimants**
Indicators	In-program period		Post-program period						Total in- and post-program
Indicators	Program start year	Additional Year	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Former claimants
2002 to 2005 Participants (n=655)
Employment earnings ($)	1,481***	2,797***	2,702***	3,079***	2,745***	2,688***	2,747***	13,961***	18,239***
Incidence of employment (percentage points)	n/a	n/a	5.7***	4.2***	5.0***	4.1**	5.2***	n/a	n/a
EI benefits ($)	173	-228**	-42	-257**	-362***	-5	-156	-823***	-878
EI weeks	0.5	-0.7*	-0.1	-0.9**	-1.2***	-0.1	-0.5	-2.8*	-2.9
SA benefits ($)	-207***	-213***	-79	-61	-98	-79	-99	-416	-836
Dependence on income support (percentage points)	-1.1	-2.9***	-1.3	-2.1*	-3.5***	-0.4	-1.8	n/a	n/a

Significance level *** 1%; ** 5%; * 10%
SA: Social Assistance

**Table E4. Cost-benefit results from the social perspective for Employment Partnerships**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=278)	Former claimants (n=655)
Program cost	-$8,259	-$7,249
Marginal social costs of public funds	-$1,094	-$363
Employment earnings (including participant’s forgone earnings)	$14,545	$17,628
Fringe benefit	$2,182	$2,644
Net present value (By how much do the benefits exceed the costs within 6 years after participation?)	$7,373	$12,661
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	$0.50	$0.40
Payback period (How many years after participation would it take for the benefits to recover the costs?)	3.9 years after participation	1.2 years after participation

Appendix F - Detailed results Employment Assistance Services

**Table F1. Socio-Demographic and Labour Market characteristics of Employment Assistance Services participants**
	Active claimants		Former claimants
	2002 to 2005	2006 to 2008	2002 to 2005	2006 to 2008
Number of observations	25,796	15,131	14,129	8,120
Gender
Male	60%	57%	54%	52%
Female	40%	43%	46%	48%
Age
Under 25	13%	12%	14%	13%
25 to 34	27%	24%	33%	31%
35 to 44	28%	25%	28%	27%
45 to 54	21%	24%	19%	21%
55 and over	11%	14%	6%	8%
Socio-demographic group
Indigenous individual*	13%	13%	17%	23%
Person with disability*	3%	5%	7%	8%
Visible minority*	8%	9%	7%	10%
Immigrant	5%	8%	5%	8%
Marital status
Married or common-law	49%	46%	36%	38%
Widow / divorced or separated	13%	13%	15%	14%
Single	37%	38%	45%	44%
Missing data / unknown	2%	3%	3%	4%
Skills level related to National Occupation Code associated with the last EI claim opened before Employment Assistance Services participation¹
Managerial	5%	6%	3%	4%
University	6%	6%	6%	4%
College or apprenticeship training	27%	26%	22%	23%
Secondary or occupational training	41%	41%	43%	43%
On-the-job training	21%	21%	26%	25%
Key labour market indicators in the year preceding the start of participation
Earnings²	$21,130	$21,633³	$10,302	$13,144³
Proportion employed	98%	99%	79%	88%
Proportion on EI	45%	44%	42%	49%
Proportion on Social Assistance	5%	5%	23%	16%

Proportions may not add up to 100% due to rounding.

*Status self-reported by participant.

²Average earnings for all individuals included in the studies. The average was calculated including participants who reported $0 earnings during that year.

³Earnings for 2006 to 2008 participants have been adjusted by the Consumer Price Index published by Statistics Canada, using 2002 as the base year.

**Table F2. Incremental impacts for Employment Assistance Services – Active claimants**
Indicators	In-program period	Post-program period						Total in- and post-program
Indicators	In-program period	1st year	2nd year	3rd year	4th year	5th year	Total post	Total in- and post-program
Active claimants
2002 to 2005 Participants (n=25,794 or a random sample of 90% of participants)
Employment earnings ($)	-2,142***	-950***	-224	-233	360**	492***	-556	-2,698***
Incidence of employment (percentage points)	0.1	0.3	0.9***	0.9***	1.3***	1.6***	n/a	n/a
EI benefits ($)	730***	-273***	-212***	-168***	-133***	-100***	-887***	-156
EI weeks	2.0***	-1.1***	-0.6***	-0.5***	-0.4***	-0.3***	-3.0***	-1.0**
SA benefits ($)	-16*	16	2	-18	-10	-14	-24	-40
Dependence on income support (percentage points)	4.5***	-1.5***	-1.2***	-1.0***	-0.8***	-0.7***	n/a	n/a
2006 to 2008 Participants (n=15,131)
Employment earnings ($)	-1,705***	-320	266	407*	n/a	n/a	353	-1,352*
Incidence of employment (percentage points)	2.3***	2.8***	3.5***	1.7***	n/a	n/a	n/a	n/a
EI benefits ($)	563***	-363***	-235***	-253***	n/a	n/a	-851***	-288***
EI weeks	1.3***	-1.4***	-0.8***	-0.8***	n/a	n/a	-3.0***	-1.7***
SA benefits ($)	20	-32*	-62***	-47**	n/a	n/a	-141***	-121*
Dependence on income support (percentage points)	2.5***	-3.6***	-2.4***	-1.7***	n/a	n/a	n/a	n/a
Sub-groups of active claimants
Youth (under 30 years old) – 2002 to 2005 participants (n=6,852)
Employment earnings ($)	-1,395***	-549***	78	400	823***	789**	1,540	146
Incidence of employment (percentage points)	0.1	0.7	1.4	0.7	1.5**	1.7***	n/a	n/a
EI benefits ($)	351***	-475***	-296***	-239***	-168***	-104***	-1,282***	-931***
EI weeks	1.1***	-1.8***	-0.8***	-0.7***	-0.5***	-0.3	-4.0***	-2.9***
SA benefits ($)	4	-33	-55**	-76***	-38	-41	-243**	-239*
Dependence on income support (percentage points)	2.6***	-3.5***	-1.5***	-1.5***	-1.5***	-0.9**	n/a	n/a
Older workers (55 years old and over) – 2002 to 2005 participants (n=2,936)
Employment earnings ($)	-2,630***	-936**	-351	120	674	674	181	-2,449
Incidence of employment (percentage points)	-0.4	-1.8*	1.7	1.9	4.4***	4.2***	n/a	n/a
EI benefits ($)	735***	-182**	-14	64	35	123**	25	761
EI weeks	2.3***	-0.8**	-0.1	0.1	0.1	0.4	-0.3	2**
SA benefits ($)	0	17	23	32	13	10	95	95
Dependence on income support (percentage points)	5.3***	-0.9	-1.2**	0.4	0.1	0.9	n/a	n/a
Long-tenured workers- 2007 to 2009 participants (n=4,998)
Employment earnings ($)	-2,751***	-769*	-739*	-454	n/a	n/a	-1,962*	-4,714***
Incidence of employment (percentage points)	0.1	1.6**	0.9	0.3	n/a	n/a	n/a	n/a
EI benefits ($)	1,008***	-174*	-274***	-227***	n/a	n/a	-675***	333
EI weeks	2.1***	-0.8***	-0.8***	-0.7***	n/a	n/a	-2.3***	-0.2
SA benefits ($)	20*	39**	45**	44**	n/a	n/a	128***	148***
Dependence on income support (percentage points)	4.8***	-0.9	-1.0**	-1.1**	n/a	n/a	n/a	n/a

Significance level *** 1%; ** 5%; * 10%

SA: Social Assistance

**Table F3. Incremental impacts related to the timing of participation in Employment Assistance Services by cohort**
Cohorts (start of EAS-only after start of an EI claim)	n=	In-program	Post-program period					Total impact post-program	Total impact in- and post- program
Cohorts (start of EAS-only after start of an EI claim)	n=	In-program	1 year	2 years	3 years	4 years	5 years	Total impact post-program	Total impact in- and post- program
Employment earnings ($)
1 to 4 weeks	5,470	-566***	-174	1,771***	2,248***	3,356***	3,896***	11,097***	10,531***
5 to 8 weeks	4,093	-1,783***	-199	846***	1,589***	2,624***	2,724***	7,585***	5,802***
9 to 12 weeks	5,051	-2,382***	-734***	211	370	1,014***	1,088***	1,950	-432
2nd quarter	4,877	-3,813***	-1,261***	-565**	-476	136	19	-2,147*	-5,960***
3rd quarter	1,508	-6,197***	-2,154***	-1,283***	-906	-330	34	-4,639**	-10,836***
4th quarter	1,059	-5,103***	270	132	809	1,348*	1,542**	4,101	-1,002
Incidence of Employment (percentage points)
1 to 4 weeks	5,470	3.0***	1.3***	1.5***	0.3	0.1	-0.7	n/a	n/a
5 to 8 weeks	4,093	1.4***	-0.1	-0.4	-1.2**	-1.3**	-2.0***	n/a	n/a
9 to 12 weeks	5,051	0.4	-1.6***	-1.6***	-2.3***	-2.6***	-3.0***	n/a	n/a
2nd quarter	4,877	-1.3***	-2.3***	-1.8***	-1.9***	-1.1*	-1.2*	n/a	n/a
3rd quarter	1,508	-5.5***	-3.1***	-3.1***	-1.2	-2.2*	-2.0*	n/a	n/a
4th quarter	1,059	-7.0***	-2.8**	-1.2	-0.3	0.9	1.1	n/a	n/a
EI Benefits ($)
1 to 4 weeks	5,470	569***	247***	-101***	-107***	-50	-29	-40	529***
5 to 8 weeks	4,093	1,571***	65	-70	-165***	-98*	-9	-277	1,294***
9 to 12 weeks	5,051	1,607***	-179***	-133***	-74*	-103**	-48	-537***	1,070***
2nd quarter	4,877	1,576***	-471***	-215***	-214***	-118**	-59	-1,077****	499**
3rd quarter	1,508	1,758***	-1,146***	-301***	-44	-15	86	-1,421***	338
4th quarter	1,059	1,019***	-1,722***	-364***	-305***	-263***	-107	-2,762***	-1,742***

* Significant at 10%; ** significant at 5%; *** significant at 1%

**Table F4. Incremental impacts of Employment Assistance Services on time of return to employment**
Cohorts (start of EAS-only after start of an EI claim)	1 to 4 weeks	5 to 8 weeks	9 to 12 week	2nd quarter	3rd quarter	4th quarter
Cohorts (start of EAS-only after start of an EI claim)	(N=5,470)	(N=4,093)	(N=5,051)	(N=4,877)	(N=1,508)	(N=1,059)
Time of Return to Employment (in weeks)	3.6***	-0.3	-1.7***	-2.7***	-2.0***	-0.9***

* Significant at 10%; ** significant at 5%; *** significant at 1%

**Table F5. Cost-benefit results from the social perspective for Employment Assistance Services**
Total costs and benefits over participation (1 to 2 years) and 6 years post-program	Active claimants (n=25,794)
Program cost	-$242
Marginal social costs of public funds	-$117
Employment earnings (including participant’s forgone earnings)	-$2,151
Fringe benefit	-$323
Net present value (By how much do the benefits exceed the costs within 6 years after participation?)	-$2,833
Cost-benefit ratio (How much does it cost in EI part II funds to achieve $1 in benefit 6 years after participation?)	Negative benefits
Payback period (How many years after participation would it take for the benefits to recover the costs?)	11.4 years after participation

Appendix G – List of nine studies included in the synthesis report

**Table G1. Overview of studies included in this synthesis report**
Study	Evidence generated	Methods	Reference period	Observation period
Analysis of EBSM Profile, Outcomes and Medium-Term Incremental Impacts for 2002 to 2005 Participants in Manitoba (Completed in 2014)	- Incremental impacts for participants including youth and older workers - Profile and socio-demographic characteristics of participants	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Statistical profiling	2002 to 2005 participants	11 to 12 consecutive years between 1997 and 2011 (5 years pre-program, 1 to 2 years in-program and 5 years post-program)
Effects of the Timing of Participation in Employment Assistance Services in Manitoba (Completed in 2014)	- Incremental impacts	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Statistical profiling
Cost-Benefit Analysis of Employment Benefits and Support Measures Delivered in Manitoba (Completed in 2016)	- Cost-benefit analysis	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Cost analysis		7 to 8 years between 2002 and 2012 (1 to 2 years in-program and 6 years post-program)
Analysis of EBSMs Profile, Outcomes and Incremental Impacts for 2006-2008 Participants in Manitoba (Completed in 2015)	- Incremental impacts - Profile and socio-demographic characteristics of participants	- Non-experimental method using propensity score matching in combination with Difference-in-Differences - Statistical profiling	2006 to 2008 participants	9 to 10 consecutive years between 2001 and 2012 (5 years pre-program, 1 to 2 years in-program and 3 years post-program)
Analysis of EBSMs Profile, Outcomes, and Incremental Impacts for EI Claimants Category “Long-Tenured Workers” in Manitoba (Completed in 2016)	- Incremental impacts - Profile and socio-demographic characteristics of participants	- Non-experimental method using propensity score matching and Difference-in-Differences - Statistical profiling	2007 to 2009 participants	9 to 10 consecutive years between 2002 and 2013 (5 years pre-program, 1 to 2 years in-program and 3 years post-program)
Study on Employment Assistance Services in Manitoba (Completed in 2014)	- Program design and delivery - Challenges and lessons learned	- 10 key informants interviews - Literature and document review	Design and delivery at the time of the data collection (2013)
Study on the Wage Subsidy Program in Manitoba (Completed in 2016)	- Program design and delivery - Challenges and lessons learned	- 6 key informants interviews - Literature and document review -Questionnaire filled by Manitoba officials	Design and delivery at the time of the data collection (2015)
Study on the Skills Development Program in Manitoba (Completed in 2016)	- Program design and delivery - Challenges and lessons learned	- 6 key informants interviews - Literature and document review - Questionnaire filled by Manitoba officials
Study on Skills Development -Apprenticeship in Manitoba (Completed in 2016)	- Program design and delivery - Challenges and lessons learned	- 7 key informants interviews - Literature and document review - Questionnaire filled by Manitoba officials

Footnotes

Footnote 1

Long-tenured workers covered in the evaluation are individuals who had long-term attachment to the labour market but not necessarily a long tenure with the same employer.

Return to footnote 1 referrer

Footnote 2

Former claimants who received maternity or parental benefits were not covered by the evaluation given the difficulty in finding a suitable comparison group.

Return to footnote 2 referrer

Footnote 3

Full details about the incremental impact methodology can be found in the following report: Stream 1 study for 2013 to 2014: national level analysis of EBSM incremental impacts. Methodology report, Evaluation Directorate, ESDC. September 16, 2013.

Return to footnote 3 referrer

Footnote 4

Government of Manitoba. October 2016. Manitoba Consultations on Labour Market Training | What We Heard Summary Report

Return to footnote 4 referrer

Footnote 5

Government of Manitoba. November 2016. Speech From the Throne.

Return to footnote 5 referrer

Footnote 6

Incremental impacts on earnings are estimated relative to pre-participation levels and to the comparison group. They are estimated using current dollars.

Return to footnote 6 referrer

Footnote 7

Fee-payers are EI eligible individuals who are authorized to maintain their entitlement to EI benefits while attending a full-time training program at their own expense.

Return to footnote 7 referrer

Footnote 8

Red Seal. 2014. Apprenticeship Completion, Certification and Outcomes. Ottawa: Red Seal.

Return to footnote 8 referrer

Footnote 9

Patrick Coe. 2013. “Apprenticeship programme requirements and apprenticeship completion rates in Canada.” Journal of Vocational Education and Training. 65(4): 575 to 605.

Return to footnote 9 referrer

Footnote 10

Sébastien LaRochelle-Côté and Sharanjit Uppal, "The Financial Well-Being of the Self-Employed," Perspectives on Labour and Income, vol. 23, no. 4, Winter 2011.

Page details

Date modified:: 2022-04-19

Evaluation of the Canada – Manitoba Labour Market Development Agreement

On this page

Alternate formats

Executive summary

Introduction

Key findings on the effectiveness and efficiency of EBSMs

Detailed quantitative findings

Challenges and lessons learned about program design and delivery

Skills Development

Skills Development - Apprenticeship

Wage Subsidies

Recommendations

Management Response

Introduction

Background

Response from Manitoba

1. Introduction

1.1 Labour Market Development Agreement background

Employment Benefits

Support Measures

1.2 Methodology

1.2.1 Qualitative Data

1.2.2 Quantitative Methods

Incremental Impacts

Cost-Benefit Analysis

1.2.3 Strengths and Limitations from the Studies

2. Evaluation Findings

2.1 Rationale and labour market context

2.2 Skills Development

2.2.1 Program Description

2.2.2 Program delivery

2.2.3 Targeting to Labour Market Demand

2.2.4 Profile of Skills Development participants

Barriers faced by participants

2.2.5 Incremental Impacts

Active claimants

Former claimants

2.2.6 Cost-benefit results

2.2.7 Challenges and lessons learned about Skills Development design and delivery

2.3 Skills Development - Apprenticeship

2.3.1 Program description

2.3.2 Program Delivery

2.3.3 Profile of Skills Development - Apprenticeship participants

Socio-demographic characteristics

2.3.4 Labour Market outcomes

Active claimants

Former claimants

2.3.5 Challenges and lessons learned about Skills Development - Apprenticeship design and delivery

2.4 Wage Subsidies

2.4.1 Program description

2.4.2 Program delivery

2.4.3 Profile of Wage Subsidies participants

2.4.4 Incremental impacts

Active claimants

Former claimants

2.4.5 Cost-benefit results

2.4.6 Challenges and lessons learned about Wage Subsidies design and delivery

2.5 Self-Employment

2.5.1 Program description and delivery

2.5.2 Profile of Self-Employment participants

2.5.3 Challenges in measuring Self-Employment incremental impacts

2.6 Employment Partnerships

2.6.1 Program description

2.6.2 Profile of Employment Partnerships Participants

Socio-demographic characteristics

2.6.3 Incremental Impacts

Active claimants

Former claimants

2.6.4 Cost-benefit results

2.7 Employment Assistances Services

2.7.1 Program description and delivery

2.7.2 Profile of Employment Assistance Services participants

Socio-demographic characteristics

Labour Market barriers faced by Employment Assistance Services participants

2.7.3 Incremental impacts

2.7.4 Cost-Benefit results

2.7.5 Challenges and lessons learned about Employment Assistance Services design and delivery

3. Comparison of key findings by program type

4. Conclusion

5. Recommendations