Evaluation of Pathways to Education, final report

The program

Pathways to Education is a charitable organization that provides after-school programming to help at-risk youth in low-income communities graduate from high school and successfully transition into post-secondary education or training. Currently there are 20 sites providing services to over 5,000 students across Canada. Pathways to Education collaborates in partnership with governments, community partners, schools, school boards and hundreds of diverse volunteers to break the cycle of poverty through education and by enabling strategic, long-term social change.

Pathways to Education funding comes from various sources, including a named grant of $31 million to Pathways to Education between 2014 to 2015 and 2017 to 2018 from Employment and Social Development Canada.

Findings

The evaluation found that:

• Pathways to Education programming addresses a demonstrable need by providing support to disadvantaged youth to complete their education, transition to a successful future and better reach their potential.
• Pathways to Education contributed to increased enrollment to post-secondary education among participants.
• Adapted programming in Winnipeg positively affected Indigenous youth.
• Pathways to Education had a direct positive effect on high school graduation rates, enrollment to post-secondary education and educational attainment.
• The program also positively affected eligible participant's labour market outcomes.

Moreover, the evaluation determined that Pathways to Education programming has a net positive social return on investment. Using the incremental impacts for Pathways-eligible students from 1 location (Regent Park) over a comparable population of students at locations not served by Pathways to Education programming as a proxy, the total net social benefit (to both individual participants and to governments) of an intervention for a single student represented a return on investment of approximately 50% over 25 years. Based on this analysis, an investment in the program “breaks even” (where the benefits equal the costs) after 22.5 years from a government perspective and after 20.8 years from a total social net benefit perspective.

Lessons learned and conclusions

The evaluation derived 3 key lessons learned applicable not just to the Pathways to Education program, but also other similar programming:

1. Targeted early interventions positively affect vulnerable populations.
2. A key component of the success of the Pathways to Education model is the flexibility to adapt to local needs and conditions.
3. Early interventions with vulnerable populations can generate long-term net benefits not just for individuals but also for governments.

As such, the evaluation determined that programming of this nature—designed to support early interventions to help vulnerable youth complete high school—has a valuable role to play.

Finding 1: Overall, Pathways programming is contributing to a demonstrable need by providing support to disadvantaged youth in order to complete their education, transition to a successful future and better their potential

Learning Branch is pleased that Pathways' work to address the educational barriers that disadvantaged youth are facing was recognized in the evaluation. Their model, which is based on various types of supports (i.e., academic, social, financial and one-on-one mentoring), contributes to a more equitable participation of Canadians in post-secondary education.

Finding 2: Pathways programming contributed to increased enrollment to post-secondary education

Learning Branch is pleased that Pathways' impact on enrollment to post-secondary education is highlighted by this report. Given an on-time high school graduation rate of 79%, and a cumulative rate (since the launch of the program) of 73% of Pathways graduates successfully transitioning to post-secondary education, it can be inferred that the overall post-secondary education participation rate for Pathways participants is approximately 58%. By way of comparison, a recent report by Statistics Canada indicates that approximately 47% of 19 year olds in the lowest income quintile and approximately 57% of 19 year olds in the second-lowest income quintile participated in post-secondary education in 2015. ^{Footnote 1}

Finding 3: Adapted programming in Winnipeg positively affected Indigenous youth

Learning Branch agrees with this finding and recognizes the importance of offering culturally adapted programming to Indigenous students. Other Pathways' sites that also serve large proportions of Indigenous youth have been able to tailor their program as well, for example offering to middle school students to meet the needs of the community (e.g., Mashteuiatsh, Quebec).

Finding 4: Pathways positively affected high school graduation rates

Learning Branch recognizes the positive impact of Pathways on high school graduation rates. This success was demonstrated through the tracking of on-time graduation rates before and after the introduction of Pathways in the community. The introduction of the Pathways program has correlated with a significant change in graduation in some communities post-Pathways (e.g. in Kingston, Ontario, the rate of on-time graduation increased from 40% before the program was introduced to 79% for the 2015 to 2016 school year^{Footnote 2}). In fact, an average of 79% of all Pathways' participants complete high school on time.

Finding 5: Pathways had a positive effect on enrollment to post-secondary education and educational attainment

Learning Branch is pleased with the findings of the evaluation which used research commissioned by the Learning Branch to demonstrate Pathways' impact on educational attainment in Regent Park. Since the study that was originally conducted by Drs. Lavecchia and Oreopoulos, more time has passed and additional data is available making it timely to confirm the positive impact of Pathways on participants go beyond its initial site in Regent Park. As such, Learning Branch plans to commission additional research that would examine the impact of Pathways at 2 of its expansion sites in Toronto (Rexdale and Lawrence Heights). This research would also explore the data feasibility of the impact of Pathways on social outcomes such as reducing health inequality and crime as well as examining the feasibility of studying expansion sites outside of Toronto. The Rexdale and Lawrence Heights sites were selected because the maturity of their program would allow for a better longitudinal view of outcomes of Pathways participants. Having both opened in 2007, the first cohort of students from these sites would now be approximately 24 years old.

Finding 6: Pathways positively affected labour market outcomes

As noted in finding #5, a small sample was used to consider Pathways' impacts on labour market outcomes. However, given that the first expansion sites (Lawrence Heights and Rexdale) are now at an appropriate age of maturation, Learning Branch plans to pursue additional research to examine other sites to broaden the sample size and expand on research available on Pathways' impact on labour market outcomes for students.

Lesson learned 1: Targeted early interventions can positively affect vulnerable populations

Learning Branch welcomes the first lesson learned through the evaluation. Early interventions have proven to be a successful means to increase academic success and improve labour market outcomes for underrepresented populations. While the Pathways model has proven to be particularly effective, the organization continues to be very intentional in its growth. Their model requires extensive community engagement and takes an average of 2 to 3 years to open a new site. With 20 sites currently in operation, mostly in larger urban areas, both Learning Branch and the organization recognize that there are still many communities that could benefit from Pathways-like supports. In order to increase capacity, Employment and Social Development Canada will explore with Pathways and other organizations ways to address this gap and provide similar opportunities to underrepresented youth who do not have access to Pathways' programming including youth in rural, remote, and Indigenous communities.

Lesson learned 2: A key component of the success of the Pathways model is the flexibility to adapt to local needs and conditions

Learning Branch agrees with this finding. The flexibility and decentralized nature of the Pathways model is critical to allowing the program to be easily adapted to account for regional diversity and provide culturally relevant content for students. Most importantly, the Pathways model is student centered focusing on students' needs and is delivered by local, trusted community partners.

Lesson learned 3: Early interventions with vulnerable populations can generate long-term benefits not just for individuals but also for governments

As the evaluation found and consistent with research produced by Drs. Levecchia, Oreopoulos and Brown, it is clear that early learning interventions with underrepresented populations can generate long-term benefits at both the individual and societal levels. Additional research including validation of the results of the aforementioned study by expanding the sample to additional sites in Lawrence Heights and Rexdale and studying additional factors by including crime and health data, will further enhance our understanding of the long-term benefits of Pathways.

1.1 Pathways: program objectives

Program description

Pathways is a charitable organization founded in 2001 that provides comprehensive after-school programming to help at-risk youth in low-income communities graduate from high school and successfully transition into post-secondary education or training.

Figure 1: Pathways components – Text version

Pathways consists of 3 elements: Tutoring, Mentoring sessions, and Financial support. Combined, these 3 components lead directly to Improved Likelihood of Graduating High School, which flows, into Improved access to post-secondary education.

Pathways collaborates in partnership with governments, community partners, schools, school boards, and hundreds of diverse volunteers. Working with these partners, Pathways aims to help “break the cycle of poverty through education and enable strategic, long-term social change” (retrieved from Pathways).

It operates within the community alongside the school system and provides after-school tutoring, mentoring, and financial assistance to address the barriers that can stand in the way of education.

Each student benefits from a personal relationship with their Student/Parent Support Worker who motivates and guides students and their families, and ensures students' accountability to the contract they sign as a condition of participation in the Program.

Pathways has a policy of equality, inclusion, and accessibility, and is available to all students of high school age within the communities served by Pathways. With average sign-up rates of more than 80%, Pathways to Education is a collective community-based movement for education.

Pathways has the following objectives:

• Reduce poverty and its effects by lowering the high school dropout rate and increasing access to post-secondary education among disadvantaged youth in Canada
• Help youth from disadvantaged communities set higher aspirations for their future
• Ensure that a greater number of youth can fully benefit from Pathways by expanding direct supports to participants and by expanding the organization's capacity to deliver more services to more participants countrywide

Expected results of the funding agreement include:

• Improved high school graduation rates and labour market outcomes for youth at-risk of leaving high school before completion
• Expand its supports to reach more students in more areas of the country.
• Expand employability programming for participants
• Develop a sustainability plan that will support knowledge transfer and sustainable reach

Program sites

As of 2017, Pathways is located in Ontario, Quebec, Nova Scotia, Manitoba, and British Columbia with sites in 20 communities serving over 5,000 students. Programming in Alberta, Saskatchewan and New Brunswick are expected to be accepting students soon.

Source: Pathways to Education 2017 Results Summary.

Figure 2: Program site map – Text version

The image depicts Pathways locations across Canada, the year of establishment, and the number of students as of 2015 to 2016.

Pathways to Education locations	Year established	Number of students
Halifax, Nova Scotia	2010	269
Saint John, New Brunswick	Opening soon	0
6 locations within the province of Quebec	2007 to 2014	984
8 locations within the province of Ontario	2001 to 2010	3,810
Winnipeg, Manitoba	2010	321
Saskatoon, Saskatchewan	Opening soon	0
Edmonton, Alberta	Opening soon	0
Vancouver, British Columbia	2017	103

1.2 Pathways funding

As a registered charitable organization, Pathways has developed many formal and informal partnerships across Canada with local governments, private sector groups and other not-for-profit organizations to deliver their programming. Its funding comes from various sources, including the Government of Canada, provincial governments, charitable foundations and private companies^{Footnote 3}. The Department provides annual funding to Pathways through a named grant. Between 2014 to 2015 and 2017 to 2018, the Department provided $31 million to Pathways.

2.1 Evaluation scope, methodology, and limitations

This evaluation examines the extent to which Pathways is relevant, achieved its outcomes, and the impact it had on its eligible students. The Department synthesized 3 reports described below to address the evaluation reporting requirements of the Financial Administration Act and the Policy on Results.

1. The evaluation of the Pathways to Education program: In 2016 to 2017, Pathways externally contracted this evaluation as stipulated under its agreement with the Department. The report focused on general evaluation issues, program effects and the design and delivery of the program. This line of evidence applied a methodology that compared outcomes between Pathways-eligible students (treatment group) to on-eligible students (comparison group)^{Footnote 4} and generated qualitative data from key informant interviews, focus groups and 3 surveys^{Footnote 5}.
2. The long-term impacts of the Pathways to Education program: This is an academic study^{Footnote 6} prepared by Adam M. Lavecchia, Philip Oreopoulos and Robert Brown for the Department in 2018. It examined the incremental impacts of the program covering the period between 1999 and 2013. It compared the outcomes of all adolescents from Regent Park that were eligible for the program with the outcomes of adolescents from other Toronto public housing sites, both before and after the introduction of Pathways. A regression analysis was also used to estimate the impacts on earnings and employment^{Footnote 7}.
3. The returns to completing high school Education: This Department-generated line of evidence literature review focused on the economics of completing high school education as well as individual and societal benefits. It reported on the frequency of not completing high school in Canada; labour market benefits from completing high school; and the benefits of completing post-secondary education.

3.1 Relevance

Relevance is the extent to which a program, policy or other entity addresses and is responsive to a demonstrable need.

Overall, Pathways programming is contributing to a demonstrable need by providing support to disadvantaged youth in order to complete their education, transition to a successful future and better reach their potential^{Footnote 8}. The discussion that follows describes the challenges facing children living in poverty and the consequences specifically associated with not completing high school.

In 2013, more than 15% of Canadian children overall, and as high as 40% among Indigenous children, lived in poverty. According to the Goss Gilroy evaluation report, children living in poverty are at high risk of developing health issues, developmental delays and behaviour disorders and are more likely to have lower levels of education and remain in poverty as adults. This group is also most likely to experience, among other things, interrupted school attendance, lower university attendance, increased risk of illiteracy, and higher dropout rates^{Footnote 9}.

While a minority of the total population, a significant portion of the population fails to complete secondary school. For example, in 2011 to 2012, Canada's dropout rate was 7.8% nationally with the dropout rates for men being higher (9.7%) than for women (5.9%)^{Footnote 10}. Moreover, the Literature Review also suggested that Canadian youth at risk of dropping out of high school are among the most vulnerable in our society. Figure 3 illustrates the potential consequences of dropping out of high school^{Footnote 11}.

Figure 3: Potential consequences of dropping out of high school – Text version

Dropping out of high school potentially leads to Increased incidence of marital instability; Reduced lifetime earnings and Increased unemployment. It also potentially leads to Early childbearing, Delinquency and Substance abuse. As well, dropping out of high school could lead to a Reduced quality of life, Poor health, Crime, and Economic dependency.

As well, youth who drop out of high school experience lower wage level increases than those who completed high school, report lower wage satisfaction, and were less likely to be employed by age 24^{Footnote 12}. For example, the present value of lifetime earnings is $129,000 higher for men and $104,000 higher for women with a high school diploma, relative to high school dropouts^{Footnote 13}.

These factors all suggest a continuing relevance for programs and policies designed to help vulnerable youth complete secondary school.

3.2 Outcomes

An outcome, under the 2016 Policy on Results, is a change or consequence attributable to outputs or to which outputs or other outcomes of the program contribute.

This section focuses on the program's outcomes with respect to increasing the number of participants entering post-secondary education and helping Indigenous youth. The evidence presented below suggests that Pathways is achieving these outcomes.

Pathways positively affected high school graduation rates

Across observed Pathways sites, there was a noticeable improvement in graduation rates compared to non-eligible students^{Footnote 22}. Graduation rates were 19.1% higher for participants at the Ontario sites on average. The graduation rates in the Halifax and Verdun sites were also higher for participants (28.4 percentage points and 10.4 percentage points, respectively)^{Footnote 23}.

This finding is supported by a separate study, conducted in 2017, which also attributed an increase in graduation rates to participation in Pathways^{Footnote 24}. It estimated that eligibility in Pathways led to an increase of 35% in high school graduation rates for Pathways-eligible Regent Park students compared to non-eligible students from Toronto public housing projects.

Figure 4, below, also demonstrates how Pathways positively influences secondary school completion. Across 10 sites where Pathways was present in the 2015-16 academic year, there was a noticeable increase in the share of students who graduated no more than 1 year beyond the province's standard secondary school term (e.g., within 4 years where high school comprises grades 10, 11, and 12) than was the case in the year before Pathways was introduced^{Footnote 25}. This type of comparison cannot attribute the observed increases to Pathways, but the improvement does suggest that the Program is likely contributing to greater on-time graduation at these sites.

Source: Pathways to Education 2017 Results Summary.

Figure 4: On-time graduation rates by site, 2015-16 – Text version

The graph depicts the on-time graduation rates, before and after the Pathways program, by site, over 2015 to 2016. These results compare the 2015 to 2016 results of a Pathway site to the results in the year prior to the introduction of Pathways. The definition of “on-time graduation rates” appears in the in-set of the graph.

	Pre-pathways	Pathways
Halifax	54	89
Hamilton	59	72
Kingston	40	79
Kitchener	54	77
Montreal (Verdun)	19	63
Ottawa	52	80
Toronto (Lawerence Heights)	54	73
Toronto (Regent Park)	44	83
Toronto (Rexdale)	46	72
Toronto (Scarborough Village)	57	79

Pathways had a positive effect on enrollment to post-secondary education and educational attainment

In addition to the improved graduation rates noted above, the average Pathways-eligible Regent Park^{Footnote 26} student claimed more post-secondary tuition expenditures for university or college. Figure 5 illustrates that between the ages of 19 and 23, the average Pathways-eligible student had higher tuition expenditures than an average non-eligible student indicating that more Pathways-eligible students participated in a post-secondary education. As well, the regression estimates indicate that eligibility for Pathways increases postsecondary education attainment by approximately 0.5 years.

Source : Lavecchia, Adam M., Philip Oreopoulos et Robert Brown. Long-term impacts of the Pathways to Education program. 2018

Figure 5: 2013 average tuition expenditures for Pathways-eligible and non-eligible students – Text version

The graph depicts the comparison between Pathways-eligible students at Regent Park location, and the non-eligible students associated with other public housing, in terms of average expenditures on tuition, in 2013.

Age	Eligible Pathways Regent Park students	Non-eligible students (other public housing)
19	$760	$635
20	$1,265	$727
21	$1,352	$744
22	$1,320	$689
23	$950	$477
24	$587	$303
25	$259	$316
26	$139	$278

Pathways positively affected labour market outcomes

Employment earnings for Pathways-eligible Regent Park students were higher than non-eligible students following graduation^{Footnote 27}. Regression analysis from the Lavecchia study found that, on average, these students earned less than non-eligible students in the comparison group (by an average of $638 annually over 5 years) between the ages of 19 and 23, (see Figure 6); however, at age 24, they earned $12,678 compared to $11,818 for non-eligible students or approximately $950 more. By age 26, Pathways-eligible students earned $1,207 more per year on average^{Footnote 28}.

Source : Lavecchia, Adam M., Philip Oreopoulos and Robert Brown. Long-term impacts of the Pathways to Education program. 2018

Figure 6: Difference between Pathways-eligible and non-eligible students’ earnings – Text version

The graph shows that the employment earnings of Pathways-eligible Regent Park students were higher than the earnings of the non-eligible students, after graduation. On average, these students earned less than did the non-eligible students in the comparison group between the ages of 19 and 23. However, at age 24, they earned more as compared to the non-eligible students. By age 26, Pathways-eligible students continued earning comparatively more per year, on average.

Age	Difference in salary
19	- $955
20	- $563
21	- $43
22	- $1,287
23	- $340
24	$950
25	$1,179
26	$1,207

Pathways also positively affected the employment rate of young adults eligible to participate in the program (Figure 7)^{Footnote 29}. At age 19, there was no difference between the Pathways-eligible and non-eligible students. By age 21, their employment rate was 5 percentage points higher compared to non-eligible students; and by age 26, it was more than 6 percentage points higher than non-eligible students.

Source: Lavecchia, Adam M., Philip Oreopoulos and Robert Brown. Long-term impacts of the Pathways to Education program. 2018

Figure 7: Difference of employment rate between Pathways-eligible students and non-eligible students – Text version

The graph depicts how Pathways positively affected the employment rate of young adults eligible to participate in the program. At age 19, there was no difference between the Pathways-eligible and non-eligible students. By age 21, their employment rate was 5 percentage points higher compared to non-eligible students; and by age 26, it was more than 6 percentage points higher than that of the non-eligible students.

Age	Difference in employment rate
19	0%
20	2%
21	5%
22	3%
23	6%
24	6%
25	2%
26	6%

In addition to the improved incidence of employment, the regression analysis found that Pathways-eligible students made less use of social assistance and employment insurance benefits than ineligible students, on average. Specifically, they decreased their likelihood of receiving social assistance by 5.1 percentage points and employment insurance by 5.8 percentage points when compared to non-eligible students. As well, they received $525 less social assistance and $183 less employment insurance than similar non-eligible students^{Footnote 30}.

3.4 Costs and benefits

This section presents benefits and costs associated with participation in the program that arise from improving high school completion rates and increasing post-secondary participation. As the program is focused on early interventions, the effects and benefits can be long-lasting and require a longer time horizon to fully assess the impacts.

The costs and benefits of Pathways accumulate to both individual participants and to governments over the short- and long-term (see Appendix B for further details). The impacts of the program begin with participation in Pathways, which results in students improving their high school completion rates.

Using Pathways-eligible Regent Park students as a proxy and comparing them to similar student population without access to Pathways programming, participation in Pathways increased the individual's lifetime earnings, even after factoring in the additional costs associated with higher levels of education and reductions in Employment Insurance and Social Assistance benefits received.

From the individual's perspective, there are short-term costs that arise from participation, such as the direct costs of post-secondary education and the opportunity costs of foregoing employment to increase educational attainment, which can be offset by short-term benefits such as education subsidies and improved skills/employability.

Long-term benefits for individuals consist of greater lifetime earnings and improved social outcomes. Pathways helps at-risk youth graduate high school, thus improving their skills and outcomes. Pathways participants are subsequently less likely to rely on government supports and may pay more taxes as their labour market attachment strengthens, which in turn may moderate (but does not completely offset) their life-time improvement in earnings.

From the government's perspective, short-term costs consist of program costs and costs associated with supporting higher levels of education. Benefits to the government begin in the medium-term and extend into the long-term. These benefits include increased tax revenues from greater lifetime earnings of participants; decreased outlay on Employment Insurance, Social Assistance and other labour market programming; and, a potential decrease in spending on social programs, such as justice and health, reflecting the improved outcomes of participants. Not all of these benefits—particularly those reflecting improved social outcomes—are easily quantifiable.

Figure 8 illustrates the costs and benefits of Pathways associated to governments and an individual student over a longer period of time. The incremental impacts were determined by comparing the outcomes of Pathways-eligible students to non-eligible students in Regent Park^{Footnote 31}. This analysis was estimated by comparing costs and benefits associated with individuals participating in Pathways compared to individuals who did not participate.

• * Components may not sum to reported total due to rounding
• Chart Source: ESDC staff estimates based on results found in Lavecchia et al. (2018). “Long Term Impacts of the Pathways Education Program” for ESDC.

Figure 8: Costs and benefits per student over 25 years, net present value – Text version

The graph shows how costs and benefits of Pathways accrue to both individual participants and to governments, over the short- and long-term. From individuals' perspective, short-term benefits such as education subsidies and improved skills/employability can offset short-term costs incurred due to participation. Long-term benefits for individuals consist of greater lifetime earnings and improved social outcomes. From the government's perspective, short-term costs consist of program costs and costs of supporting higher levels of education. Benefits to the government begin in the medium-term and extend into the long-term, and include the following: increased tax revenues from greater lifetime earnings of participants; decreased outlay on Employment Insurance, Social Assistance etc. and a potential decrease in spending on social programs, such as justice and health of individuals.

Governments	Real cumulative impact (discounted at 3% per year)
Program Costs	-$17,920
Post-Secondary Tax Credit	-$1,720
Social Assistance and Employment Insurance	+$15,250
Increased Government Revenues	+$6,400
Total Governments Net Benefit	+$2,010

Individual	Real cumulative impact (discounted at 3% per year)
Reduced Wages During Studies	-$3,022
Post-Secondary Tuition	-$2,370
Post-Secondary Tax Credits	+$1,430
Net Post-studies Earnings	+$22,140
Total Individual Net Benefit	+$5,480
Total net benefit (Governments plus Individual)	+$7,490

While the Pathways' program has total costs ($17,920) in the short-term^{Footnote 32} from the government perspective (from all sources of funding), the long-term approach reveals a net benefit ($2,010) due to decreased outlays and increased revenues.

In the short-term, Pathways-eligible students incur tuition costs related to post-secondary education and foregone earnings due to decreased employment while attending school. Their benefits include a tuition tax credit remitted to individuals attending a college or university and greater net earnings from having pursued a post-secondary education.

The individual's total net benefit is about $5,480. The total net social benefit to society of an eligible student participating in Pathways over 25 years is $7,490 compared to a non-eligible student. This results in a social return on investment for Pathways of 50.1% over 25 years or a compound annual growth rate of 1.6%.

As a result, an investment in the program “breaks even” (where the benefits equal the costs) after 22.5 years from a government perspective and after 20.8 years from a total social net benefit perspective.

As noted above, other potential gains, such as improved well-being for individuals related to better health outcomes and improved social outcomes (for example, reduced interactions with the criminal justice system), may also accrue to individuals. A number of studies suggest that completing high school is linked to social and economic benefits later in life^{Footnote 33}.

Though not quantifiable for the purposes of this analysis, international studies have shown that secondary school completion is correlated with:

• Large and significant decrease in the likelihood of both conviction and incarceration
• Small, but significant, improvements in health outcomes
• Increased civic engagement with the probability of voting rising by 28 to 34 percentage points
• Generating significant social savings through crime reduction

4.1 Lessons learned

Pathways is an organization that assists Canadian youth living in poverty and who are at risk of dropping out of high school. Pathways help youth graduate from high school and transition into post-secondary education so they can reach their full potential.

The lessons learned— applicable to both Pathways and similar programming—are as follows:

Projections: Time horizon and discount rate

All estimates assume a 25-year time horizon and an annual discount rate of 3%.

A discount rate of 3% is consistent with the discounting done in Lavecchia et al. (2018)^{Footnote 34} and other similar studies that examine the impacts of education interventions on earnings (Krueger, 1999; and Chetty et. al., 2011). This rate is also in line with Boardman et al. (2008) and the interim TBS guide for Cost-Benefit Analysis (the “TBS Guide”). The TBS Guide, which cites Boardman et al. (2008), suggests a social discount rate of 3% accompanied by a shadow price of investment applied to all the costs that crowd out investment activity^{Footnote 35}. Since Pathways does not invest in capital and is not a regulation, the 3% rate is applied without modification.

Components of the net benefit analysis

The components of the net benefit analysis are discussed in the order that they appear in Figure 8 (from left to right) in section 3.4 of the report.

Below are the costs and benefits from the perspective of governments. All values include the Marginal Social Costs of Public Funds, the distortionary loss incurred by society when raising additional taxes to fund government spending. This effect increases the magnitude of both the costs and benefits associated with a government program. As a result, this effect will be positive (net benefit) if a program generates net savings, but negative (net cost) if it falls short. As in previous evaluations produced by Employment and Social Development Canada (see the 2017 Evaluation of the Labour Market Development Agreements), the effect is assumed to be 20%, this is to say that each dollar of net costs or net benefits generated by a program or an intervention is increased by 20 cents.

Program costs

This is based on the total program costs per participant ($14,935) reported in Lavecchia et al. (2018) and applies to Regent Park Pathways-eligible students.

Funding to cover these program costs includes not only the direct contributions of the Government of Canada, but also contributions from the Government of Ontario and private donations.

As these sources cannot be disentangled from the final program costs, the Marginal Social Costs of Public Funds is applied to the entire amount and may represent an overestimate as a result. The Marginal Social Costs of Public Funds adds an additional $2,987 (20% of $14,935) for a total cost of $17,920 per student. While this cost will be spread out over multiple years, it is treated as an upfront expense and the amount is not discounted.

Post-secondary tax credit

Tuition expenses are reported in Lavecchia et al. (2018). This value is the difference in annual average tuition tax credits between participants and non-participants between the ages of 19 and 26. It represents a cost to governments due to the increased take-up of post-secondary education. After discounting, this amounts to $1,433. The Marginal Social Cost of Public Funds increases this expense by 20% for a total cost of $1,719 for the analysis period.

Related, though not included, are the potential additional institutional costs associated with a student continuing their studies beyond secondary school. The incremental cost of an additional student is likely minimal for the post-secondary educational system, but it is conceivable that sufficient scaling of the program could increase system-wide pressures due to increased demand.

Social assistance and employment insurance

Lavecchia et al. (2018) reports a reduction in social assistance use among Pathways-eligible students compared to non-eligible students of $525 in 2013. For this analysis, this reduction is assumed to be constant over the entire analysis period.

Similarly, Lavecchia et al. (2018) reports that in 2013 a reduction in the use of Employment Insurance among Pathways-eligible students compared to non-eligible students of $183. This is also assumed to be constant over the entire analysis.

The total combined savings is $12,706 after discounting.

These may be middling estimates, as benefit use may change as a result of changes in an individual's income over time. Incremental impacts related to Social Assistance, for example, may widen as incomes and outcomes diverge. In the case of Employment Insurance, it is unclear if benefits received by participants would increase or decrease relative to non-participants over the 25-year period. On the one hand, participants appear to have greater labour market attachment and may earn more, entitling them to greater benefits during a job separation. On the other hand, participants may experience fewer extended periods of unemployment, leading to less dependency on Employment Insurance.

Because it is a net benefit to governments, the Marginal Social Cost of Public Funds increases the size of this benefit by 20%. This results in a net benefit to governments of $15,248 over the analysis period.

Increased government revenue

Using forecasted earnings underlying the values for Net Post-Studies Earnings of individuals (see below), the incremental taxes paid are estimated based on similar incremental impacts observed for Ontario participants in Employment Benefits and Support Measures delivered through the Canada-Ontario Labour Market Development Agreement. For example, administrative data for tax years 2002 through 2012 showed that where there was an incremental impact on earnings of just over $1000 in a year, it resulted in increased income taxes paid of roughly $140 compared to the control population. This was calculated from the age of 24 to 43, as between 19 and 23 the forecasted earnings for both participants and non-participants were below or around the basic personal amount. The total incremental amount for income taxes came to $3,807 after discounting.

In addition to the gains from income taxes, the analysis included estimates of the incremental impact in revenues due to combined Federal and Provincial sales taxes. As in the 2017 Evaluation of the Labour Market Development Agreements, it is assumed that each dollar in incremental earnings results in 48 cents of spending on taxable goods (i.e., a propensity to consume of 96% with an assumed breakdown of half on taxable and half on non-taxable goods and services). With a Harmonized Sales Tax (HST) rate of 13%, this results in a little over 6 cents of sales tax for each incremental dollar in earnings. The total incremental amount for sales taxes came to $1,526 after discounting.

Because it is a net benefit to governments, the Marginal Social Cost of Public Funds increases the size of these benefits by 20%. Combined, this results in $6,400 in benefits due to increased tax revenues.

Changes in payroll taxes, such as Employment Insurance Premiums, are not included in this estimation.

Below are the costs and benefits from the perspective of the individual participant. Unlike the costs and benefits from the governments' perspective, the costs and benefits experienced by the individual participant are not affected by Marginal Social Costs of Public Funding. As was the case above, specific values presented below are prior to application of the discount rate.

Reduced wages during studies

Lavecchia et al. (2018) report average annual earnings for eligible students and non-eligible students from ages 19 to 26. Pathways-eligible students have lower average annual earnings than non-eligible students between the ages of 19 and 23. The difference in annual earnings between these 2 groups between ages 19 and 23 is taken to be the estimated opportunity cost of participating in the program and assumed to be associated with increased likelihood participation in schooling. The total difference is $3,022 after discounting.

Post-secondary tuition

Lavecchia et al. (2018) provides average tuition costs incurred by program eligible students between ages 19 and 26. Eligible students are more likely to pursue post-secondary education, and incur greater tuition expenses. After discounting, this amounts to $1,433 for the analysis period.

This value only reflects what has been reported and claimed according to the tax records and may understate actual incremental expenditures. Further amounts are not projected beyond the age of 26.

Social assistance and employment insurance

Lavecchia et al. (2018) reports a reduction in 2013 of social assistance use among Pathways-eligible students compared to similar students at sites where the Program is unavailable of $525. This is assumed to be constant over the entire analysis period.

Similarly, Lavecchia et al. (2018) reports a reduction in the use of Employment Insurance in 2013 among Pathways-eligible students compared to similar students at sites where the Program is unavailable of $183. This is assumed to be constant over the entire analysis period.

The total combined savings is $12,706 after discounting.

From the individual participant's perspective, changes in Social Assistance and Employment Insurance income are a reduction in lifetime earnings and represent a cost.

It should be noted that this method treats a dollar collected in Social Assistance or Employment Insurance as being of equal value to a dollar earned from employment. A potential concern of this approach is that it does not consider the possibility that income from employment earnings may be preferred to income from Social Assistance and Employment Insurance, as employment earnings may be a reflection of self-sufficiency, well-being, and greater participation in society.

Net post-studies earnings

Lavecchia et al. (2018) finds that from the age of 24 to 26 (inclusive), eligible students have higher average annual earnings than non-eligible students. This increases from $950 at age 24 to $1,207 at age 27.

Earnings for participants and the divergence in earnings between the 2 group from age 27 ($1,369) to age 43 ($3,425) are extrapolated from the above results. The forecasted earnings for participants is the sum of these 2 extrapolated values. See Appendix C: Lifetime Earnings Assumptions for the Net Social Cost-Benefit Analysis for more details.

The divergence in incomes does result in a larger gain than assumed by Lavecchia et al. (2018), as the authors assume a constant gap of $1,207 after age 27.

From the total increased post-studies earnings, the additional amount paid in taxes (see Increased Government Revenues above) is removed. The net increase in post-studies earnings is $22,144 after discounting.

Taken together, the social rate of return is calculated as the total net benefits of individuals and government ($2,010 + $5,480 = $7,490) divided by the unadjusted program costs ($14,935), which equals 50.1% over 25 years or an annualized rate of return of 1.6%.

Beyond year 25

The 25-year time horizon was selected in order to illustrate the effects of early interventions over a longer period. Even at 25 years, the constrained time period likely underestimates the full discounted benefits of a program such as Pathways.

The most obvious underestimation relates to improved lifetime earnings. Pathways, by definition, involves an intervention at a point very early in a person's life. As such, the expected earnings differential should persist well beyond the 25-year time horizon (when the participant will be only 43 years old) and may even continue to expand. This would have implications not only for the individual—who would see net earnings increase—but also on the Governments perspective, which could experience further savings in Social Assistance and Employment Insurance and increased tax revenues.

Also outside the time horizon, governments may enjoy savings in seniors' programs—particularly Old Age Security/the Guaranteed Income Supplement—because of the individual's greater lifetime earnings. Expanding the analysis would require taking into account Canada Pension Plan (CPP) payments and would need to analyze the effect related to the collection of CPP premiums over the entire period (currently not included in the analysis).

Finally, intergenerational effects could play an important role in considering the long-term outcomes associated with such an early intervention. This is beyond the scope of the evaluation, but longer time horizons could reasonably wish to encompass the effects of improved outcomes on successive generations.

Further analysis: other social benefits

In addition to the related labour market costs and benefits of governments and of individuals captured above, there are social benefits of the program that would factor into a more extensive analysis of Pathway's net benefit.

As an extension of the analysis to date, it would be worthwhile to examine ways to reflect other societal gains—such as savings for governments and higher well-being for individuals related to better health outcomes and reduced criminality—that may correlate with the outcomes observed. The Department may seek to consider how best to quantify these effects for future analysis and longer time-series data may be required. Even if quantified, however, it is unlikely this would capture the full positive impact, as it would not reflect the knock-on effects within society.

References for the net social cost-benefit analysis

• Boardman et al. (2008). “Social Discount Rates for Canada”. Working paper.
• Chetty et. al. (2011). “How Does your Kindergarten Classroom Affect Your Earnings? Evidence from Project Star,” Quarterly journal of Economics, 126(4): 1593-1660
• Employment and Social Development Canada (2017). “Evaluation of the Labour Market Development Agreements — Synthesis Report”. Ottawa, ON: ESDC.
• Krueger, Alan B. (1999). “Experimental Estimates of Education Production Functions,” Quarterly Journal of Economics, 114, 497-532.
• Lavecchia et al. (2018). “Long Term Impacts of the Pathways to Education Program,” ESDC.
• Lochner, Lance. (2011). “Nonproduction Benefits of Education: Crime, Health and Good Citizenship,” In Handbook of Economics of Education, vol. 4, edited by Eric Hanushek, Steve Machin and Ludger Woessmann, Elsevier, Amsterdam.

Appendix C: lifetime earnings assumptions for the net social cost-benefit analysis

Lavecchia et al. (2018) find that from the age of 24 to 26 (inclusive), eligible students have higher average annual earnings than non- eligible students. This earnings differential increases by 6% between 24 years old ($950) and 26 years old ($1,207).

In order to estimate the program’s incremental impact over 25 years, the Net Social-Cost Benefit Analysis requires an estimate of the earnings over the analysis period of participants and non-participants. This section of the report details the assumptions made in the Social Cost-Benefit Analysis to project earnings beyond 26 years old (labelled as the “Straight-Line Projection” approach) and also an alternative method undertaken as a robustness check (the “Educational Attainment” approach).

Straight-line projection

The following steps were taken to project earnings for both participants and non-participants from age 27 to 43 (inclusive):

1. Extrapolated income levels for non-participants from age 27 to 43 using a straight trend line
2. Used the difference in average annual earnings (for each year between 24 and 26) reported in Lavecchia et al. (2018) to extrapolate the future increases using a straight trend line
3. The forecasted earnings for participants is the sum of these 2 extrapolated values

The annual post-program earnings of participants show incremental gains over the years to follow. The size of the increase continuously grew over the years and ranged from $1,369 (age 27) to $3,425 (age 43). In other words, at the end of the projection period the annual average earnings of participants were $3,425 higher than if they had not participated.

The upward growth in income is consistent with what is observe in Statistics Canada data, where average earnings appear to peak after 50 years old for all levels of educational attainment. However, this assumption of a sustained, steady rise results in a more gradual increase in the initial years than is generally observed in the data (where a sharper increase will generally occur between the 20 to 24 and the 25 to 29 age categories).

Source: Lavecchia and al. (2018) for observed results and ESDC calculations for projected and inferred income levels.

Figure 9: Straight line projection results on employment earnings – Text version

In order to estimate the program's incremental impact over 25 years, the net social-cost benefit analysis requires an estimate of the earnings over the analysis period of participants and non-participants. Figure 9 depicts the projected earnings beyond 26 years old, i.e. earnings for both participants and non-participants from age 27 to 43 (inclusive), labelled as the “Incremental Impact” approach. In addition, as an alternative method was undertaken as a robustness check, labelled as the “Income levels” approach.

Age	Non-participants (observed) $	Participants (observed) $	Non-participants (projected) $	Participants (inferred) $	Gap (observed) $	Gap (projected) $
19	3,828	2,873			-955
20	4,957	4,394			-563
21	6,183	6,140			-43
22	7,768	6,481			-1,287
23	9,833	9,493			-340
24	11,818	12,768			950
25	13,448	14,627			1,179
26	15,574	16,781	15,574	16,781	1,207	1,207
27			17,456	18,825		1,369
28			19,379	20,877		1,498
29			21,302	22,928		1,626
30			23,224	24,979		1,755
31			25,147	27,030		1,883
32			27,070	29,081		2,012
33			28,993	31,133		2,140
34			30,915	33,184		2,269
35			32,838	35,235		2,397
36			34,761	37,286		2,526
37			36,683	39,337		2,654
38			38,606	41,389		2,783
39			40,529	43,440		2,911
40			42,451	45,491		3,040
41			44,374	47,542		3,168
42			46,297	49,593		3,297
43			48,220	51,645		3,425

Robustness check: Educational attainment approach

Given the simplicity of the Straight-Line Projection method, the Evaluation Directorate wished to validate it against a method using assumptions on educational attainment and age. This was to ensure that the Straight-Line Projection:

• provided a good approximation of the pattern that might be observed under other assumptions.
• would not result in significant over-estimates in the potential gains in lifetime earnings over the analysis period.

To that end, the Evaluation Directorate used Statistics' Canada income/earnings data (2010) in order to construct the average income profile for participants and non-participants based on the average income associated with age and educational attainment, as in Table C1.

Source: Statistics Canada, Average earnings or employment income, by age group and highest certificate, diploma or degree, Canada, 2010 Visit Statistics Canada

In order to determine the weights/breakdown among different educational attainment categories, the Report on Pathways to Education produced by Goss Gilroy Inc (GGI report) was used to weight “no certificate, diploma or degree” versus those that completed secondary school or higher. As per the GGI report, 57.6% of participants and 38.5% of the comparison group completed high school.

Among those that graduated high school, distributions observed among Ontario residents aged 25 to 64 as reported in the 2016 Census was applied. The Ontario data were used to remain consistent with the Toronto-area population analyzed by Lavecchia et al (2018). These can be found in Table C2.

Note: Sums may not add up to 100.0% due to rounding. Source: Statistics Canada, 98-402-X2016010-11 Highest level of educational attainment (general) by selected age groups 25 to 64, both sexes, percentage distribution 2016, Canada, provinces and territories, 2016 Census – 25% sample data. Visit Statistics Canada

Due to the inclusion of younger and older age categories in this sample, the level of educational attainment might be biased downwards slightly^{Footnote 36}. However, given the barriers to educational attainment often faced by the target population for the Program, this issue may be moot.

The average earnings level for each 5-year age category using this approach is reported in Table C3. This results in earnings estimates that are initially higher than our Straight-Line Projection approach, except after 40 years old when the Straight-Line Projection of income levels exceed the Educational Attainment approach.

For all age categories, however, the incremental impacts exceeds the values produced under the Simple Projection, but follows a similar pattern of growth.

Figure 10: Incremental impacts (participant earnings minus non-participant earnings) of the straight-line projection approach versus the educational attainment approach – Text version

The graph depicts the incremental impacts, i.e. the participant earnings minus non-participant earnings, of the straight-line projection approach, in comparison with those of the educational attainment approach. The Straight-Line Projection approach produces a conservative estimate over the projection period and is derived from the initial observed data. As well, it follows a similar growth path in incremental impacts, and it requires fewer assumptions that need to be calibrated to the target population.

	25 to 29	30 to 34	35 to 39	40 to 44
Educational Attainment Assumption	$1,663	$2,506	$3,547	$4,259
Simple Projection	$1,376	$2,012	$2,654	$3,232