Refund to Savings Canada Pilot Study Evaluation

Written by the Social Policy Institute and the Financial Consumer Agency of Canada

Information contained in this publication or product may be reproduced, in part or in whole, and by any means, for personal or public non-commercial purposes without charge or further permission, unless otherwise specified. Commercial reproduction and distribution are prohibited except with written permission from the Financial Consumer Agency of Canada.

For more information, contact:

Financial Consumer Agency of Canada
427 Laurier Ave. West
Ottawa ON K1R 1B9

www.canada.ca/en/financial-consumer-agency

Cat. No. FC5-89/2024E-PDF (Electronic PDF, English)

ISBN 978-0-660-71124-9

© His Majesty the King in Right of Canada, as represented by the Minister of Finance Canada, April 2024.

Aussi disponible en français sous le titre : Système bancaire ouvert et protection des consommateurs : sensibilisation et attentes des Canadiens

Executive summary

Savings are one of the strongest predictors of household financial resilience and well-being, yet Canadian households struggle to save due to an array of economic, behavioural and institutional factors. The Financial Consumer Agency of Canada (FCAC) created the National Financial Literacy Strategy 2021-2026, Make Change that Counts (the National Strategy) to help motivate stakeholders in the financial ecosystem to use and measure the impact of initiatives that can help lead to greater financial resilience for Canadians. The National Strategy’s framework identifies 6 “Ecosystem Priorities” that stakeholders can use to reduce barriers and encourage actions to help consumers achieve positive financial outcomes. The intervention outlined in this study is aligned with 3 of the National Strategy’s priorities and is aimed at improving consumers' saving behaviour by leveraging the timeliness and reach of the tax filing process.

Annual tax refunds offer many households—and low- to moderate-income households in particular—an influx of cash and serve as a unique opportunity to build savings. The Refund to Savings (R2S) initiative is a long-standing collaboration between researchers at Washington University in St. Louis and Intuit Canada, Inc. (Intuit), the makers of TurboTax tax preparation software. It applies the principles of behavioural economics to restructure the online tax preparation experience with the aim of encouraging households to save their tax refund. While the R2S Initiative has undergone many iterations in the United States, the current study presents the results of a pilot study testing the feasibility of an R2S-type initiative in the Canadian tax filing context.

For this pilot study, the Social Policy Institute (SPI) at Washington University, FCAC and Intuit collaborated to administer a randomized, controlled trial that tests the impact of embedding a savings intervention directly into the online tax filing process. This intervention included 3 components:

1. an opportunity for the filer to confirm their intention to save their refund (referred to as a “pre-commitment”) while they were preparing their taxes
2. promotional messaging around the benefits of saving
3. an in-product nudge for filers to change their direct deposit location to a savings account

We assessed the study’s outcomes using a combination of administrative and post-filing survey data.

Key findings

The experiment involved 633,449 tax filers with incomes of up to $100,000. They were randomly assigned to a control group (that is, they experienced the typical tax filing process) or a treatment group (that is, they experienced a modified tax filing process which included the intervention components listed above)—creating two groups of just over 300,000 filers each (For more information on the treatment and control groups, see the Data section below). The tax preparation software asked participants where they deposited their refund (that is, in a savings or chequing account) and recorded the amount saved. The deposit location data were collected through a voluntary question added to the tax preparation process, and the refund amounts were collected through administrative data.

A total of 141,438 study participants answered the question about their deposit location. We found that 6.9% of the control group (69,116 participants) reported depositing their tax refund into a savings account, while 8.1% of the treatment group (72,322 participants) reported depositing their refund into a savings account. The 1.2% difference amounts to a 17% increase in savings rate (p<0.001). We also found that the average refund deposited into a savings account was higher for the treatment group ($1,448) compared to the control group ($1,368)—a difference of $80 (p<0.001)—due in part to the fact that filers in the treatment group with larger refunds (and typically higher incomes) were more responsive to savings nudges compared to their counterparts in the control group. Overall, the intervention had 3 key impacts:

1. It increased the number of participants who saved their refunds (across refund size and income groups)
2. It was most effective at increasing the savings rate of those with lower incomes
3. It nudged more people with higher refund amounts to save

(For more information on the treatment and control groups, see the Data section below.)

Conclusion and future considerations

Our pilot study demonstrates that it is feasible to successfully implement a tax-time savings intervention in Canada through a public-private-academic collaboration like the R2S initiative. The savings outcomes we observed were statistically significant, with higher levels of savings among the treatment group.

Among participants who reported the location of their deposit to be a savings account (5,583 participants), this intervention resulted in a $1.3 million increase in savings, suggesting that the entire treatment group of just over 300,000 participants may have set aside up to $6.9 million in additional savings. If these patterns hold in a future scaled-up version of the intervention, it could be possible to see as much as $23 million in additional savings deposits for every 1 million participating Canadians.

Building on our learnings from the pilot, we can make 3 primary recommendations for potentially improving the future impact of the R2S initiative in the Canadian context:

1. Include more questions about saving refunds in the tax product itself, so we do not rely solely on surveys (which have low response rates) to measure key outcomes.
2. Identify additional methods and data sources, such as banking or personal finance application data, to assess filers’ savings behaviours following the R2S initiative.
3. Implement a version of the R2S initiative that includes a simple way to change the direct deposit location of the refund within the online tax preparation process itself. Alternatively, make the option to leave the tax preparation system to change direct deposit options much clearer and more prominent within the tax preparation process.

The R2S Canada study provides a model for how public sector priorities (like building savings for Canadians) can be pursued at scale through private-sector platforms. This type of collaboration can serve as a model for an array of financial well-being interventions.

Introduction

FCAC created the National Financial Literacy Strategy 2021-2026, Make Change that Counts, to outline a Canada-wide approach aimed at helping consumers build financial resilience in an increasingly digital world. Its emphasis is on getting all financial ecosystem stakeholders—including governments, community groups, academics and industry organizations—to use and measure the impact of initiatives that help advance the goals of the National Strategy. By reducing barriers, encouraging action and working together, stakeholders can collectively help consumers improve their financial skills, knowledge and confidence, and engage in behaviours that will lead them to greater financial resilience.

The National Strategy’s companion piece, a measurement plan entitled Counting Change, acts as a guide for financial ecosystem stakeholders. The plan walks them through the basic steps of how to measure the impact of their initiatives through the lens of the National Strategy, so we can understand whether our efforts are successful and use that understanding to refine our approach and maximize our collective impact. Using behavioural science to simplify financial decisions and nudge consumers towards strengthening their financial resilience is more important than ever in today’s challenging economic environment and is key to fulling FCAC’s mandate.

For Canadians, having access to emergency savings or money that is set aside to use in the event of unexpected dips in income or large expenses has been shown to reduce the risk of material hardships like food insecurity or delayed payments on essential bills.^{Footnote 1}  Beyond creating economic stability, savings have other non-financial benefits, such as helping orient households toward their future goals^{Footnote 2}  and allowing for better mental health.^{Footnote 3}  However, low- and moderate-income (LMI) households often struggle with many forms of economic instability and may well lack sufficient emergency savings. In these households, income volatility, periods of unemployment and unplanned expenses can be common occurrences.^{Footnote 4}  As of November 2022, only about half of Canadian households had set aside funds sufficient to cover expenses for 3 months in case of sickness, job loss, economic downturn or other emergencies.^{Footnote 5}  Consequently, many households are now unable to respond to economic shocks by using liquid assets.^{Footnote 6}  This issue existed well before the COVID-19 pandemic but has also been exacerbated by it.^{Footnote 7} ,^{Footnote 8}

The tax filing process presents an opportunity for many Canadians including those in LMI households to accumulate emergency savings by way of their tax refund. A federal tax refund often provides LMI households with the single largest amount of money they will receive in a given year and, for that reason, tax filing presents a unique opportunity for many lower-income households to boost savings.^{Footnote 9}  Despite the opportunity to build emergency savings at tax filing, a number of barriers may prevent LMI households from saving their refund. For example, filers may prioritize other more pressing financial needs, such as overdue bills and debt repayment.^{Footnote 10}  While managing these obligations is important, research also finds that even small amounts of emergency savings can be associated with large improvements in financial resilience and financial well-being,^{Footnote 11} ,^{Footnote 12}  and that tax refund savings deposits are associated with a reduction in the risk of material hardship.^{Footnote 13}  As a result, a growing body of research focuses on designing behavioural interventions to increase the rates at which LMI tax filers deposit their refund into savings accounts.^{Footnote 14} 

Building on previous research that leverages behavioural interventions to increase refund savings among LMI tax filers in the United States, the SPI, FCAC and Intuit collaborated to administer a pilot study to test the impacts of a low-touch behavioural intervention administered through online tax preparation software in Canada. The purpose of this pilot study was to confirm the feasibility of implementing this type of intervention in the Canadian context, while also assessing the impact of the tested intervention components, and ideally informing future interventions.

Prior evidence from the Refund to Savings initiative

The R2S initiative is a long-term collaboration between Intuit and researchers at Washington University in St. Louis that has tested the extent to which applying behavioural economics principles to the TurboTax online tax preparation software can encourage LMI tax filers to save more of their refund. In both the U.S. and Canada, the TurboTax tax preparation software was viewed as an appropriate setting for savings interventions because of the availability of free tax filing options. The Canada-based R2S initiative in this report builds on earlier findings from the R2S studies in the U.S. These studies concluded that interventions featuring persuasive messaging around saving tax refunds for emergencies, emphasizing the importance of direct deposits to savings, and encouraging pre-commitment to savings are effective methods to increase tax refund savings among LMI tax filers.^{Footnote 15}  The most effective of these approaches involves making the option to save the entire tax refund more prominent, which proved to be more effective than approaches like persuasive messaging.^{Footnote 16} 

This initiative supports 3 of the 6 Ecosystem Priorities from the National Financial Literacy Strategy 2021-2026:

• Priority 2, Build and Provide for Diverse Needs, is supported by investigating consumers’ savings behaviour according to key demographic information (that is, relationship status, income, age)—which could provide insights on how to tailor such interventions in the future.
• Priority 4, Enhance Access to Trustworthy and Affordable Financial Help, is supported by relevant, timely and unbiased advice that nudges consumers to use their refund to build emergency savings.
• Priority 5, Use Behavioural Design to Simplify Financial Decisions, is supported through using behavioural insights (for example, pre-commitment) and design research to facilitate financial decisions that will be beneficial to the consumer.

This initiative also uses just-in-time knowledge and confidence-building language and promotes increased savings behaviours—both of which are identified in the National Strategy as Key Consumer Building Blocks.

Finally, this initiative incorporated a pre-commitment (that is, asking filers to declare their intention to save their tax refund), persuasive messaging elements and an in-product nudge for filers to change their direct deposit location to a savings account. In this report, we provide background on the initiative, detail its design and highlight some findings from its implementation. Results speak to the role of behavioural interventions in changing financial behaviours and outcomes, and can inform future Canadian tax refund savings interventions.

Timeline of the 2021 intervention

The initial design of the R2S Canada initiative was discussed as early as 2017 and the first pilot intervention to assess the impact of persuasive messaging and pre-commitment on the rate of tax refund savings took place between May 2020 and July 2020. Due to the unexpected impact of the COVID-19 pandemic, the initial pilot was not implemented until late in the tax season and so served as a proof-of-concept test. To address this limitation, a second pilot intervention was fully deployed between December 2020 and August 2021. (Note that interventions reflect the prior year’s taxes. That is, the 2020 tax filing intervention focused on filers’ 2019 taxes and the 2021 intervention focused on filers’ 2020 taxes.) The present paper focuses on the pilot intervention launched in 2021, although the findings from the 2020 pilot were similar.

Summary of the 2021 intervention

Due to time and programming constraints, we decided that the full intervention, originally proposed by SPI, could not be undertaken. The primary issue was the component of the intervention that involved embedding the CRA web page (where tax filers could change their direct deposit location) directly into the TurboTax tax preparation process.

Given this, FCAC, Intuit and SPI opted to use a simpler pre-commitment option, a pared-down link-out to the CRA website, and persuasive messaging about the value of saving tax refunds. This intervention was delivered to all tax filers who both used TurboTax online tax preparation software and earned less than $100,000 in income. The in-product intervention was implemented between February 2021 and June 2021, and emails to the treatment and control groups with survey links were sent out in September 2021.

Figure A1 (in the Appendix) shows the intervention flow implemented during the 2021 tax season and Figures A2a and A2b are screenshots of the treatment group’s tax preparation experience. As part of the intervention, each tax filer completing their taxes in TurboTax received a question about whether they owned tax-advantaged savings accounts, including a Tax-Free Savings Account (TFSA), Registered Retirement Savings Plan (RRSP) and Registered Retirement Income Fund (RRIF) (Box 1 in Figure A2a). After completing the sections on income, RRSPs and deductions, eligible tax filers were randomly assigned into the treatment or the control group.

The treatment group experienced several changes in their experience during the tax-filing process:

1. Immediately after reviewing the financial summary in TurboTax (including their total income, total taxable income, total taxes, amount of tax refund, etc.), treatment group tax filers were asked a pre-commitment question—that is, whether they planned to save some or all of their tax refund (Box 2 in Figure A2a).
2. Just before seeing the direct deposit screen, filers in the treatment group were given information about the different ways they could receive their refund (for example, direct deposit, paper cheque). This screen included a link to the CRA web page that would allow filers to change their direct deposit location (Box 3 in Figure A2b).
3. At the direct deposit screen, tax filers in the treatment group saw a savings-focused message that encouraged them to use direct deposit to receive their tax refund.
4. Like tax filers in the control group, tax filers in the treatment group could select where to deposit their tax refund (Box 4, Figure A2b).

In September 2021, study participants who had completed their taxes received an email from Intuit inviting them to complete a short survey focused on filers’ usage of the tax refund, access to emergency liquidity, and ownership of financial accounts (Table A1). Filers did not receive compensation for completing the survey, which, on average, took 8 minutes to fill out and send in.

Data

Data used in this study came from 2 sources: administrative tax data and post-filing survey data. The administrative tax data collected both filers’ tax indicators (for example, income, refund size, marital status) and data on the study participants’ interactions with the experiment (for example, whether they interacted with the savings pre-commitment). The raw data for this study remained with Intuit, and anonymized, deidentified, aggregate data were shared with SPI for analytical purposes.

In total, 633,449 filers met the qualification requirements for the experiment and were randomly assigned to the treatment (N=324,040) and control groups (N=309,409). Table 1 compares the administrative tax data between those who were randomly assigned to the treatment and control groups. Overall, the treatment and control groups were highly similar, suggesting that the experiment’s randomization worked as intended. Though we do observe statistically significant differences between the treatment and control groups on certain metrics, this is primarily driven by the large number of people in the study. For example, though the treatment group had a significantly higher income than the control group, this difference is smaller than 1%. These very modest differences are unlikely to be drivers in any outcome differences we observe between the groups.

*p<0.05; **p<0.01; ***p<0.001; n.s.: “not significant”

In addition to the administrative data collected through the tax filing program (noted above), additional data were also collected from a survey sent to study participants in the weeks after tax filing. This survey asked questions about refund receipt and usage, savings behaviours, and financial account ownership. In total, 10,272 of the study participants responded to the survey, for a response rate of 1.6%.

Findings

Experimental findings

Finding: A high percentage of filers pre-committed to save their refund

Of the 324,040 tax filers assigned to the treatment group, 154,854 pre-committed to save their refund (47.8%). For comparison, a study embedding a pre-commitment prompt in a U.S.-based tax filing product aimed at low-income tax filers found that only 7.6% of filers pre-committed to save their refund,^{Footnote 17}  though there were differences in the study population and pre-commitment prompt between the U.S. and Canada studies that may explain these differences.

Finding: The R2S initiative increased overall savings deposit rates and savings amounts

Since the experiment focused on encouraging filers to deposit their refunds into savings vehicles, the primary outcome in this experiment is deposit location. The savings deposit location was self-reported, as users were asked where they deposited their refund immediately after they filed their taxes. In total, 72,322 control group participants (23.3%) and 69,116 treatment group participants (21.3%) answered the question about their refund deposit location.

Figures 1 and 2 highlight the refund savings rates and savings amounts of the control and treatment groups.

Figure 1 shows a higher share of those in the treatment group deposited their refund into a savings account (8.1%; n=5,583) compared to the control group (6.9%; n=4,956). This difference was significant at the p<0.001 level.

Figure 1. Refund savings deposit rates, by intervention group

Figure 2 shows that the mean dollar amount deposited into savings accounts was greater in the treatment group (average/mean: m=$1,448; variability/standard deviation: sd=$1,755) compared to the control group (m=$1,368; sd=$2,042). This difference was also significant at the p<0.001 level.

Figure 2. Average refund savings deposit amounts, by intervention group

Finding: Intervention impacts were observed across refund sizes and income levels

Figure 3 shows that, interestingly, tax filers with smaller refunds were more likely to deposit their refunds into savings accounts. For example, 7.5% of the control group with refunds of $1,000 or less saved their refund compared to 6.3% of those with refunds of more than $1,000. However, there was a larger treatment effect for participants with higher incomes. Compared to the control group, the treatment was associated with a 1.6% increase in savings deposits for those with refunds of $1,000 or more (7.9%) and a 0.9% increase for those with refunds of less than $1,000 (8.4%). Both differences were significant at the p<0.001 level. This pattern may indicate that, while tax filers with smaller refunds are more likely to deposit their refunds into savings, those with larger refunds are more responsive to savings nudges.

Figure 3. Refund savings deposit rates, by refund size

Figure 4 illustrates that the treatment was effective at increasing savings deposit rates across most categories. For example, of treatment group participants with the lowest incomes ($0-$10,000), 9.8% (n=736) of the treatment group made savings deposits compared to only 8.3% (n=652) of the control group in the same income category. These differences were significant for all income categories except the highest ($75,001 - $100,000)—partly due to the smaller sample size in this category.

Figure 4. Refund savings deposit rates, by intervention group and income

Figure 5 demonstrates that savings deposit amounts were also significantly greater for the treatment group than for the control group, across all income categories. For example, while the lowest income earners in the control group had an average savings deposit amount of $691, the same income earners in the treatment group had an average savings deposit amount of $743. These differences persist and even grow across all income categories.

Figure 5. Average refund savings deposit amounts, by intervention group and income

Finding: Pre-commitment may lead to increases in saving rates and amount saved

Of those who pre-committed to save their refund and who responded to the direct deposit question, 8.3% (n=3,988) reported making savings deposits compared to 6.3% (n=1,593) of those who did not pre-commit to save. This difference was significant at the p<0.001 level. The mean deposit amount was also higher for those who pre-committed to save (m=$1,511; sd=$1,805) compared to those who did not pre-commit to save (m=$1,287; sd=$1,613). As filers could not split their tax refund into separate direct deposit accounts, the higher average savings is likely related to those with higher refunds being more influenced to save. This difference was significant at the p<0.001 level, though these results should be interpreted with caution as tax filers who pre-committed to save their refund may have differed in important ways from those who did not.

Survey findings

Although the survey response rate was quite low (1.6%), Intuit reports that the demographic information (for example, age, marital status, gender, income, employment status) of the 10,272 survey respondents was similar to that of the broader Canadian tax-filing population whose annual incomes were less than $100,000. Although just 7.5% of tax filers in the initiative reported (in-product) that they were depositing their refund into a savings account, about half of survey takers in both treatment and control groups reported that they were able to save at least part of their refund (see Table A1 in Appendix for survey instrument). Of those who received a tax refund, over one-quarter of both treatment and control groups (27.4% and 27.1%, respectively) said they had used their refund to save for emergencies or unexpected needs. Of those who had yet to receive their refund, 39% of the treatment group and 36% of the control group said they were planning to use their tax refund to save for emergencies or other unexpected needs. Additionally, over half of survey respondents in both the treatment group (52%) and control group (53%) planned to save one-quarter or more of their refund for more than 6 months.

Interestingly, almost two-thirds of survey-takers in both the treatment and control groups reported that they saved (or planned to save) their refund in a savings account, and roughly 20% of survey takers in both groups reported saving their refund in Tax Free Savings Accounts. The fact that more tax filers reported saving their refund in the survey than reported depositing their refund into a savings vehicle at tax time may indicate that many filers prefer to deposit their refund into a chequing account and then transfer it to a savings vehicle after receipt. This is consistent with the fact that, in Canada, there is considerable additional work (or added “friction”) to change one’s direct deposit account location within the tax filing software. We also asked survey takers if they planned to open an array of different financial accounts (for example, savings accounts, chequing accounts, TFSA, etc.). Almost 90% of respondents did not plan to open any accounts, though roughly 2% planned to open a savings account, 2% planned to open a chequing account, 5% planned to open a TFSA and 3% planned to open an RRSP.

Discussion

This pilot intervention shows that it is possible to successfully implement a tax time savings intervention in Canada through a public-private-academic collaboration. Field experiments are often costly in terms of both time and resources, and the R2S Canada pilot represents a model for how public-sector priorities (for example, building savings for Canadians) can be pursued at scale through private-sector platforms. This type of collaboration can form a model for an array of financial well-being interventions, delivered at tax time and beyond.

Moreover, this is an example of an initiative that FCAC took on with collaborators in support of the National Financial Literacy Strategy 2021-2026. As discussed above, this initiative supports 3 of the National Strategy’s 6 Ecosystem Priorities: Priority 2 (Build and Provide for Diverse Needs), Priority 4 (Enhance Access to Trustworthy and Affordable Financial Help) and Priority 5 (Use Behavioural Design to Simplify Financial Decisions). What’s more, the current initiative also uses just-in-time knowledge and confidence-building language and promotes increased savings behaviours, which are identified in the National Strategy as Key Consumer Building Blocks.

We found that the treatment group was significantly more likely to self-report depositing their refund into a savings vehicle and to show a higher savings average than the control group, and that these effects were present across a wide range of household income levels. However, while these effects were statistically significant, they were relatively modest. The treatment group as a whole was 17% more likely to deposit their refund to savings (8.1% vs 6.9%) and deposited 6% more into savings ($1,448 vs $1,368). These effects are similar to what was seen in early versions of the U.S.-based R2S studies, which used suggested savings amounts and persuasive messaging to motivate savings.^{Footnote 18} 

We make recommendations on how to strengthen these effects in the next section of the report.

Recommendations and next steps

For future iterations of the intervention, it may be possible to implement a design that incorporates additional features.

That is, we recommend implementing an intervention that features a savings pre-commitment, emergency savings messaging, messaging to build awareness around TFSAs and, most importantly, a restructured choice architecture that makes changing the direct deposit location for the tax refund much simpler. From the conversations we have had with stakeholders in this study, as well as from the behavioural diagnoses we have conducted on the Canadian online tax filing system, making it easy for filers to access the CRA direct deposit location page appears to be the intervention component that is most likely to impact tax filers’ refund savings behaviours. The 2 primary methods to achieve this are:

• Improving the prominence of the direct link to the CRA webpage. The current link is somewhat hidden in the tax preparation process, since it is on a page that shares other information on financial health and details about the tax filing process. As an alternative, this link could be highlighted better and given its own page in the tax preparation process, to better communicate both the opportunity to change direct deposit locations and the benefits of doing so.
• Directly embedding the CRA webpage into the tax-preparation software itself, so that filers can change their deposit location without having to leave the software.

Of these 2 approaches, our expectation is that the second would be the most effective by far, though it is also likely more technically difficult to implement.

The second recommendation concerns the low response rate to the post-filing survey, which diminishes our ability to detect significant effects in post-filing refund savings behaviours. While there are ways of increasing the survey response rate—such as offering financial or other incentives—these are often costly. Instead, we propose identifying a subset of essential outcome measures from the survey and embedding questions that capture these measures directly into the tax-preparation experience, which will likely greatly increase the response rate to these questions.

We look forward to continuing to find new ways of increasing tax refund savings among Canadian tax filers. If any organizations are interested in incorporating an R2S intervention into their tax preparation software, they are encouraged to contact FCAC directly.

Conclusion

This 2021 R2S intervention resulted in a $1.3 million increase in savings for those who self-reported their direct deposit location (5,583 participants), suggesting that the entire treatment group of just over 300,000 participants may have set aside up to $6.9 million in additional savings. If these patterns hold in a future scaled-up version of the intervention, it could be possible to see as much as $23 million in additional savings deposits for every 1 million participating Canadians.

This experiment has provided evidence that it is possible to implement low-cost, low-touch, just-in-time interventions in Canada that incorporate behavioural design techniques tailored to consumers’ diverse needs and that can promote savings behaviours that will help Canadians build financial resilience and improve their financial well-being.

Appendix

Figure A1: Final 2021 pilot intervention design

Note: TT=TurboTax

Figure A2a: Final 2021 pilot in-product intervention

Figure A2b: Final 2021 pilot in-product intervention

Note: Used with permission from Intuit. All rights reserved.

Table A1. Survey instrument