Detecting Effects on Net Worth Is Nettlesome Work: Fragility in a Randomized Experiment with Individual Development Accounts

Abstract

This article analyzes data from a randomized experiment on how matched-savings accounts affect the net worth of low-income people. We find that estimates are sensitive to probable data errors and to the handling of missing values. In contrast to the findings of Mills et al. 2008, some of the estimated impacts on net worth are large, both economically and statistically. We argue that the jury is still out on the impacts of Individual Development Accounts (IDA) on net worth, and we highlight that data quality and modeling assumptions are crucial even (or especially) for randomized experiments.