Research

Job Market Paper

Composition Bias and Covariate Adjustment in Regression Discontinuity Designs

Abstract

In regression discontinuity designs covariates can be imbalanced in the samples used for estimation, even if their distribution is continuous in the running variable. I demonstrate that under such covariate imbalance the standard local polynomial RD estimator (base RD) implicitly compares units with different covariate values. This induces an additive term in the bias of the base RD estimator, which I refer to as composition bias. To reduce composition bias, I argue that researchers should adjust for pre-treatment covariates so that the estimator compares only units with similar covariate values. Specifically, I propose the Stratified RD estimator and prove that it has asymptotically weakly lower variance and worst-case bias than the base RD estimator, which leads to shorter bias-aware confidence intervals for the average treatment effect at the cutoff. In a simulation calibrated to replication data from a study of a merit-based financial aid program in Colombia, stratifying on parental education reduces the bias relative to the base RD estimator by 14–23%, and stratifying on both parental education and whether a student lives in a metropolitan area reduces it by 25–30%, depending on the bandwidth.

Work in Progress

Segment Fixed Effects in Spatial Regression Discontinuity Designs

with Soonwoo Kwon, Michael Pollmann and Aurel Rochell

Abstract

When a policy varies discontinuously across geographic space, researchers often identify its effects by comparing outcomes on either side of the policy border in a spatial regression discontinuity design. For estimation, most regression specifications control for location along the border using border-segment fixed effects, and for location away from the border using a bandwidth and sometimes linear adjustments in distance. We clarify the conditions under which segment fixed effects are needed, and propose a modeling technique to characterize the benefits and costs of fixed effects in finite samples. We provide theoretical and practical guidance: Many prior studies chose border segments that are likely longer than optimal, and a common local linear adjustment for distance to the border can yield distant comparisons. We illustrate the proposed auxiliary analyses and robustness checks by reanalyzing empirical studies.

Making a Difference: Unobservables and Pre-Treatment Outcomes in Regression Discontinuity Designs

Abstract

In many Regression Discontinuity Designs researchers observe the outcomes of units before and after treatment, and outcomes generally depend on the running variable as well as unobserved covariates. Such unobservables can increase not only the variance of the standard local polynomial RD estimator but also its bias if they are imbalanced within the estimation sample. This paper demonstrates that differencing post- and pre-treatment outcomes eliminates the contribution of time-invariant unobservables to the outcome under a separability condition. As a result, estimating the average treatment effect at the cutoff using the differenced outcome rather than the post-treatment outcome weakly reduces worst-case bias and can also reduce variance. Differencing can thus lead to substantially shorter bias-aware confidence intervals, depending on the correlation of the outcome over time and the researcher’s assumptions about how much bias is caused by time-invariant unobservables.

Generic Machine Learning Inference on Heterogeneous Treatment Effects in Regression Discontinuity Designs

with Paul Goldsmith-Pinkham

Abstract

Applied researchers routinely ask whether RD effects differ across individuals and whether these differences are explained by observed covariates. Current practice relies on splitting the sample based on a single covariate and testing for differences in treatment effect estimates across subsamples. In applied studies there are often many potential covariates to split on, which invites multiple testing, and researchers might not have strong priors about which observable characteristics matter. We propose a simple approach that combines machine learning with sample splitting to test for heterogeneous treatment effects in RD designs. The key idea is to use any flexible machine learning method to construct a proxy for the conditional average treatment effect, and then use this proxy to form subgroups that maximize between-group heterogeneity. Because the proxy is estimated on an auxiliary sample, standard nonparametric RD inference applies within each subgroup on the main sample. Using a simulation design from the literature, we find substantial improvements in power over existing methods.

Narratives: The Effect of Political Advertising on Attitudes towards Migration

with Johannes Matzat

Abstract

This paper explores the effect of substantial shifts in political narratives on attitudes towards migration in the US. We document a marked increase in the share of political advertising aired on TV that addresses the topic of migration, from 1% in 2012 to 12% in 2018. Exploiting the geographic segmentation of media markets and timing of the advertisements, we study the causal effect of political advertising on the volume and content of migration-related posts on Twitter.