Semiparametric Efficient Designs with Staggered Rollouts Restricted; Files Only

Abstract

Randomized Controlled Trials (RCTs) have long been recognized as the gold standard for unbiased causal effect estimation in various fields, including medicine and agriculture, due to their relatively weaker assumptions in comparison to observational studies. In recent years, the technology industry has increasingly embraced online controlled trials, commonly referred to as A/B testing, as a means to evaluate and optimize user experiences. These trials often involve longitudinal panel data, comprising numerous units observed across fixed time periods. In this paper, we initially examine the semiparametric efficiency bound as a fundamental building block. Following this, we extend the semiparametric efficiency bound to a panel data setting, incorporating the presence of serial correlation. Our primary objective centers on the design and analysis of experiments conducted with multiple units over various time periods, wherein the start times of treatments may differ among units. We specifically focus on the experimental design of staggered rollouts. The challenge lies in determining the optimal initial treatment time for each unit to maximize the precision of instantaneous treatment effect estimation. In this context, we consider non-adaptive experiments, wherein treatment assignments are established prior to the experiment's inception. To address this challenge, we propose the Efficiency-Guided Non-Adaptive Experiment (EGNAE) algorithm, available in both stratified and unstratified versions, to devise optimal experimental designs grounded in the novel semiparametric efficiency bound, which accounts for serial correlation in panel data settings. Finally, we showcase the superiority of our proposed experimental design in comparison to existing benchmark designs within the literature, highlighting the effectiveness of the EGNAE algorithm in optimizing experimental designs for staggered rollouts.

Table of Contents

1 Introduction 1

1.1 Summary of Contributions . . . . . . . . . . . . . . . . . . . . 3

1.2 Related Literature . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Panel Data Experiments and Assumptions 8

3 Estimating Average Treatment Effects 16

3.1 Difference-in-Means Estimator . . . . . . . . . . . . . . . . . . 17

3.2 Outcome Regression Estimator  . . . . . . . . . . . . . . . . . 18

3.3 Inverse Propensity Weighted Estimator . . . . . . . . . . . . . 21 

3.4 Augmented Inverse Propensity Weighted Estimator . . . . . . 23

4 Semiparametric Theory and Experimental Design 26

4.1 Semiparametric Theory  . . . . . . . . . . . . . . . . . . . . . . 26

4.2 Semiparametric Efficiency Bound . . . . . . . . . . . . . . . . 30 

4.2.1 Base Case . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.2.2 General Case with Serial Correlation . . . . . . . . . . 31

4.3 Experimental Design . . . . . . . . . . . . . . . . . . . . . . . 32

4.3.1 Benchmark Experimental Design . . . . . . . . . . . . 32

4.3.2 Optimal Experimental Design . . . . . . . . . . . . . . 33

5 Simulation Studies 37

5.1 Synthetic Data . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

6 Conclusion 45

Bibliography 51

A Notation 52

B Proof 54

B.1 Supplementary Material for Panel Data Experiments and Assumptions  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

B.1.1 Supplementary Material for Assumption . . . . . . . 54

B.2 Supplementary Material for Semiparametric Efficiency Bound and Experimental Design  . . . . . . . . . . . . . . . . . . . . . 56

B.2.1 Supplementary Material for Theorem 2  . . . . . . . . . 56

B.2.2 Supplementary Material for Lemma 2.1 . . . . . . . . . 57

B.2.3 Supplementary Material for Lemma 2.2 . . . . . . . . . 58

B.2.4 Supplementary Material for Lemma 2.3  . . . . . . . . . 60

B.2.5 Supplementary Material for Lemma 2.4 . . . . . . . . . 62

B.2.6 Supplementary Material for Theorem 3  . . . . . . . . . 63

B.2.7 Supplementary Material for Lemma 3.1  . . . . . . . . . 65

B.2.8 Supplementary Material for Lemma 3.2 . . . . . . . . . 67

B.2.9 Supplementary Material for Lemma 3.3  . . . . . . . . . 69

B.2.10 Supplementary Material for Lemma 3.4  . . . . . . . . . 71

B.2.11 Supplementary Material for Lemma 3.5  . . . . . . . . . 72

About this Honors Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Emory College
Department	Mathematics
Degree	B.S.
Submission	Honors Thesis
Language	English
Research Field	Mathematics Statistics
Keyword	Causal Inference Semiparametric Efficiency Bound Experimental Design
Committee Chair / Thesis Advisor	Xiong, Ruoxuan, Emory University
Committee Members	Manetta, Manuela, Emory University Hirshberg, David Abraham, Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	File download under embargo until 19 May 2025	2023-04-09 00:27:03 -0400	File download under embargo until 19 May 2025

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions