Monthly Archives: April 2014

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: The Estimation Strategy 2

Posted by Kathryn Schwartz on April 30, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

With stratification, observations are sorted from lowest to highest estimated propensity score. The comparison units with an estimated propensity score less than the minimum (or greater than the maximum) estimated propensity score for treated units are discarded. The strata, defined on the estimated propensity score, are chosen so that the covariates within each stratum are balanced across the treatment and comparison units (we know such strata exist from step one). Based on equation (2), within each stratum we take a difference in means of the outcome between the treatment and comparison groups, and weight these by the number of treated observations in each stratum. We also consider matching on the propensity score. Each treatment unit is matched with replacement to the comparison unit with the closest propensity score; the unmatched comparison units are discarded (see Dehejia and Wahba 1997 for more details; also Rubin 1979, and Heckman, Ichimura, and Todd 1997). ace payday loans

Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: The Estimation Strategy

Posted by Kathryn Schwartz on April 28, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

Estimation is in two steps. First, we estimate the propensity score for the sample of experimental treatment and non-experimental comparison units. We use the logistic model, but other standard models yield similar results. An issue is what functional form of the pre-intervention variables to include in the logit. We rely on the following proposition:
w6586-6
Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: IDENTIFYING AND ESTIMATING THE AVERAGE TREATMENT EFFECT

Posted by Kathryn Schwartz on April 26, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

Let y1 represent the value of the outcome when unit i is subject to regime 1 (called treatment), and Yj0 the value of the outcome when unit i is exposed to regime 0 (called control). Only one of Ую or y1 can be observed for any unit, since we can not observe the same unit under both treatment and control. Let T. be a treatment indicator (=1 if exposed to treatment, =0 otherwise). Then the observed outcome for unit i is Y. = Tya + (1-T)Ya. The treatment effect for unit i is
t = Y – Y
Li i1 i 0 ‘
Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: LALONDE’S RESULTS 4

Posted by Kathryn Schwartz on April 24, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

The fixed-effect type differencing estimator in column (3) fares somewhat better, although many estimates are still negative or deteriorate when we control for covariates in both panels. The estimates in column (5) are closest to the experimental estimate, consistently closer than those in column (2) which do not control for earnings in 1975. The treatment effect is underestimated by about $1,000 for the CPS comparison groups and $1,500 for the PSID groups. Lalonde’s conclusion from Panel A, which also holds for our version in Panel B, is that there is no consistent estimate robust to the specification of the regression or the choice of comparison group.

Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: LALONDE’S RESULTS 3

Posted by Kathryn Schwartz on April 22, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

Lalonde estimates linear regression, fixed-effect, and latent variable selection models of the treatment impact. Since our analysis focuses on the importance of pre-intervention variables, we consider primarily the first of these. Non-experimental estimates of the treatment effect are based on the two distinct comparison groups used by Lalonde (1986), the Panel Study of Income Dynamics (PSID-1) and Westat’s Matched Current Population Survey-Social Security Administration File (CPS-1). Lalonde also considers subsets of these two comparison groups, PSID2-3 and CPS2-3.

Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: LALONDE’S RESULTS 2

Posted by Kathryn Schwartz on April 20, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

докторррр
However, several years of pre-intervention earnings are viewed as important in determining the effect of job training programs (Angrist 1990, 1998; Ashenfelter 1978; Ashenfelter and Card 1985; and Card and Sullivan 1988). Thus, we further limit ourselves to a subset of this data in order to obtain data on earnings in 1974. Our subset, also defined using the month of assignment, includes 185 treated and 260 control observations. Since month of assignment is a pre-treatment variable, this selection does not affect the properties of the experimentally randomized data set: the treatment and control groups still have the same distribution of pre-intervention variables, so that a difference in means remains an unbiased estimate of the average treatment impact.

Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: LALONDE’S RESULTS

Posted by Kathryn Schwartz on April 18, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

The Data
The National Supported Work (NSW) Demonstration (see Manpower Demonstration Research Corporation 1983) was a federally-funded program implemented in the mid-1970s, with the objective of providing work experience for a period of 12 to 18 months to individuals who had faced economic and social problems prior to enrollment in the program. Those randomly selected to join the program participated in various types of work, ranging from operating a restaurant to construction work. Information on pre-intervention variables (pre-intervention earnings as well as education, age, ethnicity, and marital status) was obtained from initial surveys and Social Security Administration records.
Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: Introduction 2

Posted by Kathryn Schwartz on April 16, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

694706.fig.002b
We can easily control for differences between the treatment and non-experimental comparison groups through the estimated propensity score, a single variable on the unit interval. Using propensity score methods, we are able to replicate the experimental treatment effect for a range of specifications and estimators.

Continue reading…

Tags: , ,

CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: Introduction

Posted by Kathryn Schwartz on April 14, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: / No Comments

how-to-deal-with-anemia
This paper discusses the estimation of treatment effects in observational studies. This issue, which is of great practical importance because randomized experiments cannot always be implemented, has been addressed previously by Lalonde (1986), whose data we use in this paper. Lalonde estimates the impact of the National Supported Work (NSW) Demonstration, a labor training program, on post-intervention income levels, using data from a randomized evaluation of the program. He then examines the extent to which non-experimental estimators can replicate the unbiased experimental estimate of the treatment impact, when applied to a composite data set of experimental treatment units and non-experimental comparison units. He concludes that standard non-experimental estimators, such as regression, fixed-effect, and latent-variable-selection models, are either inaccurate (relative to the experimental benchmark), or sensitive to the specification used in the regression. Lalonde’s results have been influential in renewing the debate on experimental versus non-experimental evaluations (see Manski and Garfinkel 1992) and in spurring a search for alternative estimators and specification tests (e.g., Heckman and Hotz 1989; and Manski, Sandefur, McLanahan, and Powers 1992).

Continue reading…

Tags: , ,