CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: RESULTS USING THE PROPENSITY SCORE 2

Posted by Kathryn Schwartz on May 04, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES:

Table 3 presents the results. For the PSID sample, the stratification estimate is $1,608 and the matching estimate is $1,691, which should be compared against the benchmark randomized-experiment estimate of $1,794. The estimates from a difference in means, or regression control on the full sample, are -$15,205 and $731. The propensity score estimators yield more accurate estimates simply using a difference in means because only those comparison units similar to the treatment group have been used. In columns (5) and (8) controlling for covariates has little impact on the stratification and matching estimates. Likewise for the CPS, the propensity-score-based estimates from the CPS — $1,713 and $1,582 — are much closer to the experimental benchmark than estimates from the full comparison sample — -$8,498 and $972. Click Here

Another set of estimates to consider is from the subsets of the PSID and CPS. In Table 2, the estimates tend to improve when applied to narrower subsets. However, as noted above, the estimates still range from -$8,498 to $1,326. In Table 3, the estimates do not improve for the subsets, although the range of fluctuation is much narrower, from $587 to $2,321. Tables 1 and 4 shed light on this.

ControlSample No of Obs. Age Educ Black Hisp Nodegree Married RE74US$ RE75US$
NSW/Lalonde:aTreated 297 24.63 10.38 0.80 0.09 0.73 0.17 3,571
Control 425 24.45 10.19 0.80 0.11 0.81 0.16 3,672
RE74 subset:bTreated 185 25.81 10.35 0.84 0.059 0.71 0.19 2,096 1,532
Control 260 25.05 10.09 0.83 0.1 0.83 0.15 2,107 1,267
Comparisongroups:0PSID-1 2,490 34.85 12.11 0.25 0.032 0.31 0.87 19,429 19,063
PSID-2 253 36.10 10.77 0.39 0.067 0.49 0.74 11,027 7,569
PSID-3 128 38.25 10.30 0.45 0.18 0.51 0.70 5,566 2,611
CPS-1 15,992 33.22 12.02 0.07 0.07 0.29 0.71 14,016 13,650
CPS-2 2,369 28.25 11.24 0.11 0.08 0.45 0.46 8,728 7,397
CPS-3 429 28.03 10.23 0.21 0.14 0.60 0.51 5,619 2,467

Table 1 presents the pre-intervention characteristics of the various comparison groups. We note that the subsets PSID-2 and -3, and CPS-2 and -3, though more closely resembling the treatment group, are still considerably different along a number of important dimensions, including ethnicity, marital status, and especially earnings. Table 4 presents the characteristics of the matched subsamples from the comparison groups. The characteristics of the matched subsets of CPS-1 and PSID-1 closely correspond to the treatment group; none of the differences are statistically significant. But as we create the subsets of the comparison groups, the quality of the matches declines, most dramatically for the PSID, with PSID-2 and -3 earnings now increasing from 1974 to 1975, whereas for the treatment group they decline. The training literature has identified the “dip” in earnings as an important characteristic of participants in training programs (see Ashenfelter 1974, 1978). The CPS sub-samples retain the dip, but for the matched subset of CPS-3 earnings in 1974 are significantly higher than for the treatment group.

Tags: , ,