CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: RESULTS USING THE PROPENSITY SCORE

Posted by Kathryn Schwartz on May 02, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES:

Using the method outlined in the previous section, we estimate the propensity score for each comparison group separately. Figure 1 presents a histogram of the estimated propensity scores for the treatment and PSID-1 comparison units, and Figure 2 for CPS-1 comparison units. In Figure 2, we discard 12,611 (out of a total of 15,992) CPS units whose estimated propensity score is less than the minimum for the treatment units. Even then, the first bin (from 0-0.05) contains 2,969 of the remaining comparison units and only 26 treatment units. This provides a snapshot of the fact that the comparison group, although very large, contains relatively few units comparable to the treatment group. A similar pattern is seen in the first bin of Figure 1, but an important difference is that in Figure 1 there is limited overlap in the estimated propensity score between the treatment and PSID groups: there are 98 (more than half the total number of) treated units with an estimated propensity score in excess of 0.8, and only 7 comparison units. Instead, for the CPS, although the treatment units outnumber the comparisons for higher values of the estimated propensity scores, for most bins there are at least a few comparison units. Click Here

NSW Earnings Less Comparison Group Earnings NSW Treatment Earnings Less Comparison Group Earnings, Conditional On The Estimated Propensity Score
Quadratic in Scoreb Stratifying on the Score Matching on the Score
(1)Unadjusted (2)Adjusteda (3) (4)Un-adjusted (5)Adjust-eda (6)Obs.g (7)Unadjusted Adt)st1
NSW 1,794(633) 1,672(638)
PSID-1c -15,205(1154) 731(886) 294(1389) 1,608(1571) 1,494(1581) 1,255 1,691(2209) 1,473(809)
PSID-2d -3,647(959) 683(1028) 496(1193) 2,220(1768) 2,235(1793) 389 1,455(2303) 1,480(808)
PSID-3d 1,069(899) 825(1104) 647(1383) 2,321(1994) 1,870(2002) 247 2,120(2335) 1,549(826)
CPS-1e -8,498(712) 972(550) 1117(747) 1,713(1115) 1,774(1152) 4,117 1,582(1069) 1,616(751)
CPS-2e -3,822(670) 790(658) 505(847) 1,543(1461) 1,622(1346) 1493 1,788(1205) 1,563(753)
CPS-3e -635(657) 1,326(798) 556(951) 1,252(1617) 2,219(2082) 514 587(1496) 662(776)

We use stratification and matching on the propensity score to group the treatment units with the small number of comparison units that are comparable (namely, those comparison units whose estimated propensity scores are greater than the minimum — or less than the maximum -propensity score for treatment units). The treatment effect is estimated by summing the within-stratum difference in means between the treatment and comparison observations (of earnings in 1978), where the sum is weighted by the number of treated observations within each stratum (Table 3, column (4)). An alternative is a within-block regression, again taking a weighted sum over the strata (Table 3, column (5)). When the covariates are well balanced, such a regression should have little effect, but it can help to eliminate the remaining within-block differences. Likewise for matching, we can estimate a simple difference in means between the treatment and matched comparison group for earnings in 1978 (column (7)), and also perform a regression of 1978 earnings on covariates (column (8)).

Tags: , ,