CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES: LALONDE’S RESULTS 4

Posted by Kathryn Schwartz on April 24, 2014
CAUSAL EFFECTS IN NON-EXPERIMENTAL STUDIES:

The fixed-effect type differencing estimator in column (3) fares somewhat better, although many estimates are still negative or deteriorate when we control for covariates in both panels. The estimates in column (5) are closest to the experimental estimate, consistently closer than those in column (2) which do not control for earnings in 1975. The treatment effect is underestimated by about $1,000 for the CPS comparison groups and $1,500 for the PSID groups. Lalonde’s conclusion from Panel A, which also holds for our version in Panel B, is that there is no consistent estimate robust to the specification of the regression or the choice of comparison group.

Comp- NSW Treatment Unrestricted Contr- NSW Treatment Unrestricted Contr- NSW Treatment Unrestricted Contr-
arison Earnings Less Difference in olling Earnings Less Difference in olling Earnings Less Difference in olling
Group Comparison Differences: for All Comparison Differences: for All Comparison Differences: for All
Group Earnings, Quasi-Differ- Vari- Group Earnings, Quasi-Differ- Vari- Group Earnings, Quasi-Differ- Vari-
1978 b ence in Earni- ablesf 1978b ence in Earni- ablesf 1978 b ence in Earni- ablesf
ngs Growth: ngs Growth: ngs Growth:
1975-1978 1975-1978 1975-1978
Un- Ad- Un- Ad- Un- Ad- Un- Ad- Un- Ad- Un- Ad-
adjusted оte

st

u

j

adjust- justede adjusted justedc adjust- justede adjusted justedc adjust- justede
edd edd edd
(1) (2) (3) (4) (5) (1) (2) (3) (4) (5) (1) (2) (3) (4) (5)
NSW 886 798 879 802 820 1,794 1,672 1,750 1,631 1,612 1,794 1,688 1,750 1,672 1,655
(472) (472) (467) (468) (468) (633) (637) (632) (637) (639) (633) (636) (632) (638) (640)
PSID-1 -15,578 -8,067 -2,380 -2,119 -1,844 -15,205 -7,741 -582 -265 186 -15,205 -879 -582 218 731
(913) (990) (680) (746) (762) (1155) (1175) (841) (881) (901) (1155) (931) (841) (866) (886)
PSID-2 -4,020 -3,482 -1,364 -1,694 -1,876 -3,647 -2,810 721 298 111 -3,647 94 721 907 683
(781) (935) (729) (878) (885) (960) (1082) (886) (1004) (1032) (960) (1042) (886) (1004) (1028)
PSID-3 697 -509 629 -552 -576 1,070 35 1,370 243 298 1,070 821 1,370 822 825
(760) (967) (757) (967) (968) (900) (1101) (897) (1101) (1105) (900) (1100) (897) (1101) (1104)
CPS-1 -8,870 -4,416 -1,543 -1,102 -987 -8,498 -4,417 -78 525 709 -8,498 -8 -78 739 972
(562) (577) (426) (450) (452) (712) (714) (537) (557) (560) (712) (572) (537) (547) (550)
CPS-2 -4,195 -2,341 -1,649 -1,129 -1,149 -3,822 -2,208 -263 371 305 -3,822 615 -263 879 790
(533) (620) (459) (551) (551) (671) (746) (574) (662) (666) (671) (672) (574) (654) (658)
CPS-3 -1,008 -1 -1,204 -263 -234 -635 375 -91 844 875 -635 1,270 -91 1,326 1,326
(539) (681) (532) (677) (675) (657) (821) (641) (808) (810) (657) (798) (641) (796) (798)

The inclusion of earnings in 1974 as an additional variable in the regressions in Table 2 (Panel C) does not alter Lalonde’s basic message, although the estimates improve when compared with Panel B. In columns (1) to (3), many estimates are still negative, but less so than in Panel B. In columns (4) and (5), the estimates are also closer to the experimental benchmark, off by about $1,000 for PSID1-3 and CPS1-2 and by $400 for CPS-3. Overall, the best results in Table 2 are for CPS-3, Panel C. This raises a number of issues. The strategy of considering subsets of the comparison group more comparable to the treatment group certainly seems to improve matters, provided that we observe the key pre-intervention variables. But Lalonde creates these subsets in an informal manner, based on one or two pre-intervention variables. Table 1 reveals that significant differences remain between the comparison groups and the treatment group. A more systematic means of creating such subsets should improve the estimates from both the CPS and PSID. We undertake this in Sections 3 and 4 with propensity score methods. Link

Tags: , ,