Sensitivity to the Specification of the Propensity Score
How sensitive are the estimates presented to the specification of the estimated propensity score? For the stratification estimator, as was suggested in Section 3, the exact specification of the estimated propensity score is not important as long as, within each stratum, the pre-intervention characteristics are balanced across the treatment and comparison groups. Since this was the basis of the specification search suggested in Section 3, either one can find a specification that balances pre-intervention characteristics, or one must conclude the treatment and comparison groups are irreconcilably different. Click Here
The upper half of Table 5 demonstrates that the estimates of the treatment impact are not particularly sensitive to the specification used. Specifications 1 and 4 are the same as those in Table 3 (hence, they balance the pre-intervention characteristics). In specifications 2 to 3 and 5 to 6, we drop the squares and cubes of the covariates, and then interactions and dummy variables. In specifications 3 and 6, the logits then simply use the covariates linearly. These estimates are worse than those in Table 3, ranging from $835 to $1,774. But compared with the range of estimates from Table 2, these remain concentrated. Furthermore, we are unable to find a partition structure for the alternative specifications such that the pre-intervention characteristics are balanced within each stratum. There is a well-defined criterion to reject these alternative specifications. Indeed, the specification search begins with a linear specification, and adds higher-order and interaction terms until within-stratum balance is achieved.