This vignette teaches you how to use the Cragg package to perform the Cragg-Donald (1993) and Stock and Yogo (2005) tests for weak instruments. Skip to “Testing for Weak Instruments” if you would just like to see example syntax and interpretation.
I use the following notation to describe the instrumental variables model: Yi = β0 + β1D1i + β2D2i + β3X1i + β4X2i + εi
Where
Y is the outcome of interest / dependent variable
X1 and X2 are control variables
D1 and D2 are the endogenous variables / treatments and are instrumented by Z1 , Z2 , Z3, and Z4
An assumption of instrumental variables analysis is that the instruments have a strong influence on the treatments / endogenous variables we are examining. If this is not the case, then IV estimators become unreliable.
In this vignette I will show the output of the functions in the cragg package when instruments are both strong and weak. To do this, I artificially create a pool of data in which the instruments strongly influence the treatment variables and another dataset which is just random data.
I first create a covariance matrix for a situation in which the instruments (Z1 , Z2 , Z3, and Z4) strongly influence the treatments (D1 and D2)
strong_cov_mat <- matrix(
c(
#Y #X1 #X2 #D1 #D2 #Z1 #Z2 #Z3 #Z4
3.8635, 0.0387, 0.0922, 1.4810, 0.4778, 0.2024, 0.1085, 0.0850, -0.0368, #Y
0.0387, 0.0832, -0.0029, 0.0415, -0.0014, 0.0025, 0.0009, -0.0004, -0.0058, #X1
0.0922, -0.0029, 0.0848, 0.0068, 0.0462, 0.0029, -0.0009, 0.0028, 0.0027, #X2
1.4810, 0.0415, 0.0068, 1.4345, 0.1949, 0.2184, 0.0562, 0.0464, -0.0155, #D1
0.4778, -0.0014, 0.0462, 0.1949, 0.4042, -0.0034, 0.0909, 0.0875, -0.0079, #D2
0.2024, 0.0025, 0.0029, 0.2184, -0.0034, 0.0851, 0.0005, -0.0043, 0.0000, #Z1
0.1085, 0.0009, -0.0009, 0.0562, 0.0909, 0.0005, 0.0840, 0.0074, -0.0096, #Z2
0.0850, -0.0004, 0.0028, 0.0464, 0.0875, -0.0043, 0.0074, 0.0831, -0.0005, #Z3
-0.0368, -0.0058, 0.0027, -0.0155, -0.0079, 0.0000, -0.0096, -0.0005, 0.0827 #Z4
),
ncol = 9)I then a covariance matrix for a situation in which the instruments (Z1 , Z2 , Z3, and Z4) have no influence on the treatments (D1 and D2) - this covariance matrix calculated from a set of random variables.
weak_cov_mat <- matrix(
c(
# Y X1 X2 D1 D2 Z1 Z2 Z3 Z4
0.0795, 0.0009, -0.0003, -0.0022, 0.0053, 0.0003, -0.0009, 0.0002, 0.0010, #Y
0.0009, 0.0832, -0.0029, 0.0020, -0.0036, 0.0025, 0.0009, -0.0004, -0.0058, #X1
-0.0003, -0.0029, 0.0848, 0.0035, 0.0040, 0.0029, -0.0009, 0.0028, 0.0027, #X2
-0.0022, 0.0020, 0.0035, 0.0866, -0.0027, 0.0001, 0.0030, -0.0013, -0.0041, #D1
0.0053, -0.0036, 0.0040, -0.0027, 0.0840, -0.0037, 0.0020, -0.0002, -0.0068, #D2
0.0003, 0.0025, 0.0029, 0.0001, -0.0037, 0.0851, 0.0005, -0.0043, 0.0000, #Z1
-0.0009, 0.0009, -0.0009, 0.0030, 0.0020, 0.0005, 0.0840, 0.0074, -0.0096, #Z2
0.0002, -0.0004, 0.0028, -0.0013, -0.0002, -0.0043, 0.0074, 0.0831, -0.0005, #Z3
0.0010, -0.0058, 0.0027, -0.0041, -0.0068, 0.0000, -0.0096, -0.0005, 0.0827 #Z4
),
ncol = 9)I then use these covariance matrices to generate a dataset with
strong instruments, strong_instrument_data, and a dataset
with completely ineffectual instruments,
weak_instrument_data.
library(MASS)
set.seed(42)
mu <- rep(0,9)
stddev <- rep(1,9)
strong_instrument_data <- as.data.frame(MASS::mvrnorm(n = 250, mu = mu, Sigma = strong_cov_mat, empirical = FALSE))
names(strong_instrument_data) <- c("Y", "X1", "X2","D1","D2","Z1","Z2","Z3","Z4")
weak_instrument_data <- as.data.frame(MASS::mvrnorm(n = 250, mu = mu, Sigma = weak_cov_mat, empirical = FALSE))
names(weak_instrument_data) <- c("Y", "X1", "X2","D1","D2","Z1","Z2","Z3","Z4")With the datasets generated, we can now move on to testing for the
presence of weak instruments. The cragg package offers two main
functions to do this: cragg_donald(), and
stock_yogo_test.
cragg_donald() implements the calculation for the
Cragg-Donald statistic (1993), which can be thought of as the
matrix-analogue of the first stage F-statistic.
stock_yogo_test() implements the Stock and Yogo (2005)
test for weak instruments. The Stock and Yogo test uses the
Cragg-Daniels statistic and a set of precalculated critical values to
ensure that the bias is less than 5, 10, or 30 percent of the worst-case
error from OLS.
I demonstrate how to calculate the Cragg-Donald statistic for a
specific model using the cragg_donald() function below. The
function takes three right-sided formulas as arguments for the control,
treatment and instruments. These can be given as either formulas of
variable names, or as vectors. The function takes an additional data
argument which can be a data frame, list, or environment where the
variables can be found. If the variables cannot be found in the object
given, the function may look upwards to see if it can find the objects
in the global environment, but this is not recommended and is not
guaranteed to work.
Here, I calculate the Cragg-Donald statistic for the dataset with strong instruments. I pass the function the names of the control, treatment, and instrumental variables; and the name of the dataframe where the variables can be found.
library(cragg)
cragg_donald(X=~X1 + X2, # Control Variables
D=~D1 + D2, # Treatments
Z=~Z1+Z2+Z3+Z4,# Instruments
data = strong_instrument_data)
#> Cragg-Donald test for weak instruments:
#>
#> Data: strong_instrument_data
#> Controls: ~X1 + X2
#> Treatments: ~D1 + D2
#> Instruments: ~Z1 + Z2 + Z3 + Z4
#>
#> Cragg-Donald Statistic: 40.57936
#> Df: 243This test gives a large CD-statistic which is safely large enough to conclude the instruments are strong. However, A more substantial claim can be made by using the critical values for this test created by Stock and Yogo (2005). The process for using the stock and Yogo test is described below.
The Stock and Yogo test provides critical values for the Cragg-Daniels statistic. These tests define weak instruments as when the bias from TSLS is greater than 5, 10, 20 or 30% of the bias of OLS, depending on the acceptable bias threshold [Andrews_2019]. The tests then reject if the 95% confidence interval excludes this possibility and fail to reject if the potential for this level of bias is not excluded by the confidence interval.
The null hypothesis (H0) is that the instruments are weak, and the alternative hypothesis is that they are not (H1)
To calculate this test using the cragg package, the
stock_yogo_test() function is used. The interface is the
same as that for the cragg_donald function described above,
but it can take an optional parameter to specify the largest acceptable
level of bias. The function can also take a size_bias
parameter which can be set from the default bias option to
size in order to use an alternative definition for bias
based on the size of the largest allowable t-test distortion.
I run this test on the dataset with the strong instruments and obtain the same Cragg-Donald statistic as above - 40.76.
stock_yogo_test(X=~X1+X2 , # Control Variables
D=~D1 + D2 , # Treatments
Z=~Z1+Z2+Z3+Z4 ,# Instruments
size_bias="bias", #Default
B=.05, #Default
data = strong_instrument_data)
#> Results of Stock and Yogo test for weak instruments:
#>
#> Null Hypothesis: Instruments are weak
#> Alternative Hypothesis: Instruments are not weak
#>
#> Data: strong_instrument_data
#> Controls: ~X1 + X2
#> Treatments: ~D1 + D2
#> Instruments: ~Z1 + Z2 + Z3 + Z4
#>
#> Alpha: 0.05
#> Acceptable level of bias: 5% relative to OLS.
#> Critical Value: 11.04
#>
#> Cragg-Donald Statistic: 40.57936
#> Df: 243We can see that this is significantly higher than the critical value of 11.04, so we have sufficient statistical evidence to reject the null hypothesis that instruments are weak.
As a demonstration, I run this test on the data with completely weak instruments.
stock_yogo_test(X=~X1+X2 , # Control Variables
D=~D1 + D2 , # Treatments
Z=~Z1+Z2+Z3+Z4 ,# Instruments
size_bias="bias", #Default
B=.05, #Default
data = weak_instrument_data)
#> Results of Stock and Yogo test for weak instruments:
#>
#> Null Hypothesis: Instruments are weak
#> Alternative Hypothesis: Instruments are not weak
#>
#> Data: weak_instrument_data
#> Controls: ~X1 + X2
#> Treatments: ~D1 + D2
#> Instruments: ~Z1 + Z2 + Z3 + Z4
#>
#> Alpha: 0.05
#> Acceptable level of bias: 5% relative to OLS.
#> Critical Value: 11.04
#>
#> Cragg-Donald Statistic: 0.1596519
#> Df: 243A Cragg-Donald statistic of .12 is significantly lower than the critical value of 11.04. As such, there is not sufficient evidence to reject the null hypothesis that the instruments are weak in this case.