Ö ˆ[‚y¡€½( øªkLÌ‘FM“ ¾K7…Nãj$jjçÐ%š£*€k„Ó&F–;'£ÿÒìPùï¾Ç¤’¤i¯Ä¹¦>?.1F¥}Í3µåô_®½÷â:'6š$w/õߟ÷Ž—€xm„×^XÉ꺗bÇá Wât» è[ßlbÞh÷ÃÄÏ«à{ÙúûðÙš jéÄ;O GϪâbâ—Q? `t¾Å/$9² [uW÷ÜÓ=®+†ÙÝ ¦¨ˆ¸:üØ€€{êSÖ Gñ‘HoÍEcl#’åµ:+ÛÙÆÀÌcà&x’¡rK“ÕÚOÑÈM™ñ.ÚrØGÑ›¹jÎ=”`ãí¥y+Ðd‘g.%˲æ*ÌfjŽ¿ó Ú¥¹
š‚ŠÅÚ :` _ÒÐ9¶ÚòPìúO¿§9¥¾²D Í~.«ýŠgÐURLk¶õ[éô¥ƒ¶Ó´UÝtÑcŠA†ËÂìë‚MFzÉΓÏ.þ2scz «° ¬ª©ñuîI>0
‹ YZ clubSandwich/NAMESPACE 0000644 0001762 0000144 00000005433 14634640072 014112 0 ustar ligges users # Generated by roxygen2: do not edit by hand
S3method(as.matrix,clubSandwich)
S3method(augmented_model_matrix,default)
S3method(augmented_model_matrix,plm)
S3method(bread,geeglm)
S3method(bread,gls)
S3method(bread,lme)
S3method(bread,lmerMod)
S3method(bread,mlm)
S3method(bread,plm)
S3method(bread,rma.mv)
S3method(bread,rma.uni)
S3method(bread,robu)
S3method(coef_CS,default)
S3method(coef_CS,lme)
S3method(coef_CS,lmerMod)
S3method(coef_CS,mlm)
S3method(coef_CS,rma.ls)
S3method(coef_CS,robu)
S3method(model_matrix,default)
S3method(model_matrix,geeglm)
S3method(model_matrix,glm)
S3method(model_matrix,gls)
S3method(model_matrix,ivreg)
S3method(model_matrix,lme)
S3method(model_matrix,lmerMod)
S3method(model_matrix,mlm)
S3method(model_matrix,plm)
S3method(model_matrix,rma.ls)
S3method(model_matrix,robu)
S3method(na.action,rma)
S3method(print,Wald_test_clubSandwich)
S3method(print,clubSandwich)
S3method(print,coef_test_clubSandwich)
S3method(print,conf_int_clubSandwich)
S3method(residuals_CS,default)
S3method(residuals_CS,geeglm)
S3method(residuals_CS,glm)
S3method(residuals_CS,lme)
S3method(residuals_CS,lmerMod)
S3method(residuals_CS,mlm)
S3method(residuals_CS,plm)
S3method(residuals_CS,rma)
S3method(residuals_CS,robu)
S3method(targetVariance,default)
S3method(targetVariance,geeglm)
S3method(targetVariance,glm)
S3method(targetVariance,gls)
S3method(targetVariance,lme)
S3method(targetVariance,lmerMod)
S3method(targetVariance,mlm)
S3method(targetVariance,plm)
S3method(targetVariance,rma.mv)
S3method(targetVariance,rma.uni)
S3method(targetVariance,robu)
S3method(v_scale,default)
S3method(v_scale,geeglm)
S3method(v_scale,glm)
S3method(v_scale,lm)
S3method(v_scale,lme)
S3method(v_scale,lmerMod)
S3method(v_scale,mlm)
S3method(v_scale,plm)
S3method(v_scale,rma.mv)
S3method(v_scale,robu)
S3method(vcovCR,default)
S3method(vcovCR,geeglm)
S3method(vcovCR,glm)
S3method(vcovCR,gls)
S3method(vcovCR,ivreg)
S3method(vcovCR,lm)
S3method(vcovCR,lme)
S3method(vcovCR,lmerMod)
S3method(vcovCR,mlm)
S3method(vcovCR,plm)
S3method(vcovCR,rma.mv)
S3method(vcovCR,rma.uni)
S3method(vcovCR,robu)
S3method(weightMatrix,default)
S3method(weightMatrix,geeglm)
S3method(weightMatrix,glm)
S3method(weightMatrix,gls)
S3method(weightMatrix,lme)
S3method(weightMatrix,lmerMod)
S3method(weightMatrix,mlm)
S3method(weightMatrix,plm)
S3method(weightMatrix,rma.mv)
S3method(weightMatrix,rma.uni)
S3method(weightMatrix,robu)
S3method(weights,robu)
export(Wald_test)
export(coef_test)
export(conf_int)
export(constrain_equal)
export(constrain_pairwise)
export(constrain_zero)
export(findCluster.rma.mv)
export(impute_covariance_matrix)
export(linear_contrast)
export(pattern_covariance_matrix)
export(vcovCR)
import(stats)
importFrom(sandwich,bread)
clubSandwich/NEWS.md 0000644 0001762 0000144 00000020702 14634640466 013774 0 ustar ligges users # clubSandwich 0.5.11
* Corrected a unit test related to the plm package, for compatibility with upcoming release of plm.
* Deprecated `impute_covariance_matrix()` and `pattern_covariance_matrix()`, because they have been superseded by `metafor::vcalc()`.
# clubSandwich 0.5.10
* Fixed another bug in `linear_contrast()` to handle specified contrasts that are scalars when variance-covariance matrix is computed with a working model that is not inverse-variance.
* Fixed formatting of package version numbers in unit tests to conform to changes in `packageVersion()` in R-devel.
# clubSandwich 0.5.9
* Added support for `geepack::geeglm()` models.
* Added support for `rma.ls` models (location-scale models estimated using `metafor::rma.uni(scale = )`).
* Improved error handling of `Wald_test()` when vcov of contrasts is not positive definite.
* Fixed a bug in `linear_contrast()` to handle specified contrasts that are scalars.
* Improved internal `get_data` function for `gls` and `lme` objects to allow use of expressions in addition to object names.
# clubSandwich 0.5.8
* Added support for `ivreg::ivreg` objects when estimated by ordinary least squares (support for objects estimated by 2SM and 2SMM is not yet implemented).
* Updated unit tests for `plm::plm()` when `method = "FD"` to account for bug fixes in version 2.6-2 of plm.
# clubSandwich 0.5.7
* Fixed bug in methods for multi-variate multi-level models estimated with lme().
* Updated vignettes, examples, and unit tests so that the package can be compiled without any packages from SUGGESTS.
# clubSandwich 0.5.6
* Corrected bug in methods for `plm` objects estimated by random effects, which occurred when a user-specified clustering variable was at a higher level than the random effects.
* Added support for `plm` objects with nested random effects (`effects = "nested"`).
* Added additional syntactic options for specifying clustering variable with `plm` objects. See `?plm`.
* Corrected bug in how `Wald_test()` labeled results when `test = "Naive-Fp"`.
# clubSandwich 0.5.5
* New function `linear_contrast()` calculates robust confidence intervals and p-values for linear contrasts of regression coefficients from a fitted model. Works with `constrain_pairwise()` and other `constrain_*()` helper functions.
* Corrected precision of unit test leading to error on M1mac.
# clubSandwich 0.5.4
## New features
* `Wald_test()` gains an option for `test = "Naive-Fp"`, which uses denominator degrees of freedom equal to the number of clusters minus the number of coefficients in the fitted model.
* `coef_test()` and `conf_int()` gain an option for `test = "naive-tp"`, which uses denominator degrees of freedom equal to the number of clusters minus the number of coefficients in the fitted model.
## Minor improvements and bug fixes
* Corrected a bug in the Satterthwaite degrees of freedom calculations for models that include only an intercept.
* Output of `coef_test()` and `conf_int()` now include a variable containing the coefficient names, so that the results are "tidy."
* `conf_int()` now includes an option to report a p-value for each coefficient.
* `coef_test()` now reports degrees of freedom for `test = 'z'` and `test = 'naive-t'`.
* `vcovCR()` now provides a more informative error message when the clustering variable is a constant.
* `vcovCR()` now handles models estimated using analytic weights, where some weights are equal to zero. Results are consistent with omitting observations with weights of zero.
* Added more informative error messages for `Wald_test()` and `conf_int()`, triggered if the test argument does not match any of the available tests.
* Corrected a bug in `findCluster.rma.mv()`, which threw an error if a random effects factor in the rma.mv model had unobserved levels.
* Corrected a bug in `Wald_test()`, which threw an error for tests of intercept-only models.
* Fixed a minor bug in print method for `Wald_test()` results, which threw an error when the p-value was `NA`.
# clubSandwich 0.5.3
* Removed dependency on mathjaxr
# clubSandwich 0.5.2
* Added mathjaxr to Imports
# clubSandwich 0.5.1
## New features
* New functionality for `impute_covariance_matrix()`:
* Compute covariance matrices with AR1 correlation structure or with a combination of constant correlation and AR1 correlation structure.
* Compute covariance matrices that are blocked by subgroup.
* Average the variance estimates by cluster before computing covariance matrices.
* New function `pattern_covariance_matrix()`, which creates a covariance matrix based on a specified pattern of correlations between different categories of effects.
## Minor improvements and bug fixes
* Corrected bug in methods for `rma.mv` objects, which previously led to incorrect identification of clustering variables in models with multiple levels of random effects, where at least one set of random effects has inner | outer structure.
# clubSandwich 0.5.0
## New features: a major update to `Wald_test()`
* `Wald_test()` now uses new helper functions `constrain_zero()`, `constrain_equal()`, and `constrain_pairwise()` to specify constraint matrices.
* `Wald_test()` gains an argument `tidy`. When `TRUE`, results for a list of tests will be tidied into a single data.frame.
* Output of `Wald_test()` now includes both numerator and denominator degrees of freedom.
## Minor improvements and bug fixes
* Corrected bug in methods for `plm` objects, which occurred when "within" models included cluster-level interactions. Previously main effects of cluster-level variables were not getting dropped from `model_matrix.plm()`.
* Corrected bugs in methods for `robu` objects
* Corrected a bug that previously led to errors for models with only one column in the model matrix (i.e., intercept-only models).
* Corrected a bug in an internal function that previously led to errors in `constrain_equal()` and `constrain_zero()` when called on robu objects.
# clubSandwich 0.4.2
* Updated and streamlined unit tests for R 4.0.0.
# clubSandwich 0.4.1
* Updated unit tests to satisfy obscure CRAN checks.
# clubSandwich 0.4.0
* Added methods for `lmerMod` objects fitted by `lme4::lmer()`.
* Updated internals to use `inherits()` instead of checking `class()` directly.
# clubSandwich 0.3.5
* Added t statistics to output of `coef_test()`.
* Fixed a bug in `get_index_order()`, an internal function used with plm objects. Previously, the function assumed that both individual and time indices were specified in the `plm` call. The new function works even when zero or one indices are specified.
# clubSandwich 0.3.3
* `impute_covariance_matrix()` now drops unobserved factor levels.
* updated method for handling residuals from `rma.uni` and `rma.mv` objects, for consistency with metafor 2.1-0.
# clubSandwich 0.3.2
* Added `conf_int()` to provide easy cluster-robust confidence intervals.
* Added examples to documentation for `conf_int()` and `coef_test()`.
# clubSandwich 0.3.1
* Added `coefs` option to `coef_test()` to allow testing of subsets of coefficients.
* Updated tests to use `carData` instead of car package.
# clubSandwich 0.3.0
* Added methods for `ivreg` objects.
* Added methods for `mlm` objects.
* Updated `residuals_CS.plm` to account for changes in plm 1.6-6.
# clubSandwich 0.2.3
## New features
* Added methods for `glm` objects.
* Provide facility to cluster at higher level than highest random effects for `lme` and `gls` objects.
* Added `impute_covariance_matrix()` utility function for multivariate meta-analysis.
## Minor improvements and bug fixes
* Updated methods for plm objects to account for changes in plm 1.6-6.
* Added documentation of `type` options in `vcovCR()`.
* Added examples for all `vcovCR()` methods.
# clubSandwich 0.2.2
## New features
* Added `bread()` methods for all supported model classes.
* `vcovCR()` is now calculated using `bread()`, and carries attributes for `bread`, `est_mat`, and adjustment matrices.
* `vcovCR()` gains a `form` argument to obtain just the meat of the sandwich, or to use a user-specified bread matrix.
## Minor improvements and bug fixes
* Refactored internal functions for degrees of freedom calculation to improve speed and memory usage.
* Bug fixes:
- updated `nobs.plm()` method to handle first-differenced models
# clubSandwich 0.2.1
* First version released on CRAN.
clubSandwich/inst/ 0000755 0001762 0000144 00000000000 14635065010 013635 5 ustar ligges users clubSandwich/inst/doc/ 0000755 0001762 0000144 00000000000 14635065010 014402 5 ustar ligges users clubSandwich/inst/doc/Wald-tests-in-clubSandwich.Rmd 0000644 0001762 0000144 00000053720 14630154052 022114 0 ustar ligges users ---
title: "Wald tests of multiple-constraint null hypotheses"
author: "James E. Pustejovsky"
date: "`r Sys.Date()`"
output:
rmarkdown::html_vignette:
number_sections: true
toc: true
bibliography: bibliography.bib
csl: apa.csl
vignette: >
%\VignetteIndexEntry{Wald tests of multiple-constraint null hypotheses}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, echo = FALSE, results = "asis", message = FALSE, warning = FALSE}
library(clubSandwich)
AER_available <- requireNamespace("AER", quietly = TRUE)
knitr::opts_chunk$set(eval = AER_available)
if (!AER_available) cat("# Building this vignette requires the AER package. Please install it. {-}")
```
Version 0.5.0 of `clubSandwich` introduced a new syntax for `Wald_test()`, a function for conducting tests of multiple-constraint hypotheses. In previous versions, this function was poorly documented and, consequently, probably little used. This vignette will demonstrate the new syntax.
For purposes of illustration, I will use the `STAR` data (available in the `AER` package), which is drawn from a randomized trial evaluating the effects of elementary school class size on student achievement. The data consist of individual-level measures for students in each of several dozen schools. For purposes of illustration, I will look at effects on math performance in first grade. Treatment conditions (the variable called `stark`) were assigned at the classroom level, and consisted of either a) a regular-size class, b) a small-size class, or c) a regular-size class but with the addition of a teacher's aide. In all of what follows, I will cluster standard errors by school in order to allow for generalization to a super-population of schools.
```{r, message = FALSE, warning = FALSE}
library(clubSandwich)
data(STAR, package = "AER")
# clean up a few variables
levels(STAR$stark)[3] <- "aide"
levels(STAR$schoolk)[1] <- "urban"
STAR <- subset(STAR,
!is.na(schoolidk),
select = c(schoolidk, schoolk, stark, gender, ethnicity, math1, lunchk))
head(STAR)
```
# The Wald test function
The `Wald_test()` function can be used to conduct hypothesis tests that involve multiple constraints on the regression coefficients. Consider a linear model for an outcome $Y_{ij}$ regressed on a $1 \times p$ row vector of predictors $\mathbf{x}_{ij}$ (which might include a constant intercept term):
$$
Y_{ij} = \mathbf{x}_{ij} \boldsymbol\beta + \epsilon_{ij}
$$
The regression coefficient vector is $\boldsymbol\beta$. In quite general terms, a set of constraints on the regression coefficient vector can be expressed in terms of a $q \times p$ matrix $\mathbf{C}$, where each row of $\mathbf{C}$ corresponds to one constraint. A joint null hypothesis is then $H_0: \mathbf{C} \boldsymbol\beta = \mathbf{0}$, where $\mathbf{0}$ is a $q \times 1$ vector of zeros.[^more-general]
[^more-general]: In @pustejovsky2018small we used a more general formulation of multiple-constraint null hypotheses, expressed as $H_0: \mathbf{C} \boldsymbol\beta = \mathbf{d}$ for some fixed $q \times 1$ vector $\mathbf{d}$. In practice, it's often possible to modify the $\mathbf{C}$ matrix so that $\mathbf{d}$ can always be set to $\mathbf{0}$.
Wald-type test are based on the test statistic
$$
Q = \left(\mathbf{C}\boldsymbol{\hat\beta}\right)' \left(\mathbf{C} \mathbf{V}^{CR} \mathbf{C}'\right)^{-1} \left(\mathbf{C}\boldsymbol{\hat\beta}\right),
$$
where $\boldsymbol{\hat\beta}$ is the estimated regression coefficient vector and $\mathbf{V}^{CR}$ is a cluster-robust variance matrix. If the number of clusters is sufficiently large, then the distribution of $Q$ under the null hypothesis is approximately $\chi^2(q)$. @tipton2015small investigated a wide range of other approximations to the null distribution of $Q$, many of which are included as options in `Wald_test()`. Based on a large simulation, they (...er...we...) recommended a method called the "approximate Hotelling's $T^2$-Z" test, or "AHZ." This test approximates the distribution of $Q / q$ by a $T^2$ distribution, which is a multiple of an $F$ distribution, with numerator degrees of freedom $q$ and denominator degrees of freedom based on a generalization of the Satterthwaite approximation.
The `Wald_test()` function has three main arguments:
```{r}
args(Wald_test)
```
* The `obj` argument is used to specify the estimated regression model on which to perform the test,
* the `constraints` argument is a $\mathbf{C}$ matrix expressing the set of constraints to test, and
* the `vcov` argument is a cluster-robust variance matrix, which is used to construct the test statistic. (Alternately, `vcov` can be the type of cluster-robust variance matrix to construct, in which case it will be computed internally.)
By default, `Wald_test()` will use the HTZ small-sample approximation. Other options are available (via the `test` argument) but not recommended for routine use. The optional `tidy` argument will be demonstrated below.
## Testing treatment effects
Returning to the STAR data, let's suppose we want to examine differences in math performance across class sizes. This can be done with a simple linear regression model, while clustering the standard errors by `schoolidk`. The estimating equation is
$$
\left(\text{Math}\right)_{ij} = \beta_0 + \beta_1 \left(\text{small}\right)_{ij} + \beta_2 \left(\text{aide}\right)_{ij} + e_{ij},
$$
which can be estimated in R as follows:
```{r type-treat}
lm_trt <- lm(math1 ~ stark, data = STAR)
V_trt <- vcovCR(lm_trt, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_trt, vcov = V_trt)
```
In this estimating equation, the coefficients $\beta_1$ and $\beta_2$ represent treatment effects, or differences in average math scores relative to the reference level of `stark`, which in this case is a regular-size class. The t-statistics and p-values reported by `coef_test` are separate tests of the null hypotheses that each of these coefficients are equal to zero, meaning that there is no difference between the specified treatment condition and the reference level. We might want to instead test the _joint_ null hypothesis that there are no differences among _any_ of the conditions. This null can be expressed by a set of multiple constraints on the parameters: $\beta_1 = 0$ and $\beta_2 = 0$.
To test the null hypothesis that $\beta_1 = \beta_2 = 0$ based on the treatment effects model specification, we can use:
```{r}
C_trt <- matrix(c(0,0,1,0,0,1), 2, 3)
C_trt
Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)
```
The result includes details about the form of `test` computed, the $F$-statistic, the numerator and denominator degrees of freedom used to compute the reference distribution, and the $p$-value corresponding to the specified null hypothesis. In this example, $p = `r if (AER_available) format.pval(Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)$p_val, digits = 3)`$, so we can rule out the null hypothesis that there are no differences in math performance across conditions.
The representation of null hypotheses as arbitrary constraint matrices is useful for developing theory about how to test such hypotheses, but it is not all that helpful for actually running tests---constructing constraint matrices "by hand" is just too cumbersome of an exercise. Moreover, $\mathbf{C}$ matrices typically follow one of a small number of patterns. Two common use cases are a) constraining a set of $q > 1$ parameters to all be equal to zero and b) constraining a set of $q + 1$ parameters to be equal to a common value. The `clubSandwich` package now includes a set of helper functions to create constraint matrices for these common use cases.
## `constrain_zero()`
To constrain a set of $q$ regression coefficients to all be equal to zero, the simplest form of the $\mathbf{C}$ matrix would consist of a set of $q$ rows, where a single entry in each row would be equal to 1 and the remaining entries would all be zero. For the `lm_trt` model, the C matrix would look like this:
$$
\mathbf{C} = \left[\begin{array}{ccc} 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right],
$$
so that
$$
\mathbf{C}\boldsymbol\beta = \left[\begin{array}{ccc} 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right] \left[\begin{array}{c} \beta_0 \\ \beta_1 \\ \beta_2 \end{array} \right] = \left[\begin{array}{c} \beta_1 \\ \beta_2 \end{array} \right],
$$
which is set equal to $\left[\begin{array}{c} 0 \\ 0 \end{array} \right]$.
The `constrain_zero()` function will create matrices like this automatically. The function takes two main arguments:
```{r}
args(constrain_zero)
```
* The `constraints` argument is used to specify _which_ coefficients in a regression model to set equal to zero.
* The `coefs` argument is the set of estimated regression coefficients, for which to calculate the constraints.
Constraints can be specified by position index, by name, or via a regular expression. To test the joint null hypothesis that average math performance is equal across the three treatment conditions, we need to constrain the second and third coefficients to zero:
```{r}
constrain_zero(2:3, coefs = coef(lm_trt))
```
Or equivalently:
```{r}
constrain_zero(c("starksmall","starkaide"), coefs = coef(lm_trt))
```
or
```{r}
constrain_zero("^stark", coefs = coef(lm_trt), reg_ex = TRUE)
```
Note that if `constraints` is a regular expression, then the `reg_ex` argument needs to be set to `TRUE`.
The result of `constrain_zero()` can then be fed into the `Wald_test()` function:
```{r}
C_trt <- constrain_zero(2:3, coefs = coef(lm_trt))
Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)
```
To reduce redundancy in the syntax, we can also omit the `coefs` argument to `constrain_zero`, so long as we call it inside of `Wald_test`[^under-the-hood]:
```{r}
Wald_test(lm_trt, constraints = constrain_zero(2:3), vcov = V_trt)
```
[^under-the-hood]: How does this work? If we omit the `coefs` argument, `constrain_zero()` acts as a functional, by returning a function equivalent to `function(coefs) constrain_zero(constraints, coefs = coefs)`. If this function is fed into the `constraints` argument of `Wald_test()`, `Wald_test()` recognizes that it is a function and evaluates the function with `coef(obj)`. It's a kinda-sorta hacky substitute for lazy evaluation. If you have suggestions for how to do this more elegantly, please send them my way.
## `constrain_equal()`
Another common type of constraints involve setting a set of $q + 1$ regression coefficients to be all equal to a common (but unknown) value ($q + 1$ because it takes $q$ constraints to do this). There are many equivalent ways to express such a set of constraints in terms of a $\mathbf{C}$ matrix. One fairly simple form consists of a set of $q$ rows, where the entry corresponding to one of the coefficients of interest is equal to -1 and the entry corresponding to another coefficient of interest is equal to 1.
To see how this works, let's look at a different way of parameterizing our simple model for the STAR data, by using separate intercepts for each treatment condition. The estimating equation would then be
$$
\left(\text{Math}\right)_{ij} = \beta_0 \left(\text{regular}\right)_{ij} + \beta_1 \left(\text{small}\right)_{ij} + \beta_2 \left(\text{aide}\right)_{ij} + e_{ij}.
$$
This model can be estimated in R by dropping the intercept term:
```{r type-sep}
lm_sep <- lm(math1 ~ 0 + stark, data = STAR)
V_sep <- vcovCR(lm_sep, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_sep, vcov = V_sep)
```
In this parameterization, the coefficients $\beta_0$, $\beta_1$, and $\beta_2$ represent the average math performance levels of students in each of the treatment conditions. The t-tests and p-values now have a very different interpretation because they pertain to the null hypothesis that the average performance level for a given condition is equal to zero. With this separate-intercepts model, the joint null hypothesis that performance levels are equal across conditions amounts to constraining the intercepts to be equal to each other: $\beta_0 = \beta_1$ and $\beta_0 = \beta_2$ (note that we don't need the constraint $\beta_1 = \beta_2$ because it is implied by the first two).
For the `lm_sep` model, which has separate intercepts $\beta_0$, $\beta_1$, and $\beta_2$, the C matrix would look like this:
$$
\mathbf{C} = \left[\begin{array}{ccc} -1 & 1 & 0 \\ -1 & 0 & 1 \end{array} \right],
$$
so that
$$
\mathbf{C}\boldsymbol\beta = \left[\begin{array}{ccc} -1 & 1 & 0 \\ -1 & 0 & 1 \end{array} \right] \left[\begin{array}{c} \beta_0 \\ \beta_1 \\ \beta_2 \end{array} \right] = \left[\begin{array}{c} \beta_1 - \beta_0 \\ \beta_2 - \beta_0 \end{array} \right],
$$
which is set equal to $\left[\begin{array}{c} 0 \\ 0 \end{array} \right]$.
The `constrain_equal()` function will create matrices like this automatically, given a set of coefficients to constrain. The syntax is identical to `constrain_zero()`:
```{r}
args(constrain_equal)
```
To test the joint null hypothesis that average math performance is equal across the three treatment conditions, we can constrain all three coefficients of `lm_sep` to be equal:
```{r}
constrain_equal(1:3, coefs = coef(lm_sep))
```
Or equivalently:
```{r}
constrain_equal(c("starkregular","starksmall","starkaide"), coefs = coef(lm_sep))
```
or
```{r}
constrain_equal("^stark", coefs = coef(lm_sep), reg_ex = TRUE)
```
Just as with `constrain_zero`, if `constraints` is a regular expression, then the `reg_ex` argument needs to be set to `TRUE`.
This constraint matrix can then be fed into `Wald_test()`:
```{r}
C_sep <- constrain_equal("^stark", coefs = coef(lm_sep), reg_ex = TRUE)
Wald_test(lm_sep, constraints = C_sep, vcov = V_sep)
```
or equivalently:
```{r}
Wald_test(lm_sep, constraints = constrain_equal(1:3), vcov = V_sep)
```
Note that these test results are exactly equal to the tests based on `lm_trt` with `constrain_zero()`. They're algebraically equivalent---just different ways of parameterizing the same model and constraints.
# Testing an interaction
Let's now consider how these functions can be applied in a more complex model. Suppose that we are interested in understanding whether the effect of being in a small class is consistent across schools in different areas, where areas are categorized as urban, suburban, or rural. To answer this question, we need to test for an interaction between urbanicity and treatment condition. One estimating equation that would let us examine this question is:
$$
\begin{aligned}
\left(\text{Math}\right)_{ij} &= \beta_0 + \beta_1 \left(\text{suburban}\right)_{ij} + \beta_2 \left(\text{rural}\right)_{ij} \\
& \quad + \beta_3 \left(\text{small}\right)_{ij} + \beta_4 \left(\text{aide}\right)_{ij} \\
& \quad\quad + \beta_5 \left(\text{small}\right)(\text{suburban})_{ij} + \beta_6 \left(\text{aide}\right)(\text{suburban})_{ij} \\
& \quad\quad\quad + \beta_{7} \left(\text{small}\right)(\text{rural})_{ij} + \beta_{8} \left(\text{aide}\right)(\text{rural})_{ij} \\
& \quad\quad\quad\quad + \mathbf{x}_{ij} \boldsymbol\gamma + e_{ij},
\end{aligned}
$$
where $\mathbf{x}_{ij}$ is a row vector of student characteristics, included just to make the regression look fancier. In this specification, $\beta_3$ and $\beta_4$ represent the effects of being in a small class or aide class, compared to being in a regular class, but _only for the reference level of urbanicity_---in this case, urban schools. The coefficients $\beta_5, \beta_6, \beta_7, \beta_8$ all represent _interactions_ between treatment condition and urbanicity. Here's the model, estimated in R:
```{r}
lm_urbanicity <- lm(math1 ~ schoolk * stark + gender + ethnicity + lunchk, data = STAR)
V_urbanicity <- vcovCR(lm_urbanicity, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_urbanicity, vcov = V_urbanicity)
```
With this specification, there are several different null hypotheses that we might want to test. For one, perhaps we want to see if there is _any_ variation in treatment effects across different levels of urbanicity. This can be expressed in the null hypothesis that all four interaction terms are zero, or $H_{0A}: \beta_5 = \beta_6 = \beta_7 = \beta_8 = 0$. With Wald test:
```{r}
Wald_test(lm_urbanicity,
constraints = constrain_zero("schoolk.+:stark", reg_ex = TRUE),
vcov = V_urbanicity)
```
Another possibility is that we might want to focus on variation in the effect of being in a small class or regular class, while ignoring whatever is going on in the aide class condition. Here, the null hypothesis would be simply $H_{0B}: \beta_5 = \beta_6 = 0$, tested as:
```{r}
Wald_test(lm_urbanicity,
constraints = constrain_zero("schoolk.+:starksmall", reg_ex = TRUE),
vcov = V_urbanicity)
```
## Lists of constraints
In models like the urbanicity-by-treatment interaction specification, we may need to run multiple tests on the same estimating equation. This can be accomplished with `Wald_test` by providing a _list_ of constraints to the `constraints` argument. For example, we could test the hypotheses described above by creating a list of several constraint matrices and then passing it to `Wald_test`:
```{r}
C_list <- list(
`Any interaction` = constrain_zero("schoolk.+:stark",
coef(lm_urbanicity), reg_ex = TRUE),
`Small vs regular` = constrain_zero("schoolk.+:starksmall",
coef(lm_urbanicity), reg_ex = TRUE)
)
Wald_test(lm_urbanicity,
constraints = C_list,
vcov = V_urbanicity)
```
Setting the option `tidy = TRUE` will arrange the output of all the tests into a single data frame:
```{r}
Wald_test(lm_urbanicity,
constraints = C_list,
vcov = V_urbanicity,
tidy = TRUE)
```
The list of constraints can also be created inside `Wald_test`, so that the `coefs` argument can be omitted from `constrain_zero()`:
```{r}
Wald_test(
lm_urbanicity,
constraints = list(
`Any interaction` = constrain_zero("schoolk.+:stark", reg_ex = TRUE),
`Small vs regular` = constrain_zero("schoolk.+:starksmall", reg_ex = TRUE)
),
vcov = V_urbanicity,
tidy = TRUE
)
```
# Pairwise t-tests
The `clubSandwich` package also provides a further convenience function, `constrain_pairwise()` that can be used in combination with `Wald_test()` to conduct pairwise comparisons among a set of regression coefficients. This function differs from the other two `constrain_*()` functions because it returns a _list_ of constraint matrices, each of which corresponds to a single linear combination of covariates. Specifically, the `constrain_pairwise()` function provides a list of constraints that represent the differences between every possible pair among a specified set of coefficients. The syntax is very similar to the other `constrain_*()` functions.
To demonstrate, consider the separate-intercepts specification of the simpler regression model:
```{r}
coef_test(lm_sep, vcov = V_sep)
```
This specification is nice because it lets us simply read off the average outcomes for each group. However, we will naturally also want to know about whether there are differences between the groups, so we'll want to compare the small-class condition to the regular-size class condition, the aide condition to the regular-size class condition, and the small-class condition to the aide condition. Thus, we'll want comparisons among all three coefficients:
```{r}
C_pairs <- constrain_pairwise(1:3, coefs = coef(lm_sep))
C_pairs
```
Feeding these constraints into `Wald_test()` gives us significance tests for each pair:
```{r}
Wald_test(lm_sep, constraints = C_pairs, vcov = V_sep, tidy = TRUE)
```
The first two of these tests are equivalent to the tests of the treatment effect coefficients in the other parameterization of the model. Indeed, the denominator degrees of freedom are identical to the results of `coef_test(lm_trt, vcov = V_trt)`; the `Fstat`s here are equal to the squared t-statistics from the first model:
```{r}
t_stats <- coef_test(lm_trt, vcov = V_trt)$tstat[2:3]
F_stats <- Wald_test(lm_sep, constraints = C_pairs, vcov = V_sep, tidy = TRUE)$Fstat[1:2]
all.equal(t_stats^2, F_stats)
```
It is important to note that the p-values from the pairwise comparisons are _not_ corrected for multiplicity.[^multiplicity] For now, please correct-your-own using `p.adjust()` or your preferred method.
[^multiplicity]: Options to include multiplicity corrections (Bonferroni, Holm, Benjamini-Hochberg, etc.) might be included in a [future release](https://github.com/jepusto/clubSandwich/issues/33). Reach out if this is of interest to you.
Pairwise comparisons might also be of use in the model with treatment-by-urbanicity interactions. Here's the model results again:
```{r}
coef_test(lm_urbanicity, vcov = V_urbanicity)
```
Suppose that we are interested in the effect of small versus regular size classes, and in particular whether this effect varies across schools in different areas. The coefficients on `schoolksuburban:starksmall` and `schoolkrural:starksmall` already give us the differences in treatment effects between suburban schools versus urban schools and between rural schools versus urban schools. The difference between these coefficients gives us the difference in treatment effects between suburban schools and rural schools. We can look at all three of these contrasts using `constrain_pairwise()` by setting the option `with_zero = TRUE`:
```{r}
Wald_test(lm_urbanicity,
constraints = constrain_pairwise(":starksmall", reg_ex = TRUE, with_zero = TRUE),
vcov = V_urbanicity,
tidy = TRUE)
```
Again, the results of the first two tests are identical to the t-tests reported in `coef_test()`.
# Remark
All of the preceding examples were based on ordinary linear regression models with clustered standard errors. However, `Wald_test()` and its helper functions all work identically for all of the other models with supporting `clubSandwich` methods, including `nlme::lme()`, `nlme::gls()`, `lme4::lmer()`, `rma.uni()`, `rma.mv()`, and `robu()`, among others.
# References
clubSandwich/inst/doc/Wald-tests-in-clubSandwich.html 0000644 0001762 0000144 00000166605 14635065002 022346 0 ustar ligges users
Wald tests of multiple-constraint null hypotheses
Wald tests of multiple-constraint null
hypotheses
James E. Pustejovsky
2024-06-20
Version 0.5.0 of clubSandwich introduced a new syntax
for Wald_test(), a function for conducting tests of
multiple-constraint hypotheses. In previous versions, this function was
poorly documented and, consequently, probably little used. This vignette
will demonstrate the new syntax.
For purposes of illustration, I will use the STAR data
(available in the AER package), which is drawn from a
randomized trial evaluating the effects of elementary school class size
on student achievement. The data consist of individual-level measures
for students in each of several dozen schools. For purposes of
illustration, I will look at effects on math performance in first grade.
Treatment conditions (the variable called stark) were
assigned at the classroom level, and consisted of either a) a
regular-size class, b) a small-size class, or c) a regular-size class
but with the addition of a teacher’s aide. In all of what follows, I
will cluster standard errors by school in order to allow for
generalization to a super-population of schools.
library(clubSandwich)
data(STAR, package = "AER")
# clean up a few variables
levels(STAR$stark)[3] <- "aide"
levels(STAR$schoolk)[1] <- "urban"
STAR <- subset(STAR,
!is.na(schoolidk),
select = c(schoolidk, schoolk, stark, gender, ethnicity, math1, lunchk))
head(STAR)
## schoolidk schoolk stark gender ethnicity math1 lunchk
## 1137 63 rural small female cauc 538 non-free
## 1143 20 suburban small female afam 592 non-free
## 1183 19 urban aide male afam NA free
## 1277 69 rural regular male cauc 584 non-free
## 1292 79 rural small male cauc 545 free
## 1308 5 rural regular male cauc 553 free
The Wald test
function
The Wald_test() function can be used to conduct
hypothesis tests that involve multiple constraints on the regression
coefficients. Consider a linear model for an outcome \(Y_{ij}\) regressed on a \(1 \times p\) row vector of predictors \(\mathbf{x}_{ij}\) (which might include a
constant intercept term): \[
Y_{ij} = \mathbf{x}_{ij} \boldsymbol\beta + \epsilon_{ij}
\] The regression coefficient vector is \(\boldsymbol\beta\). In quite general terms,
a set of constraints on the regression coefficient vector can be
expressed in terms of a \(q \times p\)
matrix \(\mathbf{C}\), where each row
of \(\mathbf{C}\) corresponds to one
constraint. A joint null hypothesis is then \(H_0: \mathbf{C} \boldsymbol\beta =
\mathbf{0}\), where \(\mathbf{0}\) is a \(q \times 1\) vector of zeros.
Wald-type test are based on the test statistic \[
Q = \left(\mathbf{C}\boldsymbol{\hat\beta}\right)' \left(\mathbf{C}
\mathbf{V}^{CR} \mathbf{C}'\right)^{-1}
\left(\mathbf{C}\boldsymbol{\hat\beta}\right),
\] where \(\boldsymbol{\hat\beta}\) is the estimated
regression coefficient vector and \(\mathbf{V}^{CR}\) is a cluster-robust
variance matrix. If the number of clusters is sufficiently large, then
the distribution of \(Q\) under the
null hypothesis is approximately \(\chi^2(q)\). Tipton
& Pustejovsky (2015) investigated a wide range of other
approximations to the null distribution of \(Q\), many of which are included as options
in Wald_test(). Based on a large simulation, they (…er…we…)
recommended a method called the “approximate Hotelling’s \(T^2\)-Z†test, or “AHZ.†This test
approximates the distribution of \(Q /
q\) by a \(T^2\) distribution,
which is a multiple of an \(F\)
distribution, with numerator degrees of freedom \(q\) and denominator degrees of freedom
based on a generalization of the Satterthwaite approximation.
The Wald_test() function has three main arguments:
## function (obj, constraints, vcov, test = "HTZ", tidy = FALSE,
## ...)
## NULL
- The
obj argument is used to specify the estimated
regression model on which to perform the test,
- the
constraints argument is a \(\mathbf{C}\) matrix expressing the set of
constraints to test, and
- the
vcov argument is a cluster-robust variance matrix,
which is used to construct the test statistic. (Alternately,
vcov can be the type of cluster-robust variance matrix to
construct, in which case it will be computed internally.)
By default, Wald_test() will use the HTZ small-sample
approximation. Other options are available (via the test
argument) but not recommended for routine use. The optional
tidy argument will be demonstrated below.
Testing treatment
effects
Returning to the STAR data, let’s suppose we want to examine
differences in math performance across class sizes. This can be done
with a simple linear regression model, while clustering the standard
errors by schoolidk. The estimating equation is \[
\left(\text{Math}\right)_{ij} = \beta_0 + \beta_1
\left(\text{small}\right)_{ij} + \beta_2 \left(\text{aide}\right)_{ij} +
e_{ij},
\] which can be estimated in R as follows:
lm_trt <- lm(math1 ~ stark, data = STAR)
V_trt <- vcovCR(lm_trt, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_trt, vcov = V_trt)
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## (Intercept) 531.727 2.78 191.506 59.9 <0.001 ***
## starksmall 9.469 2.30 4.114 65.6 <0.001 ***
## starkaide -0.483 1.86 -0.259 65.6 0.796
In this estimating equation, the coefficients \(\beta_1\) and \(\beta_2\) represent treatment effects, or
differences in average math scores relative to the reference level of
stark, which in this case is a regular-size class. The
t-statistics and p-values reported by coef_test are
separate tests of the null hypotheses that each of these coefficients
are equal to zero, meaning that there is no difference between the
specified treatment condition and the reference level. We might want to
instead test the joint null hypothesis that there are no
differences among any of the conditions. This null can be
expressed by a set of multiple constraints on the parameters: \(\beta_1 = 0\) and \(\beta_2 = 0\).
To test the null hypothesis that \(\beta_1
= \beta_2 = 0\) based on the treatment effects model
specification, we can use:
C_trt <- matrix(c(0,0,1,0,0,1), 2, 3)
C_trt
## [,1] [,2] [,3]
## [1,] 0 1 0
## [2,] 0 0 1
Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)
## test Fstat df_num df_denom p_val sig
## HTZ 10.2 2 65.3 <0.001 ***
The result includes details about the form of test
computed, the \(F\)-statistic, the
numerator and denominator degrees of freedom used to compute the
reference distribution, and the \(p\)-value corresponding to the specified
null hypothesis. In this example, \(p =
0.000141\), so we can rule out the null hypothesis that there are
no differences in math performance across conditions.
The representation of null hypotheses as arbitrary constraint
matrices is useful for developing theory about how to test such
hypotheses, but it is not all that helpful for actually running
tests—constructing constraint matrices “by hand†is just too cumbersome
of an exercise. Moreover, \(\mathbf{C}\) matrices typically follow one
of a small number of patterns. Two common use cases are a) constraining
a set of \(q > 1\) parameters to all
be equal to zero and b) constraining a set of \(q + 1\) parameters to be equal to a common
value. The clubSandwich package now includes a set of
helper functions to create constraint matrices for these common use
cases.
constrain_zero()
To constrain a set of \(q\)
regression coefficients to all be equal to zero, the simplest form of
the \(\mathbf{C}\) matrix would consist
of a set of \(q\) rows, where a single
entry in each row would be equal to 1 and the remaining entries would
all be zero. For the lm_trt model, the C matrix would look
like this: \[
\mathbf{C} = \left[\begin{array}{ccc} 0 & 1 & 0 \\ 0 & 0
& 1 \end{array} \right],
\] so that \[
\mathbf{C}\boldsymbol\beta = \left[\begin{array}{ccc} 0 & 1 & 0
\\ 0 & 0 & 1 \end{array} \right] \left[\begin{array}{c} \beta_0
\\ \beta_1 \\ \beta_2 \end{array} \right] = \left[\begin{array}{c}
\beta_1 \\ \beta_2 \end{array} \right],
\] which is set equal to \(\left[\begin{array}{c} 0 \\ 0 \end{array}
\right]\).
The constrain_zero() function will create matrices like
this automatically. The function takes two main arguments:
## function (constraints, coefs, reg_ex = FALSE)
## NULL
- The
constraints argument is used to specify
which coefficients in a regression model to set equal to
zero.
- The
coefs argument is the set of estimated regression
coefficients, for which to calculate the constraints.
Constraints can be specified by position index, by name, or via a
regular expression. To test the joint null hypothesis that average math
performance is equal across the three treatment conditions, we need to
constrain the second and third coefficients to zero:
constrain_zero(2:3, coefs = coef(lm_trt))
## [,1] [,2] [,3]
## [1,] 0 1 0
## [2,] 0 0 1
Or equivalently:
constrain_zero(c("starksmall","starkaide"), coefs = coef(lm_trt))
## [,1] [,2] [,3]
## [1,] 0 1 0
## [2,] 0 0 1
or
constrain_zero("^stark", coefs = coef(lm_trt), reg_ex = TRUE)
## [,1] [,2] [,3]
## [1,] 0 1 0
## [2,] 0 0 1
Note that if constraints is a regular expression, then
the reg_ex argument needs to be set to
TRUE.
The result of constrain_zero() can then be fed into the
Wald_test() function:
C_trt <- constrain_zero(2:3, coefs = coef(lm_trt))
Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)
## test Fstat df_num df_denom p_val sig
## HTZ 10.2 2 65.3 <0.001 ***
To reduce redundancy in the syntax, we can also omit the
coefs argument to constrain_zero, so long as
we call it inside of Wald_test:
Wald_test(lm_trt, constraints = constrain_zero(2:3), vcov = V_trt)
## test Fstat df_num df_denom p_val sig
## HTZ 10.2 2 65.3 <0.001 ***
constrain_equal()
Another common type of constraints involve setting a set of \(q + 1\) regression coefficients to be all
equal to a common (but unknown) value (\(q +
1\) because it takes \(q\)
constraints to do this). There are many equivalent ways to express such
a set of constraints in terms of a \(\mathbf{C}\) matrix. One fairly simple form
consists of a set of \(q\) rows, where
the entry corresponding to one of the coefficients of interest is equal
to -1 and the entry corresponding to another coefficient of interest is
equal to 1.
To see how this works, let’s look at a different way of
parameterizing our simple model for the STAR data, by using separate
intercepts for each treatment condition. The estimating equation would
then be \[
\left(\text{Math}\right)_{ij} = \beta_0 \left(\text{regular}\right)_{ij}
+ \beta_1 \left(\text{small}\right)_{ij} + \beta_2
\left(\text{aide}\right)_{ij} + e_{ij}.
\] This model can be estimated in R by dropping the intercept
term:
lm_sep <- lm(math1 ~ 0 + stark, data = STAR)
V_sep <- vcovCR(lm_sep, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_sep, vcov = V_sep)
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## starkregular 532 2.78 192 59.9 <0.001 ***
## starksmall 541 2.89 187 65.0 <0.001 ***
## starkaide 531 2.72 195 64.3 <0.001 ***
In this parameterization, the coefficients \(\beta_0\), \(\beta_1\), and \(\beta_2\) represent the average math
performance levels of students in each of the treatment conditions. The
t-tests and p-values now have a very different interpretation because
they pertain to the null hypothesis that the average performance level
for a given condition is equal to zero. With this separate-intercepts
model, the joint null hypothesis that performance levels are equal
across conditions amounts to constraining the intercepts to be equal to
each other: \(\beta_0 = \beta_1\) and
\(\beta_0 = \beta_2\) (note that we
don’t need the constraint \(\beta_1 =
\beta_2\) because it is implied by the first two).
For the lm_sep model, which has separate intercepts
\(\beta_0\), \(\beta_1\), and \(\beta_2\), the C matrix would look like
this: \[
\mathbf{C} = \left[\begin{array}{ccc} -1 & 1 & 0 \\ -1 & 0
& 1 \end{array} \right],
\] so that \[
\mathbf{C}\boldsymbol\beta = \left[\begin{array}{ccc} -1 & 1 & 0
\\ -1 & 0 & 1 \end{array} \right] \left[\begin{array}{c} \beta_0
\\ \beta_1 \\ \beta_2 \end{array} \right] = \left[\begin{array}{c}
\beta_1 - \beta_0 \\ \beta_2 - \beta_0 \end{array} \right],
\] which is set equal to \(\left[\begin{array}{c} 0 \\ 0 \end{array}
\right]\).
The constrain_equal() function will create matrices like
this automatically, given a set of coefficients to constrain. The syntax
is identical to constrain_zero():
## function (constraints, coefs, reg_ex = FALSE)
## NULL
To test the joint null hypothesis that average math performance is
equal across the three treatment conditions, we can constrain all three
coefficients of lm_sep to be equal:
constrain_equal(1:3, coefs = coef(lm_sep))
## [,1] [,2] [,3]
## [1,] -1 1 0
## [2,] -1 0 1
Or equivalently:
constrain_equal(c("starkregular","starksmall","starkaide"), coefs = coef(lm_sep))
## [,1] [,2] [,3]
## [1,] -1 1 0
## [2,] -1 0 1
or
constrain_equal("^stark", coefs = coef(lm_sep), reg_ex = TRUE)
## [,1] [,2] [,3]
## [1,] -1 1 0
## [2,] -1 0 1
Just as with constrain_zero, if constraints
is a regular expression, then the reg_ex argument needs to
be set to TRUE.
This constraint matrix can then be fed into
Wald_test():
C_sep <- constrain_equal("^stark", coefs = coef(lm_sep), reg_ex = TRUE)
Wald_test(lm_sep, constraints = C_sep, vcov = V_sep)
## test Fstat df_num df_denom p_val sig
## HTZ 10.2 2 65.3 <0.001 ***
or equivalently:
Wald_test(lm_sep, constraints = constrain_equal(1:3), vcov = V_sep)
## test Fstat df_num df_denom p_val sig
## HTZ 10.2 2 65.3 <0.001 ***
Note that these test results are exactly equal to the tests based on
lm_trt with constrain_zero(). They’re
algebraically equivalent—just different ways of parameterizing the same
model and constraints.
Testing an
interaction
Let’s now consider how these functions can be applied in a more
complex model. Suppose that we are interested in understanding whether
the effect of being in a small class is consistent across schools in
different areas, where areas are categorized as urban, suburban, or
rural. To answer this question, we need to test for an interaction
between urbanicity and treatment condition. One estimating equation that
would let us examine this question is: \[
\begin{aligned}
\left(\text{Math}\right)_{ij} &= \beta_0 + \beta_1
\left(\text{suburban}\right)_{ij} + \beta_2
\left(\text{rural}\right)_{ij} \\
& \quad + \beta_3 \left(\text{small}\right)_{ij} + \beta_4
\left(\text{aide}\right)_{ij} \\
& \quad\quad + \beta_5
\left(\text{small}\right)(\text{suburban})_{ij} + \beta_6
\left(\text{aide}\right)(\text{suburban})_{ij} \\
& \quad\quad\quad + \beta_{7}
\left(\text{small}\right)(\text{rural})_{ij} + \beta_{8}
\left(\text{aide}\right)(\text{rural})_{ij} \\
& \quad\quad\quad\quad + \mathbf{x}_{ij} \boldsymbol\gamma +
e_{ij},
\end{aligned}
\] where \(\mathbf{x}_{ij}\) is
a row vector of student characteristics, included just to make the
regression look fancier. In this specification, \(\beta_3\) and \(\beta_4\) represent the effects of being in
a small class or aide class, compared to being in a regular class, but
only for the reference level of urbanicity—in this case, urban
schools. The coefficients \(\beta_5, \beta_6,
\beta_7, \beta_8\) all represent interactions between
treatment condition and urbanicity. Here’s the model, estimated in
R:
lm_urbanicity <- lm(math1 ~ schoolk * stark + gender + ethnicity + lunchk, data = STAR)
V_urbanicity <- vcovCR(lm_urbanicity, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_urbanicity, vcov = V_urbanicity)
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt)
## (Intercept) 542.62 5.91 91.8599 21.70 <0.001
## schoolksuburban 2.77 6.76 0.4100 28.35 0.6849
## schoolkrural 1.03 6.38 0.1616 30.74 0.8727
## starksmall 9.42 4.56 2.0649 17.10 0.0544
## starkaide -4.27 2.17 -1.9631 16.73 0.0665
## genderfemale 2.14 1.20 1.7773 67.14 0.0800
## ethnicityafam -16.79 4.19 -4.0026 34.94 <0.001
## ethnicityasian 13.19 11.02 1.1963 6.23 0.2751
## ethnicityhispanic 39.23 20.62 1.9028 1.01 0.3067
## ethnicityother 8.86 18.78 0.4720 3.02 0.6690
## lunchkfree -19.37 2.04 -9.4848 57.38 <0.001
## schoolksuburban:starksmall 3.03 6.39 0.4746 28.94 0.6386
## schoolkrural:starksmall -0.31 5.58 -0.0555 34.04 0.9560
## schoolksuburban:starkaide 5.10 3.72 1.3711 28.64 0.1810
## schoolkrural:starkaide 8.16 3.16 2.5857 34.30 0.0141
## Sig.
## ***
##
##
## .
## .
## .
## ***
##
##
##
## ***
##
##
##
## *
With this specification, there are several different null hypotheses
that we might want to test. For one, perhaps we want to see if there is
any variation in treatment effects across different levels of
urbanicity. This can be expressed in the null hypothesis that all four
interaction terms are zero, or \(H_{0A}:
\beta_5 = \beta_6 = \beta_7 = \beta_8 = 0\). With Wald test:
Wald_test(lm_urbanicity,
constraints = constrain_zero("schoolk.+:stark", reg_ex = TRUE),
vcov = V_urbanicity)
## test Fstat df_num df_denom p_val sig
## HTZ 1.96 4 37.5 0.121
Another possibility is that we might want to focus on variation in
the effect of being in a small class or regular class, while ignoring
whatever is going on in the aide class condition. Here, the null
hypothesis would be simply \(H_{0B}: \beta_5 =
\beta_6 = 0\), tested as:
Wald_test(lm_urbanicity,
constraints = constrain_zero("schoolk.+:starksmall", reg_ex = TRUE),
vcov = V_urbanicity)
## test Fstat df_num df_denom p_val sig
## HTZ 0.189 2 34.5 0.828
Lists of
constraints
In models like the urbanicity-by-treatment interaction specification,
we may need to run multiple tests on the same estimating equation. This
can be accomplished with Wald_test by providing a
list of constraints to the constraints argument.
For example, we could test the hypotheses described above by creating a
list of several constraint matrices and then passing it to
Wald_test:
C_list <- list(
`Any interaction` = constrain_zero("schoolk.+:stark",
coef(lm_urbanicity), reg_ex = TRUE),
`Small vs regular` = constrain_zero("schoolk.+:starksmall",
coef(lm_urbanicity), reg_ex = TRUE)
)
Wald_test(lm_urbanicity,
constraints = C_list,
vcov = V_urbanicity)
## $`Any interaction`
## test Fstat df_num df_denom p_val sig
## HTZ 1.96 4 37.5 0.121
##
## $`Small vs regular`
## test Fstat df_num df_denom p_val sig
## HTZ 0.189 2 34.5 0.828
Setting the option tidy = TRUE will arrange the output
of all the tests into a single data frame:
Wald_test(lm_urbanicity,
constraints = C_list,
vcov = V_urbanicity,
tidy = TRUE)
## hypothesis test Fstat df_num df_denom p_val sig
## Any interaction HTZ 1.960 4 37.5 0.121
## Small vs regular HTZ 0.189 2 34.5 0.828
The list of constraints can also be created inside
Wald_test, so that the coefs argument can be
omitted from constrain_zero():
Wald_test(
lm_urbanicity,
constraints = list(
`Any interaction` = constrain_zero("schoolk.+:stark", reg_ex = TRUE),
`Small vs regular` = constrain_zero("schoolk.+:starksmall", reg_ex = TRUE)
),
vcov = V_urbanicity,
tidy = TRUE
)
## hypothesis test Fstat df_num df_denom p_val sig
## Any interaction HTZ 1.960 4 37.5 0.121
## Small vs regular HTZ 0.189 2 34.5 0.828
Pairwise t-tests
The clubSandwich package also provides a further
convenience function, constrain_pairwise() that can be used
in combination with Wald_test() to conduct pairwise
comparisons among a set of regression coefficients. This function
differs from the other two constrain_*() functions because
it returns a list of constraint matrices, each of which
corresponds to a single linear combination of covariates. Specifically,
the constrain_pairwise() function provides a list of
constraints that represent the differences between every possible pair
among a specified set of coefficients. The syntax is very similar to the
other constrain_*() functions.
To demonstrate, consider the separate-intercepts specification of the
simpler regression model:
coef_test(lm_sep, vcov = V_sep)
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## starkregular 532 2.78 192 59.9 <0.001 ***
## starksmall 541 2.89 187 65.0 <0.001 ***
## starkaide 531 2.72 195 64.3 <0.001 ***
This specification is nice because it lets us simply read off the
average outcomes for each group. However, we will naturally also want to
know about whether there are differences between the groups, so we’ll
want to compare the small-class condition to the regular-size class
condition, the aide condition to the regular-size class condition, and
the small-class condition to the aide condition. Thus, we’ll want
comparisons among all three coefficients:
C_pairs <- constrain_pairwise(1:3, coefs = coef(lm_sep))
C_pairs
## $`starksmall - starkregular`
## [,1] [,2] [,3]
## [1,] -1 1 0
##
## $`starkaide - starkregular`
## [,1] [,2] [,3]
## [1,] -1 0 1
##
## $`starkaide - starksmall`
## [,1] [,2] [,3]
## [1,] 0 -1 1
Feeding these constraints into Wald_test() gives us
significance tests for each pair:
Wald_test(lm_sep, constraints = C_pairs, vcov = V_sep, tidy = TRUE)
## hypothesis test Fstat df_num df_denom p_val sig
## starksmall - starkregular HTZ 16.9238 1 65.6 <0.001 ***
## starkaide - starkregular HTZ 0.0673 1 65.6 0.796
## starkaide - starksmall HTZ 17.8137 1 66.9 <0.001 ***
The first two of these tests are equivalent to the tests of the
treatment effect coefficients in the other parameterization of the
model. Indeed, the denominator degrees of freedom are identical to the
results of coef_test(lm_trt, vcov = V_trt); the
Fstats here are equal to the squared t-statistics from the
first model:
t_stats <- coef_test(lm_trt, vcov = V_trt)$tstat[2:3]
F_stats <- Wald_test(lm_sep, constraints = C_pairs, vcov = V_sep, tidy = TRUE)$Fstat[1:2]
all.equal(t_stats^2, F_stats)
## [1] TRUE
It is important to note that the p-values from the pairwise
comparisons are not corrected for multiplicity. For now, please
correct-your-own using p.adjust() or your preferred
method.
Pairwise comparisons might also be of use in the model with
treatment-by-urbanicity interactions. Here’s the model results
again:
coef_test(lm_urbanicity, vcov = V_urbanicity)
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt)
## (Intercept) 542.62 5.91 91.8599 21.70 <0.001
## schoolksuburban 2.77 6.76 0.4100 28.35 0.6849
## schoolkrural 1.03 6.38 0.1616 30.74 0.8727
## starksmall 9.42 4.56 2.0649 17.10 0.0544
## starkaide -4.27 2.17 -1.9631 16.73 0.0665
## genderfemale 2.14 1.20 1.7773 67.14 0.0800
## ethnicityafam -16.79 4.19 -4.0026 34.94 <0.001
## ethnicityasian 13.19 11.02 1.1963 6.23 0.2751
## ethnicityhispanic 39.23 20.62 1.9028 1.01 0.3067
## ethnicityother 8.86 18.78 0.4720 3.02 0.6690
## lunchkfree -19.37 2.04 -9.4848 57.38 <0.001
## schoolksuburban:starksmall 3.03 6.39 0.4746 28.94 0.6386
## schoolkrural:starksmall -0.31 5.58 -0.0555 34.04 0.9560
## schoolksuburban:starkaide 5.10 3.72 1.3711 28.64 0.1810
## schoolkrural:starkaide 8.16 3.16 2.5857 34.30 0.0141
## Sig.
## ***
##
##
## .
## .
## .
## ***
##
##
##
## ***
##
##
##
## *
Suppose that we are interested in the effect of small versus regular
size classes, and in particular whether this effect varies across
schools in different areas. The coefficients on
schoolksuburban:starksmall and
schoolkrural:starksmall already give us the differences in
treatment effects between suburban schools versus urban schools and
between rural schools versus urban schools. The difference between these
coefficients gives us the difference in treatment effects between
suburban schools and rural schools. We can look at all three of these
contrasts using constrain_pairwise() by setting the option
with_zero = TRUE:
Wald_test(lm_urbanicity,
constraints = constrain_pairwise(":starksmall", reg_ex = TRUE, with_zero = TRUE),
vcov = V_urbanicity,
tidy = TRUE)
## hypothesis test Fstat df_num
## schoolksuburban:starksmall HTZ 0.22526 1
## schoolkrural:starksmall HTZ 0.00308 1
## schoolkrural:starksmall - schoolksuburban:starksmall HTZ 0.36471 1
## df_denom p_val sig
## 28.9 0.639
## 34.0 0.956
## 24.4 0.551
Again, the results of the first two tests are identical to the
t-tests reported in coef_test().
References
Pustejovsky, J. E., & Tipton, E. (2018). Small-
Sample
Methods for
Cluster-
Robust
Variance Estimation and
Hypothesis Testing in
Fixed
Effects Models.
Journal of Business &
Economic Statistics,
36(4), 672–683.
https://doi.org/10.1080/07350015.2016.1247004
Tipton, E., & Pustejovsky, J. E. (2015). Small-sample adjustments
for tests of moderators and model fit using robust variance estimation
in meta-regression.
Journal of Educational and Behavioral
Statistics,
40(6), 604–634.
https://doi.org/10.3102/1076998615606099
clubSandwich/inst/doc/panel-data-CRVE.Rmd 0000644 0001762 0000144 00000036604 14630154052 017622 0 ustar ligges users ---
title: "Cluster-robust standard errors and hypothesis tests in panel data models"
author: "James E. Pustejovsky"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Cluster-robust standard errors and hypothesis tests in panel data models}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
The importance of using cluster-robust variance estimators (i.e., "clustered standard errors") in panel models is now widely recognized.
Less widely recognized is the fact that standard methods for constructing hypothesis tests and confidence intervals based on CRVE can perform quite poorly in when based on a limited number of independent clusters.
Furthermore, it can be difficult to determine what counts as a large-enough sample to trust standard CRVE methods, because the finite-sample behavior of the variance estimators and test statistics depends on the configuration of the covariates, not just the total number of clusters.
One solution to this problem is to use bias-reduced linearization (BRL), which was proposed by Bell and McCaffrey (2002) and has recently begun to receive attention in the econometrics literature (e.g., Cameron & Miller, 2015; Imbens & Kolesar, 2015).
The idea of BRL is to correct the bias of standard CRVE based on a working model, and then to use a degrees-of-freedom correction for Wald tests based on the bias-reduced CRVE.
That may seem silly (after all, the whole point of CRVE is to avoid making distributional assumptions about the errors in your model), but it turns out that the correction can help quite a bit, even when the working model is wrong.
The degrees-of-freedom correction is based on a standard Satterthwaite-type approximation, and also relies on the working model.
A problem with Bell and McCaffrey's original formulation of BRL is that it does not work in some very common models for panel data, such as state-by-year panels that include fixed effects for each state and each year (Angrist and Pischke, 2009, point out this issue in their chapter on "non-standard standard error issues"; see also Young, 2016).
However, Pustejovsky and Tipton (2016) proposed a generalization of BRL that works even in models with arbitrary sets of fixed effects, and this generalization is implemented in `clubSandwich` as CRVE type `CR2`.
The package also implements small-sample corrections for multiple-constraint hypothesis tests based on an approximation proposed by Pustejovsky and Tipton (2016). For one-parameter constraints, the test reduces to a t-test with Satterthwaite degrees of freedom, and so it is a natural extension of BRL.
The following example demonstrates how to use `clubSandwich` to do cluster-robust inference for a state-by-year panel model with fixed effects in both dimensions, clustering by states.
## Effects of changing the minimum legal drinking age
Carpenter and Dobkin (2011) analyzed the effects of changes in the minimum legal drinking age on rates of motor vehicle fatalities among 18-20 year olds, using state-level panel data from the National Highway Traffic Administration's Fatal Accident Reporting System.
In their new textbook, Angrist and Pischke (2014) developed a stylized example based on Carpenter and Dobkin's work.
The following example uses Angrist and Pischke's data and follows their analysis because their data are [easily available](https://www.masteringmetrics.com/resources/).
The outcome is the incidence of deaths in motor vehicle crashes among 18-20 year-olds (per 100,000 residents), for each state plus the District of Columbia, over the period 1970 to 1983.
There were several changes in the minimum legal drinking age during this time period, with variability in the timing of changes across states.
Angrist and Pischke (following Carpenter and Dobkin) use a difference-in-differences strategy to estimate the effects of lowering the minimum legal drinking age from 21 to 18.
Their specification is
$$y_{it} = \alpha_i + \beta_t + \gamma b_{it} + \delta d_{it} + \epsilon_{it},$$
for $i$ = 1,...,51 and $t$ = 1970,...,1983.
In this model, $\alpha_i$ is a state-specific fixed effect, $\beta_t$ is a year-specific fixed effect, $b_{it}$ is the current rate of beer taxation in state $i$ in year $t$, $d_{it}$ is the proportion of 18-20 year-olds in state $i$ in year $t$ who are legally allowed to drink, and $\delta$ captures the effect of shifting the minimum legal drinking age from 21 to 18.
Following Angrist and Pischke's analysis, we estimate this model both by (unweighted) OLS and by weighted least squares with weights corresponding to population size in a given state and year.
We also demonstrate random effects estimation and implement a cluster-robust Hausman specification test.
## Unweighted OLS
The following code does some simple data-munging and the estimates the model by OLS:
```{r, message = FALSE, warning = FALSE}
library(clubSandwich)
data(MortalityRates)
# subset for deaths in motor vehicle accidents, 1970-1983
MV_deaths <- subset(MortalityRates, cause=="Motor Vehicle" &
year <= 1983 & !is.na(beertaxa))
# fit by OLS
lm_unweighted <- lm(mrate ~ 0 + legal + beertaxa +
factor(state) + factor(year), data = MV_deaths)
```
The `coef_test` function from `clubSandwich` can then be used to test the hypothesis that changing the minimum legal drinking age has no effect on motor vehicle deaths in this cohort (i.e., $H_0: \delta = 0$).
The usual way to test this is to cluster the standard errors by state, calculate the robust Wald statistic, and compare that to a standard normal reference distribution.
The code and results are as follows:
```{r}
coef_test(lm_unweighted, vcov = "CR1",
cluster = MV_deaths$state, test = "naive-t")[1:2,]
```
A better approach would be to use the generalized, bias-reduced linearization CRVE, together with Satterthwaite degrees of freedom.
In the `clubSandwich` package, the BRL adjustment is called "CR2" because it is directly analogous to the HC2 correction used in heteroskedasticity-robust variance estimation.
When applied to an OLS model estimated by `lm`, the default working model is an identity matrix, which amounts to the "working" assumption that the errors are all uncorrelated and homoskedastic.
Here's how to apply this approach in the example:
```{r}
coef_test(lm_unweighted, vcov = "CR2",
cluster = MV_deaths$state, test = "Satterthwaite")[1:2,]
```
The Satterthwaite degrees of freedom are different for each coefficient in the model, and so the `coef_test` function reports them right alongside the standard error.
For the effect of legal drinking age, the degrees of freedom are about half of what might be expected, given that there are 51 clusters.
The p-value for the CR2+Satterthwaite test is about twice as large as the p-value based on the standard Wald test, although the coefficient is still statistically significant at conventional levels.
Note, however, that the degrees of freedom on the beer taxation rate are considerably smaller because there are only a few states with substantial variability in taxation rates over time.
```{r, echo = FALSE, results = "asis"}
plm_available <- requireNamespace("plm", quietly = TRUE)
if (!plm_available) cat("## Building the remainder of the vignette requires the plm package. Please install it. {-}")
```
## Unweighted "within" estimation
The `plm` package in R provides another way to estimate the same model.
It is convenient because it absorbs the state and year fixed effects before estimating the effect of `legal`.
The `clubSandwich` package works with fitted `plm` models too:
```{r, message = FALSE, eval = plm_available}
library(plm)
plm_unweighted <- plm(mrate ~ legal + beertaxa, data = MV_deaths,
effect = "twoways", index = c("state","year"))
coef_test(plm_unweighted, vcov = "CR1", cluster = "individual", test = "naive-t")
coef_test(plm_unweighted, vcov = "CR2", cluster = "individual", test = "Satterthwaite")
```
## Population-weighted estimation
The difference between the standard method and the new method are not terribly exciting in the above example.
However, things change quite a bit if the model is estimated using population weights.
We go back to fitting in `lm` with dummies for all the fixed effects because `plm` does not handle weighted least squares.
```{r}
lm_weighted <- lm(mrate ~ 0 + legal + beertaxa + factor(state) + factor(year),
weights = pop, data = MV_deaths)
coef_test(lm_weighted, vcov = "CR1",
cluster = MV_deaths$state, test = "naive-t")[1:2,]
coef_test(lm_weighted, vcov = "CR2",
cluster = MV_deaths$state, test = "Satterthwaite")[1:2,]
```
Using population weights slightly reduces the point estimate of the effect, while also slightly increasing its precision.
If you were following the standard approach, you would probably be happy with the weighted estimates and wouldn't think about it any further.
However, using the CR2 variance estimator and Satterthwaite correction produces a p-value that is an order of magnitude larger (though still significant at the conventional 5% level).
The degrees of freedom are just `r round(coef_test(lm_weighted, vcov = "CR2", cluster = MV_deaths$state, test = "Satterthwaite")["legal","df_Satt"], 1)`---drastically smaller than would be expected based on the number of clusters.
Even with weights, the `coef_test` function uses an "independent, homoskedastic" working model as a default for `lm` objects.
In the present example, the outcome is a standardized rate and so a better assumption might be that the error variances are inversely proportional to population size.
The following code uses this alternate working model:
```{r}
coef_test(lm_weighted, vcov = "CR2",
cluster = MV_deaths$state, target = 1 / MV_deaths$pop,
test = "Satterthwaite")[1:2,]
```
The new working model leads to slightly smaller standard errors and a couple of additional degrees of freedom, though they remain in small-sample territory.
## Random effects estimation
If the unobserved effects $\alpha_1,...,\alpha_{51}$ are uncorrelated with the regressors, then a more efficient way to estimate $\gamma,\delta$ is by weighted least squares, with weights based on a random effects model. We still treat the year effects as fixed.
```{r, eval = plm_available}
plm_random <- plm(mrate ~ 0 + legal + beertaxa + year, data = MV_deaths,
effect = "individual", index = c("state","year"),
model = "random")
coef_test(plm_random, vcov = "CR1", test = "naive-t")[1:2,]
coef_test(plm_random, vcov = "CR2", test = "Satterthwaite")[1:2,]
```
With random effects estimation, the effect of legal drinking age is smaller by about 1 death per 100,000. As a procedural aside, note that `coef_test` infers that `state` is the clustering variable because the call to plm includes only one type of effects (random state effects).
## Robust Hausman test
CRVE is also used in specification tests, as in the artificial Hausman-type test for endogeneity of unobserved effects (Arellano, 1993).
As noted above, random effects estimation is more efficient than fixed effects estimation, but requires the assumption that the unobserved effects are uncorrelated with the regressors.
However, if the unobserved effects covary with $\mathbf{b}_i, \mathbf{d}_i$, then the random-effects estimator will be biased.
We can test for whether endogeneity is a problem by including group-centered covariates as additional regressors. Let $\tilde{d}_{it} = d_{it} - \frac{1}{T}\sum_t d_{it}$, with $\tilde{b}_{it}$ defined analogously. Now estimate the regression
$$y_{it} = \beta_t + \gamma_1 b_{it} + \gamma_2 \tilde{b}_{it} + \delta_1 d_{it} + \delta_2 \tilde{d}_{it} + \epsilon_{it},$$
which does not include state fixed effects. The parameters $\gamma_2,\delta_2$ represent the differences between the within-groups and between-groups estimands of $\gamma_1, \delta_1$. If these are both zero, then the random effects estimator is unbiased. Thus, the joint test for $H_0: \gamma_2 = \delta_2 = 0$ amounts to a test for exogeneity of the unobserved effects.
For efficiency, we estimate this specification using weighted least squares (although OLS would be valid too):
```{r, eval = plm_available}
MV_deaths <- within(MV_deaths, {
legal_cent <- legal - tapply(legal, state, mean)[factor(state)]
beer_cent <- beertaxa - tapply(beertaxa, state, mean)[factor(state)]
})
plm_Hausman <- plm(mrate ~ 0 + legal + beertaxa + legal_cent + beer_cent + factor(year),
data = MV_deaths,
effect = "individual", index = c("state","year"),
model = "random")
coef_test(plm_Hausman, vcov = "CR2", test = "Satterthwaite")[1:4,]
```
To conduct a joint test on the centered covariates, we can use the `Wald_test` function. The usual way to test this hypothesis would be to use the `CR1` variance estimator to calculate the robust Wald statistic, then use a $\chi^2_2$ reference distribution (or equivalently, compare a re-scaled Wald statistic to an $F(2,\infty)$ distribution). The `Wald_test` function reports the latter version:
```{r, eval = plm_available}
Wald_test(plm_Hausman,
constraints = constrain_zero(c("legal_cent","beer_cent")),
vcov = "CR1", test = "chi-sq")
```
The test is just shy of significance at the 5% level. If we instead use the `CR2` variance estimator and our newly proposed approximate F-test (which is the default in `Wald_test`), then we get:
```{r, eval = plm_available}
Wald_test(plm_Hausman,
constraints = constrain_zero(c("legal_cent","beer_cent")),
vcov = "CR2")
```
The low degrees of freedom of the test indicate that we're definitely in small-sample territory and should not trust the asymptotic $\chi^2$ approximation.
## References
Angrist, J. D., & Pischke, J. (2009). _Mostly harmless econometrics: An empiricist’s companion_. Princeton, NJ: Princeton University Press.
Angrist, J. D., and Pischke, J. S. (2014). _Mastering'metrics: the path from cause to effect_. Princeton, NJ: Princeton University Press.
Arellano, M. (1993). On the testing of correlated effects with panel data. Journal of Econometrics, 59(1-2), 87-97. doi: [10.1016/0304-4076(93)90040-C](https://www.sciencedirect.com/science/article/pii/030440769390040C)
Bell, R. M., & McCaffrey, D. F. (2002). Bias reduction in standard errors for linear regression with multi-stage samples. _Survey Methodology, 28_(2), 169-181.
Cameron, A. C., & Miller, D. L. (2015). A practitioner’s guide to cluster-robust inference. URL: https://cameron.econ.ucdavis.edu/research/Cameron_Miller_JHR_2015_February.pdf
Carpenter, C., & Dobkin, C. (2011). The minimum legal drinking age and public health. _Journal of Economic Perspectives, 25_(2), 133-156. doi: [10.1257/jep.25.2.133](https://doi.org/10.1257/jep.25.2.133)
Imbens, G. W., & Kolesar, M. (2015). Robust standard errors in small samples: Some practical advice. URL: https://doi.org/10.1162/REST_a_00552
Pustejovsky, J. E. & Tipton, E. (2016). Small sample methods for cluster-robust variance estimation and hypothesis testing in fixed effects models. arXiv: [1601.01981](https://arxiv.org/abs/1601.01981) [stat.ME]
Young, A. (2016). Improved, nearly exact, statistical inference with robust and clustered covariance matrices using effective degrees of freedom corrections.
clubSandwich/inst/doc/meta-analysis-with-CRVE.html 0000644 0001762 0000144 00000070626 14635065006 021565 0 ustar ligges users
Meta-analysis with cluster-robust variance estimation
Meta-analysis with cluster-robust variance
estimation
James E. Pustejovsky
2024-06-20
This vignette demonstrates how to use the clubSandwich
package to conduct a meta-analysis of dependent effect sizes with robust
variance estimation. Tests of meta-regression coefficients and F-tests
of multiple-coefficient hypotheses are calculated using small-sample
corrections proposed by Tipton (2015) and Tipton and Pustejovsky (2015).
The example uses a dataset of effect sizes from a Campbell Collaboration
systematic review of dropout prevention programs, conducted by Sandra Jo
Wilson and colleagues (2011).
The original analysis included a meta-regression with covariates that
capture methodological, participant, and program characteristics. The
regression specification used here is similar to Model III from Wilson
et al. (2011), but treats the evaluator_independence and
implementation_quality variables as categorical rather than
interval-level. Also, the original analysis clustered at the level of
the sample (some studies reported results from multiple samples),
whereas here we cluster at the study level. The meta-regression can be
fit in several different ways. We first demonstrate using the
robumeta package (Fisher & Tipton, 2015) and then using
the metafor package (Viechtbauer, 2010).
robumeta model
library(clubSandwich)
library(robumeta)
data(dropoutPrevention)
# clean formatting
names(dropoutPrevention)[7:8] <- c("eval","implement")
levels(dropoutPrevention$eval) <- c("independent","indirect","planning","delivery")
levels(dropoutPrevention$implement) <- c("low","medium","high")
levels(dropoutPrevention$program_site) <- c("community","mixed","classroom","school")
levels(dropoutPrevention$study_design) <- c("matched","unmatched","RCT")
levels(dropoutPrevention$adjusted) <- c("no","yes")
m3_robu <- robu(LOR1 ~ study_design + attrition + group_equivalence + adjusted
+ outcome + eval + male_pct + white_pct + average_age
+ implement + program_site + duration + service_hrs,
data = dropoutPrevention, studynum = studyID, var.eff.size = varLOR,
modelweights = "HIER")
print(m3_robu)
## RVE: Hierarchical Effects Model with Small-Sample Corrections
##
## Model: LOR1 ~ study_design + attrition + group_equivalence + adjusted + outcome + eval + male_pct + white_pct + average_age + implement + program_site + duration + service_hrs
##
## Number of clusters = 152
## Number of outcomes = 385 (min = 1 , mean = 2.53 , median = 1 , max = 30 )
## Omega.sq = 0.24907
## Tau.sq = 0.1024663
##
## Estimate StdErr t-value dfs P(|t|>) 95% CI.L 95% CI.U Sig
## 1 X.Intercept. 0.016899 0.615399 0.0275 16.9 0.97841541 -1.28228 1.31608
## 2 study_designunmatched -0.002626 0.185142 -0.0142 40.5 0.98875129 -0.37667 0.37141
## 3 study_designRCT -0.086872 0.140044 -0.6203 38.6 0.53869676 -0.37024 0.19650
## 4 attrition 0.118889 0.247228 0.4809 15.5 0.63732597 -0.40666 0.64444
## 5 group_equivalence 0.502463 0.195838 2.5657 28.7 0.01579282 0.10174 0.90318 **
## 6 adjustedyes -0.322480 0.125413 -2.5713 33.8 0.01470796 -0.57741 -0.06755 **
## 7 outcomeenrolled 0.097059 0.139842 0.6941 16.5 0.49727848 -0.19862 0.39274
## 8 outcomegraduation 0.147643 0.134938 1.0942 30.2 0.28253825 -0.12786 0.42315
## 9 outcomegraduation.ged 0.258034 0.169134 1.5256 16.3 0.14632629 -0.10006 0.61613
## 10 evalindirect -0.765085 0.399109 -1.9170 6.2 0.10212896 -1.73406 0.20389
## 11 evalplanning -0.920874 0.346536 -2.6574 5.6 0.04027061 -1.78381 -0.05794 **
## 12 evaldelivery -0.916673 0.304303 -3.0124 4.7 0.03212299 -1.71432 -0.11903 **
## 13 male_pct 0.167965 0.181538 0.9252 16.4 0.36824526 -0.21609 0.55202
## 14 white_pct 0.022915 0.149394 0.1534 21.8 0.87950385 -0.28704 0.33287
## 15 average_age 0.037102 0.027053 1.3715 21.2 0.18458247 -0.01913 0.09333
## 16 implementmedium 0.411779 0.128898 3.1946 26.7 0.00358205 0.14714 0.67642 ***
## 17 implementhigh 0.658570 0.123874 5.3164 34.6 0.00000635 0.40699 0.91015 ***
## 18 program_sitemixed 0.444384 0.172635 2.5741 28.6 0.01550504 0.09109 0.79768 **
## 19 program_siteclassroom 0.426658 0.159773 2.6704 37.4 0.01115192 0.10303 0.75028 **
## 20 program_siteschool 0.262517 0.160519 1.6354 30.1 0.11236814 -0.06525 0.59028
## 21 duration 0.000427 0.000873 0.4895 36.7 0.62736846 -0.00134 0.00220
## 22 service_hrs -0.003434 0.005012 -0.6852 36.7 0.49752503 -0.01359 0.00672
## ---
## Signif. codes: < .01 *** < .05 ** < .10 *
## ---
## Note: If df < 4, do not trust the results
Note that robumeta produces small-sample corrected
standard errors and t-tests, and so there is no need to repeat those
calculations with clubSandwich. The eval
variable has four levels, and it might be of interest to test whether
the average program effects differ by the degree of evaluator
independence. The null hypothesis in this case is that the 10th, 11th,
and 12th regression coefficients are all equal to zero. A small-sample
adjusted F-test for this hypothesis can be obtained as follows. The
vcov = "CR2" option means that the standard errors will be
corrected using the bias-reduced linearization estimator described in
Tipton and Pustejovsky (2015).
Wald_test(m3_robu, constraints = constrain_zero(10:12), vcov = "CR2")
## test Fstat df_num df_denom p_val sig
## HTZ 2.78 3 16.8 0.0732 .
By default, the Wald_test function provides an F-type
test with degrees of freedom estimated using the approximate Hotelling’s
\(T^2_Z\) method. The test has less
than 17 degrees of freedom, even though there are 152 independent
studies in the data, and has a p-value that is not quite significant at
conventional levels. The low degrees of freedom are a consequence of the
fact that one of the levels of evaluator independence has
only a few effect sizes in it:
table(dropoutPrevention$eval)
##
## independent indirect planning delivery
## 6 33 43 303
metafor model
clubSandwich also works with models fit using the
metafor package. Here we re-fit the same regression
specification, but use REML to estimate the variance components
(robumeta uses a method-of-moments estimator), as well as a
somewhat different weighting scheme than that used in
robumeta.
library(metafor)
m3_metafor <- rma.mv(LOR1 ~ study_design + attrition + group_equivalence + adjusted
+ outcome + eval
+ male_pct + white_pct + average_age
+ implement + program_site + duration + service_hrs,
V = varLOR, random = list(~ 1 | studyID, ~ 1 | studySample),
data = dropoutPrevention)
summary(m3_metafor)
##
## Multivariate Meta-Analysis Model (k = 385; method: REML)
##
## logLik Deviance AIC BIC AICc
## -489.0357 978.0714 1026.0714 1119.5371 1029.6217
##
## Variance Components:
##
## estim sqrt nlvls fixed factor
## sigma^2.1 0.2274 0.4769 152 no studyID
## sigma^2.2 0.1145 0.3384 317 no studySample
##
## Test for Residual Heterogeneity:
## QE(df = 363) = 1588.4397, p-val < .0001
##
## Test of Moderators (coefficients 2:22):
## QM(df = 21) = 293.8694, p-val < .0001
##
## Model Results:
##
## estimate se zval pval ci.lb ci.ub
## intrcpt 0.5296 0.7250 0.7304 0.4651 -0.8915 1.9506
## study_designunmatched -0.0494 0.1722 -0.2871 0.7741 -0.3870 0.2881
## study_designRCT 0.0653 0.1628 0.4010 0.6884 -0.2538 0.3843
## attrition -0.1366 0.2429 -0.5623 0.5739 -0.6126 0.3395
## group_equivalence 0.4071 0.1573 2.5877 0.0097 0.0988 0.7155 **
## adjustedyes -0.3581 0.1532 -2.3371 0.0194 -0.6585 -0.0578 *
## outcomeenrolled -0.2831 0.0771 -3.6709 0.0002 -0.4343 -0.1320 ***
## outcomegraduation -0.0913 0.0657 -1.3896 0.1646 -0.2201 0.0375
## outcomegraduation/ged 0.6983 0.0805 8.6750 <.0001 0.5406 0.8561 ***
## evalindirect -0.7530 0.4949 -1.5214 0.1282 -1.7230 0.2171
## evalplanning -0.7700 0.4869 -1.5814 0.1138 -1.7242 0.1843
## evaldelivery -1.0016 0.4600 -2.1774 0.0294 -1.9033 -0.1000 *
## male_pct 0.1021 0.1715 0.5951 0.5518 -0.2341 0.4382
## white_pct 0.1223 0.1804 0.6777 0.4979 -0.2313 0.4758
## average_age 0.0061 0.0291 0.2091 0.8344 -0.0509 0.0631
## implementmedium 0.4738 0.1609 2.9445 0.0032 0.1584 0.7892 **
## implementhigh 0.6318 0.1471 4.2965 <.0001 0.3436 0.9201 ***
## program_sitemixed 0.3289 0.2413 1.3631 0.1729 -0.1440 0.8019
## program_siteclassroom 0.2920 0.1736 1.6821 0.0926 -0.0482 0.6321 .
## program_siteschool 0.1616 0.1898 0.8515 0.3945 -0.2104 0.5337
## duration 0.0013 0.0009 1.3423 0.1795 -0.0006 0.0031
## service_hrs -0.0003 0.0047 -0.0654 0.9478 -0.0096 0.0090
##
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
metafor produces model-based standard errors, t-tests,
and confidence intervals. The coef_test function from
clubSandwich will calculate robust standard errors and
robust t-tests for each of the coefficients:
coef_test(m3_metafor, vcov = "CR2")
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## intrcpt 0.529569 0.724851 0.7306 20.08 0.47347
## study_designunmatched -0.049434 0.204152 -0.2421 58.42 0.80952
## study_designRCT 0.065272 0.149146 0.4376 53.17 0.66342
## attrition -0.136575 0.306429 -0.4457 10.52 0.66485
## group_equivalence 0.407108 0.210917 1.9302 23.10 0.06595 .
## adjustedyes -0.358124 0.136132 -2.6307 43.20 0.01176 *
## outcomeenrolled -0.283124 0.237199 -1.1936 7.08 0.27108
## outcomegraduation -0.091295 0.091465 -0.9981 9.95 0.34188
## outcomegraduation/ged 0.698328 0.364882 1.9138 8.02 0.09188 .
## evalindirect -0.752994 0.447670 -1.6820 6.56 0.13929
## evalplanning -0.769968 0.403898 -1.9063 6.10 0.10446
## evaldelivery -1.001648 0.355989 -2.8137 4.89 0.03834 *
## male_pct 0.102055 0.148410 0.6877 9.68 0.50782
## white_pct 0.122255 0.141470 0.8642 16.88 0.39961
## average_age 0.006084 0.033387 0.1822 15.79 0.85772
## implementmedium 0.473789 0.148660 3.1871 22.44 0.00419 **
## implementhigh 0.631842 0.138073 4.5761 28.68 < 0.001 ***
## program_sitemixed 0.328941 0.196848 1.6710 27.47 0.10607
## program_siteclassroom 0.291952 0.146014 1.9995 42.70 0.05195 .
## program_siteschool 0.161640 0.171700 0.9414 29.27 0.35420
## duration 0.001270 0.000978 1.2988 31.96 0.20332
## service_hrs -0.000309 0.004828 -0.0641 49.63 0.94915
Note that coef_test assumed that it should cluster based
on studyID, which is the outer-most random effect in the
metafor model. This can be specified explicitly by including the option
cluster = dropoutPrevention$studyID in the call.
The F-test for degree of evaluator independence uses the same syntax
as before:
Wald_test(m3_metafor, constraints = constrain_zero(10:12), vcov = "CR2")
## test Fstat df_num df_denom p_val sig
## HTZ 2.71 3 18.3 0.0753 .
Despite some differences in weighting schemes, the p-value is very
close to the result obtained using robumeta.
References
Fisher, Z., & Tipton, E. (2015). robumeta: An R-package for
robust variance estimation in meta-analysis. arXiv:1503.02220
Tipton, E. (2015). Small sample adjustments for robust variance
estimation with meta-regression. Psychological Methods, 20(3),
375-393. https://doi.org/10.1037/met0000011
Tipton, E., & Pustejovsky, J. E. (2015). Small-sample adjustments
for tests of moderators and model fit using robust variance estimation
in meta-regression. Journal of Educational and Behavioral
Statistics, 40(6), 604-634. https://doi.org/10.3102/1076998615606099
Viechtbauer, W. (2010). Conducting meta-analyses in R with the
metafor package. Journal of Statistical Software, 36(3), 1-48.
URL: https://doi.org/10.18637/jss.v036.i03
Wilson, S. J., Lipsey, M. W., Tanner-Smith, E., Huang, C. H., &
Steinka-Fry, K. T. (2011). Dropout prevention and intervention programs:
Effects on school completion and dropout Among school-aged children and
youth: A systematic review. Campbell Systematic Reviews, 7(1),
1-61. https://doi.org/10.4073/csr.2011.8
clubSandwich/inst/doc/panel-data-CRVE.R 0000644 0001762 0000144 00000007262 14635065007 017305 0 ustar ligges users ## ----message = FALSE, warning = FALSE-------------------------------------------------------------
library(clubSandwich)
data(MortalityRates)
# subset for deaths in motor vehicle accidents, 1970-1983
MV_deaths <- subset(MortalityRates, cause=="Motor Vehicle" &
year <= 1983 & !is.na(beertaxa))
# fit by OLS
lm_unweighted <- lm(mrate ~ 0 + legal + beertaxa +
factor(state) + factor(year), data = MV_deaths)
## -------------------------------------------------------------------------------------------------
coef_test(lm_unweighted, vcov = "CR1",
cluster = MV_deaths$state, test = "naive-t")[1:2,]
## -------------------------------------------------------------------------------------------------
coef_test(lm_unweighted, vcov = "CR2",
cluster = MV_deaths$state, test = "Satterthwaite")[1:2,]
## ----echo = FALSE, results = "asis"---------------------------------------------------------------
plm_available <- requireNamespace("plm", quietly = TRUE)
if (!plm_available) cat("## Building the remainder of the vignette requires the plm package. Please install it. {-}")
## ----message = FALSE, eval = plm_available--------------------------------------------------------
library(plm)
plm_unweighted <- plm(mrate ~ legal + beertaxa, data = MV_deaths,
effect = "twoways", index = c("state","year"))
coef_test(plm_unweighted, vcov = "CR1", cluster = "individual", test = "naive-t")
coef_test(plm_unweighted, vcov = "CR2", cluster = "individual", test = "Satterthwaite")
## -------------------------------------------------------------------------------------------------
lm_weighted <- lm(mrate ~ 0 + legal + beertaxa + factor(state) + factor(year),
weights = pop, data = MV_deaths)
coef_test(lm_weighted, vcov = "CR1",
cluster = MV_deaths$state, test = "naive-t")[1:2,]
coef_test(lm_weighted, vcov = "CR2",
cluster = MV_deaths$state, test = "Satterthwaite")[1:2,]
## -------------------------------------------------------------------------------------------------
coef_test(lm_weighted, vcov = "CR2",
cluster = MV_deaths$state, target = 1 / MV_deaths$pop,
test = "Satterthwaite")[1:2,]
## ----eval = plm_available-------------------------------------------------------------------------
plm_random <- plm(mrate ~ 0 + legal + beertaxa + year, data = MV_deaths,
effect = "individual", index = c("state","year"),
model = "random")
coef_test(plm_random, vcov = "CR1", test = "naive-t")[1:2,]
coef_test(plm_random, vcov = "CR2", test = "Satterthwaite")[1:2,]
## ----eval = plm_available-------------------------------------------------------------------------
MV_deaths <- within(MV_deaths, {
legal_cent <- legal - tapply(legal, state, mean)[factor(state)]
beer_cent <- beertaxa - tapply(beertaxa, state, mean)[factor(state)]
})
plm_Hausman <- plm(mrate ~ 0 + legal + beertaxa + legal_cent + beer_cent + factor(year),
data = MV_deaths,
effect = "individual", index = c("state","year"),
model = "random")
coef_test(plm_Hausman, vcov = "CR2", test = "Satterthwaite")[1:4,]
## ----eval = plm_available-------------------------------------------------------------------------
Wald_test(plm_Hausman,
constraints = constrain_zero(c("legal_cent","beer_cent")),
vcov = "CR1", test = "chi-sq")
## ----eval = plm_available-------------------------------------------------------------------------
Wald_test(plm_Hausman,
constraints = constrain_zero(c("legal_cent","beer_cent")),
vcov = "CR2")
clubSandwich/inst/doc/meta-analysis-with-CRVE.R 0000644 0001762 0000144 00000005326 14635065005 021014 0 ustar ligges users ## ----echo = FALSE, results = "asis", message = FALSE, warning = FALSE---------
robu_available <- requireNamespace("robumeta", quietly = TRUE)
meta_available <- requireNamespace("metafor", quietly = TRUE)
knitr::opts_chunk$set(eval = robu_available & meta_available)
if (!robu_available) cat("## Building this vignette requires the robumeta package. Please install it. {-} \n")
if (!meta_available) cat("## Building this vignette requires the metafor package. Please install it. {-} \n")
## ----include=FALSE--------------------------------------------------------------------------------
options(width = 100)
## ----message = FALSE------------------------------------------------------------------------------
library(clubSandwich)
library(robumeta)
data(dropoutPrevention)
# clean formatting
names(dropoutPrevention)[7:8] <- c("eval","implement")
levels(dropoutPrevention$eval) <- c("independent","indirect","planning","delivery")
levels(dropoutPrevention$implement) <- c("low","medium","high")
levels(dropoutPrevention$program_site) <- c("community","mixed","classroom","school")
levels(dropoutPrevention$study_design) <- c("matched","unmatched","RCT")
levels(dropoutPrevention$adjusted) <- c("no","yes")
m3_robu <- robu(LOR1 ~ study_design + attrition + group_equivalence + adjusted
+ outcome + eval + male_pct + white_pct + average_age
+ implement + program_site + duration + service_hrs,
data = dropoutPrevention, studynum = studyID, var.eff.size = varLOR,
modelweights = "HIER")
print(m3_robu)
## -------------------------------------------------------------------------------------------------
Wald_test(m3_robu, constraints = constrain_zero(10:12), vcov = "CR2")
## -------------------------------------------------------------------------------------------------
table(dropoutPrevention$eval)
## ----message = FALSE------------------------------------------------------------------------------
library(metafor)
m3_metafor <- rma.mv(LOR1 ~ study_design + attrition + group_equivalence + adjusted
+ outcome + eval
+ male_pct + white_pct + average_age
+ implement + program_site + duration + service_hrs,
V = varLOR, random = list(~ 1 | studyID, ~ 1 | studySample),
data = dropoutPrevention)
summary(m3_metafor)
## -------------------------------------------------------------------------------------------------
coef_test(m3_metafor, vcov = "CR2")
## -------------------------------------------------------------------------------------------------
Wald_test(m3_metafor, constraints = constrain_zero(10:12), vcov = "CR2")
clubSandwich/inst/doc/meta-analysis-with-CRVE.Rmd 0000644 0001762 0000144 00000015565 14630154052 021337 0 ustar ligges users ---
title: "Meta-analysis with cluster-robust variance estimation"
author: "James E. Pustejovsky"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Meta-analysis with cluster-robust variance estimation}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, echo = FALSE, results = "asis", message = FALSE, warning = FALSE}
robu_available <- requireNamespace("robumeta", quietly = TRUE)
meta_available <- requireNamespace("metafor", quietly = TRUE)
knitr::opts_chunk$set(eval = robu_available & meta_available)
if (!robu_available) cat("## Building this vignette requires the robumeta package. Please install it. {-} \n")
if (!meta_available) cat("## Building this vignette requires the metafor package. Please install it. {-} \n")
```
This vignette demonstrates how to use the `clubSandwich` package to conduct a meta-analysis of dependent effect sizes with robust variance estimation.
Tests of meta-regression coefficients and F-tests of multiple-coefficient hypotheses are calculated using small-sample corrections proposed by Tipton (2015) and Tipton and Pustejovsky (2015).
The example uses a dataset of effect sizes from a Campbell Collaboration systematic review of dropout prevention programs, conducted by Sandra Jo Wilson and colleagues (2011).
The original analysis included a meta-regression with covariates that capture methodological, participant, and program characteristics.
The regression specification used here is similar to Model III from Wilson et al. (2011), but treats the `evaluator_independence` and `implementation_quality` variables as categorical rather than interval-level.
Also, the original analysis clustered at the level of the sample (some studies reported results from multiple samples), whereas here we cluster at the study level.
The meta-regression can be fit in several different ways. We first demonstrate using the `robumeta` package (Fisher & Tipton, 2015) and then using the `metafor` package (Viechtbauer, 2010).
## robumeta model
```{r, include=FALSE}
options(width = 100)
```
```{r, message = FALSE}
library(clubSandwich)
library(robumeta)
data(dropoutPrevention)
# clean formatting
names(dropoutPrevention)[7:8] <- c("eval","implement")
levels(dropoutPrevention$eval) <- c("independent","indirect","planning","delivery")
levels(dropoutPrevention$implement) <- c("low","medium","high")
levels(dropoutPrevention$program_site) <- c("community","mixed","classroom","school")
levels(dropoutPrevention$study_design) <- c("matched","unmatched","RCT")
levels(dropoutPrevention$adjusted) <- c("no","yes")
m3_robu <- robu(LOR1 ~ study_design + attrition + group_equivalence + adjusted
+ outcome + eval + male_pct + white_pct + average_age
+ implement + program_site + duration + service_hrs,
data = dropoutPrevention, studynum = studyID, var.eff.size = varLOR,
modelweights = "HIER")
print(m3_robu)
```
Note that `robumeta` produces small-sample corrected standard errors and t-tests, and so there is no need to repeat those calculations with `clubSandwich`.
The `eval` variable has four levels, and it might be of interest to test whether the average program effects differ by the degree of evaluator independence.
The null hypothesis in this case is that the 10th, 11th, and 12th regression coefficients are all equal to zero.
A small-sample adjusted F-test for this hypothesis can be obtained as follows.
The `vcov = "CR2"` option means that the standard errors will be corrected using the bias-reduced linearization estimator described in Tipton and Pustejovsky (2015).
```{r}
Wald_test(m3_robu, constraints = constrain_zero(10:12), vcov = "CR2")
```
By default, the `Wald_test` function provides an F-type test with degrees of freedom estimated using the approximate Hotelling's $T^2_Z$ method. The test has less than 17 degrees of freedom, even though there are 152 independent studies in the data, and has a p-value that is not quite significant at conventional levels. The low degrees of freedom are a consequence of the fact that one of the levels of `evaluator independence` has only a few effect sizes in it:
```{r}
table(dropoutPrevention$eval)
```
## metafor model
`clubSandwich` also works with models fit using the `metafor` package. Here we re-fit the same regression specification, but use REML to estimate the variance components (`robumeta` uses a method-of-moments estimator), as well as a somewhat different weighting scheme than that used in `robumeta`.
```{r, message = FALSE}
library(metafor)
m3_metafor <- rma.mv(LOR1 ~ study_design + attrition + group_equivalence + adjusted
+ outcome + eval
+ male_pct + white_pct + average_age
+ implement + program_site + duration + service_hrs,
V = varLOR, random = list(~ 1 | studyID, ~ 1 | studySample),
data = dropoutPrevention)
summary(m3_metafor)
```
`metafor` produces model-based standard errors, t-tests, and confidence intervals. The `coef_test` function from `clubSandwich` will calculate robust standard errors and robust t-tests for each of the coefficients:
```{r}
coef_test(m3_metafor, vcov = "CR2")
```
Note that `coef_test` assumed that it should cluster based on `studyID`, which is the outer-most random effect in the metafor model. This can be specified explicitly by including the option `cluster = dropoutPrevention$studyID` in the call.
The F-test for degree of evaluator independence uses the same syntax as before:
```{r}
Wald_test(m3_metafor, constraints = constrain_zero(10:12), vcov = "CR2")
```
Despite some differences in weighting schemes, the p-value is very close to the result obtained using `robumeta`.
## References
Fisher, Z., & Tipton, E. (2015). robumeta: An R-package for robust variance estimation in meta-analysis. [arXiv:1503.02220](https://arxiv.org/abs/1503.02220)
Tipton, E. (2015). Small sample adjustments for robust variance estimation with meta-regression. _Psychological Methods, 20_(3), 375-393. https://doi.org/10.1037/met0000011
Tipton, E., & Pustejovsky, J. E. (2015). Small-sample adjustments for tests of moderators and model fit using robust variance estimation in meta-regression. _Journal of Educational and Behavioral Statistics, 40_(6), 604-634. https://doi.org/10.3102/1076998615606099
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. _Journal of Statistical Software, 36_(3), 1-48. URL: https://doi.org/10.18637/jss.v036.i03
Wilson, S. J., Lipsey, M. W., Tanner-Smith, E., Huang, C. H., & Steinka-Fry, K. T. (2011). Dropout prevention and intervention programs: Effects on school completion and dropout Among school-aged children and youth: A systematic review. _Campbell Systematic Reviews, 7_(1), 1-61. https://doi.org/10.4073/csr.2011.8
clubSandwich/inst/doc/Wald-tests-in-clubSandwich.R 0000644 0001762 0000144 00000013700 14635065002 021566 0 ustar ligges users ## ----echo = FALSE, results = "asis", message = FALSE, warning = FALSE---------
library(clubSandwich)
AER_available <- requireNamespace("AER", quietly = TRUE)
knitr::opts_chunk$set(eval = AER_available)
if (!AER_available) cat("# Building this vignette requires the AER package. Please install it. {-}")
## ----message = FALSE, warning = FALSE-----------------------------------------
library(clubSandwich)
data(STAR, package = "AER")
# clean up a few variables
levels(STAR$stark)[3] <- "aide"
levels(STAR$schoolk)[1] <- "urban"
STAR <- subset(STAR,
!is.na(schoolidk),
select = c(schoolidk, schoolk, stark, gender, ethnicity, math1, lunchk))
head(STAR)
## -----------------------------------------------------------------------------
args(Wald_test)
## ----type-treat---------------------------------------------------------------
lm_trt <- lm(math1 ~ stark, data = STAR)
V_trt <- vcovCR(lm_trt, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_trt, vcov = V_trt)
## -----------------------------------------------------------------------------
C_trt <- matrix(c(0,0,1,0,0,1), 2, 3)
C_trt
Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)
## -----------------------------------------------------------------------------
args(constrain_zero)
## -----------------------------------------------------------------------------
constrain_zero(2:3, coefs = coef(lm_trt))
## -----------------------------------------------------------------------------
constrain_zero(c("starksmall","starkaide"), coefs = coef(lm_trt))
## -----------------------------------------------------------------------------
constrain_zero("^stark", coefs = coef(lm_trt), reg_ex = TRUE)
## -----------------------------------------------------------------------------
C_trt <- constrain_zero(2:3, coefs = coef(lm_trt))
Wald_test(lm_trt, constraints = C_trt, vcov = V_trt)
## -----------------------------------------------------------------------------
Wald_test(lm_trt, constraints = constrain_zero(2:3), vcov = V_trt)
## ----type-sep-----------------------------------------------------------------
lm_sep <- lm(math1 ~ 0 + stark, data = STAR)
V_sep <- vcovCR(lm_sep, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_sep, vcov = V_sep)
## -----------------------------------------------------------------------------
args(constrain_equal)
## -----------------------------------------------------------------------------
constrain_equal(1:3, coefs = coef(lm_sep))
## -----------------------------------------------------------------------------
constrain_equal(c("starkregular","starksmall","starkaide"), coefs = coef(lm_sep))
## -----------------------------------------------------------------------------
constrain_equal("^stark", coefs = coef(lm_sep), reg_ex = TRUE)
## -----------------------------------------------------------------------------
C_sep <- constrain_equal("^stark", coefs = coef(lm_sep), reg_ex = TRUE)
Wald_test(lm_sep, constraints = C_sep, vcov = V_sep)
## -----------------------------------------------------------------------------
Wald_test(lm_sep, constraints = constrain_equal(1:3), vcov = V_sep)
## -----------------------------------------------------------------------------
lm_urbanicity <- lm(math1 ~ schoolk * stark + gender + ethnicity + lunchk, data = STAR)
V_urbanicity <- vcovCR(lm_urbanicity, cluster = STAR$schoolidk, type = "CR2")
coef_test(lm_urbanicity, vcov = V_urbanicity)
## -----------------------------------------------------------------------------
Wald_test(lm_urbanicity,
constraints = constrain_zero("schoolk.+:stark", reg_ex = TRUE),
vcov = V_urbanicity)
## -----------------------------------------------------------------------------
Wald_test(lm_urbanicity,
constraints = constrain_zero("schoolk.+:starksmall", reg_ex = TRUE),
vcov = V_urbanicity)
## -----------------------------------------------------------------------------
C_list <- list(
`Any interaction` = constrain_zero("schoolk.+:stark",
coef(lm_urbanicity), reg_ex = TRUE),
`Small vs regular` = constrain_zero("schoolk.+:starksmall",
coef(lm_urbanicity), reg_ex = TRUE)
)
Wald_test(lm_urbanicity,
constraints = C_list,
vcov = V_urbanicity)
## -----------------------------------------------------------------------------
Wald_test(lm_urbanicity,
constraints = C_list,
vcov = V_urbanicity,
tidy = TRUE)
## -----------------------------------------------------------------------------
Wald_test(
lm_urbanicity,
constraints = list(
`Any interaction` = constrain_zero("schoolk.+:stark", reg_ex = TRUE),
`Small vs regular` = constrain_zero("schoolk.+:starksmall", reg_ex = TRUE)
),
vcov = V_urbanicity,
tidy = TRUE
)
## -----------------------------------------------------------------------------
coef_test(lm_sep, vcov = V_sep)
## -----------------------------------------------------------------------------
C_pairs <- constrain_pairwise(1:3, coefs = coef(lm_sep))
C_pairs
## -----------------------------------------------------------------------------
Wald_test(lm_sep, constraints = C_pairs, vcov = V_sep, tidy = TRUE)
## -----------------------------------------------------------------------------
t_stats <- coef_test(lm_trt, vcov = V_trt)$tstat[2:3]
F_stats <- Wald_test(lm_sep, constraints = C_pairs, vcov = V_sep, tidy = TRUE)$Fstat[1:2]
all.equal(t_stats^2, F_stats)
## -----------------------------------------------------------------------------
coef_test(lm_urbanicity, vcov = V_urbanicity)
## -----------------------------------------------------------------------------
Wald_test(lm_urbanicity,
constraints = constrain_pairwise(":starksmall", reg_ex = TRUE, with_zero = TRUE),
vcov = V_urbanicity,
tidy = TRUE)
clubSandwich/inst/doc/panel-data-CRVE.html 0000644 0001762 0000144 00000112441 14635065007 020044 0 ustar ligges users
Cluster-robust standard errors and hypothesis tests in panel data models
Cluster-robust standard errors and
hypothesis tests in panel data models
James E. Pustejovsky
2024-06-20
The importance of using cluster-robust variance estimators (i.e.,
“clustered standard errorsâ€) in panel models is now widely recognized.
Less widely recognized is the fact that standard methods for
constructing hypothesis tests and confidence intervals based on CRVE can
perform quite poorly in when based on a limited number of independent
clusters. Furthermore, it can be difficult to determine what counts as a
large-enough sample to trust standard CRVE methods, because the
finite-sample behavior of the variance estimators and test statistics
depends on the configuration of the covariates, not just the total
number of clusters.
One solution to this problem is to use bias-reduced linearization
(BRL), which was proposed by Bell and McCaffrey (2002) and has recently
begun to receive attention in the econometrics literature (e.g., Cameron
& Miller, 2015; Imbens & Kolesar, 2015). The idea of BRL is to
correct the bias of standard CRVE based on a working model, and then to
use a degrees-of-freedom correction for Wald tests based on the
bias-reduced CRVE. That may seem silly (after all, the whole point of
CRVE is to avoid making distributional assumptions about the errors in
your model), but it turns out that the correction can help quite a bit,
even when the working model is wrong. The degrees-of-freedom correction
is based on a standard Satterthwaite-type approximation, and also relies
on the working model.
A problem with Bell and McCaffrey’s original formulation of BRL is
that it does not work in some very common models for panel data, such as
state-by-year panels that include fixed effects for each state and each
year (Angrist and Pischke, 2009, point out this issue in their chapter
on “non-standard standard error issuesâ€; see also Young, 2016). However,
Pustejovsky and Tipton (2016) proposed a generalization of BRL that
works even in models with arbitrary sets of fixed effects, and this
generalization is implemented in clubSandwich as CRVE type
CR2. The package also implements small-sample corrections
for multiple-constraint hypothesis tests based on an approximation
proposed by Pustejovsky and Tipton (2016). For one-parameter
constraints, the test reduces to a t-test with Satterthwaite degrees of
freedom, and so it is a natural extension of BRL.
The following example demonstrates how to use
clubSandwich to do cluster-robust inference for a
state-by-year panel model with fixed effects in both dimensions,
clustering by states.
Effects of changing the minimum legal drinking age
Carpenter and Dobkin (2011) analyzed the effects of changes in the
minimum legal drinking age on rates of motor vehicle fatalities among
18-20 year olds, using state-level panel data from the National Highway
Traffic Administration’s Fatal Accident Reporting System. In their new
textbook, Angrist and Pischke (2014) developed a stylized example based
on Carpenter and Dobkin’s work. The following example uses Angrist and
Pischke’s data and follows their analysis because their data are easily
available.
The outcome is the incidence of deaths in motor vehicle crashes among
18-20 year-olds (per 100,000 residents), for each state plus the
District of Columbia, over the period 1970 to 1983. There were several
changes in the minimum legal drinking age during this time period, with
variability in the timing of changes across states. Angrist and Pischke
(following Carpenter and Dobkin) use a difference-in-differences
strategy to estimate the effects of lowering the minimum legal drinking
age from 21 to 18. Their specification is
\[y_{it} = \alpha_i + \beta_t + \gamma
b_{it} + \delta d_{it} + \epsilon_{it},\]
for \(i\) = 1,…,51 and \(t\) = 1970,…,1983. In this model, \(\alpha_i\) is a state-specific fixed
effect, \(\beta_t\) is a year-specific
fixed effect, \(b_{it}\) is the current
rate of beer taxation in state \(i\) in
year \(t\), \(d_{it}\) is the proportion of 18-20
year-olds in state \(i\) in year \(t\) who are legally allowed to drink, and
\(\delta\) captures the effect of
shifting the minimum legal drinking age from 21 to 18. Following Angrist
and Pischke’s analysis, we estimate this model both by (unweighted) OLS
and by weighted least squares with weights corresponding to population
size in a given state and year. We also demonstrate random effects
estimation and implement a cluster-robust Hausman specification
test.
Unweighted OLS
The following code does some simple data-munging and the estimates
the model by OLS:
library(clubSandwich)
data(MortalityRates)
# subset for deaths in motor vehicle accidents, 1970-1983
MV_deaths <- subset(MortalityRates, cause=="Motor Vehicle" &
year <= 1983 & !is.na(beertaxa))
# fit by OLS
lm_unweighted <- lm(mrate ~ 0 + legal + beertaxa +
factor(state) + factor(year), data = MV_deaths)
The coef_test function from clubSandwich
can then be used to test the hypothesis that changing the minimum legal
drinking age has no effect on motor vehicle deaths in this cohort (i.e.,
\(H_0: \delta = 0\)). The usual way to
test this is to cluster the standard errors by state, calculate the
robust Wald statistic, and compare that to a standard normal reference
distribution. The code and results are as follows:
coef_test(lm_unweighted, vcov = "CR1",
cluster = MV_deaths$state, test = "naive-t")[1:2,]
## Coef. Estimate SE t-stat d.f. (naive-t) p-val (naive-t) Sig.
## legal 7.59 2.44 3.108 49 0.00313 **
## beertaxa 3.82 5.14 0.743 49 0.46128
A better approach would be to use the generalized, bias-reduced
linearization CRVE, together with Satterthwaite degrees of freedom. In
the clubSandwich package, the BRL adjustment is called
“CR2†because it is directly analogous to the HC2 correction used in
heteroskedasticity-robust variance estimation. When applied to an OLS
model estimated by lm, the default working model is an
identity matrix, which amounts to the “working†assumption that the
errors are all uncorrelated and homoskedastic. Here’s how to apply this
approach in the example:
coef_test(lm_unweighted, vcov = "CR2",
cluster = MV_deaths$state, test = "Satterthwaite")[1:2,]
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## legal 7.59 2.51 3.019 24.58 0.00583 **
## beertaxa 3.82 5.27 0.725 5.77 0.49663
The Satterthwaite degrees of freedom are different for each
coefficient in the model, and so the coef_test function
reports them right alongside the standard error. For the effect of legal
drinking age, the degrees of freedom are about half of what might be
expected, given that there are 51 clusters. The p-value for the
CR2+Satterthwaite test is about twice as large as the p-value based on
the standard Wald test, although the coefficient is still statistically
significant at conventional levels. Note, however, that the degrees of
freedom on the beer taxation rate are considerably smaller because there
are only a few states with substantial variability in taxation rates
over time.
Unweighted “within†estimation
The plm package in R provides another way to estimate
the same model. It is convenient because it absorbs the state and year
fixed effects before estimating the effect of legal. The
clubSandwich package works with fitted plm
models too:
library(plm)
plm_unweighted <- plm(mrate ~ legal + beertaxa, data = MV_deaths,
effect = "twoways", index = c("state","year"))
coef_test(plm_unweighted, vcov = "CR1", cluster = "individual", test = "naive-t")
## Coef. Estimate SE t-stat d.f. (naive-t) p-val (naive-t) Sig.
## legal 7.59 2.44 3.108 49 0.00313 **
## beertaxa 3.82 5.14 0.743 49 0.46128
coef_test(plm_unweighted, vcov = "CR2", cluster = "individual", test = "Satterthwaite")
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## legal 7.59 2.51 3.019 24.58 0.00583 **
## beertaxa 3.82 5.27 0.725 5.77 0.49663
Population-weighted estimation
The difference between the standard method and the new method are not
terribly exciting in the above example. However, things change quite a
bit if the model is estimated using population weights. We go back to
fitting in lm with dummies for all the fixed effects
because plm does not handle weighted least squares.
lm_weighted <- lm(mrate ~ 0 + legal + beertaxa + factor(state) + factor(year),
weights = pop, data = MV_deaths)
coef_test(lm_weighted, vcov = "CR1",
cluster = MV_deaths$state, test = "naive-t")[1:2,]
## Coef. Estimate SE t-stat d.f. (naive-t) p-val (naive-t) Sig.
## legal 7.78 2.01 3.87 49 <0.001 ***
## beertaxa 11.16 4.20 2.66 49 0.0106 *
coef_test(lm_weighted, vcov = "CR2",
cluster = MV_deaths$state, test = "Satterthwaite")[1:2,]
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## legal 7.78 2.13 3.64 8.52 0.00588 **
## beertaxa 11.16 4.37 2.55 6.85 0.03854 *
Using population weights slightly reduces the point estimate of the
effect, while also slightly increasing its precision. If you were
following the standard approach, you would probably be happy with the
weighted estimates and wouldn’t think about it any further. However,
using the CR2 variance estimator and Satterthwaite correction produces a
p-value that is an order of magnitude larger (though still significant
at the conventional 5% level). The degrees of freedom are just
8.5—drastically smaller than would be expected based on the number of
clusters.
Even with weights, the coef_test function uses an
“independent, homoskedastic†working model as a default for
lm objects. In the present example, the outcome is a
standardized rate and so a better assumption might be that the error
variances are inversely proportional to population size. The following
code uses this alternate working model:
coef_test(lm_weighted, vcov = "CR2",
cluster = MV_deaths$state, target = 1 / MV_deaths$pop,
test = "Satterthwaite")[1:2,]
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## legal 7.78 2.03 3.83 12.64 0.00221 **
## beertaxa 11.16 4.17 2.68 5.06 0.04333 *
The new working model leads to slightly smaller standard errors and a
couple of additional degrees of freedom, though they remain in
small-sample territory.
Random effects estimation
If the unobserved effects \(\alpha_1,...,\alpha_{51}\) are uncorrelated
with the regressors, then a more efficient way to estimate \(\gamma,\delta\) is by weighted least
squares, with weights based on a random effects model. We still treat
the year effects as fixed.
plm_random <- plm(mrate ~ 0 + legal + beertaxa + year, data = MV_deaths,
effect = "individual", index = c("state","year"),
model = "random")
coef_test(plm_random, vcov = "CR1", test = "naive-t")[1:2,]
## Coef. Estimate SE t-stat d.f. (naive-t) p-val (naive-t) Sig.
## legal 7.31 2.39 3.054 49 0.00364 **
## beertaxa 3.37 5.11 0.661 49 0.51202
coef_test(plm_random, vcov = "CR2", test = "Satterthwaite")[1:2,]
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## legal 7.31 2.46 2.966 25.18 0.00652 **
## beertaxa 3.37 5.22 0.647 5.78 0.54258
With random effects estimation, the effect of legal drinking age is
smaller by about 1 death per 100,000. As a procedural aside, note that
coef_test infers that state is the clustering
variable because the call to plm includes only one type of effects
(random state effects).
Robust Hausman test
CRVE is also used in specification tests, as in the artificial
Hausman-type test for endogeneity of unobserved effects (Arellano,
1993). As noted above, random effects estimation is more efficient than
fixed effects estimation, but requires the assumption that the
unobserved effects are uncorrelated with the regressors. However, if the
unobserved effects covary with \(\mathbf{b}_i,
\mathbf{d}_i\), then the random-effects estimator will be
biased.
We can test for whether endogeneity is a problem by including
group-centered covariates as additional regressors. Let \(\tilde{d}_{it} = d_{it} - \frac{1}{T}\sum_t
d_{it}\), with \(\tilde{b}_{it}\) defined analogously. Now
estimate the regression
\[y_{it} = \beta_t + \gamma_1 b_{it} +
\gamma_2 \tilde{b}_{it} + \delta_1 d_{it} + \delta_2 \tilde{d}_{it} +
\epsilon_{it},\]
which does not include state fixed effects. The parameters \(\gamma_2,\delta_2\) represent the
differences between the within-groups and between-groups estimands of
\(\gamma_1, \delta_1\). If these are
both zero, then the random effects estimator is unbiased. Thus, the
joint test for \(H_0: \gamma_2 = \delta_2 =
0\) amounts to a test for exogeneity of the unobserved
effects.
For efficiency, we estimate this specification using weighted least
squares (although OLS would be valid too):
MV_deaths <- within(MV_deaths, {
legal_cent <- legal - tapply(legal, state, mean)[factor(state)]
beer_cent <- beertaxa - tapply(beertaxa, state, mean)[factor(state)]
})
plm_Hausman <- plm(mrate ~ 0 + legal + beertaxa + legal_cent + beer_cent + factor(year),
data = MV_deaths,
effect = "individual", index = c("state","year"),
model = "random")
coef_test(plm_Hausman, vcov = "CR2", test = "Satterthwaite")[1:4,]
## Coef. Estimate SE t-stat d.f. (Satt) p-val (Satt) Sig.
## legal -9.180 7.62 -1.2042 24.94 0.2398
## beertaxa 3.395 9.40 0.3613 6.44 0.7295
## legal_cent 16.768 8.53 1.9665 25.44 0.0602 .
## beer_cent 0.424 9.25 0.0458 6.42 0.9648
To conduct a joint test on the centered covariates, we can use the
Wald_test function. The usual way to test this hypothesis
would be to use the CR1 variance estimator to calculate the
robust Wald statistic, then use a \(\chi^2_2\) reference distribution (or
equivalently, compare a re-scaled Wald statistic to an \(F(2,\infty)\) distribution). The
Wald_test function reports the latter version:
Wald_test(plm_Hausman,
constraints = constrain_zero(c("legal_cent","beer_cent")),
vcov = "CR1", test = "chi-sq")
## test Fstat df_num df_denom p_val sig
## chi-sq 2.93 2 Inf 0.0534 .
The test is just shy of significance at the 5% level. If we instead
use the CR2 variance estimator and our newly proposed
approximate F-test (which is the default in Wald_test),
then we get:
Wald_test(plm_Hausman,
constraints = constrain_zero(c("legal_cent","beer_cent")),
vcov = "CR2")
## test Fstat df_num df_denom p_val sig
## HTZ 2.56 2 11.7 0.12
The low degrees of freedom of the test indicate that we’re definitely
in small-sample territory and should not trust the asymptotic \(\chi^2\) approximation.
References
Angrist, J. D., & Pischke, J. (2009). Mostly harmless
econometrics: An empiricist’s companion. Princeton, NJ: Princeton
University Press.
Angrist, J. D., and Pischke, J. S. (2014). Mastering’metrics: the
path from cause to effect. Princeton, NJ: Princeton University
Press.
Arellano, M. (1993). On the testing of correlated effects with panel
data. Journal of Econometrics, 59(1-2), 87-97. doi: 10.1016/0304-4076(93)90040-C
Bell, R. M., & McCaffrey, D. F. (2002). Bias reduction in
standard errors for linear regression with multi-stage samples.
Survey Methodology, 28(2), 169-181.
Cameron, A. C., & Miller, D. L. (2015). A practitioner’s guide to
cluster-robust inference. URL: https://cameron.econ.ucdavis.edu/research/Cameron_Miller_JHR_2015_February.pdf
Carpenter, C., & Dobkin, C. (2011). The minimum legal drinking
age and public health. Journal of Economic Perspectives, 25(2),
133-156. doi: 10.1257/jep.25.2.133
Imbens, G. W., & Kolesar, M. (2015). Robust standard errors in
small samples: Some practical advice. URL: https://doi.org/10.1162/REST_a_00552
Pustejovsky, J. E. & Tipton, E. (2016). Small sample methods for
cluster-robust variance estimation and hypothesis testing in fixed
effects models. arXiv: 1601.01981 [stat.ME]
Young, A. (2016). Improved, nearly exact, statistical inference with
robust and clustered covariance matrices using effective degrees of
freedom corrections.
clubSandwich/build/ 0000755 0001762 0000144 00000000000 14635065010 013757 5 ustar ligges users clubSandwich/build/vignette.rds 0000644 0001762 0000144 00000000562 14635065010 016321 0 ustar ligges users ‹ …’ÝJÃ0dzµN7”y#Í dà…O°9dŠz›µg4&%É,»óÅužtéX§Î‹žääü“ßùè[Ò&q‘v„Û¨¦ƒß‰wHLº¸^¾r™2ÖY&KäböÄUZŠ$Nó4¨.rpœqÅåÒ
ËJá26š¾ÜnIúW YÊQ¸ 5YמE+Õsš/¤…–heáB9ªRÒlYh—.Þìù çï£÷õ
!Ì…d* clubSandwich/build/partial.rdb 0000644 0001762 0000144 00000000075 14635064767 016130 0 ustar ligges users ‹ ‹àb```b`aff`b1…À€… H02°0piÖ¼ÄÜÔb C "Éð 4¾è÷7 clubSandwich/man/ 0000755 0001762 0000144 00000000000 14634640072 013441 5 ustar ligges users clubSandwich/man/SATcoaching.Rd 0000644 0001762 0000144 00000002476 14630154051 016055 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/data-documentation.R
\docType{data}
\name{SATcoaching}
\alias{SATcoaching}
\title{Randomized experiments on SAT coaching}
\format{
A data frame with 67 rows and 11 variables:
\describe{
\item{study}{Study identifier}
\item{year}{Year of publication}
\item{test}{Character string indicating whether effect size corresponds to outcome on verbal (SATV) or math (SATM) test}
\item{d}{Effect size estimate (Standardized mean difference)}
\item{V}{Variance of effect size estimate}
\item{nT}{Sample size in treatment condition}
\item{nC}{Sample size in control condition}
\item{study_type}{Character string indicating whether study design used a matched, non-equivalent, or randomized control group}
\item{hrs}{Hours of coaching}
\item{ETS}{Indicator variable for Educational Testing Service}
\item{homework}{Indicator variable for homework}
}
}
\usage{
SATcoaching
}
\description{
Effect sizes from studies on the effects of SAT coaching,
reported in Kalaian and Raudenbush (1996)
}
\references{
Kalaian, H. A. & Raudenbush, S. W. (1996). A multivariate mixed
linear model for meta-analysis. \emph{Psychological Methods, 1}(3),
227-235.
\doi{10.1037/1082-989X.1.3.227}
}
\keyword{datasets}
clubSandwich/man/coef_test.Rd 0000644 0001762 0000144 00000005044 14630154051 015677 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/coef_test.R
\name{coef_test}
\alias{coef_test}
\title{Test all or selected regression coefficients in a fitted model}
\usage{
coef_test(
obj,
vcov,
test = "Satterthwaite",
coefs = "All",
p_values = TRUE,
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate t-tests.}
\item{vcov}{Variance covariance matrix estimated using \code{vcovCR} or a
character string specifying which small-sample adjustment should be used to
calculate the variance-covariance.}
\item{test}{Character vector specifying which small-sample corrections to
calculate. \code{"z"} returns a z test (i.e., using a standard normal
reference distribution). \code{"naive-t"} returns a t test with \code{m -
1} degrees of freedom, where \code{m} is the number of unique clusters.
\code{"naive-tp"} returns a t test with \code{m - p} degrees of freedom,
where \code{p} is the number of regression coefficients in \code{obj}.
\code{"Satterthwaite"} returns a Satterthwaite correction.
\code{"saddlepoint"} returns a saddlepoint correction. Default is
\code{"Satterthwaite"}.}
\item{coefs}{Character, integer, or logical vector specifying which
coefficients should be tested. The default value \code{"All"} will test all
estimated coefficients.}
\item{p_values}{Logical indicating whether to report p-values. The default
value is \code{TRUE}.}
\item{...}{Further arguments passed to \code{\link{vcovCR}}, which are only
needed if \code{vcov} is a character string.}
}
\value{
A data frame containing estimated regression coefficients, standard
errors, and test results. For the Satterthwaite approximation, degrees of
freedom and a p-value are reported. For the saddlepoint approximation, the
saddlepoint and a p-value are reported.
}
\description{
\code{coef_test} reports t-tests for each coefficient estimate in a fitted
linear regression model, using a sandwich estimator for the standard errors
and a small sample correction for the p-value. The small-sample correction is
based on a Satterthwaite approximation or a saddlepoint approximation.
}
\examples{
data("ChickWeight", package = "datasets")
lm_fit <- lm(weight ~ Diet * Time, data = ChickWeight)
diet_index <- grepl("Diet.:Time", names(coef(lm_fit)))
coef_test(lm_fit, vcov = "CR2", cluster = ChickWeight$Chick, coefs = diet_index)
V_CR2 <- vcovCR(lm_fit, cluster = ChickWeight$Chick, type = "CR2")
coef_test(lm_fit, vcov = V_CR2, coefs = diet_index)
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.lm.Rd 0000644 0001762 0000144 00000006155 14630154051 015541 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/lm.R
\name{vcovCR.lm}
\alias{vcovCR.lm}
\title{Cluster-robust variance-covariance matrix for an lm object.}
\usage{
\method{vcovCR}{lm}(
obj,
cluster,
type,
target = NULL,
inverse_var = NULL,
form = "sandwich",
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Expression or vector indicating which observations belong to
the same cluster. Required for \code{lm} objects.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If a vector, the target matrix is assumed to be
diagonal. If not specified, the target is taken to be an identity matrix.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from an \code{\link{lm}} object.
}
\examples{
data("ChickWeight", package = "datasets")
lm_fit <- lm(weight ~ Time + Diet:Time, data = ChickWeight)
vcovCR(lm_fit, cluster = ChickWeight$Chick, type = "CR2")
if (requireNamespace("plm", quietly = TRUE)) withAutoprint({
data("Produc", package = "plm")
lm_individual <- lm(log(gsp) ~ 0 + state + log(pcap) + log(pc) + log(emp) + unemp, data = Produc)
individual_index <- !grepl("state", names(coef(lm_individual)))
vcovCR(lm_individual, cluster = Produc$state, type = "CR2")[individual_index,individual_index]
# compare to plm()
plm_FE <- plm::plm(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp,
data = Produc, index = c("state","year"),
effect = "individual", model = "within")
vcovCR(plm_FE, type="CR2")
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.robu.Rd 0000644 0001762 0000144 00000005775 14630154051 016107 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/robu.R
\name{vcovCR.robu}
\alias{vcovCR.robu}
\title{Cluster-robust variance-covariance matrix for a robu object.}
\usage{
\method{vcovCR}{robu}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional expression or vector indicating which observations
belong to the same cluster. If not specified, will be set to the
\code{studynum} used in fitting the \code{\link[robumeta]{robu}} object.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be the
inverse of the estimated weights used in fitting the
\code{\link[robumeta]{robu}} object.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from a
\code{\link[robumeta]{robu}} object.
}
\examples{
if (requireNamespace("robumeta", quietly = TRUE)) withAutoprint({
library(robumeta)
data(hierdat)
robu_fit <- robu(effectsize ~ binge + followup + sreport + age,
data = hierdat, studynum = studyid,
var.eff.size = var, modelweights = "HIER")
robu_fit
robu_CR2 <- vcovCR(robu_fit, type = "CR2")
robu_CR2
coef_test(robu_fit, vcov = robu_CR2, test = c("Satterthwaite", "saddlepoint"))
Wald_test(robu_fit, constraints = constrain_zero(c(2,4)), vcov = robu_CR2)
Wald_test(robu_fit, constraints = constrain_zero(2:5), vcov = robu_CR2)
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.geeglm.Rd 0000644 0001762 0000144 00000005447 14630154051 016374 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/geeglm.R
\name{vcovCR.geeglm}
\alias{vcovCR.geeglm}
\title{Cluster-robust variance-covariance matrix for a geeglm object.}
\usage{
\method{vcovCR}{geeglm}(
obj,
cluster,
type,
target = NULL,
inverse_var = NULL,
form = "sandwich",
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Expression or vector indicating which observations belong to
the same cluster. Required for \code{geeglm} objects.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If a vector, the target matrix is assumed to be
diagonal. If not specified, the target is taken to be the estimated variance function.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from an \code{\link[geepack]{geeglm}} object.
}
\examples{
if (requireNamespace("geepack", quietly = TRUE)) {
library(geepack)
data(dietox, package = "geepack")
dietox$Cu <- as.factor(dietox$Cu)
mf <- formula(Weight ~ Cu * (Time + I(Time^2) + I(Time^3)))
gee1 <- geeglm(mf, data=dietox, id=Pig, family=poisson("identity"), corstr="ar1")
V_CR <- vcovCR(gee1, cluster = dietox$Pig, type = "CR2")
coef_test(gee1, vcov = V_CR, test = "Satterthwaite")
}
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.lme.Rd 0000644 0001762 0000144 00000006067 14630154051 015710 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/lme.R
\name{vcovCR.lme}
\alias{vcovCR.lme}
\title{Cluster-robust variance-covariance matrix for an lme object.}
\usage{
\method{vcovCR}{lme}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional expression or vector indicating which observations
belong to the same cluster. If not specified, will be set to
\code{getGroups(obj)}.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be the
estimated variance-covariance structure of the \code{lme} object.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from a \code{\link[nlme]{lme}} object.
}
\examples{
if (requireNamespace("nlme", quietly = TRUE)) {
library(nlme)
rat_weight <- lme(weight ~ Time * Diet, data=BodyWeight, ~ Time | Rat)
vcovCR(rat_weight, type = "CR2")
}
pkgs_available <-
requireNamespace("nlme", quietly = TRUE) &
requireNamespace("mlmRev", quietly = TRUE)
if (pkgs_available) {
data(egsingle, package = "mlmRev")
subset_ids <- levels(egsingle$schoolid)[1:10]
egsingle_subset <- subset(egsingle, schoolid \%in\% subset_ids)
math_model <- lme(math ~ year * size + female + black + hispanic,
random = list(~ year | schoolid, ~ 1 | childid),
data = egsingle_subset)
vcovCR(math_model, type = "CR2")
}
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/MortalityRates.Rd 0000644 0001762 0000144 00000003032 14635054714 016714 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/data-documentation.R
\docType{data}
\name{MortalityRates}
\alias{MortalityRates}
\title{State-level annual mortality rates by cause among 18-20 year-olds}
\format{
A data frame with 5508 rows and 12 variables: \describe{
\item{year}{Year of observation}
\item{state}{identifier for state}
\item{count}{Number of deaths}
\item{pop}{Population size}
\item{legal}{Proportion of 18-20 year-old population that is legally allowed to drink}
\item{beertaxa}{Beer taxation rate}
\item{beerpercap}{Beer consumption per capita}
\item{winepercap}{Wine consumption per capita}
\item{spiritpercap}{Spirits consumption per capita}
\item{totpercap}{Total alcohol consumption per capita}
\item{mrate}{Mortality rate per 10,000}
\item{cause}{Cause of death}
}
}
\source{
\href{https://masteringmetrics.com/wp-content/uploads/2015/01/deaths.dta}{Mastering
'Metrics data archive}
}
\usage{
MortalityRates
}
\description{
A dataset containing state-level annual mortality rates for select causes of
death, as well as data related to the minimum legal drinking age and alcohol
consumption.
}
\references{
Angrist, J. D., and Pischke, J. S. (2014). _Mastering'metrics: the path from
cause to effect_. Princeton University Press, 2014.
Carpenter, C., & Dobkin, C. (2011). The minimum legal drinking age and public
health. _Journal of Economic Perspectives, 25_(2), 133-156.
\doi{10.1257/jep.25.2.133}
}
\keyword{datasets}
clubSandwich/man/pattern_covariance_matrix.Rd 0000644 0001762 0000144 00000013267 14630154051 021165 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/rma-mv.R
\name{pattern_covariance_matrix}
\alias{pattern_covariance_matrix}
\title{Impute a patterned block-diagonal covariance matrix}
\usage{
pattern_covariance_matrix(
vi,
cluster,
pattern_level,
r_pattern,
r,
smooth_vi = FALSE,
subgroup = NULL,
return_list = identical(as.factor(cluster), sort(as.factor(cluster))),
check_PD = TRUE
)
}
\arguments{
\item{vi}{Vector of variances}
\item{cluster}{Vector indicating which effects belong to the same cluster.
Effects with the same value of `cluster` will be treated as correlated.}
\item{pattern_level}{Vector of categories for each effect size, used to
determine which entry of the pattern matrix will be used to impute a
correlation.}
\item{r_pattern}{Patterned correlation matrix with row and column names
corresponding to the levels of \code{pattern}.}
\item{r}{Vector or numeric value of assumed constant correlation(s) between
effect size estimates from each study.}
\item{smooth_vi}{Logical indicating whether to smooth the marginal variances
by taking the average \code{vi} within each cluster. Defaults to
\code{FALSE}.}
\item{subgroup}{Vector of category labels describing sub-groups of effects.
If non-null, effects that share the same category label and the same
cluster will be treated as correlated, but effects with different category
labels will be treated as uncorrelated, even if they come from the same
cluster.}
\item{return_list}{Optional logical indicating whether to return a list of
matrices (with one entry per block) or the full variance-covariance matrix.}
\item{check_PD}{Optional logical indicating whether to check whether each
covariance matrix is positive definite. If \code{TRUE} (the default), the
function will display a warning if any covariance matrix is not positive
definite.}
}
\value{
If \code{cluster} is appropriately sorted, then a list of matrices,
with one entry per cluster, will be returned by default. If \code{cluster}
is out of order, then the full variance-covariance matrix will be returned
by default. The output structure can be controlled with the optional
\code{return_list} argument.
}
\description{
`r lifecycle::badge("superseded")`
This function is superseded by the \code{\link[metafor]{vcalc}} provided by
the \code{metafor} package. Compared to \code{pattern_covariance_matrix},
\code{\link[metafor]{vcalc}} provides many further features, includes a
\code{data} argument, and uses syntax that is consistent with other
functions in \code{metafor}.
\code{pattern_covariance_matrix} calculates a
block-diagonal covariance matrix, given the marginal variances, the block
structure, and an assumed correlation structure defined by a patterned
correlation matrix.
}
\details{
A block-diagonal variance-covariance matrix (possibly represented as
a list of matrices) with a specified correlation structure, defined by a
patterned correlation matrix. Let \eqn{v_{ij}}{v-ij} denote the specified
variance for effect \eqn{i}{i} in cluster \eqn{j}{j} and
\eqn{C_{hij}}{C-hij} be the covariance between effects \eqn{h}{h} and
\eqn{i}{i} in cluster \eqn{j}{j}. Let \eqn{p_{ij}}{p-ij} be the level
of the pattern variable for effect \eqn{i}{i} in cluster \eqn{j}{j},
taking a value in \eqn{1,...,C}{1,...,C}. A patterned correlation matrix
is defined as a set of correlations between pairs of effects taking each
possible combination of patterns. Formally, let \eqn{r_{cd}}{r-cd} be the
correlation between effects in categories \eqn{c}{c} and \eqn{d}{d},
respectively, where \eqn{r_{cd} = r_{dc}}{r-cd = r-dc}. Then the
covariance between effects \eqn{h}{h} and \eqn{i}{i} in cluster
\eqn{j}{j} is taken to be \deqn{C_{hij} = \sqrt{v_{hj} v_{ij}} \times
r_{p_{hj} p_{ij}}.}{C-hij = sqrt(v-hj v-ij) * r[p-hj, p-ij].}
Correlations between effect sizes within the same category are defined by the diagonal
values of the pattern matrix, which may take values less than one.
Combinations of pattern levels that do not occur in the patterned correlation matrix will be set equal to \code{r}.
If \code{smooth_vi = TRUE}, then all of the variances within cluster
\eqn{j}{j} will be set equal to the average variance of cluster
\eqn{j}{j}, i.e., \deqn{v'_{ij} = \frac{1}{n_j} \sum_{i=1}^{n_j}
v_{ij}}{v-ij' = (v-1j + ... + v-nj,j) / n-j} for
\eqn{i=1,...,n_j}{i=1,...,n-j} and \eqn{j=1,...,k}{j=1,...,k}.
}
\examples{
pkgs_available <-
requireNamespace("metafor", quietly = TRUE) &
requireNamespace("robumeta", quietly = TRUE)
if (pkgs_available) {
library(metafor)
data(oswald2013, package = "robumeta")
dat <- escalc(data = oswald2013, measure = "ZCOR", ri = R, ni = N)
subset_ids <- unique(dat$Study)[1:20]
dat <- subset(dat, Study \%in\% subset_ids)
# make a patterned correlation matrix
p_levels <- levels(dat$Crit.Cat)
r_pattern <- 0.7^as.matrix(dist(1:length(p_levels)))
diag(r_pattern) <- seq(0.75, 0.95, length.out = 6)
rownames(r_pattern) <- colnames(r_pattern) <- p_levels
# impute the covariance matrix using patterned correlations
V_list <- pattern_covariance_matrix(vi = dat$vi,
cluster = dat$Study,
pattern_level = dat$Crit.Cat,
r_pattern = r_pattern,
smooth_vi = TRUE)
# fit a model using imputed covariance matrix
MVFE <- rma.mv(yi ~ 0 + Crit.Cat, V = V_list,
random = ~ Crit.Cat | Study,
data = dat)
conf_int(MVFE, vcov = "CR2")
}
}
clubSandwich/man/conf_int.Rd 0000644 0001762 0000144 00000005156 14630154051 015527 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/conf_int.R
\name{conf_int}
\alias{conf_int}
\title{Calculate confidence intervals for all or selected regression coefficients in
a fitted model}
\usage{
conf_int(
obj,
vcov,
level = 0.95,
test = "Satterthwaite",
coefs = "All",
...,
p_values = FALSE
)
}
\arguments{
\item{obj}{Fitted model for which to calculate confidence intervals.}
\item{vcov}{Variance covariance matrix estimated using \code{vcovCR} or a
character string specifying which small-sample adjustment should be used to
calculate the variance-covariance.}
\item{level}{Desired coverage level for confidence intervals.}
\item{test}{Character vector specifying which small-sample corrections to
calculate. \code{"z"} returns a z test (i.e., using a standard normal
reference distribution). \code{"naive-t"} returns a t test with \code{m -
1} degrees of freedom, where \code{m} is the number of unique clusters.
\code{"naive-tp"} returns a t test with \code{m - p} degrees of freedom,
where \code{p} is the number of regression coefficients in \code{obj}.
\code{"Satterthwaite"} returns a Satterthwaite correction. Unlike in
\code{coef_test()}, \code{"saddlepoint"} is not currently supported in
\code{conf_int()} because saddlepoint confidence intervals do not have a
closed-form solution.}
\item{coefs}{Character, integer, or logical vector specifying which
coefficients should be tested. The default value \code{"All"} will test all
estimated coefficients.}
\item{...}{Further arguments passed to \code{\link{vcovCR}}, which are only
needed if \code{vcov} is a character string.}
\item{p_values}{Logical indicating whether to report p-values. The default
value is \code{FALSE}.}
}
\value{
A data frame containing estimated regression coefficients, standard
errors, confidence intervals, and (optionally) p-values.
}
\description{
\code{conf_int} reports confidence intervals for each coefficient estimate in
a fitted linear regression model, using a sandwich estimator for the standard
errors and a small sample correction for the critical values. The
small-sample correction is based on a Satterthwaite approximation.
}
\examples{
data("ChickWeight", package = "datasets")
lm_fit <- lm(weight ~ Diet * Time, data = ChickWeight)
diet_index <- grepl("Diet.:Time", names(coef(lm_fit)))
conf_int(lm_fit, vcov = "CR2", cluster = ChickWeight$Chick, coefs = diet_index)
V_CR2 <- vcovCR(lm_fit, cluster = ChickWeight$Chick, type = "CR2")
conf_int(lm_fit, vcov = V_CR2, level = .99, coefs = diet_index)
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.gls.Rd 0000644 0001762 0000144 00000005226 14630154051 015714 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/gls.R
\name{vcovCR.gls}
\alias{vcovCR.gls}
\title{Cluster-robust variance-covariance matrix for a gls object.}
\usage{
\method{vcovCR}{gls}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional expression or vector indicating which observations
belong to the same cluster. If not specified, will be set to
\code{getGroups(obj)}.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be the
estimated variance-covariance structure of the \code{gls} object.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from a \code{\link[nlme]{gls}} object.
}
\examples{
if (requireNamespace("nlme", quietly = TRUE)) {
library(nlme)
data(Ovary, package = "nlme")
Ovary$time_int <- 1:nrow(Ovary)
lm_AR1 <- gls(follicles ~ sin(2*pi*Time) + cos(2*pi*Time), data = Ovary,
correlation = corAR1(form = ~ time_int | Mare))
vcovCR(lm_AR1, type = "CR2")
}
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.ivreg.Rd 0000644 0001762 0000144 00000007261 14630154051 016244 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/ivreg.R
\name{vcovCR.ivreg}
\alias{vcovCR.ivreg}
\title{Cluster-robust variance-covariance matrix for an ivreg object.}
\usage{
\method{vcovCR}{ivreg}(
obj,
cluster,
type,
target = NULL,
inverse_var = FALSE,
form = "sandwich",
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Expression or vector indicating which observations belong to
the same cluster. Required for \code{ivreg} objects.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If a vector, the target matrix is assumed to be
diagonal. If not specified, the target is taken to be an identity matrix.}
\item{inverse_var}{Not used for \code{ivreg} objects.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from an ivreg object fitted
from the \CRANpkg{AER} package or the \CRANpkg{ivreg} package.
}
\details{
For any "ivreg" objects fitted via the \code{\link[ivreg]{ivreg}}
function from the \CRANpkg{ivreg} package, only traditional 2SLS
regression method (method = "OLS") is supported.
clubSandwich currently cannot support robust-regression methods such as
M-estimation (method = "M") or MM-estimation (method = "MM").
}
\examples{
if (requireNamespace("AER", quietly = TRUE)) withAutoprint({
library(AER)
data("CigarettesSW")
Cigs <- within(CigarettesSW, {
rprice <- price/cpi
rincome <- income/population/cpi
tdiff <- (taxs - tax)/cpi
})
iv_fit_AER <- AER::ivreg(log(packs) ~ log(rprice) + log(rincome) |
log(rincome) + tdiff + I(tax/cpi), data = Cigs)
vcovCR(iv_fit_AER, cluster = Cigs$state, type = "CR2")
coef_test(iv_fit_AER, vcov = "CR2", cluster = Cigs$state)
})
pkgs_available <-
requireNamespace("AER", quietly = TRUE) &
requireNamespace("ivreg", quietly = TRUE)
if (pkgs_available) withAutoprint ({
data("CigarettesSW")
Cigs <- within(CigarettesSW, {
rprice <- price/cpi
rincome <- income/population/cpi
tdiff <- (taxs - tax)/cpi
})
iv_fit_ivreg <- ivreg::ivreg(log(packs) ~ log(rprice) + log(rincome) |
log(rincome) + tdiff + I(tax/cpi), data = Cigs)
vcovCR(iv_fit_ivreg, cluster = Cigs$state, type = "CR2")
coef_test(iv_fit_ivreg, vcov = "CR2", cluster = Cigs$state)
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/dropoutPrevention.Rd 0000644 0001762 0000144 00000005664 14630154051 017502 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/data-documentation.R
\docType{data}
\name{dropoutPrevention}
\alias{dropoutPrevention}
\title{Dropout prevention/intervention program effects}
\format{
A data frame with 385 rows and 18 variables: \describe{
\item{LOR1}{log-odds ratio measuring the intervention effect}
\item{varLOR}{estimated sampling variance of the log-odds ratio}
\item{studyID}{unique identifier for each study} \item{studySample}{unique
identifier for each sample within a study} \item{study_design}{study design
(randomized, matched, or non-randomized and unmatched)}
\item{outcome}{outcome measure for the intervention effect is estimated
(school dropout, school enrollment, graduation, graduation or GED receipt)}
\item{evaluator_independence}{degree of evaluator independence
(independent, indirect but influential, involved in planning but not
delivery, involved in delivery)} \item{implementation_quality}{level of
implementation quality (clear problems, possible problems, no apparent
problems)} \item{program_site}{Program delivery site (community, mixed,
school classroom, school but outside of classroom)}
\item{attrition}{Overall attrition (proportion)}
\item{group_equivalence}{pretest group-equivalence log-odds ratio}
\item{adjusted}{adjusted or unadjusted data used to calculate intervention
effect} \item{male_pct}{proportion of the sample that is male}
\item{white_pct}{proportion of the sample that is white}
\item{average_age}{average age of the sample} \item{duration}{program
duration (in weeks)} \item{service_hrs}{program contact hours per week}
\item{big_study}{indicator for the 32 studies with 3 or more effect sizes}
}
}
\source{
Wilson, S. J., Lipsey, M. W., Tanner-Smith, E., Huang, C. H., &
Steinka-Fry, K. T. (2011). Dropout prevention and intervention programs:
Effects on school completion and dropout Among school-aged children and
youth: A systematic review. _Campbell Systematic Reviews, 7_(1), 1-61.
\doi{10.4073/csr.2011.8}
}
\usage{
dropoutPrevention
}
\description{
A dataset containing estimated effect sizes, variances, and covariates from a
meta-analysis of dropout prevention/intervention program effects, conducted
by Wilson et al. (2011). Missing observations were imputed.
}
\references{
Wilson, S. J., Lipsey, M. W., Tanner-Smith, E., Huang, C. H., &
Steinka-Fry, K. T. (2011). Dropout prevention and intervention programs:
Effects on school completion and dropout Among school-aged children and
youth: A systematic review. _Campbell Systematic Reviews, 7_(1), 1-61.
\doi{10.4073/csr.2011.8}
Tipton, E., & Pustejovsky, J. E. (2015). Small-sample adjustments for tests
of moderators and model fit using robust variance estimation in
meta-regression. _Journal of Educational and Behavioral Statistics, 40_(6), 604-634.
\doi{10.3102/1076998615606099}
}
\keyword{datasets}
clubSandwich/man/linear_contrast.Rd 0000644 0001762 0000144 00000007351 14630154051 017116 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/conf_int.R
\name{linear_contrast}
\alias{linear_contrast}
\title{Calculate confidence intervals and p-values for linear contrasts of
regression coefficients in a fitted model}
\usage{
linear_contrast(
obj,
vcov,
contrasts,
level = 0.95,
test = "Satterthwaite",
...,
p_values = FALSE
)
}
\arguments{
\item{obj}{Fitted model for which to calculate confidence intervals.}
\item{vcov}{Variance covariance matrix estimated using \code{vcovCR} or a
character string specifying which small-sample adjustment should be used to
calculate the variance-covariance.}
\item{contrasts}{A contrast matrix, or a list of multiple contrast matrices
to test. See details and examples.}
\item{level}{Desired coverage level for confidence intervals.}
\item{test}{Character vector specifying which small-sample corrections to
calculate. \code{"z"} returns a z test (i.e., using a standard normal
reference distribution). \code{"naive-t"} returns a t test with \code{m -
1} degrees of freedom, where \code{m} is the number of unique clusters.
\code{"naive-tp"} returns a t test with \code{m - p} degrees of freedom,
where \code{p} is the number of regression coefficients in \code{obj}.
\code{"Satterthwaite"} returns a Satterthwaite correction. Unlike in
\code{coef_test()}, \code{"saddlepoint"} is not currently supported in
\code{conf_int()} because saddlepoint confidence intervals do not have a
closed-form solution.}
\item{...}{Further arguments passed to \code{\link{vcovCR}}, which are only
needed if \code{vcov} is a character string.}
\item{p_values}{Logical indicating whether to report p-values. The default
value is \code{FALSE}.}
}
\value{
A data frame containing estimated contrasts, standard
errors, confidence intervals, and (optionally) p-values.
}
\description{
\code{linear_contrast} reports confidence intervals and (optionally) p-values
for linear contrasts of regression coefficients from a fitted model, using a
sandwich estimator for the standard errors and (optionally) a small sample
correction for the critical values. The default small-sample correction is
based on a Satterthwaite approximation.
}
\details{
Constraints can be specified directly as q X p matrices or
indirectly through \code{\link{constrain_pairwise}},
\code{\link{constrain_equal}}, or \code{\link{constrain_zero}}.
}
\examples{
data("ChickWeight", package = "datasets")
lm_fit <- lm(weight ~ 0 + Diet + Time:Diet, data = ChickWeight)
# Pairwise comparisons of diet-by-time slopes
linear_contrast(lm_fit, vcov = "CR2", cluster = ChickWeight$Chick,
contrasts = constrain_pairwise("Diet.:Time", reg_ex = TRUE))
if (requireNamespace("carData", quietly = TRUE)) withAutoprint({
data(Duncan, package = "carData")
Duncan$cluster <- sample(LETTERS[1:8], size = nrow(Duncan), replace = TRUE)
Duncan_fit <- lm(prestige ~ 0 + type + income + type:income + type:education, data=Duncan)
# Note that type:income terms are interactions because main effect of income is included
# but type:education terms are separate slopes for each unique level of type
# Pairwise comparisons of type-by-education slopes
linear_contrast(Duncan_fit, vcov = "CR2", cluster = Duncan$cluster,
contrasts = constrain_pairwise(":education", reg_ex = TRUE),
test = "Satterthwaite")
# Pairwise comparisons of type-by-income interactions
linear_contrast(Duncan_fit, vcov = "CR2", cluster = Duncan$cluster,
contrasts = constrain_pairwise(":income", reg_ex = TRUE, with_zero = TRUE),
test = "Satterthwaite")
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.plm.Rd 0000644 0001762 0000144 00000011514 14630154051 015714 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/plm.R
\name{vcovCR.plm}
\alias{vcovCR.plm}
\title{Cluster-robust variance-covariance matrix for a plm object.}
\usage{
\method{vcovCR}{plm}(
obj,
cluster,
type,
target,
inverse_var,
form = "sandwich",
ignore_FE = FALSE,
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional character string, expression, or vector indicating
which observations belong to the same cluster. For fixed-effect models that
include individual effects or time effects (but not both), the cluster will
be taken equal to the included fixed effects if not otherwise specified.
Clustering on individuals can also be obtained by specifying the name of
the individual index (e.g., \code{cluster = "state"}) or \code{cluster =
"individual"}; clustering on time periods can be obtained by specifying the
name of the time index (e.g., \code{cluster = "year"}) or \code{cluster =
"time"}; if a group index is specified, clustering on groups (in which
individuals are nested) can be obtained by specifying the name of the group
index or \code{cluster = "group"}. For random-effects models, the cluster
will be taken equal to the included random effect identifier if not
otherwise specified.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. By default, the target is taken to be an identity
matrix for fixed effect models or the estimated compound-symmetric
covariance matrix for random effects models.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{ignore_FE}{Optional logical controlling whether fixed effects are
ignored when calculating small-sample adjustments in models where fixed
effects are estimated through absorption.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from a \code{\link[plm]{plm}}
object.
}
\examples{
if (requireNamespace("plm", quietly = TRUE)) withAutoprint({
library(plm)
# fixed effects
data("Produc", package = "plm")
plm_FE <- plm(log(gsp) ~ log(pcap) + log(pc) + log(emp) + unemp,
data = Produc, index = c("state","year","region"),
effect = "individual", model = "within")
vcovCR(plm_FE, type="CR2")
vcovCR(plm_FE, type = "CR2", cluster = Produc$region) # clustering on region
# random effects
plm_RE <- update(plm_FE, model = "random")
vcovCR(plm_RE, type = "CR2")
vcovCR(plm_RE, type = "CR2", cluster = Produc$region) # clustering on region
# nested random effects
plm_nested <- update(plm_FE, effect = "nested", model = "random")
vcovCR(plm_nested, type = "CR2") # clustering on region
})
pkgs_available <- requireNamespace("plm", quietly = TRUE) & requireNamespace("AER", quietly = TRUE)
if (pkgs_available) withAutoprint({
# first differencing
data(Fatalities, package = "AER")
Fatalities <- within(Fatalities, {
frate <- 10000 * fatal / pop
drinkagec <- cut(drinkage, breaks = 18:22, include.lowest = TRUE, right = FALSE)
drinkagec <- relevel(drinkagec, ref = 4)
})
plm_FD <- plm(frate ~ beertax + drinkagec + miles + unemp + log(income),
data = Fatalities, index = c("state", "year"),
model = "fd")
vcovHC(plm_FD, method="arellano", type = "sss", cluster = "group")
vcovCR(plm_FD, type = "CR1S")
vcovCR(plm_FD, type = "CR2")
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.lmerMod.Rd 0000644 0001762 0000144 00000005761 14630154051 016532 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/lmer.R
\name{vcovCR.lmerMod}
\alias{vcovCR.lmerMod}
\title{Cluster-robust variance-covariance matrix for an lmerMod object.}
\usage{
\method{vcovCR}{lmerMod}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional expression or vector indicating which observations
belong to the same cluster. If not specified, will be set to
\code{getGroups(obj)}.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be the
estimated variance-covariance structure of the \code{lmerMod} object.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from \code{\link[lme4:merMod-class]{merMod}} object.
}
\examples{
if (requireNamespace("lme4", quietly = TRUE)) {
library(lme4)
sleep_fit <- lmer(Reaction ~ Days + (Days | Subject), sleepstudy)
vcovCR(sleep_fit, type = "CR2")
}
pkgs_available <-
requireNamespace("lme4", quietly = TRUE) &
requireNamespace("mlmRev", quietly = TRUE)
if (pkgs_available) {
data(egsingle, package = "mlmRev")
subset_ids <- levels(egsingle$schoolid)[1:10]
math_model <- lmer(math ~ year * size + female + black + hispanic
+ (1 | schoolid) + (1 | childid),
data = egsingle, subset = schoolid \%in\% subset_ids)
vcovCR(math_model, type = "CR2")
}
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.mlm.Rd 0000644 0001762 0000144 00000004724 14630154051 015716 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/mlm.R
\name{vcovCR.mlm}
\alias{vcovCR.mlm}
\title{Cluster-robust variance-covariance matrix for an mlm object.}
\usage{
\method{vcovCR}{mlm}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional expression or vector indicating which observations
belong to the same cluster. If not specified, each row of the data will be
treated as a separate cluster.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be an
identity matrix.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from an \code{mlm} object.
}
\examples{
iris_fit <- lm(cbind(Sepal.Length, Sepal.Width) ~ Species +
Petal.Length + Petal.Width, data = iris)
Vcluster <- vcovCR(iris_fit, type = "CR2")
Vcluster
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/constraint_matrices.Rd 0000644 0001762 0000144 00000007354 14630154051 020005 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/Wald_test.R
\name{constraint_matrices}
\alias{constraint_matrices}
\alias{constrain_zero}
\alias{constrain_equal}
\alias{constrain_pairwise}
\title{Create constraint matrices}
\usage{
constrain_zero(constraints, coefs, reg_ex = FALSE)
constrain_equal(constraints, coefs, reg_ex = FALSE)
constrain_pairwise(constraints, coefs, reg_ex = FALSE, with_zero = FALSE)
}
\arguments{
\item{constraints}{Set of constraints to test. Can be logical (using
\code{TRUE} to specify which coefficients to constrain), integer (specify
the index of coefficients to constrain), character (specify the names of
the coefficients to constrain), or a regular expression.}
\item{coefs}{Vector of coefficient estimates, used to determine the column
dimension of the constraint matrix. Can be omitted if the function is
called inside \code{Wald_test()}.}
\item{reg_ex}{Logical indicating whether \code{constraints} should be
interpreted as a regular expression. Defaults to \code{FALSE}.}
\item{with_zero}{Logical indicating whether coefficients should also be
compared to zero. Defaults to \code{FALSE}.}
}
\value{
A matrix or list of matrices encoding the specified set of
constraints.
}
\description{
Helper functions to create common types of constraint matrices,
for use with \code{\link{Wald_test}} to conduct Wald-type tests of linear
contrasts from a fitted regression model.
}
\details{
Constraints can be specified as character vectors, regular
expressions (with \code{reg_ex = TRUE}), integer vectors, or logical
vectors.
\code{constrain_zero()} Creates a matrix that constrains a specified set of
coefficients to all be equal to zero.
\code{constrain_equal()} Creates a matrix that constrains a specified set
of coefficients to all be equal.
\code{constrain_pairwise()} Creates a list of constraint matrices
consisting of all pairwise comparisons between a specified set of
coefficients. If \code{with_zero = TRUE}, then the list will also include a
set of constraint matrices comparing each coefficient to zero.
}
\examples{
if (requireNamespace("carData", quietly = TRUE)) withAutoprint({
data(Duncan, package = "carData")
Duncan$cluster <- sample(LETTERS[1:8], size = nrow(Duncan), replace = TRUE)
Duncan_fit <- lm(prestige ~ 0 + type + income + type:income + type:education, data=Duncan)
# Note that type:income terms are interactions because main effect of income is included
# but type:education terms are separate slopes for each unique level of type
Duncan_coefs <- coef(Duncan_fit)
# The following are all equivalent
constrain_zero(constraints = c("typeprof:income","typewc:income"),
coefs = Duncan_coefs)
constrain_zero(constraints = ":income", coefs = Duncan_coefs,
reg_ex = TRUE)
constrain_zero(constraints = 5:6, coefs = Duncan_coefs)
constrain_zero(constraints = c(FALSE, FALSE, FALSE, FALSE, TRUE, TRUE, FALSE, FALSE, FALSE),
coefs = Duncan_coefs)
# The following are all equivalent
constrain_equal(c("typebc:education","typeprof:education","typewc:education"),
Duncan_coefs)
constrain_equal(":education", Duncan_coefs, reg_ex = TRUE)
constrain_equal(7:9, Duncan_coefs)
constrain_equal(c(FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE),
Duncan_coefs)
# Test pairwise equality of the education slopes
constrain_pairwise(":education", Duncan_coefs,
reg_ex = TRUE)
# Test pairwise equality of the income slopes, plus compare against zero
constrain_pairwise(":income", Duncan_coefs,
reg_ex = TRUE, with_zero = TRUE)
})
}
\seealso{
\code{\link{Wald_test}}
}
clubSandwich/man/vcovCR.rma.uni.Rd 0000644 0001762 0000144 00000005641 14630154051 016501 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/rma-uni.R
\name{vcovCR.rma.uni}
\alias{vcovCR.rma.uni}
\title{Cluster-robust variance-covariance matrix for a rma.uni object.}
\usage{
\method{vcovCR}{rma.uni}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Expression or vector indicating which observations
belong to the same cluster. Required for \code{rma.uni} objects.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be diagonal
with entries equal to the estimated marginal variance of the effect sizes.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from a
\code{\link[metafor]{rma.uni}} object.
}
\examples{
pkgs_available <-
requireNamespace("metafor", quietly = TRUE) &
requireNamespace("metadat", quietly = TRUE)
if (pkgs_available) withAutoprint({
library(metafor)
data(dat.assink2016, package = "metadat")
mfor_fit <- rma.uni(yi ~ year + deltype, vi = vi,
data = dat.assink2016)
mfor_fit
mfor_CR2 <- vcovCR(mfor_fit, type = "CR2", cluster = dat.assink2016$study)
mfor_CR2
coef_test(mfor_fit, vcov = mfor_CR2, test = c("Satterthwaite", "saddlepoint"))
Wald_test(mfor_fit, constraints = constrain_zero(2:4), vcov = mfor_CR2)
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/vcovCR.Rd 0000644 0001762 0000144 00000015573 14630154051 015136 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/clubSandwich.R
\name{vcovCR}
\alias{vcovCR}
\alias{vcovCR.default}
\title{Cluster-robust variance-covariance matrix}
\usage{
vcovCR(obj, cluster, type, target, inverse_var, form, ...)
\method{vcovCR}{default}(
obj,
cluster,
type,
target = NULL,
inverse_var = FALSE,
form = "sandwich",
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Expression or vector indicating which observations belong to
the same cluster. For some classes, the cluster will be detected
automatically if not specified.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If a vector, the target matrix is assumed to be
diagonal. If not specified, \code{vcovCR} will attempt to infer a value.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates. The matrix has several attributes:
\describe{ \item{type}{indicates which small-sample adjustment was used}
\item{cluster}{contains the factor vector that defines independent
clusters} \item{bread}{contains the bread matrix} \item{v_scale}{constant
used in scaling the sandwich estimator} \item{est_mats}{contains a list of
estimating matrices used to calculate the sandwich estimator}
\item{adjustments}{contains a list of adjustment matrices used to calculate
the sandwich estimator} \item{target}{contains the working
variance-covariance model used to calculate the adjustment matrices. This
is needed for calculating small-sample corrections for Wald tests.} }
}
\description{
This is a generic function, with specific methods defined for
\code{\link[stats]{lm}}, \code{\link[plm]{plm}}, \code{\link[stats]{glm}},
\code{\link[nlme]{gls}}, \code{\link[nlme]{lme}},
\code{\link[robumeta]{robu}}, \code{\link[metafor]{rma.uni}}, and
\code{\link[metafor]{rma.mv}} objects.
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates.
}
\details{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates.
Several different small sample corrections are available, which run
parallel with the "HC" corrections for heteroskedasticity-consistent
variance estimators, as implemented in \code{\link[sandwich]{vcovHC}}. The
"CR2" adjustment is recommended (Pustejovsky & Tipton, 2017; Imbens &
Kolesar, 2016). See Pustejovsky and Tipton (2017) and Cameron and Miller
(2015) for further technical details. Available options include: \describe{
\item{"CR0"}{is the original form of the sandwich estimator (Liang & Zeger,
1986), which does not make any small-sample correction.}
\item{"CR1"}{multiplies CR0 by \code{m / (m - 1)}, where \code{m} is the
number of clusters.}
\item{"CR1p"}{multiplies CR0 by \code{m / (m - p)}, where \code{m} is the
number of clusters and \code{p} is the number of covariates.}
\item{"CR1S"}{multiplies CR0 by \code{(m (N-1)) / [(m -
1)(N - p)]}, where \code{m} is the number of clusters, \code{N} is the
total number of observations, and \code{p} is the number of covariates.
Some Stata commands use this correction by default.}
\item{"CR2"}{is the
"bias-reduced linearization" adjustment proposed by Bell and McCaffrey
(2002) and further developed in Pustejovsky and Tipton (2017). The
adjustment is chosen so that the variance-covariance estimator is exactly
unbiased under a user-specified working model.}
\item{"CR3"}{approximates the leave-one-cluster-out jackknife variance estimator (Bell & McCaffrey,
2002).} }
}
\examples{
# simulate design with cluster-dependence
m <- 8
cluster <- factor(rep(LETTERS[1:m], 3 + rpois(m, 5)))
n <- length(cluster)
X <- matrix(rnorm(3 * n), n, 3)
nu <- rnorm(m)[cluster]
e <- rnorm(n)
y <- X \%*\% c(.4, .3, -.3) + nu + e
dat <- data.frame(y, X, cluster, row = 1:n)
# fit linear model
lm_fit <- lm(y ~ X1 + X2 + X3, data = dat)
vcov(lm_fit)
# cluster-robust variance estimator with CR2 small-sample correction
vcovCR(lm_fit, cluster = dat$cluster, type = "CR2")
# compare small-sample adjustments
CR_types <- paste0("CR",c("0","1","1S","2","3"))
sapply(CR_types, function(type)
sqrt(diag(vcovCR(lm_fit, cluster = dat$cluster, type = type))))
}
\references{
Bell, R. M., & McCaffrey, D. F. (2002). Bias reduction in
standard errors for linear regression with multi-stage samples. Survey
Methodology, 28(2), 169-181.
Cameron, A. C., & Miller, D. L. (2015). A Practitioner's Guide to
Cluster-Robust Inference. \emph{Journal of Human Resources, 50}(2), 317-372.
\doi{10.3368/jhr.50.2.317}
Imbens, G. W., & Kolesar, M. (2016). Robust standard errors in small samples:
Some practical advice. \emph{Review of Economics and Statistics, 98}(4),
701-712. \doi{10.1162/rest_a_00552}
Liang, K.-Y., & Zeger, S. L. (1986). Longitudinal data analysis using
generalized linear models. \emph{Biometrika, 73}(1), 13-22.
\doi{10.1093/biomet/73.1.13}
Pustejovsky, J. E. & Tipton, E. (2018). Small sample methods for
cluster-robust variance estimation and hypothesis testing in fixed effects
models. \emph{Journal of Business and Economic Statistics, 36}(4), 672-683.
\doi{10.1080/07350015.2016.1247004}
}
\seealso{
\code{\link{vcovCR.lm}}, \code{\link{vcovCR.plm}},
\code{\link{vcovCR.glm}}, \code{\link{vcovCR.gls}},
\code{\link{vcovCR.lme}}, \code{\link{vcovCR.lmerMod}}, \code{\link{vcovCR.robu}},
\code{\link{vcovCR.rma.uni}}, \code{\link{vcovCR.rma.mv}}
}
clubSandwich/man/vcovCR.glm.Rd 0000644 0001762 0000144 00000005305 14630154051 015704 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/glm.R
\name{vcovCR.glm}
\alias{vcovCR.glm}
\title{Cluster-robust variance-covariance matrix for a glm object.}
\usage{
\method{vcovCR}{glm}(
obj,
cluster,
type,
target = NULL,
inverse_var = NULL,
form = "sandwich",
...
)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Expression or vector indicating which observations belong to
the same cluster. Required for \code{glm} objects.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If a vector, the target matrix is assumed to be
diagonal. If not specified, the target is taken to be the estimated variance function.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from an \code{\link{glm}} object.
}
\examples{
if (requireNamespace("geepack", quietly = TRUE)) {
data(dietox, package = "geepack")
dietox$Cu <- as.factor(dietox$Cu)
weight_fit <- glm(Weight ~ Cu * poly(Time, 3), data=dietox, family = "quasipoisson")
V_CR <- vcovCR(weight_fit, cluster = dietox$Pig, type = "CR2")
coef_test(weight_fit, vcov = V_CR, test = "Satterthwaite")
}
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/impute_covariance_matrix.Rd 0000644 0001762 0000144 00000013413 14635054543 021016 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/rma-mv.R
\name{impute_covariance_matrix}
\alias{impute_covariance_matrix}
\title{Impute a block-diagonal covariance matrix}
\usage{
impute_covariance_matrix(
vi,
cluster,
r,
ti,
ar1,
smooth_vi = FALSE,
subgroup = NULL,
return_list = identical(as.factor(cluster), sort(as.factor(cluster))),
check_PD = TRUE
)
}
\arguments{
\item{vi}{Vector of variances}
\item{cluster}{Vector indicating which effects belong to the same cluster.
Effects with the same value of `cluster` will be treated as correlated.}
\item{r}{Vector or numeric value of assumed constant correlation(s) between
effect size estimates from each study.}
\item{ti}{Vector of time-points describing temporal spacing of effects, for
use with auto-regressive correlation structures.}
\item{ar1}{Vector or numeric value of assumed AR(1) auto-correlation(s)
between effect size estimates from each study. If specified, then \code{ti}
argument must be specified.}
\item{smooth_vi}{Logical indicating whether to smooth the marginal variances
by taking the average \code{vi} within each cluster. Defaults to
\code{FALSE}.}
\item{subgroup}{Vector of category labels describing sub-groups of effects.
If non-null, effects that share the same category label and the same
cluster will be treated as correlated, but effects with different category
labels will be treated as uncorrelated, even if they come from the same
cluster.}
\item{return_list}{Optional logical indicating whether to return a list of
matrices (with one entry per block) or the full variance-covariance matrix.}
\item{check_PD}{Optional logical indicating whether to check whether each
covariance matrix is positive definite. If \code{TRUE} (the default), the
function will display a warning if any covariance matrix is not positive
definite.}
}
\value{
If \code{cluster} is appropriately sorted, then a list of matrices,
with one entry per cluster, will be returned by default. If \code{cluster}
is out of order, then the full variance-covariance matrix will be returned
by default. The output structure can be controlled with the optional
\code{return_list} argument.
}
\description{
`r lifecycle::badge("superseded")`
This function is superseded by the \code{\link[metafor]{vcalc}} provided by
the \code{metafor} package. Compared to \code{impute_covariance_matrix},
\code{\link[metafor]{vcalc}} provides many further features, includes a
\code{data} argument, and uses syntax that is consistent with other
functions in \code{metafor}.
\code{impute_covariance_matrix} calculates a block-diagonal covariance
matrix, given the marginal variances, the block structure, and an assumed
correlation structure. Can be used to create compound-symmetric structures,
AR(1) auto-correlated structures, or combinations thereof.
}
\details{
A block-diagonal variance-covariance matrix (possibly represented as
a list of matrices) with a specified structure. The structure depends on
whether the \code{r} argument, \code{ar1} argument, or both arguments are
specified. Let \eqn{v_{ij}}{v-ij} denote the specified variance for effect
\eqn{i}{i} in cluster \eqn{j}{j} and \eqn{C_{hij}}{C-hij} be the covariance
between effects \eqn{h}{h} and \eqn{i}{i} in cluster
\eqn{j}{j}.
\itemize{
\item{If only \code{r} is specified, each block of the variance-covariance
matrix will have a constant (compound symmetric) correlation, so that
\deqn{C_{hij} = r_j \sqrt{v_{hj} v_{ij},}}{C-hij = r-j * sqrt(v-hj v-ij),}
where \eqn{r_j}{r-j} is the specified correlation
for cluster \eqn{j}{j}. If only a single value is given in \code{r}, then
it will be used for every cluster.}
\item{If only \code{ar1} is specified, each block of the variance-covariance matrix will have an
AR(1) auto-correlation structure, so that
\deqn{C_{hij} = \phi_j^{|t_{hj}- t_{ij}|} \sqrt{v_{hj} v_{ij},}}{C-hij = (ar1-j)^|t-hj - t-ij| * sqrt(v-hj v-ij),}
where \eqn{\phi_j}{ar1-j} is the specified auto-correlation
for cluster \eqn{j}{j} and \eqn{t_{hj}}{t-hj} and \eqn{t_{ij}}{t-ij}
are specified time-points corresponding to effects \eqn{h}{h} and
\eqn{i}{i} in cluster \eqn{j}{j}. If only a single value is given in
\code{ar1}, then it will be used for every cluster.}
\item{If both \code{r} and \code{ar1} are specified, each block of the variance-covariance matrix will have combination of compound symmetric and an AR(1)
auto-correlation structures, so that
\deqn{C_{hij} = \left[r_j + (1 - r_j)\phi_j^{|t_{hj} - t_{ij}|}\right] \sqrt{v_{hj} v_{ij},}}{C-hij = [r-j + (1 - r-j)(ar1-j)^|t-hj - t-ij|] * sqrt(v-hj v-ij),}
where \eqn{r_j}{r-j} is the specified constant correlation for cluster
\eqn{j}{j}, \eqn{\phi_j}{ar1-j} is the specified auto-correlation for
cluster \eqn{j}{j} and \eqn{t_{hj}}{t-hj} and \eqn{t_{ij}}{t-ij} are
specified time-points corresponding to effects \eqn{h}{h} and
\eqn{i}{i} in cluster \eqn{j}{j}. If only single values are given in
\code{r} or \code{ar1}, they will be used for every cluster.}
}
If \code{smooth_vi = TRUE}, then all of the variances within cluster
\eqn{j}{j} will be set equal to the average variance of cluster
\eqn{j}{j}, i.e., \deqn{v'_{ij} = \frac{1}{n_j} \sum_{i=1}^{n_j}
v_{ij}}{v-ij' = (v-1j + ... + v-nj,j) / n-j} for
\eqn{i=1,...,n_j}{i=1,...,n-j} and \eqn{j=1,...,k}{j=1,...,k}.
}
\examples{
if (requireNamespace("metafor", quietly = TRUE)) {
library(metafor)
# Constant correlation
data(SATcoaching)
V_list <- impute_covariance_matrix(vi = SATcoaching$V, cluster = SATcoaching$study, r = 0.66)
MVFE <- rma.mv(d ~ 0 + test, V = V_list, data = SATcoaching)
conf_int(MVFE, vcov = "CR2", cluster = SATcoaching$study)
}
}
clubSandwich/man/AchievementAwardsRCT.Rd 0000644 0001762 0000144 00000004040 14630154051 017662 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/data-documentation.R
\docType{data}
\name{AchievementAwardsRCT}
\alias{AchievementAwardsRCT}
\title{Achievement Awards Demonstration program}
\format{
A data frame with 16526 rows and 21 variables: \describe{
\item{school_id}{Fictitious school identification number}
\item{school_type}{Factor identifying the school type (Arab religious, Jewish religious, Jewish secular)}
\item{pair}{Number of treatment pair. Note that 7 is a triple.}
\item{treated}{Indicator for whether school was in treatment group}
\item{year}{Cohort year}
\item{student_id}{Fictitious student identification number}
\item{sex}{Factor identifying student sex}
\item{siblings}{Number of siblings}
\item{immigrant}{Indicator for immigrant status}
\item{father_ed}{Father's level of education}
\item{mother_ed}{Mother's level of education}
\item{Bagrut_status}{Indicator for Bagrut attainment}
\item{attempted}{Number of Bagrut units attempted}
\item{awarded}{Number of Bagrut units awarded}
\item{achv_math}{Indicator for satisfaction of math requirement}
\item{achv_english}{Indicator for satisfaction of English requirement}
\item{achv_hebrew}{Indicator for satisfaction of Hebrew requirement}
\item{lagscore}{Lagged Bagrut score}
\item{qrtl}{Quartile within distribution of lagscore, calculated by cohort and sex}
\item{half}{Lower or upper half within distribution of lagscore, calculated by cohort and sex}
}
}
\source{
\href{https://economics.mit.edu/people/faculty/josh-angrist/angrist-data-archive}{Angrist Data Archive}
}
\usage{
AchievementAwardsRCT
}
\description{
Data from a randomized trial of the Achievement Awards
Demonstration program, reported in Angrist & Lavy (2009).
}
\references{
Angrist, J. D., & Lavy, V. (2009). The effects of high stakes
high school achievement awards : Evidence from a randomized trial.
\emph{American Economic Review, 99}(4), 1384-1414.
\doi{10.1257/aer.99.4.1384}
}
\keyword{datasets}
clubSandwich/man/vcovCR.rma.mv.Rd 0000644 0001762 0000144 00000006100 14630154051 016317 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/rma-mv.R
\name{vcovCR.rma.mv}
\alias{vcovCR.rma.mv}
\title{Cluster-robust variance-covariance matrix for a rma.mv object.}
\usage{
\method{vcovCR}{rma.mv}(obj, cluster, type, target, inverse_var, form = "sandwich", ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate the variance-covariance matrix}
\item{cluster}{Optional expression or vector indicating which observations
belong to the same cluster. If not specified, will be set to the factor in
the random-effects structure with the fewest distinct levels. Caveat
emptor: the function does not check that the random effects are nested.}
\item{type}{Character string specifying which small-sample adjustment should
be used, with available options \code{"CR0"}, \code{"CR1"}, \code{"CR1p"},
\code{"CR1S"}, \code{"CR2"}, or \code{"CR3"}. See "Details" section of
\code{\link{vcovCR}} for further information.}
\item{target}{Optional matrix or vector describing the working
variance-covariance model used to calculate the \code{CR2} and \code{CR4}
adjustment matrices. If not specified, the target is taken to be the
estimated variance-covariance structure of the \code{rma.mv} object.}
\item{inverse_var}{Optional logical indicating whether the weights used in
fitting the model are inverse-variance. If not specified, \code{vcovCR}
will attempt to infer a value.}
\item{form}{Controls the form of the returned matrix. The default
\code{"sandwich"} will return the sandwich variance-covariance matrix.
Alternately, setting \code{form = "meat"} will return only the meat of the
sandwich and setting \code{form = B}, where \code{B} is a matrix of
appropriate dimension, will return the sandwich variance-covariance matrix
calculated using \code{B} as the bread. \code{form = "estfun"} will return the
(appropriately scaled) estimating function, the transposed crossproduct of
which is equal to the sandwich variance-covariance matrix.}
\item{...}{Additional arguments available for some classes of objects.}
}
\value{
An object of class \code{c("vcovCR","clubSandwich")}, which consists
of a matrix of the estimated variance of and covariances between the
regression coefficient estimates.
}
\description{
\code{vcovCR} returns a sandwich estimate of the variance-covariance matrix
of a set of regression coefficient estimates from a
\code{\link[metafor]{rma.mv}} object.
}
\examples{
pkgs_available <-
requireNamespace("metafor", quietly = TRUE) &
requireNamespace("metadat", quietly = TRUE)
if (pkgs_available) withAutoprint({
library(metafor)
data(dat.assink2016, package = "metadat")
mfor_fit <- rma.mv(yi ~ year + deltype,
V = vi, random = ~ 1 | study / esid,
data = dat.assink2016)
mfor_fit
mfor_CR2 <- vcovCR(mfor_fit, type = "CR2")
mfor_CR2
coef_test(mfor_fit, vcov = mfor_CR2, test = c("Satterthwaite", "saddlepoint"))
Wald_test(mfor_fit, constraints = constrain_zero(3:4), vcov = mfor_CR2)
})
}
\seealso{
\code{\link{vcovCR}}
}
clubSandwich/man/findCluster.rma.mv.Rd 0000644 0001762 0000144 00000001535 14630154051 017406 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/rma-mv.R
\name{findCluster.rma.mv}
\alias{findCluster.rma.mv}
\title{Detect cluster structure of an rma.mv object}
\usage{
findCluster.rma.mv(obj)
}
\arguments{
\item{obj}{A fitted \code{rma.mv} object.}
}
\value{
A a vector of ID variables for the highest level of clustering in \code{obj}.
}
\description{
\code{findCluster.rma.mv} returns a vector of ID variables for the highest level of clustering in a fitted \code{rma.mv} model.
}
\examples{
if (requireNamespace("metafor", quietly = TRUE)) {
library(metafor)
data(dat.assink2016, package = "metadat")
mfor_fit <- rma.mv(yi ~ year + deltype,
V = vi, random = ~ 1 | study / esid,
data = dat.assink2016)
findCluster.rma.mv(mfor_fit)
}
}
clubSandwich/man/Wald_test.Rd 0000644 0001762 0000144 00000006413 14630154051 015653 0 ustar ligges users % Generated by roxygen2: do not edit by hand
% Please edit documentation in R/Wald_test.R
\name{Wald_test}
\alias{Wald_test}
\title{Test parameter constraints in a fitted linear regression model}
\usage{
Wald_test(obj, constraints, vcov, test = "HTZ", tidy = FALSE, ...)
}
\arguments{
\item{obj}{Fitted model for which to calculate Wald tests.}
\item{constraints}{List of one or more constraints to test. See details and
examples.}
\item{vcov}{Variance covariance matrix estimated using \code{vcovCR} or a
character string specifying which small-sample adjustment should be used to
calculate the variance-covariance.}
\item{test}{Character vector specifying which small-sample correction(s) to
calculate. The following corrections are available: \code{"chi-sq"},
\code{"Naive-F"}, \code{"Naive-Fp"}, \code{"HTA"}, \code{"HTB"}, \code{"HTZ"}, \code{"EDF"},
\code{"EDT"}. Default is \code{"HTZ"}.}
\item{tidy}{Logical value controlling whether to tidy the test results. If
\code{constraints} is a list with multiple constraints, the result will
be coerced into a data frame when \code{tidy = TRUE}.}
\item{...}{Further arguments passed to \code{\link{vcovCR}}, which are only
needed if \code{vcov} is a character string.}
}
\value{
A list of test results.
}
\description{
\code{Wald_test} reports Wald-type tests of linear contrasts from a fitted
linear regression model, using a sandwich estimator for the
variance-covariance matrix and a small sample correction for the p-value.
Several different small-sample corrections are available.
}
\details{
Constraints can be specified directly as q X p matrices or
indirectly through \code{\link{constrain_equal}},
\code{\link{constrain_zero}}, or \code{\link{constrain_pairwise}}
}
\examples{
if (requireNamespace("carData", quietly = TRUE)) withAutoprint({
data(Duncan, package = "carData")
Duncan$cluster <- sample(LETTERS[1:8], size = nrow(Duncan), replace = TRUE)
Duncan_fit <- lm(prestige ~ 0 + type + income + type:income + type:education, data=Duncan)
# Note that type:income terms are interactions because main effect of income is included
# but type:education terms are separate slopes for each unique level of type
# Test equality of intercepts
Wald_test(Duncan_fit,
constraints = constrain_equal(1:3),
vcov = "CR2", cluster = Duncan$cluster)
# Test equality of type-by-education slopes
Wald_test(Duncan_fit,
constraints = constrain_equal(":education", reg_ex = TRUE),
vcov = "CR2", cluster = Duncan$cluster)
# Pairwise comparisons of type-by-education slopes
Wald_test(Duncan_fit,
constraints = constrain_pairwise(":education", reg_ex = TRUE),
vcov = "CR2", cluster = Duncan$cluster)
# Test type-by-income interactions
Wald_test(Duncan_fit,
constraints = constrain_zero(":income", reg_ex = TRUE),
vcov = "CR2", cluster = Duncan$cluster)
# Pairwise comparisons of type-by-income interactions
Wald_test(Duncan_fit,
constraints = constrain_pairwise(":income", reg_ex = TRUE, with_zero = TRUE),
vcov = "CR2", cluster = Duncan$cluster)
})
}
\seealso{
\code{\link{vcovCR}}, \code{\link{constrain_equal}},
\code{\link{constrain_zero}}, \code{\link{constrain_pairwise}}
}
clubSandwich/DESCRIPTION 0000644 0001762 0000144 00000004214 14635072452 014377 0 ustar ligges users Package: clubSandwich
Title: Cluster-Robust (Sandwich) Variance Estimators with Small-Sample
Corrections
Version: 0.5.11
Authors@R: person("James", "Pustejovsky", email = "jepusto@gmail.com", role = c("aut", "cre"), comment = c(ORCID = "0000-0003-0591-9465"))
Description: Provides several cluster-robust variance estimators (i.e.,
sandwich estimators) for ordinary and weighted least squares linear regression
models, including the bias-reduced linearization estimator introduced by Bell
and McCaffrey (2002)
and
developed further by Pustejovsky and Tipton (2017)
. The package includes functions for estimating
the variance- covariance matrix and for testing single- and multiple-
contrast hypotheses based on Wald test statistics. Tests of single regression
coefficients use Satterthwaite or saddle-point corrections. Tests of multiple-
contrast hypotheses use an approximation to Hotelling's T-squared distribution.
Methods are provided for a variety of fitted models, including lm() and mlm
objects, glm(), geeglm() (from package 'geepack'), ivreg() (from package 'AER'), ivreg() (from package 'ivreg' when
estimated by ordinary least squares), plm() (from package 'plm'), gls() and
lme() (from 'nlme'), lmer() (from `lme4`), robu() (from 'robumeta'), and rma.uni()
and rma.mv() (from 'metafor').
URL: http://jepusto.github.io/clubSandwich/
BugReports: https://github.com/jepusto/clubSandwich/issues
Depends: R (>= 3.0.0)
License: GPL-3
VignetteBuilder: knitr
LazyData: true
Imports: stats, sandwich, lifecycle
Suggests: Formula, knitr, carData, geepack, metafor, metadat, robumeta,
nlme, mlmRev, AER, plm (>= 1.6-4), Matrix, lme4, zoo, testthat,
rmarkdown, covr, ivreg
RoxygenNote: 7.3.1
Encoding: UTF-8
Language: en-US
NeedsCompilation: no
Packaged: 2024-06-20 17:42:32 UTC; jamespustejovsky
Author: James Pustejovsky [aut, cre] ()
Maintainer: James Pustejovsky
Repository: CRAN
Date/Publication: 2024-06-20 18:30:02 UTC