The metaSEM package: Examples
Mike W.-L. Cheung
22 January, 2017
- Introduction
- Univariate meta-analysis
- Multivariate meta-analysis
- Three-level meta-analysis
- Comparisons between two- and three-level models with Cooper et al.’s (2003) dataset
- More complex 3-level meta-analyses with Bornmann et al.’s (2007) dataset
- Inspecting the data
- Model 0: Intercept
- Model 1:
Typeas a covariate - Model 2:
YearandYear^2as covariates - Model 3:
Disciplineas a covariate - Model 4:
Countryas a covariate - Model 5:
TypeandDisciplineas covariates - Model 6:
TypeandCountryas covariates - Model 7:
DisciplineandCountryas covariates - Model 8:
Type,DisciplineandCountryas covariates - Handling missing covariates with FIML
- Two-stage structural equation modeling
- Installation and help
Introduction
The
metaSEMpackage provides functions to conducting univariate and multivariate meta-analysis using a structural equation modeling approach via the OpenMx package. It also implemented the two-stage structural equation modeling (TSSEM) approach to conducting fixed- and random-effects meta-analytic structural equation modeling (MASEM) on correlation/covariance matrices.- The
metaSEMpackage is based on the following papers:- Cheung, M.W.-L. (2008). A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. Psychological Methods, 13, 182-202.
- Cheung, M.W.-L. (2009). Constructing approximate confidence intervals for parameters with structural equation models. Structural Equation Modeling, 16, 267–294.
- Cheung, M.W.-L. (2010). Fixed-effects meta-analyses as multiple-group structural equation models. Structural Equation Modeling, 17, 481-509.
- Cheung, M.W.-L. (2013). Implementing restricted maximum likelihood estimation in structural equation models. Structural Equation Modeling, 20, 157-167.
- Cheung, M.W.-L. (2013). Multivariate meta-analysis as structural equation models. Structural Equation Modeling, 20, 429-454.
- Cheung, M.W.-L. (2014). Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46, 29-40.
- Cheung, M.W.-L. (2014). Modeling dependent effect sizes with three-level meta-analyses: A structural equation modeling approach. Psychological Methods, 19, 211-229.
- Cheung, M.W.-L. (2015). MetaSEM: an R package for meta-analysis using structural equation modeling. Frontiers in Psychology, 5 (1521).
- Cheung, M.W.-L. (2015). Meta-Analysis: A Structural Equation Modeling Approach. Wiley.
- Cheung, M.W.-L., & Chan, W. (2004). Testing dependent correlation coefficients via structural equation modeling. Organizational Research Methods, 7, 206–223.
- Cheung, M.W.-L., & Chan, W. (2005). Meta-analytic structural equation modeling: A two-stage approach. Psychological Methods, 10, 40-64.
- Cheung, M.W.-L., & Chan, W. (2009). A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Structural Equation Modeling, 6, 28-53.
- Cheung, M. W.-L., & Cheung, S. F. (2016). Random-effects models for meta-analytic structural equation modeling: Review, issues, and illustrations. Research Synthesis Methods, 7, 140-155.
Univariate meta-analysis
Random-effects model
## Load the library
library(metaSEM)
## Show the first few studies of the data set
head(Becker83) study di vi percentage items
1 1 -0.33 0.03 25 2
2 2 0.07 0.03 25 2
3 3 -0.30 0.02 50 2
4 4 0.35 0.02 100 38
5 5 0.69 0.07 100 30
6 6 0.81 0.22 100 45## Random-effects meta-analysis with ML
summary( meta(y=di, v=vi, data=Becker83) )
Call:
meta(y = di, v = vi, data = Becker83)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.174734 0.113378 -0.047482 0.396950 1.5412 0.1233
Tau2_1_1 0.077376 0.054108 -0.028674 0.183426 1.4300 0.1527
Q statistic on the homogeneity of effect sizes: 30.64949
Degrees of freedom of the Q statistic: 9
P value of the Q statistic: 0.0003399239
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.6718
Number of studies (or clusters): 10
Number of observed statistics: 10
Number of estimated parameters: 2
Degrees of freedom: 8
-2 log likelihood: 7.928307
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Fixed-effects model
## Fixed-effects meta-analysis by fixiing the heterogeneity variance at 0
summary( meta(y=di, v=vi, data=Becker83, RE.constraints=0) )
Call:
meta(y = di, v = vi, data = Becker83, RE.constraints = 0)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.100640 0.060510 -0.017957 0.219237 1.6632 0.09627 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 30.64949
Degrees of freedom of the Q statistic: 9
P value of the Q statistic: 0.0003399239
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0
Number of studies (or clusters): 10
Number of observed statistics: 10
Number of estimated parameters: 1
Degrees of freedom: 9
-2 log likelihood: 17.86043
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Mixed-effects model
## Mixed-effects meta-analysis with "log(items)" as the predictor
summary( meta(y=di, v=vi, x=log(items), data=Becker83) )
Call:
meta(y = di, v = vi, x = log(items), data = Becker83)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept1 -3.2015e-01 1.0981e-01 -5.3539e-01 -1.0492e-01 -2.9154
Slope1_1 2.1088e-01 4.5084e-02 1.2251e-01 2.9924e-01 4.6774
Tau2_1_1 1.0000e-10 2.0095e-02 -3.9386e-02 3.9386e-02 0.0000
Pr(>|z|)
Intercept1 0.003552 **
Slope1_1 2.905e-06 ***
Tau2_1_1 1.000000
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 30.64949
Degrees of freedom of the Q statistic: 9
P value of the Q statistic: 0.0003399239
Explained variances (R2):
y1
Tau2 (no predictor) 0.0774
Tau2 (with predictors) 0.0000
R2 1.0000
Number of studies (or clusters): 10
Number of observed statistics: 10
Number of estimated parameters: 3
Degrees of freedom: 7
-2 log likelihood: -4.208024
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Multivariate meta-analysis
Random-effects model
## Show the data set
Berkey98 trial pub_year no_of_patients PD AL var_PD cov_PD_AL var_AL
1 1 1983 14 0.47 -0.32 0.0075 0.0030 0.0077
2 2 1982 15 0.20 -0.60 0.0057 0.0009 0.0008
3 3 1979 78 0.40 -0.12 0.0021 0.0007 0.0014
4 4 1987 89 0.26 -0.31 0.0029 0.0009 0.0015
5 5 1988 16 0.56 -0.39 0.0148 0.0072 0.0304## Multivariate meta-analysis with a random-effects model
mult1 <- meta(y=cbind(PD, AL), v=cbind(var_PD, cov_PD_AL, var_AL), data=Berkey98)
summary(mult1)
Call:
meta(y = cbind(PD, AL), v = cbind(var_PD, cov_PD_AL, var_AL),
data = Berkey98)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.3448392 0.0536312 0.2397239 0.4499544 6.4298 1.278e-10
Intercept2 -0.3379381 0.0812479 -0.4971812 -0.1786951 -4.1593 3.192e-05
Tau2_1_1 0.0070020 0.0090497 -0.0107351 0.0247391 0.7737 0.4391
Tau2_2_1 0.0094607 0.0099698 -0.0100797 0.0290010 0.9489 0.3427
Tau2_2_2 0.0261445 0.0177409 -0.0086270 0.0609161 1.4737 0.1406
Intercept1 ***
Intercept2 ***
Tau2_1_1
Tau2_2_1
Tau2_2_2
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 128.2267
Degrees of freedom of the Q statistic: 8
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.6021
Intercept2: I2 (Q statistic) 0.9250
Number of studies (or clusters): 5
Number of observed statistics: 10
Number of estimated parameters: 5
Degrees of freedom: 5
-2 log likelihood: -11.68131
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)## Plot the effect sizes
plot(mult1)
## Plot the effect sizes with the forest plots
## Load the library for forest plots
library("metafor")
## Create extra panels for the forest plots
plot(mult1, diag.panel=TRUE, main="Multivariate meta-analysis",
axis.label=c("PD", "AL"))
## Forest plot for PD
forest( rma(yi=PD, vi=var_PD, data=Berkey98) )
title("Forest plot of PD")
## Forest plot for AL
forest( rma(yi=AL, vi=var_AL, data=Berkey98) )
title("Forest plot of AL")
Fixed-effects model
## Fixed-effects meta-analysis by fixiing the heterogeneity variance component at
## a 2x2 matrix of 0.
summary( meta(y=cbind(PD, AL), v=cbind(var_PD, cov_PD_AL, var_AL), data=Berkey98,
RE.constraints=matrix(0, nrow=2, ncol=2)) )
Call:
meta(y = cbind(PD, AL), v = cbind(var_PD, cov_PD_AL, var_AL),
data = Berkey98, RE.constraints = matrix(0, nrow = 2, ncol = 2))
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.307219 0.028575 0.251212 0.363225 10.751 < 2.2e-16 ***
Intercept2 -0.394377 0.018649 -0.430929 -0.357825 -21.147 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 128.2267
Degrees of freedom of the Q statistic: 8
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0
Intercept2: I2 (Q statistic) 0
Number of studies (or clusters): 5
Number of observed statistics: 10
Number of estimated parameters: 2
Degrees of freedom: 8
-2 log likelihood: 90.88326
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Mixed-effects model
## Multivariate meta-analysis with "publication year-1979" as a predictor
summary( meta(y=cbind(PD, AL), v=cbind(var_PD, cov_PD_AL, var_AL), data=Berkey98,
x=scale(pub_year, center=1979)) )
Call:
meta(y = cbind(PD, AL), v = cbind(var_PD, cov_PD_AL, var_AL),
x = scale(pub_year, center = 1979), data = Berkey98)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.3440001 0.0857659 0.1759021 0.5120981 4.0109 6.048e-05
Intercept2 -0.2918175 0.1312796 -0.5491207 -0.0345142 -2.2229 0.02622
Slope1_1 0.0063540 0.1078235 -0.2049761 0.2176841 0.0589 0.95301
Slope2_1 -0.0705888 0.1620965 -0.3882920 0.2471145 -0.4355 0.66322
Tau2_1_1 0.0080405 0.0101206 -0.0117955 0.0278766 0.7945 0.42692
Tau2_2_1 0.0093413 0.0105515 -0.0113392 0.0300218 0.8853 0.37599
Tau2_2_2 0.0250135 0.0170788 -0.0084603 0.0584873 1.4646 0.14303
Intercept1 ***
Intercept2 *
Slope1_1
Slope2_1
Tau2_1_1
Tau2_2_1
Tau2_2_2
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 128.2267
Degrees of freedom of the Q statistic: 8
P value of the Q statistic: 0
Explained variances (R2):
y1 y2
Tau2 (no predictor) 0.0070020 0.0261
Tau2 (with predictors) 0.0080405 0.0250
R2 0.0000000 0.0433
Number of studies (or clusters): 5
Number of observed statistics: 10
Number of estimated parameters: 7
Degrees of freedom: 3
-2 log likelihood: -12.00859
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Three-level meta-analysis
This section illustrates how to conduct three-level meta-analyses using the metaSEM package implemented in the R environment. The
metaSEMpackage was written to simplify the procedures to conduct meta-analysis. Most readers may only need to use themetaSEMpackage to conduct the analysis. The next section shows how to conduct two- and three-level meta-analyses with themeta()andmeta3()functions. The third section demonstrates more complicated three-level meta-analyses using a dataset with more predictors. The final section shows how to implement three-level meta-analyses as structural equation models using theOpenMxpackage. It provides detailed steps on how three-level meta-analyses can be formulated as structural equation models.This file also demonstrates the advantages of using the SEM approach to conduct three-level meta-analyses. These include flexibility on imposing constraints for model comparisons and construction of likelihood-based confidence interval (LBCI). I also demonstrate how to conduct three-level meta-analysis with restricted (or residual) maximum likelihood (REML) using the
reml3()function and handling missing covariates with full information maximum likelihood (FIML) using themeta3X()function. Readers may refer to Cheung (2015) for the design and implementation of themetaSEMpackage and Cheung (2014) for the theory and issues on how to formulate three-level meta-analyses as structural equation models.Two datasets from published meta-analyses were used in the illustrations. The first dataset was based on Cooper et al. (2003) and Konstantopoulos (2011). Konstantopoulos (2011) selected part of the dataset to illustrate how to conduct three-level meta-analysis. The second dataset was reported by Bornmann et al. (2007) and Marsh et al. (2009). They conducted a three-level meta-analysis on gender effects in peer reviews of grant proposals.
Comparisons between two- and three-level models with Cooper et al.’s (2003) dataset
As an illustration, I first conduct the tradition (two-level) meta-analysis using the meta() function. Then I conduct a three-level meta-analysis using the meta3() function. We may compare the similarities and differences between these two sets of results.
Inspecting the data
Before running the analyses, we need to load the metaSEM library. The datasets are stored in the library. It is always a good idea to inspect the data before the analyses. We may display the first few cases of the dataset by using the head() command.
#### Cooper et al. (2003)
library("metaSEM")
head(Cooper03) District Study y v Year
1 11 1 -0.18 0.118 1976
2 11 2 -0.22 0.118 1976
3 11 3 0.23 0.144 1976
4 11 4 -0.30 0.144 1976
5 12 5 0.13 0.014 1989
6 12 6 -0.26 0.014 1989Two-level meta-analysis
Similar to other R packages, we may use summary() to extract the results after running the analyses. I first conduct a random-effects meta-analysis and then a fixed- and mixed-effects meta-analyses.
- Random-effects model The Q statistic on testing the homogeneity of effect sizes was \(578.86, df = 55, p < .001\). The estimated heterogeneity \(\tau^2\) (labeled
Tau2_1_1in the output) and \(I^2\) were 0.0866 and 0.9459, respectively. This indicates that the between-study effect explains about 95% of the total variation. The average population effect (labeledIntercept1in the output; and its 95% Wald CI) was 0.1280 (0.0428, 0.2132).
#### Two-level meta-analysis
## Random-effects model
summary( meta(y=y, v=v, data=Cooper03) )
Call:
meta(y = y, v = v, data = Cooper03)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.128003 0.043472 0.042799 0.213207 2.9445 0.003235 **
Tau2_1_1 0.086537 0.019485 0.048346 0.124728 4.4411 8.949e-06 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.9459
Number of studies (or clusters): 56
Number of observed statistics: 56
Number of estimated parameters: 2
Degrees of freedom: 54
-2 log likelihood: 33.2919
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Fixed-effects model A fixed-effects meta-analysis can be conducted by fixing the heterogeneity of the random effects at 0 with the
RE.constraintsargument (random-effects constraints). The estimated common effect (and its 95% Wald CI) was 0.0464 (0.0284, 0.0644).
## Fixed-effects model
summary( meta(y=y, v=v, data=Cooper03, RE.constraints=0) )
Call:
meta(y = y, v = v, data = Cooper03, RE.constraints = 0)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.0464072 0.0091897 0.0283957 0.0644186 5.0499 4.42e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0
Number of studies (or clusters): 56
Number of observed statistics: 56
Number of estimated parameters: 1
Degrees of freedom: 55
-2 log likelihood: 434.2075
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Mixed-effects model
Yearwas used as a covariate. It is easier to interpret the intercept by centering theYearwithscale(Year, scale=FALSE). Thescale=FALSEargument means that it is centered, but not standardized. The estimated regression coefficient (labeledSlope1_1in the output; and its 95% Wald CI) was 0.0051 (-0.0033, 0.0136) which is not significant at \(\alpha=.05\). The \(R^2\) is 0.0164.
## Mixed-effects model
summary( meta(y=y, v=v, x=scale(Year, scale=FALSE), data=Cooper03) )
Call:
meta(y = y, v = v, x = scale(Year, scale = FALSE), data = Cooper03)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept1 0.1259125 0.0432028 0.0412367 0.2105884 2.9145 0.003563
Slope1_1 0.0051307 0.0043248 -0.0033457 0.0136071 1.1864 0.235483
Tau2_1_1 0.0851153 0.0190462 0.0477856 0.1224451 4.4689 7.862e-06
Intercept1 **
Slope1_1
Tau2_1_1 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Explained variances (R2):
y1
Tau2 (no predictor) 0.0865
Tau2 (with predictors) 0.0851
R2 0.0164
Number of studies (or clusters): 56
Number of observed statistics: 56
Number of estimated parameters: 3
Degrees of freedom: 53
-2 log likelihood: 31.88635
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Three-level meta-analysis
- Random-effects model The Q statistic on testing the homogeneity of effect sizes was the same as that under the two-level meta-analysis. The estimated heterogeneity at level 2 \(\tau^2_{(2)}\) (labeled
Tau2_2in the output) and at level 3 \(\tau^2_{(3)}\) (labeledTau2_3in the output) were 0.0329 and 0.0577, respectively. The level 2 \(I^2_{(2)}\) (labeledI2_2in the output) and the level 3 \(I^2_{(3)}\) (labeledI2_3in the output) were 0.3440 and 0.6043, respectively. Schools (level 2) and districts (level 3) explain about 34% and 60% of the total variation, respectively. The average population effect (and its 95% Wald CI) was 0.1845 (0.0266, 0.3423).
#### Three-level meta-analysis
## Random-effects model
summary( meta3(y=y, v=v, cluster=District, data=Cooper03) )
Call:
meta3(y = y, v = v, cluster = District, data = Cooper03)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept 0.1844554 0.0805411 0.0265977 0.3423130 2.2902 0.022010 *
Tau2_2 0.0328648 0.0111397 0.0110314 0.0546982 2.9502 0.003175 **
Tau2_3 0.0577384 0.0307423 -0.0025154 0.1179921 1.8781 0.060362 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
I2_2 (Typical v: Q statistic) 0.3440
I2_3 (Typical v: Q statistic) 0.6043
Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 3
Degrees of freedom: 53
-2 log likelihood: 16.78987
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Mixed-effects model
Yearwas used as a covariate. The estimated regression coefficient (labeledSlope_1in the output; and its 95% Wald CI) was 0.0051 (-0.0116, 0.0218) which is not significant at \(\alpha=.05\). The estimated level 2 \(R^2_{(2)}\) and level 3 \(R^2_{(3)}\) were 0.0000 and 0.0221, respectively.
## Mixed-effects model
summary( meta3(y=y, v=v, cluster=District, x=scale(Year, scale=FALSE),
data=Cooper03) )
Call:
meta3(y = y, v = v, cluster = District, x = scale(Year, scale = FALSE),
data = Cooper03)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept 0.1780268 0.0805219 0.0202067 0.3358469 2.2109 0.027042 *
Slope_1 0.0050737 0.0085266 -0.0116382 0.0217856 0.5950 0.551814
Tau2_2 0.0329390 0.0111620 0.0110618 0.0548162 2.9510 0.003168 **
Tau2_3 0.0564628 0.0300330 -0.0024007 0.1153264 1.8800 0.060104 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.032865 0.0577
Tau2 (with predictors) 0.032939 0.0565
R2 0.000000 0.0221
Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 4
Degrees of freedom: 52
-2 log likelihood: 16.43629
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Model comparisons Many research hypotheses involve model comparisons among nested models.
anova(), a generic function to comparing nested models, may be used to conduct a likelihood ratio test which is also known as a chi-square difference test.- Testing \(H_0: \tau^2_{(3)} = 0\)
- Based on the data structure, it is clear that a 3-level meta-analysis is preferred to a traditional 2-level meta-analysis. It is still of interest to test whether the 3-level model is statistically better than the 2-level model by testing \(H_0: \tau^2_{(3)}=0\). Since the models with \(\tau^2_{(3)}\) being freely estimated and with \(\tau^2_{(3)}=0\) are nested, we may compare them by the use of a likelihood ratio test.
- It should be noted, however, that \(H_0: \tau^2_{(3)}=0\) is tested on the boundary. The likelihood ratio test is not distributed as a chi-square variate with 1 df. A simple strategy to correct this bias is to reject the null hypothesis when the observed p value is larger than .10 for \(\alpha=.05\).
- The likelihood-ratio test was 16.5020 (df =1), p < .001. This clearly demonstrates that the three-level model is statistically better than the two-level model.
## Model comparisons
model2 <- meta(y=y, v=v, data=Cooper03, model.name="2 level model", silent=TRUE)
#### An equivalent model by fixing tau2 at level 3=0 in meta3()
## model2 <- meta3(y=y, v=v, cluster=District, data=Cooper03,
## model.name="2 level model", RE3.constraints=0)
model3 <- meta3(y=y, v=v, cluster=District, data=Cooper03,
model.name="3 level model", silent=TRUE)
anova(model3, model2) base comparison ep minus2LL df AIC diffLL diffdf
1 3 level model <NA> 3 16.78987 53 -89.21013 NA NA
2 3 level model 2 level model 2 33.29190 54 -74.70810 16.50203 1
p
1 NA
2 4.859793e-05- Testing \(H_0: \tau^2_{(2)} = \tau^2_{(3)}\)
- From the results of the 3-level random-effects meta-analysis, it appears the level 3 heterogeneity is much larger than that at level 2.
- We may test the null hypothesis \(H_0: \tau^2_{(2)} = \tau^2_{(3)}\) by the use of a likelihood-ratio test.
- We may impose an equality constraint on \(\tau^2_{(2)} = \tau^2_{(3)}\) by using the same label in
meta3(). For example,Eq_tau2is used as the label inRE2.constraintsandRE3.constraintsmeaning that both the level 2 and level 3 random effects heterogeneity variances are constrained equally. The value of0.1was used as the starting value in the constraints. - The likelihood-ratio test was 0.6871 (df = 1), p = 0.4072. This indicates that there is not enough evidence to reject \(H_0: \tau^2_2=\tau^2_3\).
## Testing \tau^2_2 = \tau^2_3
modelEqTau2 <- meta3(y=y, v=v, cluster=District, data=Cooper03,
model.name="Equal tau2 at both levels",
RE2.constraints="0.1*Eq_tau2", RE3.constraints="0.1*Eq_tau2")
anova(model3, modelEqTau2) base comparison ep minus2LL df AIC
1 3 level model <NA> 3 16.78987 53 -89.21013
2 3 level model Equal tau2 at both levels 2 17.47697 54 -90.52303
diffLL diffdf p
1 NA NA NA
2 0.6870959 1 0.4071539- Likelihood-based confidence interval
- A Wald CI is constructed by \(\hat{\theta} \pm 1.96 SE\) where \(\hat{\theta}\) and \(SE\) are the parameter estimate and its estimated standard error.
- A LBCI can be constructed by the use of the likelihood ratio statistic (e.g., Cheung, 2009; Neal & Miller, 1997).
- It is well known that the performance of Wald CI on variance components is very poor. For example, the 95% Wald CI on \(\hat{\tau}^2_{(3)}\) in the three-level random-effects meta-analysis was (-0.0025, 0.1180). The lower bound falls outside 0.
- A LBCI on the heterogeneity variance is preferred. Since \(I^2_{(2)}\) and \(I^2_{(3)}\) are functions of \(\tau^2_{(2)}\) and \(\tau^2_{(3)}\), LBCI on these indices may also be requested and used to indicate the precision of these indices.
- LBCI may be requested by specifying
LBin theintervals.typeargument. - The 95% LBCI on \(\hat{\tau}^2_{(3)}\) is (0.0198, 0.1763) that stay inside the meaningful boundaries. Regarding the \(I^2\), the 95% LBCIs on \(I^2_{(2)}\) and \(I^2_{(3)}\) were (0.1274, 0.6573) and (0.2794, 0.8454), respectively.
## LBCI for random-effects model
summary( meta3(y=y, v=v, cluster=District, data=Cooper03,
I2=c("I2q", "ICC"), intervals.type="LB") )
Call:
meta3(y = y, v = v, cluster = District, data = Cooper03, intervals.type = "LB",
I2 = c("I2q", "ICC"))
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept 0.184455 NA 0.012025 0.358179 NA NA
Tau2_2 0.032865 NA 0.016331 0.060489 NA NA
Tau2_3 0.057738 NA 0.019810 0.082072 NA NA
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Heterogeneity indices (I2) and their 95% likelihood-based CIs:
lbound Estimate ubound
I2_2 (Typical v: Q statistic) 0.12754 0.34396 NA
ICC_2 (tau^2/(tau^2+tau^3)) NA 0.36273 NA
I2_3 (Typical v: Q statistic) NA 0.60429 NA
ICC_3 (tau^3/(tau^2+tau^3)) NA 0.63727 NA
Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 3
Degrees of freedom: 53
-2 log likelihood: 16.78987
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)## LBCI for mixed-effects model
summary( meta3(y=y, v=v, cluster=District, x=scale(Year, scale=FALSE),
data=Cooper03, intervals.type="LB") )
Call:
meta3(y = y, v = v, cluster = District, x = scale(Year, scale = FALSE),
data = Cooper03, intervals.type = "LB")
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept 0.1780268 NA 0.0056477 0.3512066 NA NA
Slope_1 0.0050737 NA -0.0128322 0.0237932 NA NA
Tau2_2 0.0329390 NA 0.0163748 0.0632634 NA NA
Tau2_3 0.0564628 NA 0.0192365 0.1716327 NA NA
Q statistic on the homogeneity of effect sizes: 578.864
Degrees of freedom of the Q statistic: 55
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.032865 0.0577
Tau2 (with predictors) 0.032939 0.0565
R2 0.000000 0.0221
Number of studies (or clusters): 11
Number of observed statistics: 56
Number of estimated parameters: 4
Degrees of freedom: 52
-2 log likelihood: 16.43629
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Restricted maximum likelihood estimation
- REML may also be used in three-level meta-analysis. The parameter estimates for \(\tau^2_{(2)}\) and \(\tau^2_{(3)}\) were 0.0327 and 0.0651, respectively.
## REML
summary( reml1 <- reml3(y=y, v=v, cluster=District, data=Cooper03) )
Call:
reml3(y = y, v = v, cluster = District, data = Cooper03)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Tau2_2 0.0327365 0.0110922 0.0109963 0.0544768 2.9513 0.003164 **
Tau2_3 0.0650619 0.0355102 -0.0045368 0.1346607 1.8322 0.066921 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Number of studies (or clusters): 56
Number of observed statistics: 55
Number of estimated parameters: 2
Degrees of freedom: 53
-2 log likelihood: -81.14044
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- We may impose an equality constraint on \(\tau^2_{(2)}\) and \(\tau^2_{(3)}\) and test whether this constraint is statistically significant. The estimated value for \(\tau^2_{(2)}=\tau^2_{(3)}\) was 0.0404. When this model is compared against the unconstrained model, the test statistic was 1.0033 (df = 1), p = .3165, which is not significant.
summary( reml0 <- reml3(y=y, v=v, cluster=District, data=Cooper03,
RE.equal=TRUE, model.name="Equal Tau2") )
Call:
reml3(y = y, v = v, cluster = District, data = Cooper03, RE.equal = TRUE,
model.name = "Equal Tau2")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Tau2 0.040418 0.010290 0.020249 0.060587 3.9277 8.576e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Number of studies (or clusters): 56
Number of observed statistics: 55
Number of estimated parameters: 1
Degrees of freedom: 54
-2 log likelihood: -80.1371
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)anova(reml1, reml0) base comparison ep minus2LL df AIC diffLL
1 Variance component with REML <NA> 2 -81.14044 -2 NA NA
2 Variance component with REML Equal Tau2 1 -80.13710 -1 NA 1.003336
diffdf p
1 NA NA
2 1 0.3165046- We may also estimate the residual heterogeneity after controlling for the covariate. The estimated residual heterogeneity for \(\tau^2_{(2)}\) and \(\tau^2_{(3)}\) were 0.0327 and 0.0723, respectively.
summary( reml3(y=y, v=v, cluster=District, x=scale(Year, scale=FALSE),
data=Cooper03) )
Call:
reml3(y = y, v = v, cluster = District, x = scale(Year, scale = FALSE),
data = Cooper03)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Tau2_2 0.0326502 0.0110529 0.0109870 0.0543134 2.9540 0.003137 **
Tau2_3 0.0722656 0.0405349 -0.0071813 0.1517125 1.7828 0.074619 .
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Number of studies (or clusters): 56
Number of observed statistics: 54
Number of estimated parameters: 2
Degrees of freedom: 52
-2 log likelihood: -72.09405
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)More complex 3-level meta-analyses with Bornmann et al.’s (2007) dataset
This section replicates the findings in Table 3 of Marsh et al. (2009). Several additional analyses on model comparisons were conducted. Missing data were artificially introduced to illustrate how missing data might be handled with FIML.
Inspecting the data
The effect size and its sampling variance are logOR (log of the odds ratio) and v, respectively. Cluster is the variable representing the cluster effect, whereas the potential covariates are Year (year of publication), Type (Grants vs. Fellowship), Discipline (Physical sciences, Life sciences/biology, Social sciences/humanities and Multidisciplinary) and Country (United States, Canada, Australia, United Kingdom and Europe).
#### Bornmann et al. (2007)
head(Bornmann07) Id Study Cluster logOR v Year
1 1 Ackers (2000a; Marie Curie) 1 -0.40108 0.01391692 1996
2 2 Ackers (2000b; Marie Curie) 1 -0.05727 0.03428793 1996
3 3 Ackers (2000c; Marie Curie) 1 -0.29852 0.03391122 1996
4 4 Ackers (2000d; Marie Curie) 1 0.36094 0.03404025 1996
5 5 Ackers (2000e; Marie Curie) 1 -0.33336 0.01282103 1996
6 6 Ackers (2000f; Marie Curie) 1 -0.07173 0.01361189 1996
Type Discipline Country
1 Fellowship Physical sciences Europe
2 Fellowship Physical sciences Europe
3 Fellowship Physical sciences Europe
4 Fellowship Physical sciences Europe
5 Fellowship Social sciences/humanities Europe
6 Fellowship Physical sciences EuropeModel 0: Intercept
The Q statistic was 221.2809 (df = 65), p < .001. The estimated average effect (and its 95% Wald CI) was -0.1008 (-0.1794, -0.0221). The \(\hat{\tau}^2_{(2)}\) and \(\hat{\tau}^3_{(3)}\) were 0.0038 and 0.0141, respectively. The \(I^2_{(2)}\) and \(I^2_{(3)}\) were 0.1568 and 0.5839, respectively.
## Model 0: Intercept
summary( Model0 <- meta3(y=logOR, v=v, cluster=Cluster, data=Bornmann07,
model.name="3 level model") )
Call:
meta3(y = logOR, v = v, cluster = Cluster, data = Bornmann07,
model.name = "3 level model")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.1007784 0.0401327 -0.1794371 -0.0221198 -2.5111 0.01203 *
Tau2_2 0.0037965 0.0027210 -0.0015367 0.0091297 1.3952 0.16295
Tau2_3 0.0141352 0.0091445 -0.0037877 0.0320580 1.5458 0.12216
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
I2_2 (Typical v: Q statistic) 0.1568
I2_3 (Typical v: Q statistic) 0.5839
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 3
Degrees of freedom: 63
-2 log likelihood: 25.80256
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Testing \(H_0: \tau^2_3 = 0\) We may test whether the three-level model is necessary by testing \(H_0: \tau^2_{(3)} = 0\). The likelihood ratio statistic was 10.2202 (df = 1), p < .01. Thus, the three-level model is statistically better than the two-level model.
## Testing tau^2_3 = 0
Model0a <- meta3(logOR, v, cluster=Cluster, data=Bornmann07,
RE3.constraints=0, model.name="2 level model")
anova(Model0, Model0a) base comparison ep minus2LL df AIC diffLL diffdf
1 3 level model <NA> 3 25.80256 63 -100.19744 NA NA
2 3 level model 2 level model 2 36.02279 64 -91.97721 10.22024 1
p
1 NA
2 0.001389081- Testing \(H_0: \tau^2_2 = \tau^2_3\) The likelihood-ratio statistic in testing \(H_0: \tau^2_{(2)} = \tau^2_{(3)}\) was 1.3591 (df = 1), p = 0.2437. Thus, there is no evidence to reject the null hypothesis.
## Testing tau^2_2 = tau^2_3
Model0b <- meta3(logOR, v, cluster=Cluster, data=Bornmann07,
RE2.constraints="0.1*Eq_tau2", RE3.constraints="0.1*Eq_tau2",
model.name="tau2_2 equals tau2_3")
anova(Model0, Model0b) base comparison ep minus2LL df AIC diffLL
1 3 level model <NA> 3 25.80256 63 -100.1974 NA
2 3 level model tau2_2 equals tau2_3 2 27.16166 64 -100.8383 1.359103
diffdf p
1 NA NA
2 1 0.243693Model 1: Type as a covariate
- Conventionally, one level (e.g.,
Grants) is used as the reference group. The estimated intercept (labeledInterceptin the output) represents the estimated effect size forGrantsand the regression coefficient (labeledSlope_1in the output) is the difference betweenFellowshipandGrants.- The estimated slope on
Type(and its 95% Wald CI) was -0.1956 (-0.3018, -0.0894) which is statistically significant at \(\alpha=.05\). This is the difference betweenFellowshipandGrants. The \(R^2_{(2)}\) and \(R^2_{(3)}\) were 0.0693 and 0.7943, respectively.
- The estimated slope on
## Model 1: Type as a covariate
## Convert characters into a dummy variable
## Type2=0 (Grants); Type2=1 (Fellowship)
Type2 <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
summary( Model1 <- meta3(logOR, v, x=Type2, cluster=Cluster, data=Bornmann07))
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = Type2, data = Bornmann07)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.0066071 0.0371125 -0.0793462 0.0661320 -0.1780 0.8587001
Slope_1 -0.1955869 0.0541649 -0.3017483 -0.0894256 -3.6110 0.0003051
Tau2_2 0.0035335 0.0024306 -0.0012303 0.0082974 1.4538 0.1460058
Tau2_3 0.0029079 0.0031183 -0.0032039 0.0090197 0.9325 0.3510704
Intercept
Slope_1 ***
Tau2_2
Tau2_3
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0035335 0.0029
R2 0.0692595 0.7943
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 4
Degrees of freedom: 62
-2 log likelihood: 17.62569
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Alternative model:
GrantsandFellowshipas indicator variables- If we want to estimate the effects for both
GrantsandFellowship, we may create two indicator variables to represent them. Since we cannot estimate the intercept and these coefficients at the same time, we need to fix the intercept at 0 by specifying theintercept.constraints=0argument inmeta3(). We are now able to include bothGrantsandFellowshipin the analysis. The estimated effects (and their 95% CIs) forGrantsandFellowshipwere -0.0066 (-0.0793, 0.0661) and -0.2022 (-0.2805, -0.1239), respectively.
- If we want to estimate the effects for both
## Alternative model: Grants and Fellowship as indicators
## Indicator variables
Grants <- ifelse(Bornmann07$Type=="Grants", yes=1, no=0)
Fellowship <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
Model1b <- meta3(logOR, v, x=cbind(Grants, Fellowship),
cluster=Cluster, data=Bornmann07,
intercept.constraints=0, model.name="Model 1")
## Rerun to remove error code
Model1b <- rerun(Model1b)
summary(Model1b)
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Grants,
Fellowship), data = Bornmann07, intercept.constraints = 0,
model.name = "Model 1")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Slope_1 0.34917952 NA NA NA NA NA
Slope_2 -0.20209280 0.03916901 -0.27886265 -0.12532295 -5.1595 2.476e-07
Tau2_2 0.00357518 0.00040076 0.00278971 0.00436066 8.9210 < 2.2e-16
Tau2_3 0.00271391 0.00263601 -0.00245258 0.00788040 1.0296 0.3032
Slope_1
Slope_2 ***
Tau2_2 ***
Tau2_3
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0035752 0.0027
R2 0.0582930 0.8080
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 4
Degrees of freedom: 62
-2 log likelihood: 17.65814
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Model 2: Year and Year^2 as covariates
- When there are several covariates, we may combine them with the
cbind()command. For example,cbind(Year, Year^2)includes bothYearand its squared as covariates. In the output,Slope_1andSlope_2refer to the regression coefficients forYearandYear^2, respectively. To increase the numerical stability, the covariates are usually centered before creating the quadratic terms. Since Marsh et al. (2009) standardizedYearin their analysis, I follow this practice here. - The estimated regression coefficients (and their 95% CIs) for
YearandYear^2were -0.0010 (-0.0473, 0.0454) and -0.0118 (-0.0247, 0.0012), respectively. The \(R^2_{(2)}\) and \(R^2_{(3)}\) were 0.2430 and 0.0000, respectively.
## Model 2: Year and Year^2 as covariates
summary( Model2 <- meta3(logOR, v, x=cbind(scale(Year), scale(Year)^2),
cluster=Cluster, data=Bornmann07,
model.name="Model 2") )
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(scale(Year),
scale(Year)^2), data = Bornmann07, model.name = "Model 2")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.08627312 0.04125581 -0.16713303 -0.00541321 -2.0912 0.03651
Slope_1 -0.00095287 0.02365224 -0.04731040 0.04540466 -0.0403 0.96786
Slope_2 -0.01176840 0.00659995 -0.02470407 0.00116727 -1.7831 0.07457
Tau2_2 0.00287389 0.00206817 -0.00117965 0.00692744 1.3896 0.16466
Tau2_3 0.01479446 0.00926095 -0.00335666 0.03294558 1.5975 0.11015
Intercept *
Slope_1
Slope_2 .
Tau2_2
Tau2_3
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0028739 0.0148
R2 0.2430134 0.0000
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 5
Degrees of freedom: 61
-2 log likelihood: 22.3836
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Testing \(H_0: \beta_{Year} = \beta_{Year^2}=0\) The test statistic was 3.4190 (df = 2), p = 0.1810. Thus, there is no evidence supporting that =Year= has a quadratic effect on the effect size.
## Testing beta_{Year} = beta_{Year^2}=0
anova(Model2, Model0) base comparison ep minus2LL df AIC diffLL diffdf p
1 Model 2 <NA> 5 22.38360 61 -99.6164 NA NA NA
2 Model 2 3 level model 3 25.80256 63 -100.1974 3.418955 2 0.1809603Model 3: Discipline as a covariate
- There are four categories in
Discipline.multidisciplinaryis used as the reference group in the analysis. - The estimated regression coefficients (and their 95% Wald CIs) for
DisciplinePhy,DisciplineLifeandDisciplineSocwere -0.0091 (-0.2041, 0.1859), -0.1262 (-0.2804, 0.0280) and -0.2370 (-0.4746, 0.0007), respectively. The \(R^2_2\) and \(R^2_3\) were 0.0000 and 0.4975, respectively.
## Model 3: Discipline as a covariate
## Create dummy variables using multidisciplinary as the reference group
DisciplinePhy <- ifelse(Bornmann07$Discipline=="Physical sciences", yes=1, no=0)
DisciplineLife <- ifelse(Bornmann07$Discipline=="Life sciences/biology", yes=1, no=0)
DisciplineSoc <- ifelse(Bornmann07$Discipline=="Social sciences/humanities", yes=1, no=0)
summary( Model3 <- meta3(logOR, v, x=cbind(DisciplinePhy, DisciplineLife, DisciplineSoc),
cluster=Cluster, data=Bornmann07,
model.name="Model 3") )
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(DisciplinePhy,
DisciplineLife, DisciplineSoc), data = Bornmann07, model.name = "Model 3")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.01474783 0.06389946 -0.13998847 0.11049280 -0.2308 0.81747
Slope_1 -0.00913064 0.09949197 -0.20413133 0.18587004 -0.0918 0.92688
Slope_2 -0.12617958 0.07866275 -0.28035574 0.02799659 -1.6041 0.10870
Slope_3 -0.23695698 0.12123091 -0.47456519 0.00065123 -1.9546 0.05063
Tau2_2 0.00390942 0.00283948 -0.00165587 0.00947471 1.3768 0.16857
Tau2_3 0.00710338 0.00643210 -0.00550331 0.01971006 1.1044 0.26944
Intercept
Slope_1
Slope_2
Slope_3 .
Tau2_2
Tau2_3
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0039094 0.0071
R2 0.0000000 0.4975
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 6
Degrees of freedom: 60
-2 log likelihood: 20.07571
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Testing whether
Disciplineis significant- The test statistic was 5.7268 (df = 3), p = 0.1257 which is not significant. Therefore, there is no evidence supporting that =Discipline= explains the variation of the effect sizes.
## Testing whether Discipline is significant
anova(Model3, Model0) base comparison ep minus2LL df AIC diffLL diffdf
1 Model 3 <NA> 6 20.07571 60 -99.92429 NA NA
2 Model 3 3 level model 3 25.80256 63 -100.19744 5.726842 3
p
1 NA
2 0.1256832Model 4: Country as a covariate
- There are five categories in
Country.United Statesis used as the reference group in the analysis. - The estimated regression coefficients (and their 95% Wald CIs) for
CountryAus,CountryCan,CountryEur, andCountryUKare -0.0240 (-0.2405, 0.1924), -0.1341 (-0.3674, 0.0993), -0.2211 (-0.3660, -0.0762) and 0.0537 (-0.1413, 0.2487), respectively. The \(R^2_2\) and \(R^2_3\) were 0.1209 and 0.6606, respectively.
## Model 4: Country as a covariate
## Create dummy variables using the United States as the reference group
CountryAus <- ifelse(Bornmann07$Country=="Australia", yes=1, no=0)
CountryCan <- ifelse(Bornmann07$Country=="Canada", yes=1, no=0)
CountryEur <- ifelse(Bornmann07$Country=="Europe", yes=1, no=0)
CountryUK <- ifelse(Bornmann07$Country=="United Kingdom", yes=1, no=0)
summary( Model4 <- meta3(logOR, v, x=cbind(CountryAus, CountryCan, CountryEur,
CountryUK), cluster=Cluster, data=Bornmann07,
model.name="Model 4") )
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(CountryAus,
CountryCan, CountryEur, CountryUK), data = Bornmann07, model.name = "Model 4")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept 0.0025681 0.0597768 -0.1145922 0.1197284 0.0430 0.965732
Slope_1 -0.0240109 0.1104328 -0.2404552 0.1924333 -0.2174 0.827876
Slope_2 -0.1340800 0.1190667 -0.3674465 0.0992865 -1.1261 0.260127
Slope_3 -0.2210801 0.0739174 -0.3659555 -0.0762046 -2.9909 0.002782 **
Slope_4 0.0537251 0.0994803 -0.1412527 0.2487030 0.5401 0.589157
Tau2_2 0.0033376 0.0023492 -0.0012667 0.0079420 1.4208 0.155383
Tau2_3 0.0047979 0.0044818 -0.0039862 0.0135820 1.0705 0.284379
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0033376 0.0048
R2 0.1208598 0.6606
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 7
Degrees of freedom: 59
-2 log likelihood: 14.18259
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Testing whether
Disciplineis significant- The test statistic was 11.6200 (df = 4), p = 0.0204 which is statistically significant.
## Testing whether Discipline is significant
anova(Model4, Model0) base comparison ep minus2LL df AIC diffLL diffdf
1 Model 4 <NA> 7 14.18259 59 -103.8174 NA NA
2 Model 4 3 level model 3 25.80256 63 -100.1974 11.61996 4
p
1 NA
2 0.02041284Model 5: Type and Discipline as covariates
- The \(R^2_{(2)}\) and \(R^2_{(3)}\) were 0.3925 and 1.0000, respectively. The \(\hat{\tau}^2_{(3)}\) was near 0 after controlling for the covariates.
## Model 5: Type and Discipline as covariates
Model5 <- meta3(logOR, v, x=cbind(Type2, DisciplinePhy, DisciplineLife,
DisciplineSoc), cluster=Cluster, data=Bornmann07,
model.name="Model 5")
## Rerun to remove error code
Model5 <- rerun(Model5)
summary(Model5)
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Type2, DisciplinePhy,
DisciplineLife, DisciplineSoc), data = Bornmann07, model.name = "Model 5")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept 6.7036e-02 1.8553e-02 3.0672e-02 1.0340e-01 3.6132
Slope_1 -1.9004e-01 4.0234e-02 -2.6890e-01 -1.1118e-01 -4.7233
Slope_2 1.9511e-02 6.5942e-02 -1.0973e-01 1.4875e-01 0.2959
Slope_3 -1.2779e-01 3.5914e-02 -1.9818e-01 -5.7400e-02 -3.5582
Slope_4 -2.3950e-01 9.4054e-02 -4.2384e-01 -5.5154e-02 -2.5464
Tau2_2 2.3062e-03 1.4270e-03 -4.9059e-04 5.1030e-03 1.6162
Tau2_3 1.0000e-10 NA NA NA NA
Pr(>|z|)
Intercept 0.0003025 ***
Slope_1 2.32e-06 ***
Slope_2 0.7673209
Slope_3 0.0003734 ***
Slope_4 0.0108849 *
Tau2_2 0.1060586
Tau2_3 NA
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0023062 0.0000
R2 0.3925434 1.0000
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 7
Degrees of freedom: 59
-2 log likelihood: 4.66727
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Testing whether
Disciplineis significant after controlling forType- The test statistic was 12.9584 (df = 3), p = 0.0047 which is significant. Therefore,
Disciplineis still significant after controlling forType.
- The test statistic was 12.9584 (df = 3), p = 0.0047 which is significant. Therefore,
## Testing whether Discipline is significant after controlling for Type
anova(Model5, Model1) base comparison ep minus2LL df AIC diffLL diffdf
1 Model 5 <NA> 7 4.66727 59 -113.3327 NA NA
2 Model 5 Meta analysis with ML 4 17.62569 62 -106.3743 12.95842 3
p
1 NA
2 0.004727388Model 6: Type and Country as covariates
- The \(R^2_{(2)}\) and \(R^2_{(3)}\) were 0.3948 and 1.0000, respectively. The \(\hat{\tau}^2_{(3)}\) was near 0 after controlling for the covariates.
## Model 6: Type and Country as covariates
Model6 <- meta3(logOR, v, x=cbind(Type2, CountryAus, CountryCan,
CountryEur, CountryUK), cluster=Cluster, data=Bornmann07,
model.name="Model 6")
## Rerun to remove error code
Model6 <- rerun(Model6)
summary(Model6)
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Type2, CountryAus,
CountryCan, CountryEur, CountryUK), data = Bornmann07, model.name = "Model 6")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept 6.7507e-02 1.8933e-02 3.0399e-02 1.0461e-01 3.5656
Slope_1 -1.5167e-01 4.1113e-02 -2.3225e-01 -7.1092e-02 -3.6892
Slope_2 -6.9580e-02 8.5164e-02 -2.3650e-01 9.7339e-02 -0.8170
Slope_3 -1.4231e-01 7.5204e-02 -2.8970e-01 5.0878e-03 -1.8923
Slope_4 -1.6116e-01 4.0203e-02 -2.3995e-01 -8.2361e-02 -4.0086
Slope_5 9.0419e-03 7.0074e-02 -1.2830e-01 1.4639e-01 0.1290
Tau2_2 2.2976e-03 1.4407e-03 -5.2618e-04 5.1213e-03 1.5947
Tau2_3 1.0000e-10 NA NA NA NA
Pr(>|z|)
Intercept 0.0003631 ***
Slope_1 0.0002250 ***
Slope_2 0.4139267
Slope_3 0.0584497 .
Slope_4 6.108e-05 ***
Slope_5 0.8973315
Tau2_2 0.1107693
Tau2_3 NA
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0022976 0.0000
R2 0.3948192 1.0000
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 8
Degrees of freedom: 58
-2 log likelihood: 5.076592
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- Testing whether
Countryis significant after controlling forType- The test statistic was 12.5491 (df = 4), p = 0.0137. Thus,
Countryis significant after controlling forType.
- The test statistic was 12.5491 (df = 4), p = 0.0137. Thus,
## Testing whether Country is significant after controlling for Type
anova(Model6, Model1) base comparison ep minus2LL df AIC diffLL diffdf
1 Model 6 <NA> 8 5.076592 58 -110.9234 NA NA
2 Model 6 Meta analysis with ML 4 17.625692 62 -106.3743 12.5491 4
p
1 NA
2 0.01370262Model 7: Discipline and Country as covariates
- The \(R^2_{(2)}\) and \(R^2_{(3)}\) were 0.1397 and 0.7126, respectively.
## Model 7: Discipline and Country as covariates
summary( meta3(logOR, v, x=cbind(DisciplinePhy, DisciplineLife, DisciplineSoc,
CountryAus, CountryCan, CountryEur, CountryUK),
cluster=Cluster, data=Bornmann07,
model.name="Model 7") )
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(DisciplinePhy,
DisciplineLife, DisciplineSoc, CountryAus, CountryCan, CountryEur,
CountryUK), data = Bornmann07, model.name = "Model 7")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept 0.0029305 0.0576743 -0.1101091 0.1159700 0.0508 0.95948
Slope_1 0.1742169 0.1702555 -0.1594777 0.5079116 1.0233 0.30618
Slope_2 0.0826806 0.1599803 -0.2308751 0.3962362 0.5168 0.60528
Slope_3 -0.0462265 0.1715775 -0.3825121 0.2900592 -0.2694 0.78761
Slope_4 -0.0486321 0.1306919 -0.3047836 0.2075193 -0.3721 0.70981
Slope_5 -0.2169133 0.1915704 -0.5923844 0.1585579 -1.1323 0.25751
Slope_6 -0.3036578 0.1670722 -0.6311132 0.0237976 -1.8175 0.06914 .
Slope_7 -0.0605272 0.1809420 -0.4151671 0.2941127 -0.3345 0.73799
Tau2_2 0.0032661 0.0022784 -0.0011994 0.0077317 1.4335 0.15171
Tau2_3 0.0040618 0.0038459 -0.0034759 0.0115996 1.0562 0.29090
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0032661 0.0041
R2 0.1396974 0.7126
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 10
Degrees of freedom: 56
-2 log likelihood: 10.31105
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Model 8: Type, Discipline and Country as covariates
- The \(R^2_{(2)}\) and \(R^2_{(3)}\) were 0.4466 and 1.0000, respectively. The \(\hat{\tau}^2_{(3)}\) was near 0 after controlling for the covariates.
## Model 8: Type, Discipline and Country as covariates
Model8 <- meta3(logOR, v, x=cbind(Type2, DisciplinePhy, DisciplineLife, DisciplineSoc,
CountryAus, CountryCan, CountryEur, CountryUK),
cluster=Cluster, data=Bornmann07,
model.name="Model 8")
## There was an estimation error. The model was rerun again.
summary(rerun(Model8))
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = cbind(Type2, DisciplinePhy,
DisciplineLife, DisciplineSoc, CountryAus, CountryCan, CountryEur,
CountryUK), data = Bornmann07, model.name = "Model 8")
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept 6.8563e-02 1.8630e-02 3.2049e-02 1.0508e-01 3.6802
Slope_1 -1.6885e-01 4.1545e-02 -2.5028e-01 -8.7425e-02 -4.0643
Slope_2 2.5329e-01 1.5814e-01 -5.6670e-02 5.6324e-01 1.6016
Slope_3 1.2689e-01 1.4774e-01 -1.6268e-01 4.1646e-01 0.8589
Slope_4 -8.3549e-03 1.5796e-01 -3.1795e-01 3.0124e-01 -0.0529
Slope_5 -1.1530e-01 1.1147e-01 -3.3377e-01 1.0317e-01 -1.0344
Slope_6 -2.6412e-01 1.6402e-01 -5.8559e-01 5.7344e-02 -1.6103
Slope_7 -2.9029e-01 1.4859e-01 -5.8152e-01 9.5193e-04 -1.9536
Slope_8 -1.5975e-01 1.6285e-01 -4.7893e-01 1.5943e-01 -0.9810
Tau2_2 2.1010e-03 1.2925e-03 -4.3226e-04 4.6342e-03 1.6255
Tau2_3 1.0000e-10 NA NA NA NA
Pr(>|z|)
Intercept 0.000233 ***
Slope_1 4.818e-05 ***
Slope_2 0.109239
Slope_3 0.390410
Slope_4 0.957818
Slope_5 0.300948
Slope_6 0.107323
Slope_7 0.050754 .
Slope_8 0.326609
Tau2_2 0.104051
Tau2_3 NA
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 221.2809
Degrees of freedom of the Q statistic: 65
P value of the Q statistic: 0
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0021010 0.0000
R2 0.4466073 1.0000
Number of studies (or clusters): 21
Number of observed statistics: 66
Number of estimated parameters: 11
Degrees of freedom: 55
-2 log likelihood: -1.645211
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Handling missing covariates with FIML
When there are missing data in the covariates, data with missing values are excluded before the analysis in meta3(). The missing covariates can be handled by the use of FIML in meta3X(). We illustrate two examples on how to analyze data with missing covariates with missing completely at random (MCAR) and missing at random (MAR) data.
MCAR
- About 25% of the level-2 covariate
Typewas introduced by the MCAR mechanism.
#### Handling missing covariates with FIML
## MCAR
## Set seed for replication
set.seed(1000000)
## Copy Bornmann07 to my.df
my.df <- Bornmann07
## "Fellowship": 1; "Grant": 0
my.df$Type_MCAR <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
## Create 17 out of 66 missingness with MCAR
my.df$Type_MCAR[sample(1:66, 17)] <- NA
summary(meta3(y=logOR, v=v, cluster=Cluster, x=Type_MCAR, data=my.df))
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = Type_MCAR, data = my.df)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept -0.00484542 0.03934429 -0.08195881 0.07226797 -0.1232
Slope_1 -0.21090081 0.05346222 -0.31568483 -0.10611679 -3.9449
Tau2_2 0.00446788 0.00549282 -0.00629784 0.01523361 0.8134
Tau2_3 0.00092884 0.00336491 -0.00566625 0.00752394 0.2760
Pr(>|z|)
Intercept 0.9020
Slope_1 7.985e-05 ***
Tau2_2 0.4160
Tau2_3 0.7825
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 151.643
Degrees of freedom of the Q statistic: 48
P value of the Q statistic: 1.115552e-12
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0042664 0.0145
Tau2 (with predictors) 0.0044679 0.0009
R2 0.0000000 0.9361
Number of studies (or clusters): 20
Number of observed statistics: 49
Number of estimated parameters: 4
Degrees of freedom: 45
-2 log likelihood: 13.13954
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- There is no need to specify whether the covariates are level 2 or level 3 in
meta3()because the covariates are treated as a design matrix. Whenmeta3X()is used, users need to specify whether the covariates are at level 2 (x2) or level 3 (x3).
summary(meta3X(y=logOR, v=v, cluster=Cluster, x2=Type_MCAR, data=my.df))
Call:
meta3X(y = logOR, v = v, cluster = Cluster, x2 = Type_MCAR, data = my.df)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.0106318 0.0397685 -0.0885766 0.0673131 -0.2673 0.789207
SlopeX2_1 -0.1753249 0.0582619 -0.2895162 -0.0611336 -3.0093 0.002619 **
Tau2_2 0.0030338 0.0026839 -0.0022266 0.0082941 1.1304 0.258324
Tau2_3 0.0036839 0.0042817 -0.0047082 0.0120759 0.8604 0.389586
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0037965 0.0141
Tau2 (with predictors) 0.0030338 0.0037
R2 0.2009069 0.7394
Number of studies (or clusters): 21
Number of observed statistics: 115
Number of estimated parameters: 7
Degrees of freedom: 108
-2 log likelihood: 49.76372
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)MAR
- For the case for missing covariates with MAR, the missingness in
Typedepends on the values ofYear.Typeis missing whenYearis smaller than 1996.
## MAR
Type_MAR <- ifelse(Bornmann07$Type=="Fellowship", yes=1, no=0)
## Create 27 out of 66 missingness with MAR for cases Year<1996
index_MAR <- ifelse(Bornmann07$Year<1996, yes=TRUE, no=FALSE)
Type_MAR[index_MAR] <- NA
summary(meta3(logOR, v, x=Type_MAR, cluster=Cluster, data=Bornmann07))
Call:
meta3(y = logOR, v = v, cluster = Cluster, x = Type_MAR, data = Bornmann07)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.01587052 0.03952547 -0.09333901 0.06159796 -0.4015 0.688032
Slope_1 -0.17573648 0.06328327 -0.29976941 -0.05170354 -2.7770 0.005487
Tau2_2 0.00259266 0.00171596 -0.00077056 0.00595588 1.5109 0.130811
Tau2_3 0.00278384 0.00267150 -0.00245221 0.00801989 1.0421 0.297388
Intercept
Slope_1 **
Tau2_2
Tau2_3
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 151.11
Degrees of freedom of the Q statistic: 38
P value of the Q statistic: 1.998401e-15
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0029593 0.0097
Tau2 (with predictors) 0.0025927 0.0028
R2 0.1238926 0.7121
Number of studies (or clusters): 12
Number of observed statistics: 39
Number of estimated parameters: 4
Degrees of freedom: 35
-2 log likelihood: -24.19956
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- It is possible to include level 2 (
av2) and level 3 (av3) auxiliary variables. Auxiliary variables are those that predict the missing values or are correlated with the variables that contain missing values. The inclusion of auxiliary variables can improve the efficiency of the estimation and the parameter estimates.
## Include auxiliary variable
summary(meta3X(y=logOR, v=v, cluster=Cluster, x2=Type_MAR, av2=Year, data=my.df))
Call:
meta3X(y = logOR, v = v, cluster = Cluster, x2 = Type_MAR, av2 = Year,
data = my.df)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Intercept -0.0264058 0.0571997 -0.1385151 0.0857036 -0.4616 0.644339
SlopeX2_1 -0.2003999 0.0691051 -0.3358434 -0.0649564 -2.8999 0.003732 **
Tau2_2 0.0029970 0.0022371 -0.0013877 0.0073816 1.3397 0.180358
Tau2_3 0.0030212 0.0032462 -0.0033413 0.0093837 0.9307 0.352018
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Explained variances (R2):
Level 2 Level 3
Tau2 (no predictor) 0.0049237 0.0088
Tau2 (with predictors) 0.0029970 0.0030
R2 0.3913243 0.6571
Number of studies (or clusters): 21
Number of observed statistics: 171
Number of estimated parameters: 14
Degrees of freedom: 157
-2 log likelihood: 377.3479
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)Two-stage structural equation modeling
Inspect the data: Digman (1997)
The correlation matrices and the sample sizes were stored in Digman97$data and Digman97$n, respectively. We may list the first few cases of the data by using the head() command.
#### Load the metaSEM library for TSSEM
library(metaSEM)
#### Inspect the data for inspection
head(Digman97$data)$`Digman 1 (1994)`
A C ES E I
A 1.00 0.62 0.41 -0.48 0.00
C 0.62 1.00 0.59 -0.10 0.35
ES 0.41 0.59 1.00 0.27 0.41
E -0.48 -0.10 0.27 1.00 0.37
I 0.00 0.35 0.41 0.37 1.00
$`Digman 2 (1994)`
A C ES E I
A 1.00 0.39 0.53 -0.30 -0.05
C 0.39 1.00 0.59 0.07 0.44
ES 0.53 0.59 1.00 0.09 0.22
E -0.30 0.07 0.09 1.00 0.45
I -0.05 0.44 0.22 0.45 1.00
$`Digman 3 (1963c)`
A C ES E I
A 1.00 0.65 0.35 0.25 0.14
C 0.65 1.00 0.37 -0.10 0.33
ES 0.35 0.37 1.00 0.24 0.41
E 0.25 -0.10 0.24 1.00 0.41
I 0.14 0.33 0.41 0.41 1.00
$`Digman & Takemoto-Chock (1981b)`
A C ES E I
A 1.00 0.65 0.70 -0.26 -0.03
C 0.65 1.00 0.71 -0.16 0.24
ES 0.70 0.71 1.00 0.01 0.11
E -0.26 -0.16 0.01 1.00 0.66
I -0.03 0.24 0.11 0.66 1.00
$`Graziano & Ward (1992)`
A C ES E I
A 1.00 0.64 0.35 0.29 0.22
C 0.64 1.00 0.27 0.16 0.22
ES 0.35 0.27 1.00 0.32 0.36
E 0.29 0.16 0.32 1.00 0.53
I 0.22 0.22 0.36 0.53 1.00
$`Yik & Bond (1993)`
A C ES E I
A 1.00 0.66 0.57 0.35 0.38
C 0.66 1.00 0.45 0.20 0.31
ES 0.57 0.45 1.00 0.49 0.31
E 0.35 0.20 0.49 1.00 0.59
I 0.38 0.31 0.31 0.59 1.00 head(Digman97$n)[1] 102 149 334 162 91 656Fixed-effects TSSEM
Stage 1 analysis
To conduct a fixed-effects TSSEM, we may specify method=FEM argument (the default method) in calling the tssem1() function. The results are stored in an object named fixed1. It can be displayed by the summary() command. The \(\chi^2(130, N=4,496) = 1,499.73, p < .001\), CFI=0.6825, RMSEA=0.1812 and SRMR=0.1750. Based on the test statistic and the goodness-of-fit indices, the assumption of homogeneity of correlation matrices was rejected.
## Fixed-effects model: Stage 1 analysis
fixed1 <- tssem1(my.df=Digman97$data, n=Digman97$n, method="FEM")
summary(fixed1)
Call:
tssem1FEM(my.df = my.df, n = n, cor.analysis = cor.analysis,
model.name = model.name, cluster = cluster, suppressWarnings = suppressWarnings,
silent = silent, run = run)
Coefficients:
Estimate Std.Error z value Pr(>|z|)
S[1,2] 0.363278 0.013368 27.1756 < 2.2e-16 ***
S[1,3] 0.390375 0.012881 30.3071 < 2.2e-16 ***
S[1,4] 0.103572 0.015048 6.8829 5.866e-12 ***
S[1,5] 0.092286 0.015047 6.1330 8.624e-10 ***
S[2,3] 0.416070 0.012520 33.2336 < 2.2e-16 ***
S[2,4] 0.135148 0.014776 9.1462 < 2.2e-16 ***
S[2,5] 0.141431 0.014866 9.5134 < 2.2e-16 ***
S[3,4] 0.244459 0.014153 17.2720 < 2.2e-16 ***
S[3,5] 0.138339 0.014834 9.3258 < 2.2e-16 ***
S[4,5] 0.424566 0.012376 34.3064 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 4496.0000
Chi-square of target model 1505.4443
DF of target model 130.0000
p value of target model 0.0000
Chi-square of independence model 4471.4242
DF of independence model 140.0000
RMSEA 0.1815
RMSEA lower 95% CI 0.1736
RMSEA upper 95% CI 0.1901
SRMR 0.1621
TLI 0.6580
CFI 0.6824
AIC 1245.4443
BIC 412.0217
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)The pooled correlation matrix (the parameter estimates) can be extracted by the use of the coef() command.
coef(fixed1) A C ES E I
A 1.00000000 0.3632782 0.3903748 0.1035716 0.09228557
C 0.36327824 1.0000000 0.4160695 0.1351477 0.14143058
ES 0.39037483 0.4160695 1.0000000 0.2444593 0.13833895
E 0.10357155 0.1351477 0.2444593 1.0000000 0.42456626
I 0.09228557 0.1414306 0.1383390 0.4245663 1.00000000Stage 2 analysis
As an illustration, I continued to fit the structural model based on the pooled correlation matrix. We may specify the two-factor model with the RAM formulation
## Factor covariance among latent factors
Phi <- matrix(c(1,"0.3*cor","0.3*cor",1), ncol=2, nrow=2)
## Error covariance matrix
Psi <- Diag(c("0.2*e1","0.2*e2","0.2*e3","0.2*e4","0.2*e5"))
## S matrix
S1 <- bdiagMat(list(Psi, Phi))
## This step is not necessary but it is useful for inspecting the model.
dimnames(S1)[[1]] <- dimnames(S1)[[2]] <- c("A","C","ES","E","I","Alpha","Beta")
## Display S1
S1 A C ES E I Alpha Beta
A "0.2*e1" "0" "0" "0" "0" "0" "0"
C "0" "0.2*e2" "0" "0" "0" "0" "0"
ES "0" "0" "0.2*e3" "0" "0" "0" "0"
E "0" "0" "0" "0.2*e4" "0" "0" "0"
I "0" "0" "0" "0" "0.2*e5" "0" "0"
Alpha "0" "0" "0" "0" "0" "1" "0.3*cor"
Beta "0" "0" "0" "0" "0" "0.3*cor" "1" ## A matrix
Lambda <-
matrix(c(".3*Alpha_A",".3*Alpha_C",".3*Alpha_ES",rep(0,5),".3*Beta_E",".3*Beta_I"),
ncol=2, nrow=5)
A1 <- rbind( cbind(matrix(0,ncol=5,nrow=5), Lambda),
matrix(0, ncol=7, nrow=2) )
## This step is not necessary but it is useful for inspecting the model.
dimnames(A1)[[1]] <- dimnames(A1)[[2]] <- c("A","C","ES","E","I","Alpha","Beta")
## Display A1
A1 A C ES E I Alpha Beta
A "0" "0" "0" "0" "0" ".3*Alpha_A" "0"
C "0" "0" "0" "0" "0" ".3*Alpha_C" "0"
ES "0" "0" "0" "0" "0" ".3*Alpha_ES" "0"
E "0" "0" "0" "0" "0" "0" ".3*Beta_E"
I "0" "0" "0" "0" "0" "0" ".3*Beta_I"
Alpha "0" "0" "0" "0" "0" "0" "0"
Beta "0" "0" "0" "0" "0" "0" "0" ## F matrix to select the observed variables
F1 <- create.Fmatrix(c(1,1,1,1,1,0,0), as.mxMatrix=FALSE)
## Display F1
F1 [,1] [,2] [,3] [,4] [,5] [,6] [,7]
[1,] 1 0 0 0 0 0 0
[2,] 0 1 0 0 0 0 0
[3,] 0 0 1 0 0 0 0
[4,] 0 0 0 1 0 0 0
[5,] 0 0 0 0 1 0 0- We may also specify the model using
lavann’s syntax.
model1 <- "## Factor loadings
Alpha=~A+C+ES
Beta=~E+I
## Factor correlation
Alpha~~Beta"
RAM1 <- lavaan2RAM(model1, obs.variables=c("A","C","ES","E","I"),
A.notation="on", S.notation="with")
RAM1$A
A C ES E I Alpha Beta
A "0" "0" "0" "0" "0" "0*AonAlpha" "0"
C "0" "0" "0" "0" "0" "0*ConAlpha" "0"
ES "0" "0" "0" "0" "0" "0*ESonAlpha" "0"
E "0" "0" "0" "0" "0" "0" "0*EonBeta"
I "0" "0" "0" "0" "0" "0" "0*IonBeta"
Alpha "0" "0" "0" "0" "0" "0" "0"
Beta "0" "0" "0" "0" "0" "0" "0"
$S
A C ES E I
A "0*AwithA" "0" "0" "0" "0"
C "0" "0*CwithC" "0" "0" "0"
ES "0" "0" "0*ESwithES" "0" "0"
E "0" "0" "0" "0*EwithE" "0"
I "0" "0" "0" "0" "0*IwithI"
Alpha "0" "0" "0" "0" "0"
Beta "0" "0" "0" "0" "0"
Alpha Beta
A "0" "0"
C "0" "0"
ES "0" "0"
E "0" "0"
I "0" "0"
Alpha "1" "0*AlphawithBeta"
Beta "0*AlphawithBeta" "1"
$F
A C ES E I Alpha Beta
A 1 0 0 0 0 0 0
C 0 1 0 0 0 0 0
ES 0 0 1 0 0 0 0
E 0 0 0 1 0 0 0
I 0 0 0 0 1 0 0A1 <- RAM1$A
S1 <- RAM1$S
F1 <- RAM1$F- Since we are conducting a correlation structure analysis, the error variances are not free parameters. We need to use the
diag.constraintsto constrain the diagonals as one. When there are nonlinear constraints, standard errors are not available inOpenMx. We may request the likelihood-based confidence intervals (LBCI) with theintervalsargument. The chi-square test statistic of the Stage 2 analysis was \(\chi^2(4, N=4,496) = 65.06, p < .001\), CFI=0.9802, RMSEA=0.0583 and SRMR=0.0284.
fixed2 <- tssem2(fixed1, Amatrix=A1, Smatrix=S1, Fmatrix=F1,
diag.constraints=TRUE, intervals="LB",
model.name="TSSEM2 Digman97")
summary(fixed2)
Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), asyCov = vcov.tssem1FEM(tssem1.obj),
n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix,
Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis,
intervals.type = intervals.type, mx.algebras = mx.algebras,
model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
AonAlpha 0.56280 NA 0.53270 0.59302 NA NA
ConAlpha 0.60522 NA 0.57525 0.63535 NA NA
EonBeta 0.78141 NA 0.71872 0.85497 NA NA
ESonAlpha 0.71920 NA 0.68876 0.75032 NA NA
IonBeta 0.55137 NA 0.50000 0.60270 NA NA
AwithA 0.68326 NA 0.64833 0.71623 NA NA
AlphawithBeta 0.36268 NA 0.31859 0.40646 NA NA
CwithC 0.63371 NA 0.59633 0.66909 NA NA
EwithE 0.38939 NA 0.26902 0.48346 NA NA
ESwithES 0.48275 NA 0.43702 0.52560 NA NA
IwithI 0.69599 NA 0.63675 0.75000 NA NA
Goodness-of-fit indices:
Value
Sample size 4496.0000
Chi-square of target model 65.4528
DF of target model 4.0000
p value of target model 0.0000
Number of constraints imposed on "Smatrix" 5.0000
DF manually adjusted 0.0000
Chi-square of independence model 3112.4037
DF of independence model 10.0000
RMSEA 0.0585
RMSEA lower 95% CI 0.0465
RMSEA upper 95% CI 0.0713
SRMR 0.0284
TLI 0.9505
CFI 0.9802
AIC 57.4528
BIC 31.8090
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)Fixed-effects TSSEM with cluster
Stage 1 analysis
There are 4 types of sample characteristics in the original cluster. We may group them into either younger or older samples.
#### Display the frequencies of "cluster"
table(Digman97$cluster)
Adolescents Children Mature adults Young adults
1 4 6 3 #### Fixed-effects TSSEM with several clusters
#### Create a variable for different cluster
#### Younger participants: Children and Adolescents
#### Older participants: others
clusters <- ifelse(Digman97$cluster %in% c("Children","Adolescents"),
yes="Younger participants", no="Older participants")
#### Show the clusters
clusters [1] "Younger participants" "Younger participants" "Younger participants"
[4] "Younger participants" "Younger participants" "Older participants"
[7] "Older participants" "Older participants" "Older participants"
[10] "Older participants" "Older participants" "Older participants"
[13] "Older participants" "Older participants" - We may conduct a fixed-effects model by specifying the
cluster=clustersargument. Fixed-effects TSSEM will be conducted according to the labels in theclusters. The goodness-of-fit indices of the Stage 1 analysis for the older and younger participants were \(\chi^2(80, N=3,658) = 823.88, p < .001\), CFI=0.7437, RMSEA=0.1513 and SRMR=0.1528, and \(\chi^2(40, N=838) = 344.18, p < .001\), CFI=0.7845, RMSEA=0.2131 and SRMR=0.1508, respectively.
## Example of Fixed-effects TSSEM with several clusters
cluster1 <- tssem1(Digman97$data, Digman97$n, method="FEM",
cluster=clusters)
summary(cluster1)$`Older participants`
Call:
tssem1FEM(my.df = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis,
model.name = model.name, suppressWarnings = suppressWarnings)
Coefficients:
Estimate Std.Error z value Pr(>|z|)
S[1,2] 0.297471 0.015436 19.2714 < 2.2e-16 ***
S[1,3] 0.370248 0.014533 25.4772 < 2.2e-16 ***
S[1,4] 0.137694 0.016403 8.3944 < 2.2e-16 ***
S[1,5] 0.098061 0.016724 5.8636 4.528e-09 ***
S[2,3] 0.393692 0.014146 27.8303 < 2.2e-16 ***
S[2,4] 0.183045 0.016055 11.4009 < 2.2e-16 ***
S[2,5] 0.092774 0.016643 5.5743 2.485e-08 ***
S[3,4] 0.260753 0.015554 16.7644 < 2.2e-16 ***
S[3,5] 0.096141 0.016574 5.8009 6.598e-09 ***
S[4,5] 0.411756 0.013900 29.6223 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 3658.0000
Chi-square of target model 825.9826
DF of target model 80.0000
p value of target model 0.0000
Chi-square of independence model 3000.9731
DF of independence model 90.0000
RMSEA 0.1515
RMSEA lower 95% CI 0.1424
RMSEA upper 95% CI 0.1611
SRMR 0.1459
TLI 0.7117
CFI 0.7437
AIC 665.9826
BIC 169.6088
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)
$`Younger participants`
Call:
tssem1FEM(my.df = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis,
model.name = model.name, suppressWarnings = suppressWarnings)
Coefficients:
Estimate Std.Error z value Pr(>|z|)
S[1,2] 0.604330 0.022125 27.3142 < 2.2e-16 ***
S[1,3] 0.465536 0.027493 16.9327 < 2.2e-16 ***
S[1,4] -0.031381 0.035939 -0.8732 0.38257
S[1,5] 0.061508 0.034547 1.7805 0.07500 .
S[2,3] 0.501432 0.026348 19.0311 < 2.2e-16 ***
S[2,4] -0.060678 0.034557 -1.7559 0.07911 .
S[2,5] 0.320987 0.031064 10.3330 < 2.2e-16 ***
S[3,4] 0.175437 0.033675 5.2097 1.891e-07 ***
S[3,5] 0.305149 0.031586 9.6610 < 2.2e-16 ***
S[4,5] 0.478640 0.026883 17.8045 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 838.0000
Chi-square of target model 346.2810
DF of target model 40.0000
p value of target model 0.0000
Chi-square of independence model 1470.4511
DF of independence model 50.0000
RMSEA 0.2139
RMSEA lower 95% CI 0.1939
RMSEA upper 95% CI 0.2355
SRMR 0.1411
TLI 0.7305
CFI 0.7844
AIC 266.2810
BIC 77.0402
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- The pooled correlation matrix for each cluster can be extracted by the use of the
coef()command.
coef(cluster1)$`Older participants`
A C ES E I
A 1.00000000 0.29747123 0.37024803 0.1376942 0.09806125
C 0.29747123 1.00000000 0.39369157 0.1830450 0.09277411
ES 0.37024803 0.39369157 1.00000000 0.2607530 0.09614072
E 0.13769424 0.18304500 0.26075304 1.0000000 0.41175622
I 0.09806125 0.09277411 0.09614072 0.4117562 1.00000000
$`Younger participants`
A C ES E I
A 1.00000000 0.6043300 0.4655359 -0.0313810 0.06150839
C 0.60433002 1.0000000 0.5014319 -0.0606784 0.32098713
ES 0.46553592 0.5014319 1.0000000 0.1754367 0.30514853
E -0.03138100 -0.0606784 0.1754367 1.0000000 0.47864004
I 0.06150839 0.3209871 0.3051485 0.4786400 1.00000000Stage 2 analysis
The goodness-of-fit indices of the Stage 2 analysis for the older and younger participants were \(\chi^2(4, N=3,658) = 21.92, p < .001\), CFI=0.9921, RMSEA=0.0350 and SRMR=0.0160, and \(\chi^2(4, N=838) = 144.87, p < .001\), CFI=0.9427, RMSEA=0.2051 and SRMR=0.1051, respectively.
cluster2 <- tssem2(cluster1, Amatrix=A1, Smatrix=S1, Fmatrix=F1,
diag.constraints=FALSE, intervals.type="z")
#### Please note that the estimates for the younger participants are problematic.
summary(cluster2)$`Older participants`
Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), asyCov = vcov.tssem1FEM(tssem1.obj),
n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix,
Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis,
intervals.type = intervals.type, mx.algebras = mx.algebras,
model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
AonAlpha 0.512657 0.018206 0.476973 0.548341 28.158 < 2.2e-16 ***
ConAlpha 0.549968 0.017946 0.514795 0.585140 30.646 < 2.2e-16 ***
EonBeta 0.967136 0.058752 0.851985 1.082287 16.461 < 2.2e-16 ***
ESonAlpha 0.732174 0.018929 0.695074 0.769273 38.680 < 2.2e-16 ***
IonBeta 0.430649 0.029634 0.372568 0.488730 14.532 < 2.2e-16 ***
AlphawithBeta 0.349236 0.028118 0.294125 0.404346 12.420 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 3658.0000
Chi-square of target model 21.9954
DF of target model 4.0000
p value of target model 0.0002
Number of constraints imposed on "Smatrix" 0.0000
DF manually adjusted 0.0000
Chi-square of independence model 2273.2284
DF of independence model 10.0000
RMSEA 0.0351
RMSEA lower 95% CI 0.0217
RMSEA upper 95% CI 0.0500
SRMR 0.0160
TLI 0.9801
CFI 0.9920
AIC 13.9954
BIC -10.8233
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)
$`Younger participants`
Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), asyCov = vcov.tssem1FEM(tssem1.obj),
n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix,
Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis,
intervals.type = intervals.type, mx.algebras = mx.algebras,
model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
AonAlpha 0.747648 0.023880 0.700843 0.794452 31.3082 <2e-16 ***
ConAlpha 0.911706 0.019864 0.872773 0.950639 45.8970 <2e-16 ***
EonBeta 0.152572 0.159108 -0.159275 0.464419 0.9589 0.3376
ESonAlpha 0.677437 0.025863 0.626746 0.728128 26.1931 <2e-16 ***
IonBeta 3.283639 3.362434 -3.306611 9.873889 0.9766 0.3288
AlphawithBeta 0.117264 0.125407 -0.128529 0.363057 0.9351 0.3498
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 838.0000
Chi-square of target model 145.6164
DF of target model 4.0000
p value of target model 0.0000
Number of constraints imposed on "Smatrix" 0.0000
DF manually adjusted 0.0000
Chi-square of independence model 2480.2902
DF of independence model 10.0000
RMSEA 0.2057
RMSEA lower 95% CI 0.1778
RMSEA upper 95% CI 0.2350
SRMR 0.1051
TLI 0.8567
CFI 0.9427
AIC 137.6164
BIC 118.6923
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)We may also plot the parameter estimates of these two groups.
library(semPlot)
## Convert the model to semPlotModel object with 2 plots
my.plots <- lapply(X=cluster2, FUN=meta2semPlot, latNames=c("Alpha","Beta"))
## Setup two plots
layout(t(1:2))
semPaths(my.plots[[1]], whatLabels="est", nCharNodes=10, color="green")
title("Younger")
semPaths(my.plots[[2]], whatLabels="est", nCharNodes=10, color="yellow")
title("Older")
Random-effects TSSEM
Stage 1 analysis
- Since there is not enough data to estimate the full variance component of the random effects, I estimate the variance component with a diagonal matrix in the
RE.type="Diag"argument. The range of \(I^2\) indices, the percentage of total variance that can be explained by the between study effect, are from .84 to .95.
#### Random-effects TSSEM with random effects on the diagonals
random1 <- tssem1(Digman97$data, Digman97$n, method="REM", RE.type="Diag")
summary(random1)
Call:
meta(y = ES, v = acovR, RE.constraints = Diag(x = paste(RE.startvalues,
"*Tau2_", 1:no.es, "_", 1:no.es, sep = "")), RE.lbound = RE.lbound,
I2 = I2, model.name = model.name, suppressWarnings = TRUE,
silent = silent, run = run)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept1 3.9460e-01 5.4212e-02 2.8835e-01 5.0085e-01 7.2789
Intercept2 4.4005e-01 4.1242e-02 3.5922e-01 5.2089e-01 10.6701
Intercept3 5.4511e-02 6.1731e-02 -6.6479e-02 1.7550e-01 0.8830
Intercept4 9.8646e-02 4.6203e-02 8.0894e-03 1.8920e-01 2.1350
Intercept5 4.2958e-01 4.0146e-02 3.5090e-01 5.0827e-01 10.7005
Intercept6 1.2852e-01 4.0839e-02 4.8474e-02 2.0856e-01 3.1469
Intercept7 2.0521e-01 4.9554e-02 1.0809e-01 3.0234e-01 4.1412
Intercept8 2.3988e-01 3.1927e-02 1.7731e-01 3.0246e-01 7.5134
Intercept9 1.8907e-01 4.2999e-02 1.0480e-01 2.7335e-01 4.3972
Intercept10 4.4403e-01 3.2524e-02 3.8029e-01 5.0778e-01 13.6524
Tau2_1_1 3.7222e-02 1.5000e-02 7.8227e-03 6.6622e-02 2.4815
Tau2_2_2 2.0313e-02 8.4336e-03 3.7839e-03 3.6843e-02 2.4086
Tau2_3_3 4.8286e-02 1.9739e-02 9.5976e-03 8.6974e-02 2.4462
Tau2_4_4 2.4630e-02 1.0625e-02 3.8052e-03 4.5455e-02 2.3181
Tau2_5_5 1.8744e-02 8.2473e-03 2.5795e-03 3.4908e-02 2.2727
Tau2_6_6 1.8317e-02 8.8017e-03 1.0662e-03 3.5568e-02 2.0811
Tau2_7_7 2.9412e-02 1.2252e-02 5.3990e-03 5.3425e-02 2.4006
Tau2_8_8 9.6827e-03 4.8934e-03 9.1679e-05 1.9274e-02 1.9787
Tau2_9_9 2.0951e-02 9.1286e-03 3.0590e-03 3.8842e-02 2.2951
Tau2_10_10 1.1156e-02 5.0453e-03 1.2673e-03 2.1045e-02 2.2111
Pr(>|z|)
Intercept1 3.366e-13 ***
Intercept2 < 2.2e-16 ***
Intercept3 0.37721
Intercept4 0.03276 *
Intercept5 < 2.2e-16 ***
Intercept6 0.00165 **
Intercept7 3.456e-05 ***
Intercept8 5.751e-14 ***
Intercept9 1.097e-05 ***
Intercept10 < 2.2e-16 ***
Tau2_1_1 0.01308 *
Tau2_2_2 0.01601 *
Tau2_3_3 0.01444 *
Tau2_4_4 0.02044 *
Tau2_5_5 0.02304 *
Tau2_6_6 0.03743 *
Tau2_7_7 0.01637 *
Tau2_8_8 0.04785 *
Tau2_9_9 0.02173 *
Tau2_10_10 0.02703 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 2390.085
Degrees of freedom of the Q statistic: 130
P value of the Q statistic: 0
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.9489
Intercept2: I2 (Q statistic) 0.9086
Intercept3: I2 (Q statistic) 0.9417
Intercept4: I2 (Q statistic) 0.8898
Intercept5: I2 (Q statistic) 0.9010
Intercept6: I2 (Q statistic) 0.8545
Intercept7: I2 (Q statistic) 0.9096
Intercept8: I2 (Q statistic) 0.7726
Intercept9: I2 (Q statistic) 0.8751
Intercept10: I2 (Q statistic) 0.8437
Number of studies (or clusters): 14
Number of observed statistics: 140
Number of estimated parameters: 20
Degrees of freedom: 120
-2 log likelihood: -110.8756
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)- The pooled correlation coefficients (fixed effects) and the variance components (the random effects) can be extracted by the use of the
coef()command via theselectargument.
## Fixed effects
coef(random1, select="fixed") Intercept1 Intercept2 Intercept3 Intercept4 Intercept5 Intercept6
0.39460162 0.44005280 0.05451102 0.09864569 0.42958489 0.12851613
Intercept7 Intercept8 Intercept9 Intercept10
0.20521180 0.23988099 0.18907356 0.44403450 ## Random effects
coef(random1, select="random") Tau2_1_1 Tau2_2_2 Tau2_3_3 Tau2_4_4 Tau2_5_5 Tau2_6_6
0.037222112 0.020313405 0.048285571 0.024629946 0.018743972 0.018317119
Tau2_7_7 Tau2_8_8 Tau2_9_9 Tau2_10_10
0.029412031 0.009682657 0.020950714 0.011155919 Stage 2 analysis
- The steps are exactly the same as those in the fixed-effects model. The chi-square test statistic of the Stage 2 analysis was \(\chi^2(4, N=4,496) = 8.51, p < .001\), CFI=0.9911, RMSEA=0.0158 and SRMR=0.0463.
random2 <- tssem2(random1, Amatrix=A1, Smatrix=S1, Fmatrix=F1,
diag.constraints=FALSE, intervals="LB")
summary(random2)
Call:
wls(Cov = pooledS, asyCov = asyCov, n = tssem1.obj$total.n, Amatrix = Amatrix,
Smatrix = Smatrix, Fmatrix = Fmatrix, diag.constraints = diag.constraints,
cor.analysis = cor.analysis, intervals.type = intervals.type,
mx.algebras = mx.algebras, model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
AonAlpha 0.57256 NA 0.47381 0.67679 NA NA
ConAlpha 0.59013 NA 0.49059 0.69479 NA NA
EonBeta 0.69329 NA 0.56260 0.84722 NA NA
ESonAlpha 0.77029 NA 0.65987 0.90394 NA NA
IonBeta 0.64008 NA 0.50848 0.78605 NA NA
AlphawithBeta 0.39366 NA 0.30248 0.49017 NA NA
Goodness-of-fit indices:
Value
Sample size 4496.0000
Chi-square of target model 8.5067
DF of target model 4.0000
p value of target model 0.0747
Number of constraints imposed on "Smatrix" 0.0000
DF manually adjusted 0.0000
Chi-square of independence model 515.0303
DF of independence model 10.0000
RMSEA 0.0158
RMSEA lower 95% CI 0.0000
RMSEA upper 95% CI 0.0308
SRMR 0.0463
TLI 0.9777
CFI 0.9911
AIC 0.5067
BIC -25.1371
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)- We may also plot the models with the labels or the parameter estimates.
library(semPlot)
## Convert the model to semPlotModel object
my.plot <- meta2semPlot(random2, latNames=c("Alpha","Beta"))
## Plot the model with labels
semPaths(my.plot, whatLabels="path", nCharEdges=10, nCharNodes=10, color="red")
## Plot the parameter estimates
semPaths(my.plot, whatLabels="est", nCharNodes=10, color="green")
Becker and Schram (1994)
Inspect the data
- We may list the first few cases of the data by using the
head()command.
#### Load the metaSEM library for TSSEM
library(metaSEM)
#### Inspect the data for inspection (not required for the analysis)
head(Becker94$data)$`Becker (1978) Females`
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.00 0.47 -0.21
Spatial 0.47 1.00 -0.15
SAT (Verbal) -0.21 -0.15 1.00
$`Becker (1978) Males`
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.00 0.28 0.19
Spatial 0.28 1.00 0.18
SAT (Verbal) 0.19 0.18 1.00
$`Berry (1957) Females`
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.00 0.48 0.41
Spatial 0.48 1.00 0.26
SAT (Verbal) 0.41 0.26 1.00
$`Berry (1957) Males`
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.00 0.37 0.40
Spatial 0.37 1.00 0.27
SAT (Verbal) 0.40 0.27 1.00
$`Rosenberg (1981) Females`
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.00 0.42 0.48
Spatial 0.42 1.00 0.23
SAT (Verbal) 0.48 0.23 1.00
$`Rosenberg (1981) Males`
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.00 0.41 0.74
Spatial 0.41 1.00 0.44
SAT (Verbal) 0.74 0.44 1.00head(Becker94$n)[1] 74 153 48 55 51 18Fixed-effects TSSEM
Stage 1 analysis
- The test statistic of the Stage 1 analysis based on the TSSEM approach was \(\chi^2(27, N=538) = 62.50, p < .001\), CFI=0.7943, RMSEA=0.1565 and SRMR=0.2011. Based on the test statistic and the goodness-of-fit indices, the hypothesis of homogeneity of correlation matrices was rejected.
#### Fixed-effects model
## Stage 1 analysis
fixed1 <- tssem1(Becker94$data, Becker94$n, method="FEM")
summary(fixed1)
Call:
tssem1FEM(my.df = my.df, n = n, cor.analysis = cor.analysis,
model.name = model.name, cluster = cluster, suppressWarnings = suppressWarnings,
silent = silent, run = run)
Coefficients:
Estimate Std.Error z value Pr(>|z|)
S[1,2] 0.379961 0.037123 10.2351 < 2.2e-16 ***
S[1,3] 0.334562 0.039947 8.3752 < 2.2e-16 ***
S[2,3] 0.176461 0.042334 4.1683 3.069e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 538.0000
Chi-square of target model 63.6553
DF of target model 27.0000
p value of target model 0.0001
Chi-square of independence model 207.7894
DF of independence model 30.0000
RMSEA 0.1590
RMSEA lower 95% CI 0.1096
RMSEA upper 95% CI 0.2117
SRMR 0.1586
TLI 0.7709
CFI 0.7938
AIC 9.6553
BIC -106.1169
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)coef(fixed1) SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.0000000 0.3799605 0.334562
Spatial 0.3799605 1.0000000 0.176461
SAT (Verbal) 0.3345620 0.1764610 1.000000Stage 2 analysis
We may specify the model via the RAM formulation. If all variables are observed, there is no need to specify the F matrix. Since the df of the regression model is 0, the proposed model always fits the data perfectly.
#### Prepare models for stage 2 analysis
## A2: asymmetric matrix (regression coefficients)
A2 <- create.mxMatrix(c(0,"0.2*Spatial2Math","0.2*Verbal2Math",
0,0,0,
0,0,0),
type="Full", ncol=3, nrow=3, as.mxMatrix=FALSE, byrow=TRUE)
dimnames(A2) <- list( c("Math", "Spatial", "Verbal"), c("Math", "Spatial", "Verbal") )
A2 Math Spatial Verbal
Math "0" "0.2*Spatial2Math" "0.2*Verbal2Math"
Spatial "0" "0" "0"
Verbal "0" "0" "0" ## S2: symmetric matrix (variance covariance matrix among variables)
S2 <- create.mxMatrix(c("0.2*ErrorVarMath",0,0,1,"0.2*SpatialCorVerbal",1),
type="Symm", as.mxMatrix=FALSE)
dimnames(S2) <- list( c("Math", "Spatial", "Verbal"), c("Math", "Spatial", "Verbal") )
S2 Math Spatial Verbal
Math "0.2*ErrorVarMath" "0" "0"
Spatial "0" "1" "0.2*SpatialCorVerbal"
Verbal "0" "0.2*SpatialCorVerbal" "1" ## Using lavaan's syntax
model2 <- "## Math is modeled by Spatial and Verbal
Math ~ Spatial2Math*Spatial + Verbal2Math*Verbal
## Variances of predictors are fixed at 1
Spatial ~~ 1*Spatial
Verbal ~~ 1*Verbal
## Correlation between the predictors
Spatial ~~ SpatialCorVerbal*Verbal
## Error variance
Math ~~ ErrorVarMath*Math"
RAM2 <- lavaan2RAM(model2)
RAM2$A
Math Spatial Verbal
Math "0" "0*Spatial2Math" "0*Verbal2Math"
Spatial "0" "0" "0"
Verbal "0" "0" "0"
$S
Math Spatial Verbal
Math "0*ErrorVarMath" "0" "0"
Spatial "0" "1" "0*SpatialCorVerbal"
Verbal "0" "0*SpatialCorVerbal" "1"
$F
Math Spatial Verbal
Math 1 0 0
Spatial 0 1 0
Verbal 0 0 1## Stage 2 analysis
fixed2 <- tssem2(fixed1, Amatrix=RAM2$A, Smatrix=RAM2$S, diag.constraints=TRUE,
intervals="LB")
summary(fixed2)
Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), asyCov = vcov.tssem1FEM(tssem1.obj),
n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix,
Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis,
intervals.type = intervals.type, mx.algebras = mx.algebras,
model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Spatial2Math 0.331238 NA 0.258216 0.404145 NA NA
Verbal2Math 0.276111 NA 0.199502 0.352776 NA NA
ErrorVarMath 0.781766 NA 0.714798 0.840626 NA NA
SpatialCorVerbal 0.176461 NA 0.093483 0.259434 NA NA
Goodness-of-fit indices:
Value
Sample size 538.00
Chi-square of target model 0.00
DF of target model 0.00
p value of target model 0.00
Number of constraints imposed on "Smatrix" 1.00
DF manually adjusted 0.00
Chi-square of independence model 160.47
DF of independence model 3.00
RMSEA 0.00
RMSEA lower 95% CI 0.00
RMSEA upper 95% CI 0.00
SRMR 0.00
TLI -Inf
CFI 1.00
AIC 0.00
BIC 0.00
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)Fixed-effects TSSEM with cluster
Stage 1 analysis
- The goodness-of-fit indices of the Stage 1 analysis for the female and male participants were \(\chi^2(12, N=235) = 42.41, p < .001\), CFI=0.7116, RMSEA=0.2327 and SRMR=0.2339, and \(\chi^2(12, N=303) = 16.13, p = .1852\), CFI=0.9385, RMSEA=0.0755 and SRMR=0.1508, respectively.
#### Fixed-effects model with cluster
## Stage 1 analysis
cluster1 <- tssem1(Becker94$data, Becker94$n, method="FEM", cluster=Becker94$gender)
summary(cluster1)$Females
Call:
tssem1FEM(my.df = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis,
model.name = model.name, suppressWarnings = suppressWarnings)
Coefficients:
Estimate Std.Error z value Pr(>|z|)
S[1,2] 0.455896 0.051993 8.7684 < 2.2e-16 ***
S[1,3] 0.341583 0.061944 5.5144 3.5e-08 ***
S[2,3] 0.171931 0.064731 2.6561 0.007905 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 235.0000
Chi-square of target model 43.1898
DF of target model 12.0000
p value of target model 0.0000
Chi-square of independence model 123.4399
DF of independence model 15.0000
RMSEA 0.2357
RMSEA lower 95% CI 0.1637
RMSEA upper 95% CI 0.3161
SRMR 0.2141
TLI 0.6405
CFI 0.7124
AIC 19.1898
BIC -22.3252
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)
$Males
Call:
tssem1FEM(my.df = data.cluster[[i]], n = n.cluster[[i]], cor.analysis = cor.analysis,
model.name = model.name, suppressWarnings = suppressWarnings)
Coefficients:
Estimate Std.Error z value Pr(>|z|)
S[1,2] 0.318051 0.051698 6.1521 7.646e-10 ***
S[1,3] 0.328286 0.052226 6.2859 3.261e-10 ***
S[2,3] 0.179549 0.055944 3.2094 0.00133 **
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Goodness-of-fit indices:
Value
Sample size 303.0000
Chi-square of target model 16.4819
DF of target model 12.0000
p value of target model 0.1701
Chi-square of independence model 84.3496
DF of independence model 15.0000
RMSEA 0.0786
RMSEA lower 95% CI 0.0000
RMSEA upper 95% CI 0.1643
SRMR 0.1025
TLI 0.9192
CFI 0.9354
AIC -7.5181
BIC -52.0829
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)coef(cluster1)$Females
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.0000000 0.4558958 0.3415826
Spatial 0.4558958 1.0000000 0.1719309
SAT (Verbal) 0.3415826 0.1719309 1.0000000
$Males
SAT (Math) Spatial SAT (Verbal)
SAT (Math) 1.0000000 0.3180507 0.3282856
Spatial 0.3180507 1.0000000 0.1795489
SAT (Verbal) 0.3282856 0.1795489 1.0000000Stage 2 analysis
## Stage 2 analysis
cluster2 <- tssem2(cluster1, Amatrix=A2, Smatrix=S2, diag.constraints=FALSE, intervals="LB")
summary(cluster2)$Females
Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), asyCov = vcov.tssem1FEM(tssem1.obj),
n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix,
Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis,
intervals.type = intervals.type, mx.algebras = mx.algebras,
model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Spatial2Math 0.40926 NA 0.30529 0.51254 NA NA
Verbal2Math 0.27122 NA 0.15613 0.38656 NA NA
SpatialCorVerbal 0.17193 NA 0.04506 0.29880 NA NA
Goodness-of-fit indices:
Value
Sample size 235.00
Chi-square of target model 0.00
DF of target model 0.00
p value of target model 0.00
Number of constraints imposed on "Smatrix" 0.00
DF manually adjusted 0.00
Chi-square of independence model 105.57
DF of independence model 3.00
RMSEA 0.00
RMSEA lower 95% CI 0.00
RMSEA upper 95% CI 0.00
SRMR 0.00
TLI -Inf
CFI 1.00
AIC 0.00
BIC 0.00
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)
$Males
Call:
wls(Cov = coef.tssem1FEM(tssem1.obj), asyCov = vcov.tssem1FEM(tssem1.obj),
n = sum(tssem1.obj$n), Amatrix = Amatrix, Smatrix = Smatrix,
Fmatrix = Fmatrix, diag.constraints = diag.constraints, cor.analysis = tssem1.obj$cor.analysis,
intervals.type = intervals.type, mx.algebras = mx.algebras,
model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Spatial2Math 0.267739 NA 0.166907 0.368546 NA NA
Verbal2Math 0.280213 NA 0.178024 0.382428 NA NA
SpatialCorVerbal 0.179549 NA 0.069891 0.289197 NA NA
Goodness-of-fit indices:
Value
Sample size 303.000
Chi-square of target model 0.000
DF of target model 0.000
p value of target model 0.000
Number of constraints imposed on "Smatrix" 0.000
DF manually adjusted 0.000
Chi-square of independence model 68.564
DF of independence model 3.000
RMSEA 0.000
RMSEA lower 95% CI 0.000
RMSEA upper 95% CI 0.000
SRMR 0.000
TLI -Inf
CFI 1.000
AIC 0.000
BIC 0.000
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)- We may also plot the parameter estimates of these two groups.
## Convert the model to semPlotModel object with 2 plots
my.plots <- lapply(X=cluster2, FUN=meta2semPlot)
## Setup two plots
layout(t(1:2))
semPaths(my.plots[[1]], whatLabels="est", nCharNodes=10, color="green")
title("Females")
semPaths(my.plots[[2]], whatLabels="est", nCharNodes=10, color="yellow")
title("Males")
Random-effects TSSEM
Stage 1 analysis
- The \(I^2\) indices for the correlations between spatial and math, verbal and math, and spatial and verbal are .00, .81 and .23, respectively.
#### Random-effects model
## Stage 1 analysis: A diagonal matrix for random effects
random1 <- tssem1(Becker94$data, Becker94$n, method="REM", RE.type="Diag")
summary(random1)
Call:
meta(y = ES, v = acovR, RE.constraints = Diag(x = paste(RE.startvalues,
"*Tau2_", 1:no.es, "_", 1:no.es, sep = "")), RE.lbound = RE.lbound,
I2 = I2, model.name = model.name, suppressWarnings = TRUE,
silent = silent, run = run)
95% confidence intervals: z statistic approximation
Coefficients:
Estimate Std.Error lbound ubound z value
Intercept1 3.7179e-01 3.6499e-02 3.0025e-01 4.4333e-01 10.1863
Intercept2 4.3173e-01 7.7451e-02 2.7993e-01 5.8354e-01 5.5743
Intercept3 2.0412e-01 4.6874e-02 1.1224e-01 2.9599e-01 4.3546
Tau2_1_1 1.0000e-10 4.7098e-03 -9.2311e-03 9.2311e-03 0.0000
Tau2_2_2 4.8060e-02 2.6340e-02 -3.5652e-03 9.9684e-02 1.8246
Tau2_3_3 5.8428e-03 9.8463e-03 -1.3456e-02 2.5141e-02 0.5934
Pr(>|z|)
Intercept1 < 2.2e-16 ***
Intercept2 2.486e-08 ***
Intercept3 1.333e-05 ***
Tau2_1_1 1.00000
Tau2_2_2 0.06806 .
Tau2_3_3 0.55291
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Q statistic on the homogeneity of effect sizes: 83.65808
Degrees of freedom of the Q statistic: 27
P value of the Q statistic: 1.041483e-07
Heterogeneity indices (based on the estimated Tau2):
Estimate
Intercept1: I2 (Q statistic) 0.000
Intercept2: I2 (Q statistic) 0.811
Intercept3: I2 (Q statistic) 0.254
Number of studies (or clusters): 10
Number of observed statistics: 30
Number of estimated parameters: 6
Degrees of freedom: 24
-2 log likelihood: -23.66615
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values may indicate problems.)coef(random1, select="fixed")Intercept1 Intercept2 Intercept3
0.3717903 0.4317347 0.2041156 coef(random1, select="random") Tau2_1_1 Tau2_2_2 Tau2_3_3
0.0000000001 0.0480595552 0.0058427867 Stage 2 analysis
## Stage 2 analysis
random2 <- tssem2(random1, Amatrix=A2, Smatrix=S2, diag.constraints=FALSE, intervals="LB")
summary(random2)
Call:
wls(Cov = pooledS, asyCov = asyCov, n = tssem1.obj$total.n, Amatrix = Amatrix,
Smatrix = Smatrix, Fmatrix = Fmatrix, diag.constraints = diag.constraints,
cor.analysis = cor.analysis, intervals.type = intervals.type,
mx.algebras = mx.algebras, model.name = model.name, suppressWarnings = suppressWarnings,
silent = silent, run = run)
95% confidence intervals: Likelihood-based statistic
Coefficients:
Estimate Std.Error lbound ubound z value Pr(>|z|)
Spatial2Math 0.29600 NA 0.21537 0.37418 NA NA
Verbal2Math 0.37132 NA 0.21244 0.52949 NA NA
SpatialCorVerbal 0.20412 NA 0.11224 0.29599 NA NA
Goodness-of-fit indices:
Value
Sample size 538.00
Chi-square of target model 0.00
DF of target model 0.00
p value of target model 0.00
Number of constraints imposed on "Smatrix" 0.00
DF manually adjusted 0.00
Chi-square of independence model 132.21
DF of independence model 3.00
RMSEA 0.00
RMSEA lower 95% CI 0.00
RMSEA upper 95% CI 0.00
SRMR 0.00
TLI -Inf
CFI 1.00
AIC 0.00
BIC 0.00
OpenMx status1: 0 ("0" or "1": The optimization is considered fine.
Other values indicate problems.)- We may also plot the models with the labels or the parameter estimates.
## Convert the model to semPlotModel object
my.plot <- meta2semPlot(random2)
## Plot the model with labels
semPaths(my.plot, whatLabels="path", nCharEdges=10, nCharNodes=10, color="red")
## Plot the parameter estimates
semPaths(my.plot, whatLabels="est", nCharNodes=10, color="green")
Installation and help
- First of most important, you need R to run it. You can install the
metaSEMpackage by running the following command inside anRsession:
install.packages("metaSEM")- The developmental version can be installed from GitHub by running:
library(devtools)
install_github("mikewlcheung/metasem")Potential issues with the OpenMx available at CRAN
- The OpenMx available at CRAN includes the open source
SLSQPandCSOLNPoptimizers. - If the
SLSQPoptimizer (default in themetaSEMpackage) does not work well for you, e.g., there are many error codes, you may try to rerun the analysis. For example,
random1 <- meta(y=yi, v=vi, data=Hox02)
random1 <- rerun(random1)
summary(random1)- If you still prefer the non-free
NPSOLoptimizer, you may install it from the OpenMx website and call it by running:
mxOption(NULL, "Default optimizer", "NPSOL")Help
- Reference manual
- Vignette
- If you need help,
OpenMx discussion forum: A discussion forum for the
metaSEMpackage inOpenMx. You may post technical questions related tometaSEMthere. Please include information on the session. It will be helpful if you can include a reproducible example. You may save a copy of your data, saymy.df, and attach the content ofmyData.Rin the post by usingsessionInfo() dump(c("my.df"), file="myData.R")Meta-analysis resources: A discussion forum related to the conceptual issues of meta-analytic structural equation modeling.
The files are based on the following versions of R and R packages:
sessionInfo()R version 3.3.2 (2016-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Linux Mint 18.1
locale:
[1] LC_CTYPE=en_SG.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_SG.UTF-8 LC_COLLATE=en_SG.UTF-8
[5] LC_MONETARY=en_SG.UTF-8 LC_MESSAGES=en_SG.UTF-8
[7] LC_PAPER=en_SG.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_SG.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods
[8] base
other attached packages:
[1] semPlot_1.0.1 metafor_1.9-9 metaSEM_0.9.11-8 OpenMx_2.7.4
[5] Rcpp_0.12.9 Matrix_1.2-7.1 MASS_7.3-45 digest_0.6.11
[9] knitr_1.15.1 rmarkdown_1.3
loaded via a namespace (and not attached):
[1] splines_3.3.2 ellipse_0.3-8 gtools_3.5.0
[4] network_1.13.0 Formula_1.2-1 assertthat_0.1
[7] stats4_3.3.2 latticeExtra_0.6-28 d3Network_0.5.2.1
[10] yaml_2.1.14 lisrelToR_0.1.4 pbivnorm_0.6.0
[13] backports_1.0.5 lattice_0.20-34 quantreg_5.29
[16] quadprog_1.5-5 RColorBrewer_1.1-2 checkmate_1.8.2
[19] ggm_2.3 minqa_1.2.4 colorspace_1.3-2
[22] htmltools_0.3.5 plyr_1.8.4 psych_1.6.12
[25] XML_3.98-1.5 SparseM_1.74 corpcor_1.6.8
[28] scales_0.4.1 whisker_0.3-2 glasso_1.8
[31] sna_2.4 jpeg_0.1-8 fdrtool_1.2.15
[34] lme4_1.1-12 MatrixModels_0.4-1 huge_1.2.7
[37] arm_1.9-3 tibble_1.2 htmlTable_1.8
[40] rockchalk_1.8.101 mgcv_1.8-16 car_2.1-4
[43] ggplot2_2.2.1 nnet_7.3-12 lazyeval_0.2.0
[46] pbkrtest_0.4-6 mnormt_1.5-5 survival_2.40-1
[49] magrittr_1.5 statnet.common_3.3.0 evaluate_0.10
[52] nlme_3.1-129 foreign_0.8-67 tools_3.3.2
[55] data.table_1.10.0 stringr_1.1.0 munsell_0.4.3
[58] cluster_2.0.5 sem_3.1-8 grid_3.3.2
[61] nloptr_1.0.4 rjson_0.2.15 igraph_1.0.1
[64] lavaan_0.5-22 base64enc_0.1-3 boot_1.3-18
[67] mi_1.0 gtable_0.2.0 abind_1.4-5
[70] reshape2_1.4.2 qgraph_1.4.1 gridExtra_2.2.1
[73] Hmisc_4.0-2 rprojroot_1.2 stringi_1.1.2
[76] matrixcalc_1.0-3 rpart_4.1-10 acepack_1.4.1
[79] png_0.1-7 coda_0.19-1