Tải bản đầy đủ
8 Example 2: Evaluating the Efficacy of a Treatment

8 Example 2: Evaluating the Efficacy of a Treatment

Tải bản đầy đủ

570

â†œæ¸€å±®

â†œæ¸€å±®

Hierarchical Linear Modeling

small-scale cluster-randomized trial where clusters or groups (not individuals) have
been randomly assigned to experimental conditions and scores for an outcome and
covariate have been collected from participants. Note that the two counseling methods
do not constitute a separate level as method is a fixed factor that describes the clusters,
as the counseling method conditions do not represent a sample from some larger population of possible counseling methods. Even if they did, two levels would be much too
small to serve as the upper level of a multilevel model. Thus, this cluster randomized
trial is a two-level nested design, with clients (level 1) nested within clusters (level 2).
Counseling method is a fixed level-2 (cluster-level) variable.
Given the relatively small number of observations in the data set, we present the following data set. Shown are the client id, the cluster id, client empathy (which is the
outcome of interest), client scores on a measure of contentment (which is intended to
serve as a covariate), and counseling method (method) employed in the relevant clusters coded either as 0 for the new treatment or 1 for control.
Note that in the online data set, group- and grand-mean centered forms of contentment
are present, labeled respectively, groupcontent and grandcontent, as well as meancontent, which was obtained by computing the cluster means for the contentment variable.
ClientId
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

Cluid

Empathy

Contentment

Method

1
1
1
1
2
2
2
2
3
3
3
3
4
4
4
4
5
5
5
5
6
6
6

23
22
20
19
16
17
18
19
25
28
29
31
27
23
22
21
32
31
28
26
13
12
14

33
33
27
25
22
21
28
31
28
38
35
34
38
27
28
25
28
37
33
30
27
22
34

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
1
1

Chapter 13

ClientId
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

â†œæ¸€å±®

Cluid

Empathy

Contentment

Method

6
7
7
7
7
8
8
8
8
9
9
9
9
10
10
10
10

15
16
17
14
12
11
10
20
15
21
18
19
23
18
17
16
23

28
30
37
27
25
28
23
34
33
29
31
30
39
27
36
36
32

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

â†œæ¸€å±®

TableÂ€13.11 shows some basic descriptive statistics for each counseling method based
on the client scores (without regard to cluster, as will be considered in the multilevel
analysis). Inspecting TableÂ€13.11 indicates that mean empathy is greater by about 6.5
points for the new treatment condition, the two treatment groups have similar mean
scores on contentment (which is expected due to the random assignment of clusters),
and that variability for each variable is similar across the two methods.
Due to the limited number of clusters and participants in this example, statistical power
to detect treatment effects will, in general, not be sufficient unless there are large treatment effects. So, while we will include contentment as a covariate shortly, we first estimate a multilevel model with only method included. Note that a null model (i.e., with
no predictors) could also be estimated but our presentation here focuses on treatment
effects. The client- or level-1 modelÂ€is
empathyij = β0 j + rij , (24)
where the outcome empathy is modeled as a function of a cluster intercept and residual
term rij where rij ~ N(0, σ2). With no predictor in EquationÂ€24, β0j represents a given
cluster j’s empathy mean. The cluster-level model, which includes the dummy-coded
method predictor,Â€is
β0 j = γ 00 + γ 01 (method ) + u0 j , (25)

571

572

â†œæ¸€å±®

â†œæ¸€å±®

Hierarchical Linear Modeling

 Table 13.11:â•‡ Descriptive Statistics for the Study Variables
Empathy

Method
New treatment (n = 20)
Control (n = 20)

Contentment

M

SD

M

SD

23.85
17.40

4.96
4.95

30.05
30.40

5.00
4.69

where γ00, given the coding for method, represents the empathy mean for the new treatment condition, and γ01 represents the difference in empathy means for the two treatment conditions. The residuals are assumed to be normally distributed, with a mean
of zero, and have homogeneous variance, or u0j ~ N(0, τ00). When we estimated this
model, we found that γ00 is estimated to be 23.85 (which is the same as in TableÂ€13.11
due to the design being completely balanced) and that the estimate for γ01 is −6.45
(SE = 3.02, pÂ€=Â€.065). Thus, using an alpha of .05, we could not conclude that the difference of about 6.5 points, which favors the new treatment condition, is statistically
significant. The nonsignificance is somewhat expected given the small sample size in
this study.
To improve statistical power, we now consider the covariate contentment. This predictor is at the client level and so it is possible that the within-cluster and between-cluster
associations between contentment and empathy differ. If so, including both the client
and mean form of this covariate may provide for greater power than may be obtained
by just adding the client-level predictor alone. So, for now, we include client contentment and cluster mean contentment. With group-mean centered contentment, the
client-level model becomes
empathyij = β0 j + β1 j contentmentij + rij . 

(26)

With group mean centering, β0j remains a given clusters j’s empathy mean and β1j
captures the within-cluster association between empathy and contentment. Since adding the group-mean centered contentment will not explain any variance at the cluster level, we now include mean contentment (uncentered) in the cluster-level model,
which isÂ€now

(

)

β0 j = γ 00 + γ 01 method j + γ 02 meancontent j + u0 j
. (27)

β1 j = γ 10
Note that the within-cluster slope (β1j) is specified as a fixed effect at the cluster level,
as its variance (τ11) is assumed to be zero, an assumption we will check shortly. The
combined equation isÂ€then
empathyij = γ 00 + γ 01method j + γ 02 meancontent j + γ 10 contentment j + u0 j + rij(28)

Chapter 13

â†œæ¸€å±®

â†œæ¸€å±®

Thus, the fixed effects of interest are γ01, which represents the difference in empathy
means between the two treatment conditions, controlling for mean contentment, γ02,
which represents the change in mean empathy given a unit increase in mean contentment, holding treatment condition constant, and γ10 is the within-cluster association
between empathy and contentment.
TableÂ€13.12 presents SAS and SPSS syntax that was used to estimate EquationÂ€28,
and TableÂ€13.13 reports the SAS results (as results obtained using SPSS were similar).
Focusing on the parameter of interest, the treatment effect estimate of −7.03 (p < .001)
indicates that after adjusting for differences in mean contentment, the new treatment
mean is about 7 points greater than the control mean, with this difference being statistically significant. Note that by including the contentment variables, the standard
error of the treatment effect is now 1.25, compared to 3.02 in the model without any
covariates, with this power increase due to adding these covariates. Note that both
client contentment and mean contentment are positively related to empathy. Also, the
difference in these latter coefficients, γ02 − γ10Â€=Â€1.66 − .33Â€=Â€1.33, is indicative of a
contextual effect (which can be tested for significance if desired).
If desired, we can compute adjusted means for the two counseling conditions by combining parameter estimates, covariate means (zero for contentment and 30.23 for mean
contentment) and the dummy codes for method using EquationÂ€28, while inserting
means (zeros) for the random effects. So, to compute the adjusted mean empathy for
the control group, the computation is −26.15 − 7.03(1) + 1.66(30.23)Â€=Â€17.00. For the
new treatment, the adjusted empathy mean is −26.15 − 7.03(0) + 1.66(30.23)Â€=Â€24.03.
This difference, 17.0 − 24.3= −7.03, is the treatment effect estimate, of course, and has
already been found to be statistically significant.
Although the analysis is largely concluded, we estimate a couple of models, the first to
check for the possibility of variable within-cluster slopes and the second to compare
the results of the previous model with those obtained by using grand-mean centering
for client contentment (without inclusion of mean contentment). Testing for variable
slopes (β1j of EquationÂ€26) is of interest for two reasons. First, finding such variation
would be of interest for those who hypothesize that the treatment may interact with client contentment, as it may be hypothesized that clients experiencing the new treatment
will have relatively high empathy regardless of their prior level of contentment. As such,
within-cluster slopes in the new treatment condition may be much flatter or smaller than
the positive association obtained in the previous analysis (i.e., γ10Â€=Â€.33). Observing variation in these slopes, although not a prerequisite for testing such an interaction, suggests
the possibility of such an interaction. In addition, the standard error of the treatment
effect, as previously estimated, may be misestimated if slope variation were present,
so including such variation may provide for more accurate inference for the treatment effect. To test for the possibility that the within-cluster or client-level association
between empathy and contentment varies across clusters, we estimated EquationsÂ€26
and 27, except that we modified the cluster-level equation, keeping EquationÂ€27 as is for
β0j but including a residual term for the slope that so the slope equation is β1jÂ€=Â€γ10 + u1j.

573

574

â†œæ¸€å±®

â†œæ¸€å±®

Hierarchical Linear Modeling

 Table 13.12:â•‡ SAS and SPSS Control Lines for Estimating EquationÂ€28
SAS

SPSS

PROC MIXED METHODÂ€=Â€REML NOCLPRINT
MIXED empathy WITH method groupcon(1) 
COVTEST NOITPRINT;
tent meancontent
/FIXED= method groupcontent mean
CLASS cluid;
MODEL empathyÂ€=Â€method groupcontent
content | SSTYPE(3)
(1) 
meancontent / ddfm=kenwardroger
/METHOD=REML
SOLUTION;
/PRINT=G SOLUTION TESTCOV
/RANDOM=INTERCEPT | SUBJECT(cluid)

RANDOM intercept / typeÂ€=Â€vc SUBJECT
COVTYPE(VC).
=cluid;
RUN;

(1) In the MODEL (SAS) and MIXED (SPSS) statements, the variable groupcontent is the within-cluster
centered client contentment variable and meancontent is the cluster mean contentment variable. Also for
SAS, the ddfmÂ€=Â€kenwardroger option requests that the denominator degrees of freedom for fixed
effect tests be calculated using the Kenward-Roger method. SPSS MIXED does not offer this option but by default uses the Satterthwaite method to compute these degrees of freedom. Each of these methods is intended
to provide for better inference when sample size is small.

When we estimated EquationsÂ€26 and 27 but now allowing for variable slopes,
we initially requested estimates for a full variance-covariance matrix for the cluster random effects, which includes estimates of the intercept variance (τ00), slope
variance (τ11), and the covariance (τ01) of the random effects. However, the estimated model did not converge (for both SAS and SPSS), which is often indicative of variance-covariance components that are near zero. We then estimated the
same model but constrained the covariance (τ01) to zero. This can be done in SAS
by replacing the RANDOM line that appears in TableÂ€13.12 with the statement
RANDOM intercept groupcontent / typeÂ€=Â€vc SUBJECT=cluid; and in SPSS by
replacing the RANDOM statement with /RANDOMÂ€=Â€INTERCEPT groupcontent|
SUBJECT(cluid) COVTYPE(VC).
When this was done, convergence was attained, and the estimate of the slope variance
τ11 is .002 (SEÂ€=Â€.04, pÂ€=Â€.48), suggesting no variation in slopes. Of course, this p value
is obtained from the z test, and we know that deviance testing is preferred over the z
test for variances. Note that EquationsÂ€26 and 27 are nested in the current equations
because EquationsÂ€26 and 27 are identical to the current equations except that the slope
variance is constrained to be zero. The deviance associated with EquationsÂ€26 and 27,
as shown in TableÂ€13.13, is 183.000 and the deviance for the variable slope model is
also 183.000. We can readily see that there is no improvement in fit by allowing for
slope variation. Formally, we would compare this difference in fit (here, zero) to a
corresponding chi-square critical value of 2.706, again doubling the alpha of .05 given
we are testing a single variance with 1 degree of freedom. So, there is no support for
variable slopes. Note that a conventional chi-square critical value can be used here

Chapter 13

â†œæ¸€å±®

â†œæ¸€å±®

 Table 13.13: SAS Output for EquationÂ€28 (or Equivalently EquationsÂ€26 andÂ€27)
Fit Statistics
-2 Res Log Likelihood
AIC (smaller is better)
AICC (smaller is better)
BIC (smaller is better)

183.0
187.0
187.3
187.6
Solution for Fixed Effects

Effect
Intercept
METHOD
groupCONTENT
MEANCONTENT

Estimate

Standard
Error

-26.1484
-7.0323
0.3274
1.6638

7.9531
1.2483
0.08212
0.2630

DF
7
7
29
7

tÂ€Value
-3.29
-5.63
3.99
6.33

Pr > |t|
0.0133
0.0008
0.0004
0.0004

Covariance Parameter Estimates
Cov Parm
Intercept
Residual

Subject

Â€

CLUID

Estimate
2.7454
4.5164

Standard
Error
2.0921
1.1861

Z Value
1.31
3.81

Pr > Z
0.0947
<.0001

Note: Predictor variable groupCONTENT is the group-mean centered client contentment variable and MEANCONTENT is the cluster mean contentment variable.

because we are testing one parameter (i.e., τ11), as opposed to the two parameters (i.e.,
τ11 and τ01) that were tested in sectionÂ€13.5.3.
Finally, we might wonder whether using grand-mean centering and excluding mean
content would provide for a more powerful analysis than obtained by EquationÂ€28. To
test this idea, we replaced the group-mean centered client contentment in EquationÂ€28
with grand-mean centered contentment (referred to as grandcontent in the online data
set) and removed mean contentment from the model. Recall that a grand-mean centered
level-1 variable can explain variation in an outcome at level 2, while a group-mean
centered level-1 predictor cannot. Further, by not including the mean of the predictor
in the grand-mean centered model, we could potentially increase the power for the test
of the treatment effect because the degrees of freedom for this effect are larger (providing a lower critical value) with the omission of the variable. When we estimated
this new grand-mean centered model, the treatment effect estimate (−6.6) was somewhat different than that obtained with EquationÂ€28, and the standard error was larger
(2.48). Given this larger standard error, there is no advantage to using this grand-mean
centered model. Also, EquationÂ€28 is arguably a better model because it provides for
valid estimates of the within- and between-cluster associations of empathy and contentment when a contextual effect is present, whereas the grand-mean centered model
just described blends these effects.

575

576

â†œæ¸€å±®

â†œæ¸€å±®

Hierarchical Linear Modeling

13.9â•‡SUMMARY
In this chapter, we provided an introduction to multilevel modeling as well as the use of
SAS and SPSS to estimate model parameters for a two-level cross-sectional design. It
should be noted that while it is relatively easy to use software to estimate model parameters, it is more challenging to understand the model being estimated, which is necessary, of course, to properly interpret the resulting parameter estimates and associated
significance tests. Examining the equations for multilevel models in both forms, that
is, equations expressed separately for each level and the combined equation, is helpful
for understanding the effects that are being estimated. In addition, graphical displays of
results, particularly for interactions, helps you achieve and convey understanding of study
findings. It is also helpful to recognize that the fixed effects in such models are essentially
regression coefficients. It is the random effects and their associated variance-covariance
components that may be initially challenging to understand. Further, while not demonstrated in this chapter, because this is an introductory treatment, residuals can be estimated to allow for an examination of statistical assumptions. As in any analysis, one
should attempt to determine if the assumptions of the procedure are reasonably satisfied,
whether outlying and influential observation are present, and whether important interactions or nonlinear associations have been left out of the model.
Consulting multilevel modeling texts, many of which were cited in this chapter, will
you with additional worked examples, fuller descriptions of the estimation processes
used, as well as other important multilevel modeling techniques. These include models for growth across time, dichotomous or ordinal outcomes, multivariate outcomes,
meta-analysis, and use with more complicated data structures, such as those with three
or more levels, cross-classification, and multiple membership, each involving multiple
random effects. You should also be aware that in addition to SAS and SPSS, several other software programs can be used to estimate multilevel models including, for
example, HLM (Raudenbush, Bryk, Cheong, Congdon Jr.,Â€& du Toit, 2011), MLwiN
(Rasbash, Browne, Healy, Cameron,Â€& Charlton, 2012), Mplus (MuthénÂ€& Muthén,
1998–2013), and R (R Development Core Team, 2014).
We hope that you continue learning about multilevel modeling, as this technique is
being increasingly applied to a wide variety of research designs.
REFERENCES
Arnold, C.â•›L. (1992). An introduction to hierarchical linear models. Measurement and Evaluation in Counseling and Development, 25, 58–90.
Bell, B.â•›A., Morgan, G.â•›B., Schoeneberger, J.â•›A., Kromrey, J.â•›D.,Â€& Ferron, J.â•›M. (2014). How
low can you go? An investigation of the influence of sample size and model complexity on
point and interval estimates in two-level linear models. Methodology, 10, 1–11.
Burstein, L. (1980). The analysis of multilevel data in educational research and evaluation.
Review of Research in Education, 8, 158–233.

Chapter 13

â†œæ¸€å±®

â†œæ¸€å±®

Enders, C.â•›K.,Â€& Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel
models: AÂ€new look at an old issue. Psychological Methods, 12, 121–138.
Heck, R.â•›H., Thomas, S.â•›L.,Â€& Tabata, L.â•›N. (2014). Multilevel and longitudinal modeling with
IBM SPSS (2nd ed.). New York, NY: Routledge.
Hedges, L.â•›
V.,Â€& Hedberg, E.â•›C. (2007). Intraclass correlation values for planning
group-randomized trials in education. Educational Evaluation and Policy Analysis, 29,
60–87.
Hox, J.â•›J. (2010). Multilevel analysis: Techniques and applications (2nd ed.). New York, NY:
Routledge.
Kenny, D.,Â€& Judd, C. (1986). Consequences of violating the independent assumption in
analysis of variance. Psychological Bulletin, 99, 422–431.
Kreft, I.G.G. (1996). Are multilevel techniques necessary? An overview, including simulation
studies. Unpublished report, California State University, Los Angeles. Available at http://
www.eric.ed.gov
Kreft, I.,Â€& de Leeuw, J. (1998). Introducing multilevel modeling. Thousand Oaks, CA:Â€Sage.
Mathieu, J.â•›E., Aguinis, H., Culpepper, S.â•›A.,Â€& Chen, G. (2012). Understanding and estimating the power to detect cross-level interaction effects in multilevel modeling. Journal of
Applied Psychology, 97, 951–966.
Muthén, L.â•›K.,Â€& Muthén, B.â•›O. (1998–2013). Mplus user’s guide (7th ed.). Los Angeles, CA:
Author.
Rasbash, J., Browne, W.â•›J., Healy, M., Cameron, B.,Â€& Charlton, C. (2012). MLwiN Version
2.25. Bristol, England: Centre for Multilevel Modelling, University of Bristol.
Raudenbush, S.,Â€& Bryk, A.â•›S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA:Â€Sage.
Raudenbush, S.â•›
W., Bryk, A.â•›
S., Cheong, Y.â•›
F., Congdon, R.â•›T., Jr.,Â€& du Toit, M. (2011).
HLM 7: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software
International.
R Development Core Team. (2014). R: AÂ€language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Retrieved from http://
www.R-project.org/
Scariano, S.,Â€& Davenport, J. (1987). The effects of violations of the independence assumption in the one way ANOVA. American Statistician, 41, 123–129.
Snijders, T.A.B.,Â€& Bosker, R.â•›J. (2012). Multilevel analysis: An introduction to basic and
advanced multilevel modeling (2nd ed.). Los Angeles, CA:Â€Sage.
Spybrook, J.,Â€& Raudenbush, S.â•›W. (2009). An examination of the precision and technical
accuracy of the first wave of group-randomized trials funded by the Institute of Education
Sciences. Educational Evaluation and Policy Analysis, 31, 298–318.
Van Breukelen, G.,Â€& Moorbeek, M. (2013). Design considerations in multilevel studies. In
M.â•›A. Scott, J.â•›S. Simonoff,Â€& B.â•›D. Marx (Eds.), The SAGE handbook of multilevel modelÂ�ing
(pp.Â€183–199). Thousand Oaks, CA:Â€Sage.
West, B.â•›T., Welch, K.â•›B.,Â€& Galecki, A.â•›T. (2014). Linear mixed models: AÂ€practical guide using
statistical software (2nd ed.). New York, NY: CRC Press.

577

Chapter 14

MULTIVARIATE MULTILEVEL
MODELING
14.1â•‡INTRODUCTION
Previous chapters in this text have addressed the use of multivariate analysis of variance (MANOVA) and hierarchical linear modeling or, more generally, multilevel modeling. Traditional applications of these procedures have limitations that restrict their
use. In particular, standard use of MANOVA assumes that responses of individuals are
independently distributed, an assumption that may be violated when participants are
nested in organizations or settings (such as students nested in schools, clients nested
in therapists, workers nested in workplaces). When such dependence is present, use of
MANOVA may result in unacceptably high type IÂ€error rates associated with the effects
of explanatory variables, as detailed in ChapterÂ€6. For its part, multilevel modeling
accommodates the dependence arising from such clustered data that MANOVA does
not. However, standard multilevel modeling is able to incorporate only one dependent
variable from units, often participants, at the lower level. Thus, such use of multilevel
modeling is not able to take advantage of the benefits associated with multivariate
analysis that have been described previously in thisÂ€book.
An extension of traditional MANOVA and multilevel analysis, multivariate multilevel
modeling (MVMM) can accommodate dependence of responses that results from the
nesting of participants in settings while simultaneously modeling multiple outcomes.
More generally, MVMM may be employed in a variety of research designs that involve
repeated measures analysis, multivariate growth curve modeling, multilevel structural
equation modeling, and multilevel mediation analysis. MVMM also shares key features of models where items comprise the lowest level of the data structure, such as
with applications of multilevel item response theory and those where researchers wish
to form an overall scale using responses, for example, from several survey items. As
such, MVMM can be viewed as a gateway technique to other advanced applications
that enable investigators to address a wide range of research questions.
This chapter focuses on some basic applications of multivariate multilevel modeling where multiple outcomes have been collected from individuals. After presenting

CHaPter 14

â†œæ¸€å±®

â†œæ¸€å±®

motivation for using this multivariate procedure, we explain the format of the data
required to conduct MVMM and show how the SAS and SPSS software programs
can reorganize data into the needed format. We then show how standard multilevel
models can be modified to include multiple outcome variables, where scores for these
variables have been collected from individuals. We then present a research example
with simulated data that we use to illustrate two sets of analyses. The first set of analyses, with two-level models, is designed to ease you into MVMM but also to show that
MVMM can replicate the results produced by standard MANOVA when no organizational nesting is present. This is important because an investigator may wish to use
MVMM instead of MANOVA in such a design because of the ability of MVMM to
include individuals in the analysis who have some missing data on the outcomes and
to readily test for the equivalence of effects. In the second set, various three-level analyses using MVMM are conducted, with multiple outcomes nested within students who
are nested in schools. In these analyses, we show how covariates and interactions can
be modeled when multiple outcomes are present in a multilevel design.
14.2â•‡BENEFITS OF CONDUCTING A MULTIVARIATE MULTILEVEL
ANALYSIS
When data are collected on multiple outcomes, researchers have a choice to conduct
univariate or multivariate analysis. As stated earlier in the text, one reason for considering a multivariate analysis is to help guard against the inflation of the overall type
IÂ€error rate by using an initial global multivariate test as a protected testing approach.
AÂ€second reason is that instead of examining univariate group differences using a total
score, obtained by summing or averaging scores across multiple subtests, investigators can compare group differences on the multiple subtests, which may provide more
insight into the nature of group differences.
These advantages for multivariate analysis are also applicable to MVMM. However,
1. The MVMM approach does not require that a participant provide scores for each
dependent variable. Rather, if a participant provides a score for at least one of the
dependent variables, that participant may be included in the analysis. Thus, compared to the standard MANOVA approach, MVMM makes greater use of available
data, which may provide for increased power. Further, SAS and SPSS provide
maximum likelihood treatment of missing data for MVMM, which we noted in
ChapterÂ€1, provides for optimal estimates of parameters when the missing data
mechanism is Missing Completely at Random (MCAR) or Missing at Random
(MAR).
2. Snijders and Bosker (2012) note that use of MVMM may result in smaller standard errors for the tests of predictors on a given outcome compared to a univariate analysis. They note that the additional precision and increase in power for
the multivariate approach may be substantial when the dependent variables are

579

580

â†œæ¸€å±®

â†œæ¸€å±®

Multivariate Multilevel Modeling

more highly correlated and participants have missing data on some of the outcome
variables.
3. When the dependent variables are similarly scaled, MVMM can be used to test
whether the effects of an explanatory variable are the same or differ across the
multiple outcomes. In an experimental setting, for example, an investigator may
learn if treatment effects are stronger for some outcomes than others, which may
suggest revising the nature and/or implementation of the intervention.
4. When participants are clustered in organizations, MVMM can be used to describe
the associations between the outcome variables at the participant and cluster levels
due to the partitioning of variability that is obtained with multilevel modeling.
Instead of learning about how scores for a single outcome vary across participants
and clusters, as with traditional multilevel modeling, MVMM can inform investigators of the associations between outcome variables that are within and between
clusters.
Of course, MVMM is a more complicated analysis procedure compared to univariate
analysis. As such, instead of proceeding immediately into an analysis with MVMM,
an investigator may wish to conduct preliminary analysis using one outcome at a time
in order to obtain an initial understanding of how a given outcome is related to the
explanatory variables of interest. Once that is attained, MVMM could be conducted to
make use potentially of a greater number of observations, provide the formal significance testing needed for the study, and decompose the correlations among outcomes at
the participant and cluster (or other) levels.
14.3â•‡ RESEARCH EXAMPLE
This chapter presents two sets of illustrative analyses involving MVMM that each
use the same hypothetical research example. In this example, we suppose a study is
being conducted to assess the effectiveness of a new component of an existing health
curriculum that is being introduced to fifth graders in a large school district. The new
component, delivered by a computer-based type of game, focuses on nutrition education. The program is intended to complement the regular health curriculum but, due to
its perhaps more engaging delivery, is expected to impart greater knowledge of proper
nutrition and motivation for adhering to a healthier diet. Ultimately, the goal of the
intervention is that students will begin (or continue) a lifetime habit of proper nutrition. Each set of analyses will focus on estimating and testing treatment effects for the
multiple outcome variables.
In order to minimize potential contamination between students in the same school,
the researchers have selected a cluster randomized trial where schools are randomly
assigned to the new computer-based instruction or regular nutrition education as provided in the existing curriculum. The researchers, we suppose, were able to recruit 40
elementary schools and randomly assigned 20 schools to each condition. To simplify
the presentation, only one class per school was selected to be included in the study.