3 Power of Test, Mean Degree and Sample Size
Tải bản đầy đủ - 0trang
42
S. Farber et al.
Fig. 4 The impact of mean degree on small networks
Fig. 5 The impact of mean degree on large networks
degree to sample size, the denser the weight matrix and the weaker the power. The
key question is how much so. Figures 4 and 5 show the differences in test power
between networks with the largest and smallest mean degree and sample size. Interestingly, the figures suggest that sample size and mean degree interact and impact
test strength in a variety of ways. The first observation is that the power of the LR test
Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models
43
for SAR models on networks with exponentially distributed degrees is quite robust
to changes in mean degree except for 0:01 < < 0:1. The LR test for the SAR
model using a Poisson distributed degree function appears to be the most sensitive to
increases in mean degree, especially in the range of 0:01 < < 0:3 when the test is
at least 30 and up to 90 percentage points weaker in more connected networks. The
shapes of the curves for the SEM specifications differ substantially to those for the
SAR specifications. While the SEM-Poisson combination produces slightly weaker
LR strengths than the SEM-Exponential tests, the most striking observation is that
both SEM curves are right-shifted indicating a relative immunity to mean degree in
the lower ranges of , and a vulnerability to degree in the higher investigated ranges
of the spatial parameter (0:2 < < 0:5).
The curves in Fig. 5 can be used to determine the degree to which increasing
sample size mitigates the impact of mean degree on test strength. Since increasing
sample size decreases weight matrix density for any given mean degree, we expect
to find the curves in Fig. 5 to be flatter than those in Fig. 4. It is clearly seen that
the effect of increasing sample size is an overall scale reduction in test weakness
and a left-shift of the curves’ minima. While the curves in Fig. 4 appear clustered
by model specification (SEM versus SAR curves), those in Fig. 5 either appear clustered by degree distribution function (Poisson versus exponential) or not clustered at
all; specifically, the test strengths on Poisson networks are more negatively impacted
by mean degree than for exponential networks.
4.4 Power of Test and Clustering
Results from Farber et al. (2009) indicate that the clustering coefficient, a measure of overall transitivity, does not impact test strength for SAR models using a
Poisson degree distribution function. As seen in Fig. 6, the impact of clustering is
Fig. 6 The impact of clustering on rejection frequency
44
S. Farber et al.
negligible on SAR models with both distribution function specifications. Whereas
clustering does not have a discernable impact on tests for SAR dependence, there
does seem to be a weak positive relationship between clustering and test strength
for SEM dependence. To date there are no analytically driven results that would
predict these simulation results, and these observations necessitate the investigation of the likelihood functions with respect to clustering before they can be fully
understood.
4.5 Power of Test and Matrix Density
Recent work has identified matrix density to be an important determinant of estimation bias in spatial and network regression models (Smith 2009). Given a binary
weight matrix W of size n x n, density is calculated as:
P P
DD
i
n.n
j
wij
1/
which is simply the sum of all the entries divided by the total number of possible
entries.
Figure 7 shows the relationship between average test power and matrix density
for each combination of model and degree distribution function. Each data point on
Fig. 7 The impact of matrix density on rejection frequency
Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models
45
the vertical axis is the LR rejection rate averaged over all and C for each sample size and mean degree. The data points are clustered along the horizontal access
due to the three disjoint sample-sizes used in the simulations. It appears that test
strength generally declined with matrix density for all four data series. In support
of the findings above, the LR tests are strongest for the SAR-exponential case and
are quite insensitive to changes in matrix density. While the curve for SAR-Poisson
is shifted upward from the SEM curves, the average rate of decline among the three
curves appears to be equivalent. Notice that the exponential matrices contain fewer
highly connected nodes in comparison to their Poisson distributed counterparts with
similar mean degree and this may be one reason for improved LR test strengths for
exponential network specifications.
4.6 Estimation Bias
Theoretically, one would expect the likelihood ratio tests to fail to reject the null
hypothesis when the estimated parameter is downward biased, as was shown
to occur in dense networks by Smith (2009) and Mizruchi and Neuman (2008).
However, neither Smith nor Mizruchi and Neuman investigate the possible impact
of network structure (besides link density) on estimation bias. In this section we
explore the simulation results to verify the previous findings regarding network
density, and extend the frontier by investigating the impact of other topological
properties.
In this context, the mean bias of the estimate for a given network and model
specification is defined as:
PR
r
. Or
R
/
;
where r is an iteration index, R is the total number of iterations (1,000 in our
experiments), is the true network dependence parameter, and Or is the estimated
dependence parameter in iteration r.
To begin, Fig. 8 displays the average mean bias for each level of and combination of model/network specification. In concordance with previously published
results, the amount of negative estimation bias increases with (Mizruchi and
Neuman 2008). This indicates that bias is a decreasing function of the spatial dependence parameter (at least in the range of parameters tested). Interestingly, the SEM
estimates are far more biased than the SAR estimates which on average show a
very weak negative bias. Theoretically, both the SEM and SAR estimates should be
unbiased when using a properly specified weight matrix. In an effort to explore its
possible causes, estimation bias is investigated with respect to the same topological
properties used to explore LR test strength above.
46
S. Farber et al.
Fig. 8 Dependence parameter estimation bias for different levels of dependence
Fig. 9 The impact of sample size on dependence parameter estimation bias
4.7 Estimation Bias and Sample Size
The difference in estimation bias between the largest and smallest networks is displayed in Fig. 9. The figure shows that bias is not impacted by sample size for the
SAR-exponential iterations. However, for the SAR-Poisson models the bias difference increases with until 0.4 and then starts to decrease slightly. This indicates that
as increases the bias-reducing impact of increasing sample size gets stronger. This
is true in general for both SEM specifications as well, however the inflection points
Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models
47
occur slightly earlier at D 0:35. The fact that increasing sample size reduces
bias is not surprising since the strength of most statistical tests increases with sample size. Additionally however, bias is known to increase with density, which is
inversely related to sample size. So in this specific case, it is difficult to separate and
differentiate between the effects of increasing sample size and decreasing matrix
density (see below for more). The fact that the impact of sample-size varies with
is difficult to explain and suggests that bias is functionally related to an interacting
term containing sample-size and dependence.
4.8 Estimation Bias and Mean Degree Distribution
Figure 10 illustrates the impact of mean degree on estimation bias for the different levels of dependence. The trends for the different specifications are quite
unique. As before, estimates for the SAR-exponential specification seem immune
to degree distribution. On the other hand, the impact of high mean degree on
the SAR-Poisson estimates grows more negative with increasing dependence up
to
D 0:45 at which point the trend seems to reverse. The impact on SEMexponential estimates decreases consistently with increasing dependence, while the
impact on SEM-Poisson estimates is erratic for < 0:2 and then decreases smoothly
with increasing dependence. As before, it is not surprising that bias is negatively
impacted by increasing mean degree; however the interaction between mean degree
and dependence is puzzling.
Fig. 10 The impact of mean degree on dependence parameter estimation bias
48
S. Farber et al.
Fig. 11 The effect of clustering on dependence parameter estimation bias
4.9 Estimation Bias and Clustering Coefficient
Figure 11 shows the mean estimation bias for each combination of
and C ,
averaging over all sample sizes and mean degrees. The figure clearly shows that
while bias increases with autocorrelation, the scale of bias decreases with increasing clustering. Moreover, the impact of clustering on estimation bias appears to
be far more pronounced than that on LR test rejection above (Fig. 6), indicating
that density is not necessarily the only relevant topological characteristic. While
the impact of clustering seems to be strongly positive on both SEM specifications, it is only mildly effective and seemingly ineffective on the SAR-Poisson and
SAR-exponential specifications respectively.
4.10 Estimation Bias and Matrix Density
The impact of matrix density on estimation bias is shown in Fig. 12. Each charted
observation corresponds to the mean matrix density and estimation bias for each
combination of sample size and degree distribution. This was done to simplify
the chart especially since matrix density does not vary with and only varies
slightly with clustering. As expected, the chart confirms previous results by indicating that weight matrix density increases estimation bias, except in the case of
SAR-exponential, which does not seem to be impacted by matrix density. While
Mizruchi and Neuman’s findings that network density introduces bias in the estimates of are confirmed by the experiments herein, other topological properties
also seem relevant to the bias discussion. Of course, if density causes bias, then
so would degree distribution and sample size since they are functionally related
Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models
49
Fig. 12 The relationship between matrix density and estimation bias
properties. But the relationship between bias and network clustering displayed in
Fig. 11 is unrelated to density and suggests that the causes of bias are related to
topology in a more complex manner than previously suggested.
5 Regression Analysis
5.1 Logistic Regression for LR Test Results
Table 2 contains the results from a series of simple logistic regression models used
to explore the relative influence of the various topological properties on LR rejection
frequency. A separate regression is calibrated for each model type and degree distribution function in order to capture the heterogeneities between cases as observed
in Sect. 4. In each case, the dependent variable is the frequency of LR test rejections
over 1,000 trials. The purpose of the regressions is to organize the vast amount of
simulation results into a parsimoniously defined functional relationship between LR
rejection frequency and the topology of the networks. A linear model is a simple way
to quantify and compare the impact of the various topological factors investigated
herein. In order to allow for non-linear relationships, the topological characteristics of the networks, namely size, mean degree, and clustering, have been coded
into dummy variables representing each factor level. While not displayed for the
sake of brevity, several other model specifications were experimented with before
50
S. Farber et al.
settling with the ones in Table 2. In particular, sample-size and mean degree were
replaced by the continuous measure of network density. Also, the dummy variables
representing different levels of the clustering coefficient were replaced with the continuous measure of actual achieved clustering. In both cases, the signs and scales of
the continuous variables were commensurate with the results in Table 2, however the
model-fits did decrease slightly, and the imposition of the linearity constraint associated with the use of a single continuous variable was unfavourable. Furthermore, the
Table 2 Results of rejection frequency logistic regression
SAR-Poisson
b
Constant
-4.12
Sample size
n D 100
n D 500
2.46
n D 1000
3.18
Mean degree
z D 1:5
z D 3:5
-1.15
z D 5:5
-3.11
z D 7:5
-4.06
Clustering coefficient
c D 0:2
c D 0:3
-0.26
c D 0:4
-0.23
c D 0:5
-0.14
c D 0:6
0.08
c D 0:7
-0.14
Lag strength
D 0:00
D 0:01
0.95
D 0:05
4.51
D 0:10
6.50
D 0:15
7.71
D 0:20
8.40
D 0:25
9.02
D 0:30
9.67
D 0:35
10.38
D 0:40
11.18
D 0:45
12.12
D 0:50
13.16
Summary statistics
Deviance
40949.0
0.9765
Pseudo-R2
SSE
2.8605
SAR-Exponential
t
B
t
166:1
-4.24
165:5
178:8
209:3
Reference
1.76
2.23
88:3
112:3
69:0
171:1
214:9
Reference
-0.26
-0.62
-0.80
1:7E C 07
35:2
45:1
15:2
13:5
8:2
4:9
8:2
Reference
0.03
0.06
0.09
0.02
0.01
2:2E C 06
3:8E C 06
6:0E C 06
1:1E C 06
8:7E C 05
41:3
195:7
251:4
268:2
274:5
276:2
270:5
251:4
215:0
163:3
111:3
Reference
1.70
5.92
10.54
39.19
39.19
39.19
39.19
39.19
39.19
39.19
39.19
86:2
235:0
96:0
2168:9
2168:9
2168:9
2168:9
2168:9
2168:9
2168:9
2168:9
9263.1
0.9938
0.5747
(continued)
Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models
51
Table 2 (continued)
SEM-Poisson
Constant
Sample size
n D 100
n D 500
n D 1000
Mean degree
z D 1:5
z D 3:5
z D 5:5
z D 7:5
Clustering coefficient
c D 0:2
c D 0:3
c D 0:4
c D 0:5
c D 0:6
c D 0:7
Lag strength
D 0:00
D 0:01
D 0:05
D 0:10
D 0:15
D 0:20
D 0:25
D 0:30
D 0:35
D 0:40
D 0:45
D 0:50
Summary statistics
Deviance
Pseudo-R2
SSE
SEM-Exponential
B
-5.92
t
242:8
B
-6.47
t
258:3
3.05
4.02
266:2
308:0
2.97
3.96
253:6
296:2
-0.6
-1.26
-1.81
51:4
112:4
158:4
-0.36
-0.78
-0.73
32:8
70:3
65:6
0.07
0.16
0.36
0.50
0.52
5:4
12:4
27:3
37:8
39:5
0.43
0.55
0.76
0.82
0.84
32:9
41:6
57:2
61:4
62:6
-0.01
1.29
3.06
4.54
5.58
6.35
6.94
7.51
8.07
8.57
9.10
0:2
56:4
141:2
202:8
237:9
260:0
275:0
286:4
293:0
292:9
284:7
0.05
1.28
3.06
4.62
5.67
6.37
6.93
7.51
8.13
8.73
9.39
1:8
56:9
143:1
206:6
240:6
261:1
275:6
286:2
289:3
281:2
258:8
29271.0
0.9887
1.6064
32195.0
0.9845
2.192
binary nature of the final set of variables displayed in Table 2 allows for the direct
interpretation and comparison of coefficients. The unfortunate functional relationship, densityD N (mean degree)/(N N–N), obviates the possibility to include
these three variables simultaneously into a regression model without introducing
high levels of multicolinearity. To this end, including the two terms (size and degree)
was an appropriate way to model the rejection frequencies although it does inhibit
the ability to estimate the impact of density directly.
52
S. Farber et al.
All of the coefficients in all the models are significant with a very high degree
of confidence as indicated by their associated t-values. It is noteworthy that such a
high level of significance is achieved in part due to the extremely large sample size.
One thousand repetitions for each of the 72 networks (per model and distribution
function) and 12 levels of rho results in an overall sample size of 864,000 observations. Given such a large sample size it becomes necessary to use the t-values as a
relative measure of significance between coefficients. In this way it becomes quite
clear that the coefficients for clustering are the least reliable, and caution should
be used when drawing conclusions based on them.1 As an interesting side-note,
the replacement of the clustering dummy variables with a single continuous variable produced a similarly low t-value for the clustering variable. Conversely, the
single continuous density measure obtained an extremely high t-value when it
replaced the sample size and mean degree variables in Table 2.
The coefficients themselves can be used to judge the relative influence of each
variable on the rejection frequency. In all the models, the level of dependence
obtains the highest regression coefficients. This confirms the above visual analysis whereby rejection frequency uniformly increased with the size of . It is also
important to observe the diminishing rate of increase of the regression coefficients
as increases. This indicates that the relationship between dependence level and LR
rejection frequency is non-linear, an important finding that can be used in the future
to validate analytical attempts at exploring the likelihood-ratio. Comparing the coefficients between the models, we observe that influence of dependence is stronger in
the SAR models than in the SEM specifications.
Following dependence, sample size and degree distribution are the next most
influential factors. Sample size seems to be a consistently strong positive influence
on test-strength. Mean degree on the other hand has a negative influence, but its
impact is for more pronounced in the Poisson cases, and especially in the SARPoisson case. It is difficult to determine exactly why this is, but the differences may
be derived from the differences between the shapes of the distribution density functions. In particular, the Poisson distribution is more concentrated around its mean
so most nodes obtain the mean number of connections. The exponential distribution is more dispersed and positively skewed, with most nodes obtaining a small
number of connections, and some obtaining a very large number. The net effect is
that for a given mean-degree and sample size, the simulated Poisson networks are
more connected than the exponential ones. This might explain why mean-degree
has a stronger influence on the Poisson networks than on the exponential ones. The
models achieve a very high pseudo-R2, an indication that a strong linear relationship exists between the observed and estimated rejection probabilities as seen in
the scatterplots in Fig. 13. A second summary statistic, the sum of squared errors
(SSE) is useful in comparing the strength of the models’ fit. Considering that the
1
The visualizations in Sect. 4 indicate that the LR test for SAR-exponential is extremely strong,
even for quite small values of dependence. This would explain the extremely high coefficients on
the dependence parameters, and perhaps the extremely significant but extremely small coefficients
for clustering.