Tải bản đầy đủ - 0 (trang)
3 Power of Test, Mean Degree and Sample Size

3 Power of Test, Mean Degree and Sample Size

Tải bản đầy đủ - 0trang

42



S. Farber et al.



Fig. 4 The impact of mean degree on small networks



Fig. 5 The impact of mean degree on large networks



degree to sample size, the denser the weight matrix and the weaker the power. The

key question is how much so. Figures 4 and 5 show the differences in test power

between networks with the largest and smallest mean degree and sample size. Interestingly, the figures suggest that sample size and mean degree interact and impact

test strength in a variety of ways. The first observation is that the power of the LR test



Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models



43



for SAR models on networks with exponentially distributed degrees is quite robust

to changes in mean degree except for 0:01 < < 0:1. The LR test for the SAR

model using a Poisson distributed degree function appears to be the most sensitive to

increases in mean degree, especially in the range of 0:01 < < 0:3 when the test is

at least 30 and up to 90 percentage points weaker in more connected networks. The

shapes of the curves for the SEM specifications differ substantially to those for the

SAR specifications. While the SEM-Poisson combination produces slightly weaker

LR strengths than the SEM-Exponential tests, the most striking observation is that

both SEM curves are right-shifted indicating a relative immunity to mean degree in

the lower ranges of , and a vulnerability to degree in the higher investigated ranges

of the spatial parameter (0:2 < < 0:5).

The curves in Fig. 5 can be used to determine the degree to which increasing

sample size mitigates the impact of mean degree on test strength. Since increasing

sample size decreases weight matrix density for any given mean degree, we expect

to find the curves in Fig. 5 to be flatter than those in Fig. 4. It is clearly seen that

the effect of increasing sample size is an overall scale reduction in test weakness

and a left-shift of the curves’ minima. While the curves in Fig. 4 appear clustered

by model specification (SEM versus SAR curves), those in Fig. 5 either appear clustered by degree distribution function (Poisson versus exponential) or not clustered at

all; specifically, the test strengths on Poisson networks are more negatively impacted

by mean degree than for exponential networks.



4.4 Power of Test and Clustering

Results from Farber et al. (2009) indicate that the clustering coefficient, a measure of overall transitivity, does not impact test strength for SAR models using a

Poisson degree distribution function. As seen in Fig. 6, the impact of clustering is



Fig. 6 The impact of clustering on rejection frequency



44



S. Farber et al.



negligible on SAR models with both distribution function specifications. Whereas

clustering does not have a discernable impact on tests for SAR dependence, there

does seem to be a weak positive relationship between clustering and test strength

for SEM dependence. To date there are no analytically driven results that would

predict these simulation results, and these observations necessitate the investigation of the likelihood functions with respect to clustering before they can be fully

understood.



4.5 Power of Test and Matrix Density

Recent work has identified matrix density to be an important determinant of estimation bias in spatial and network regression models (Smith 2009). Given a binary

weight matrix W of size n x n, density is calculated as:

P P

DD



i



n.n



j



wij

1/



which is simply the sum of all the entries divided by the total number of possible

entries.

Figure 7 shows the relationship between average test power and matrix density

for each combination of model and degree distribution function. Each data point on



Fig. 7 The impact of matrix density on rejection frequency



Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models



45



the vertical axis is the LR rejection rate averaged over all and C for each sample size and mean degree. The data points are clustered along the horizontal access

due to the three disjoint sample-sizes used in the simulations. It appears that test

strength generally declined with matrix density for all four data series. In support

of the findings above, the LR tests are strongest for the SAR-exponential case and

are quite insensitive to changes in matrix density. While the curve for SAR-Poisson

is shifted upward from the SEM curves, the average rate of decline among the three

curves appears to be equivalent. Notice that the exponential matrices contain fewer

highly connected nodes in comparison to their Poisson distributed counterparts with

similar mean degree and this may be one reason for improved LR test strengths for

exponential network specifications.



4.6 Estimation Bias

Theoretically, one would expect the likelihood ratio tests to fail to reject the null

hypothesis when the estimated parameter is downward biased, as was shown

to occur in dense networks by Smith (2009) and Mizruchi and Neuman (2008).

However, neither Smith nor Mizruchi and Neuman investigate the possible impact

of network structure (besides link density) on estimation bias. In this section we

explore the simulation results to verify the previous findings regarding network

density, and extend the frontier by investigating the impact of other topological

properties.

In this context, the mean bias of the estimate for a given network and model

specification is defined as:

PR

r



. Or

R



/



;



where r is an iteration index, R is the total number of iterations (1,000 in our

experiments), is the true network dependence parameter, and Or is the estimated

dependence parameter in iteration r.

To begin, Fig. 8 displays the average mean bias for each level of and combination of model/network specification. In concordance with previously published

results, the amount of negative estimation bias increases with (Mizruchi and

Neuman 2008). This indicates that bias is a decreasing function of the spatial dependence parameter (at least in the range of parameters tested). Interestingly, the SEM

estimates are far more biased than the SAR estimates which on average show a

very weak negative bias. Theoretically, both the SEM and SAR estimates should be

unbiased when using a properly specified weight matrix. In an effort to explore its

possible causes, estimation bias is investigated with respect to the same topological

properties used to explore LR test strength above.



46



S. Farber et al.



Fig. 8 Dependence parameter estimation bias for different levels of dependence



Fig. 9 The impact of sample size on dependence parameter estimation bias



4.7 Estimation Bias and Sample Size

The difference in estimation bias between the largest and smallest networks is displayed in Fig. 9. The figure shows that bias is not impacted by sample size for the

SAR-exponential iterations. However, for the SAR-Poisson models the bias difference increases with until 0.4 and then starts to decrease slightly. This indicates that

as increases the bias-reducing impact of increasing sample size gets stronger. This

is true in general for both SEM specifications as well, however the inflection points



Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models



47



occur slightly earlier at D 0:35. The fact that increasing sample size reduces

bias is not surprising since the strength of most statistical tests increases with sample size. Additionally however, bias is known to increase with density, which is

inversely related to sample size. So in this specific case, it is difficult to separate and

differentiate between the effects of increasing sample size and decreasing matrix

density (see below for more). The fact that the impact of sample-size varies with

is difficult to explain and suggests that bias is functionally related to an interacting

term containing sample-size and dependence.



4.8 Estimation Bias and Mean Degree Distribution

Figure 10 illustrates the impact of mean degree on estimation bias for the different levels of dependence. The trends for the different specifications are quite

unique. As before, estimates for the SAR-exponential specification seem immune

to degree distribution. On the other hand, the impact of high mean degree on

the SAR-Poisson estimates grows more negative with increasing dependence up

to

D 0:45 at which point the trend seems to reverse. The impact on SEMexponential estimates decreases consistently with increasing dependence, while the

impact on SEM-Poisson estimates is erratic for < 0:2 and then decreases smoothly

with increasing dependence. As before, it is not surprising that bias is negatively

impacted by increasing mean degree; however the interaction between mean degree

and dependence is puzzling.



Fig. 10 The impact of mean degree on dependence parameter estimation bias



48



S. Farber et al.



Fig. 11 The effect of clustering on dependence parameter estimation bias



4.9 Estimation Bias and Clustering Coefficient

Figure 11 shows the mean estimation bias for each combination of

and C ,

averaging over all sample sizes and mean degrees. The figure clearly shows that

while bias increases with autocorrelation, the scale of bias decreases with increasing clustering. Moreover, the impact of clustering on estimation bias appears to

be far more pronounced than that on LR test rejection above (Fig. 6), indicating

that density is not necessarily the only relevant topological characteristic. While

the impact of clustering seems to be strongly positive on both SEM specifications, it is only mildly effective and seemingly ineffective on the SAR-Poisson and

SAR-exponential specifications respectively.



4.10 Estimation Bias and Matrix Density

The impact of matrix density on estimation bias is shown in Fig. 12. Each charted

observation corresponds to the mean matrix density and estimation bias for each

combination of sample size and degree distribution. This was done to simplify

the chart especially since matrix density does not vary with and only varies

slightly with clustering. As expected, the chart confirms previous results by indicating that weight matrix density increases estimation bias, except in the case of

SAR-exponential, which does not seem to be impacted by matrix density. While

Mizruchi and Neuman’s findings that network density introduces bias in the estimates of are confirmed by the experiments herein, other topological properties

also seem relevant to the bias discussion. Of course, if density causes bias, then

so would degree distribution and sample size since they are functionally related



Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models



49



Fig. 12 The relationship between matrix density and estimation bias



properties. But the relationship between bias and network clustering displayed in

Fig. 11 is unrelated to density and suggests that the causes of bias are related to

topology in a more complex manner than previously suggested.



5 Regression Analysis

5.1 Logistic Regression for LR Test Results

Table 2 contains the results from a series of simple logistic regression models used

to explore the relative influence of the various topological properties on LR rejection

frequency. A separate regression is calibrated for each model type and degree distribution function in order to capture the heterogeneities between cases as observed

in Sect. 4. In each case, the dependent variable is the frequency of LR test rejections

over 1,000 trials. The purpose of the regressions is to organize the vast amount of

simulation results into a parsimoniously defined functional relationship between LR

rejection frequency and the topology of the networks. A linear model is a simple way

to quantify and compare the impact of the various topological factors investigated

herein. In order to allow for non-linear relationships, the topological characteristics of the networks, namely size, mean degree, and clustering, have been coded

into dummy variables representing each factor level. While not displayed for the

sake of brevity, several other model specifications were experimented with before



50



S. Farber et al.



settling with the ones in Table 2. In particular, sample-size and mean degree were

replaced by the continuous measure of network density. Also, the dummy variables

representing different levels of the clustering coefficient were replaced with the continuous measure of actual achieved clustering. In both cases, the signs and scales of

the continuous variables were commensurate with the results in Table 2, however the

model-fits did decrease slightly, and the imposition of the linearity constraint associated with the use of a single continuous variable was unfavourable. Furthermore, the



Table 2 Results of rejection frequency logistic regression

SAR-Poisson

b

Constant

-4.12

Sample size

n D 100

n D 500

2.46

n D 1000

3.18

Mean degree

z D 1:5

z D 3:5

-1.15

z D 5:5

-3.11

z D 7:5

-4.06

Clustering coefficient

c D 0:2

c D 0:3

-0.26

c D 0:4

-0.23

c D 0:5

-0.14

c D 0:6

0.08

c D 0:7

-0.14

Lag strength

D 0:00

D 0:01

0.95

D 0:05

4.51

D 0:10

6.50

D 0:15

7.71

D 0:20

8.40

D 0:25

9.02

D 0:30

9.67

D 0:35

10.38

D 0:40

11.18

D 0:45

12.12

D 0:50

13.16

Summary statistics

Deviance

40949.0

0.9765

Pseudo-R2

SSE

2.8605



SAR-Exponential

t



B



t



166:1



-4.24



165:5



178:8

209:3



Reference

1.76

2.23



88:3

112:3



69:0

171:1

214:9



Reference

-0.26

-0.62

-0.80



1:7E C 07

35:2

45:1



15:2

13:5

8:2

4:9

8:2



Reference

0.03

0.06

0.09

0.02

0.01



2:2E C 06

3:8E C 06

6:0E C 06

1:1E C 06

8:7E C 05



41:3

195:7

251:4

268:2

274:5

276:2

270:5

251:4

215:0

163:3

111:3



Reference

1.70

5.92

10.54

39.19

39.19

39.19

39.19

39.19

39.19

39.19

39.19



86:2

235:0

96:0

2168:9

2168:9

2168:9

2168:9

2168:9

2168:9

2168:9

2168:9



9263.1

0.9938

0.5747

(continued)



Topology, Dependency Tests and Estimation Bias in Network Autoregressive Models



51



Table 2 (continued)

SEM-Poisson



Constant

Sample size

n D 100

n D 500

n D 1000

Mean degree

z D 1:5

z D 3:5

z D 5:5

z D 7:5

Clustering coefficient

c D 0:2

c D 0:3

c D 0:4

c D 0:5

c D 0:6

c D 0:7

Lag strength

D 0:00

D 0:01

D 0:05

D 0:10

D 0:15

D 0:20

D 0:25

D 0:30

D 0:35

D 0:40

D 0:45

D 0:50

Summary statistics

Deviance

Pseudo-R2

SSE



SEM-Exponential



B

-5.92



t

242:8



B

-6.47



t

258:3



3.05

4.02



266:2

308:0



2.97

3.96



253:6

296:2



-0.6

-1.26

-1.81



51:4

112:4

158:4



-0.36

-0.78

-0.73



32:8

70:3

65:6



0.07

0.16

0.36

0.50

0.52



5:4

12:4

27:3

37:8

39:5



0.43

0.55

0.76

0.82

0.84



32:9

41:6

57:2

61:4

62:6



-0.01

1.29

3.06

4.54

5.58

6.35

6.94

7.51

8.07

8.57

9.10



0:2

56:4

141:2

202:8

237:9

260:0

275:0

286:4

293:0

292:9

284:7



0.05

1.28

3.06

4.62

5.67

6.37

6.93

7.51

8.13

8.73

9.39



1:8

56:9

143:1

206:6

240:6

261:1

275:6

286:2

289:3

281:2

258:8



29271.0

0.9887

1.6064



32195.0

0.9845

2.192



binary nature of the final set of variables displayed in Table 2 allows for the direct

interpretation and comparison of coefficients. The unfortunate functional relationship, densityD N (mean degree)/(N N–N), obviates the possibility to include

these three variables simultaneously into a regression model without introducing

high levels of multicolinearity. To this end, including the two terms (size and degree)

was an appropriate way to model the rejection frequencies although it does inhibit

the ability to estimate the impact of density directly.



52



S. Farber et al.



All of the coefficients in all the models are significant with a very high degree

of confidence as indicated by their associated t-values. It is noteworthy that such a

high level of significance is achieved in part due to the extremely large sample size.

One thousand repetitions for each of the 72 networks (per model and distribution

function) and 12 levels of rho results in an overall sample size of 864,000 observations. Given such a large sample size it becomes necessary to use the t-values as a

relative measure of significance between coefficients. In this way it becomes quite

clear that the coefficients for clustering are the least reliable, and caution should

be used when drawing conclusions based on them.1 As an interesting side-note,

the replacement of the clustering dummy variables with a single continuous variable produced a similarly low t-value for the clustering variable. Conversely, the

single continuous density measure obtained an extremely high t-value when it

replaced the sample size and mean degree variables in Table 2.

The coefficients themselves can be used to judge the relative influence of each

variable on the rejection frequency. In all the models, the level of dependence

obtains the highest regression coefficients. This confirms the above visual analysis whereby rejection frequency uniformly increased with the size of . It is also

important to observe the diminishing rate of increase of the regression coefficients

as increases. This indicates that the relationship between dependence level and LR

rejection frequency is non-linear, an important finding that can be used in the future

to validate analytical attempts at exploring the likelihood-ratio. Comparing the coefficients between the models, we observe that influence of dependence is stronger in

the SAR models than in the SEM specifications.

Following dependence, sample size and degree distribution are the next most

influential factors. Sample size seems to be a consistently strong positive influence

on test-strength. Mean degree on the other hand has a negative influence, but its

impact is for more pronounced in the Poisson cases, and especially in the SARPoisson case. It is difficult to determine exactly why this is, but the differences may

be derived from the differences between the shapes of the distribution density functions. In particular, the Poisson distribution is more concentrated around its mean

so most nodes obtain the mean number of connections. The exponential distribution is more dispersed and positively skewed, with most nodes obtaining a small

number of connections, and some obtaining a very large number. The net effect is

that for a given mean-degree and sample size, the simulated Poisson networks are

more connected than the exponential ones. This might explain why mean-degree

has a stronger influence on the Poisson networks than on the exponential ones. The

models achieve a very high pseudo-R2, an indication that a strong linear relationship exists between the observed and estimated rejection probabilities as seen in

the scatterplots in Fig. 13. A second summary statistic, the sum of squared errors

(SSE) is useful in comparing the strength of the models’ fit. Considering that the



1



The visualizations in Sect. 4 indicate that the LR test for SAR-exponential is extremely strong,

even for quite small values of dependence. This would explain the extremely high coefficients on

the dependence parameters, and perhaps the extremely significant but extremely small coefficients

for clustering.



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

3 Power of Test, Mean Degree and Sample Size

Tải bản đầy đủ ngay(0 tr)

×