Tải bản đầy đủ - 0trang
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



The presentation

The workshop began with a brief introduction to what REMI can do, in

terms of the issues it can be used to address and how this is accomplished.

Dr Treyz provided a description of the key elements of the model:

●



an input output structure;



●



general equilibrium elements, with long-term responses to prices and

wages;



●



an econometric component based on time series modelling to incorporate

dynamic effects;



●



new economic geography, features including agglomeration and clustering

effects.



This was followed by examples of the inputs used to drive a typical model

scenario and the outputs that can then be reviewed. These focused on the

demands for goods and services, the effects on wage and price changes, and

demographic and labour market effects.

Examples of the policy insights which can be derived were then

summarised for the US (with examples in a number of different contexts) and

also in other countries.

The pros and cons of a model-based approach were discussed, both in

Dr. Treyz’s presentation and in subsequent discussion. The following key

points were made:



192



●



it was argued that macroeconomic insights are essential for rational policy

decision making. micro-studies take us only part of the way to a complete

evaluation;



●



a quantitative macro-model can provide a common framework for comparing

the overall fiscal and employment effects of alternative policy interventions;



●



such an all encompassing and comprehensive approach can also deal with

displacement effects across geographical boundaries;



●



such a general model enables the incorporation of theoretical insights

supported by empirical testing from a variety of disciplines;



●



the model framework facilitates transparency in terms of assumptions and

implications;



●



the dynamic element of the model framework is essential to evaluating

when effects take place.



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



The need to regard the use of such methods as a process, with feedback

from users and subsequent modifications influencing the final outcomes, was

emphasised. Such analysis should not just be a “one off” exercise.

If done appropriately, this can help to empower users and ensure

involvement and ownership of the results and outcomes.



Alternative macro-econometric models

Other macro econometric models have also been developed which can be

used to help evaluate labour market interventions. In the United Kingdom, for

example, the Institute for Employment Research (IER), in collaboration with

Cambridge Econometrics (CE), have developed the Local Economy Forecasting

Model (LEFM). As with REMI, LEFM is based on an assumption of common

behavioural patterns and technical linkages applying at national, regional and

local levels, while recognising the structural differences that make each

geographical area unique.

LEFM is a tailored software tool that provides local economy analysis and

forecasting in the United Kingdom. Since its inception in 1993 this has been

set up for hundreds of areas in the UK, as well as a few in mainland Europe. Its

prime function is to guide policy makers and analysts on fundamental

economic and labour market trends at a local level.

LEFM has been designed to fulfil the following criteria:

●



an efficient means of generating tailored local economy projections that

makes maximum use of the national, regional and local information

available;



●



easy updating, allowing the user to draw on and reassess previous analysis;



●



a rigorous and transparent method of analysis, yielding results that can be

readily traced back to assumptions;



●



easy links to regularly updated, authoritative forecasts at the national and

regional levels;



●



an explicit way to introduce local knowledge and views;



●



substantial sectoral detail, so that projections can be identified closely with

major local firms;



●



labour market detail (sector, gender, occupation);



●



the ability to implement scenarios and sensitivity testing;



●



the ability to carry out impact studies (e.g. opening/closure of a large

establishment);



●



easy access to results for evaluation, plotting and file transfer to other

software for report writing, presentations, etc.



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



193



7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



In LEFM employment data play a central role, since it is on this variable

that the most detailed and reliable information is available at local level. Most

of the economic indicators at local level are derived by combining information

on the corresponding regional level indicator and the local area’s share of

regional employment or population.

The model’s basic structure (simplified) shares a number of features in

common with REMI (see Figure 7.1). It is based around a detailed Leontieff input

output table. It provides a dynamic solution using annual historical data from

1981 with prospectus forward to 2015. However, in contrast to REMI, LEFM is

focussed primarily upon the demand side. LEFM distinguishes 50 employing

activities and 6 status/gender types. In addition, the occupational results

provide a breakdown by 2 occupational categories, for males and females

separately. It is comprised of a series of behavioural and technical relationships

at local level which mirror those at national and regional level in terms of

parameter values, etc. The main elements are:

●



Consumer demand.



●



Investment demand.



Figure 7.1. LEFM – A local economy forecasting model



Rest of the

world

Outside the

local area



Activity, prices



Government



Inside the

local area



National activity

and prices



Exports



Imports



Total demand

Consumers

expenditure



194



Investment,

inputs to



Incomes



Employment



Unemployment



Occupations



Output



Working

population



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



●



Government expenditure.



●



Exports and imports.



●



Intermediate demand.



●



Total commodity demand (Q).



●



Industrial gross output (Y).



●



Value o.



●



Employment ondustry (YEO).



●



Employment by gender and type.



●



Employment by occupation.



●



Labour supply, etc.



●



Unemployment.



●



Incomes.



LEFM is calibrated so that employment in the local area follows the

observed historical patterns, while the forecast moves in line with regional or

national trends. This is based on a simple econometric analysis. Calibration is

achieved by altering local gross output levels. This in turn is achieved by

amending the export share ratio. This ensures that, over the historical period,

the model tracks the observed employment data and that the initial forecast

is “sensible”. The adjustments to the “net commuting” residual also ensures

that the model traces the observed unemployment data and that the initial

base forecast changes in line with more general regional trends. Complete

details of the model can be found in Wilson et al. (1995).



Initial discussion: statistical significance and validation

The statistical significant and robustness of the results from REMI were

queried and questions were also raised about the applicability of US

parameters to European contexts. This can be seen as part of a more general

debate about the applicability of model results based on particular historical

experiences to other contexts and periods.

Dr Treyz emphasised that considerable care was taken to ensure that

estimated parameters were statistically robust, but recognised that, when

looking at forecast results, it is much more difficult to make precise

statements about statistical accuracy. Rather, the emphasis should be on

producing reasonable or sensible outcomes and focusing on the difference

between policy “on” and “off” or differences between different policies.

There was a lengthy discussion on the problems of applying REMI in a

European context, given the different social and economic context. As well as

problems of data classification and consistency, this assumes that the

performance and behaviour are common across international boundaries.



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



195



7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



While this may be true in a very general sense, different institutional and legal

frameworks undoubtedly affect the way people behave, the incentives they

face, etc.

The social context of a given situation, social cohesion and similar factors

are not readily incorporated into a quantitative model framework. The general

parameters should be regarded as a starting point subject to modification to

reflect local circumstances. This reinforces the importance of getting local

involvement when applying these tools, so that such local factors can be taken

into account. Models such as REMI and LEFM enable local evaluation analysts

and policy makers to gain insights based on general findings from economic

analysis and from other disciplines.

There was some discussion of the complexities of using such models,

which does undoubtedly require a certain minimum level of expertise. The

results from applying such approaches can be quite sensitive to how they are

operated and the input assumptions used. As always there is the danger of

“garbage in garbage out”.

A further concern was raised about the timeliness of some key data

inputs, especially elements such as input output tables. However, such values

do tend to change quite slowly over time, so this was not felt to be a major

issue. This again emphasises the importance of focusing on relative effects (of

one policy compared to another, policy on compared to policy off) rather than

absolute changes.



Usefulness of macro models

Earlier sessions in the conference, particularly the contribution by Bartik,

emphasised the importance, when evaluating particular policies and

programmes, of moving beyond the immediate impact on individuals or firms.

In particular, a comprehensive evaluation requires an assessment of the

overall effect on revenues and total employment levels. While the various

approaches to evaluation advocated in previous sessions focused on issues

such as deadweight and displacement at a micro level, they generally provide

no mechanism for reaching a macroeconomic overview of the effects. Equally,

while these other methods provide various means by which the past impact of

such possible interventions may be assessed they provide no insight into

possible future effects.

In order to obtain these additional insights a macroeconomic forecasting

model is needed. This provides the mechanism by which the macroeconomic

consequences of individual effects can be summed (building in multiplier and

input-output linkage effects) and also providing a useful counterfactual of

what might have happened in the absence of the policy intervention. The key

conclusion from the discussion is that such models, while not a panacea, and



196



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



while providing just one leg of a complete evaluation approach, can deliver

important insights into the evaluation process. They help to remind us of all

the complex interactions that should be taken into account but which are

sometimes forgotten.

Other papers in the conference (e.g. the contribution by Wong) emphasised

the need to take into account geographical displacement effects. Macro models

cannot by themselves answer this question but they provide an important

framework within which it can be addressed.

Bartik also emphasised that rigorous evaluation is possible through a

variety of methods but including the need to link in regional econometrics

with fiscal impact and local labour market models. A mixture of techniques,

quantitative and qualitative, is needed with a focus on improving programmes

rather than “killing” poor ones.

Smith also raised the importance of synergy and macroeconomic effects

which might otherwise be neglected in a purely microeconomic evaluation.

Oldsman emphasised the importance of establishing robust baseline

data. A key aspect of this is providing a counter-factual – what would have

happened anyway, even if the policy had not been introduced/changed?

Eberts and O’Leary made the important point that social programmes

that are intended to affect local and regional economies are especially difficult

to evaluate. Their scope is much broader than simply looking at impacts on

individual actors or organisations. On the other hand, the scale of interventions

are often so small as to have indiscernible effects. A macro modelling tool

provides a means to at least get a handle on these kinds of outcomes.

Walker emphasised the need to collect longitudinal data to facilitate model

building and the development of better, more sophisticated forecasting tools.

Finally, another important issue is that polices and subsequent

evaluations may interact and evolve, changing behaviour and generating

additional capacity (Stern). This requires quite careful handling in the model

framework, in which the parameters are based on previous patterns of

behaviour and institutional and related frameworks.

Wong reminded us that intervention in the social sciences is rarely a

matter of repeating a controlled experiment. Often the carrying out of the

experiment, as well as any evaluation of it, will influence the outcomes and

behaviours of those involved. This applies with special force when using

macro models of the REMI or LEFM type. Changing behaviour may result in the

need to reassess the explicit assumption of fixed parameters based on

historical experience. Robson emphasised a similar point, citing the

importance of action orientated research, perceptual indicators and softer,

qualitative, rather than hard quantitative approaches.



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



197



7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...



Reference

WILSON, R., A. ASSEFA and J. BEARD (1995), “A Local Economy Forecasting Model

(LEFM) for the UK Economy”. Paper presented at the European Symposium of

Labour Market Developments, University of Warwick, 18th-19th May, 1995.



198



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



ISBN 92-64-01708-9

Evaluating Local Economic and Employment Development

How to Assess What Works among Programmes and Policies

© OECD 2004



Chapter 8



Area-based Policy Evaluation

by

Brian Robson,

Centre for Urban Policy Studies,

University of Manchester,

United Kingdom



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



199



8.



AREA-BASED POLICY EVALUATION



The policy context

Formal urban policies in Britain date from the late 1960s and, over the last

three decades, policy has placed great faith in area-based initiatives (ABIs) as

a mechanism to tackle the problems associated with dereliction and

deprivation in the large ex-industrial towns and cities of Britain (for example,

Hall and Nevin, 1999).

Since the 1980s there has been a bewildering array of different types of

policy instrument, amongst which have been (or are):

●



Action for Cities (AfC) in the 1980s which focused additional resources on

57 local authority districts (LADs).



●



City Challenge (CC) which operated between 1992 and 1998 and targeted

resources at 31 sub-district areas.



●



the Single Regeneration Budget (SRB) which rolled together the expenditures

from 21 earlier programmes, went through 6 rounds of allocation during the

1990s, and was largely targeted at small geographically defined areas which

received funding for periods of up to seven years.



●



New Deal for Communities (NDC) which currently supports 39 sub-district

areas with populations of up to 4 000 over a period of ten years.



●



Urban Regeneration Companies (URC), the first of which were created in

1999, with 11 having now been established to develop master plans mainly

for city centre areas, and with running costs funded through partnerships

between Regional Development Agencies (RDAs), LADs and English

Partnerships (EP).



Each of these programmes has been funded by the government department

with principal responsibility for urban regeneration – successively named the

Department of Environment (DoE), Department of Environment Transport and

the Regions (DETR), Department of Transport, Local Government and the Regions

(DTLR), and now the Office of the Deputy Prime Minister (ODPM).

In addition, in the last five years, there has been an increasing number of

complementary area-based initiatives from other central government

departments. Examples include the Crime Reduction Programme, Sure Start,

Education Action Zones, Employment Zones, Sports Action Zones, and Health

Action Zones – from government departments such as the Home Office,

Employment, Education, Sport and Culture, and Health.



200



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



8.



AREA-BASED POLICY EVALUATION



Such area-based approaches are predicated on the belief either that there

are area-related processes that compound the problems that are faced by

deprived individuals, or that there are efficiencies associated with the delivery

of policy within defined targeted areas. While there has been much agonising

in the academic literature about whether or not an “area effect” can be

demonstrated (Dorling et al., 2001; Atkinson and Kintrea, 2001), there seems to

be general agreement that there are administrative benefits associated with

spatial targeting – not least the potential synergy that can be achieved across

different policy domains, and the efficiency of deploying personnel and

resources within a limited number of areas. The case for area-targeting is that,

given the wide disparities in deprivation, the neighbourhood is the most

appropriate scale for fostering community identity and involvement and that

resources concentrated at small areas over a number of years can achieve a

step improvement in the circumstances of deprived areas (Lawless et al., 2000).

Over time, the focus of ABIs has changed. At the outset, in the 1970s, most

programmes broadly covered economic, social and environmental objectives,

but the additional targeted resources were generally small. In the 1980s, the

majority of funding was directed to the physical improvement of derelict areas

in an attempt to revive the working of property markets. Typical interventions

included programmes such as Enterprise Zones which offered financial

incentives to firms to locate in decayed urban sub-areas, or the Urban

Development Corporations which were run by private-sector boards with the

aim of re-furbishing the infrastructure of derelict ex-industrial sites. In the

1990s, City Challenge and the SRB programmes brought local authorities back

into the frame and broadened the focus to encompass social and economic

issues as well as environmental and property-related aims. With this change of

emphasis, two principal features have come to dominate recent approaches to

urban regeneration; a stress on partnership working and, related to this, the aim

of developing better co-ordination across policy domains.

The recent Urban White Paper (DETR, 2000) fundamentally changed the

main thrust of policy. It moved away from the previous almost exclusive

dependence on area-based initiatives and espoused the aim of “mainstreaming”

as a way of better tackling urban problems. Mainstreaming can be thought of

in three distinct ways:

●



the attempt to bend resources from main spending programmes (such as

education, social services, housing) to target areas of especial need or to

improve the quality of service delivery to such areas;



●



the attempt to learn lessons from what works in specific programmes and

projects and apply them more generally to other areas; and



●



the attempt to incorporate into mainstream services the policy lessons that

arise from specific initiatives.



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004



201



8.



AREA-BASED POLICY EVALUATION



While mainstreaming has long been a mantra within urban policy, the new

policy framework is the first to put it centre stage. Local Strategic Partnerships are

now in process of being established in local authorities and they are charged with

developing Community Strategies that consciously use mainstream as well as

specific resources from funds such as the Neighbourhood Renewal Fund (which is

targeted at 88 local authorities) to reverse the fortunes of deprived areas.

So, even though many area-based initiatives remain – not least the NDCs,

URCs and some of the later rounds of SRB funding – English urban policy has

now drawn a line under its almost exclusive dependence on area-based

initiatives.



The growth of monitoring and evaluation

From the outset of this 30-year history of urban policy, there has been a

continuous development of monitoring and evaluation under the auspices of

central government. The total resources channelled at monitoring and

evaluation have grown considerably. On the face of it, evaluation has now

become an integral part of the policy environment. As well as the evaluation

projects sponsored by funding departments of central government, generic

policy reviews are undertaken by bodies such as the Audit Commission (for

example, Audit Commission, 1989; 2002) and the National Audit Office; and

the newly-established co-ordinating units established by central government

such as the Social Exclusion Unit and the Regional Co-ordination Unit also

draw on research-based evidence with the aim of steering the direction of

policy. A recent example is the Regional Co-ordination Unit’s review of areabased initiatives (RCU, 2002) which makes recommendations about merging

and mainstreaming several of the existing separately funded regeneration

programmes from across a range of government departments. In addition to

such government-sponsored evaluation, there have been programmes

evaluating area-based policies both by research charities – especially the Area

Regeneration Programme of the Joseph Rowntree Foundation (see, for

example, Maclennan, 2000) – and by the Economic and Social Research

Council through its Cities Programme (see, for example, Begg, 2002). In this

paper, however, the focus is restricted more narrowly to governmentsponsored research.

Much of the research supported by the Department of the Environment in

the 1970s and 1980s was relatively haphazard. Most of the evaluations were ex

post and researchers had some difficulty in assembling data that could

identify the initial conditions when programmes started and in recreating

data on outputs during the lifespan of initiatives. However, since the 1990s

(and especially since City Challenge and the SRB programmes), most funding

of policy interventions has usually been contingent on continuous monitoring



202



EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004