Chapter 7. A Commentary on Frederick and George Treyz’s Paper and the Workshop “Analysis Policies for Local Development Using Forecasting Models”
Tải bản đầy đủ - 0trang
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
The presentation
The workshop began with a brief introduction to what REMI can do, in
terms of the issues it can be used to address and how this is accomplished.
Dr Treyz provided a description of the key elements of the model:
●
an input output structure;
●
general equilibrium elements, with long-term responses to prices and
wages;
●
an econometric component based on time series modelling to incorporate
dynamic effects;
●
new economic geography, features including agglomeration and clustering
effects.
This was followed by examples of the inputs used to drive a typical model
scenario and the outputs that can then be reviewed. These focused on the
demands for goods and services, the effects on wage and price changes, and
demographic and labour market effects.
Examples of the policy insights which can be derived were then
summarised for the US (with examples in a number of different contexts) and
also in other countries.
The pros and cons of a model-based approach were discussed, both in
Dr. Treyz’s presentation and in subsequent discussion. The following key
points were made:
192
●
it was argued that macroeconomic insights are essential for rational policy
decision making. micro-studies take us only part of the way to a complete
evaluation;
●
a quantitative macro-model can provide a common framework for comparing
the overall fiscal and employment effects of alternative policy interventions;
●
such an all encompassing and comprehensive approach can also deal with
displacement effects across geographical boundaries;
●
such a general model enables the incorporation of theoretical insights
supported by empirical testing from a variety of disciplines;
●
the model framework facilitates transparency in terms of assumptions and
implications;
●
the dynamic element of the model framework is essential to evaluating
when effects take place.
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
The need to regard the use of such methods as a process, with feedback
from users and subsequent modifications influencing the final outcomes, was
emphasised. Such analysis should not just be a “one off” exercise.
If done appropriately, this can help to empower users and ensure
involvement and ownership of the results and outcomes.
Alternative macro-econometric models
Other macro econometric models have also been developed which can be
used to help evaluate labour market interventions. In the United Kingdom, for
example, the Institute for Employment Research (IER), in collaboration with
Cambridge Econometrics (CE), have developed the Local Economy Forecasting
Model (LEFM). As with REMI, LEFM is based on an assumption of common
behavioural patterns and technical linkages applying at national, regional and
local levels, while recognising the structural differences that make each
geographical area unique.
LEFM is a tailored software tool that provides local economy analysis and
forecasting in the United Kingdom. Since its inception in 1993 this has been
set up for hundreds of areas in the UK, as well as a few in mainland Europe. Its
prime function is to guide policy makers and analysts on fundamental
economic and labour market trends at a local level.
LEFM has been designed to fulfil the following criteria:
●
an efficient means of generating tailored local economy projections that
makes maximum use of the national, regional and local information
available;
●
easy updating, allowing the user to draw on and reassess previous analysis;
●
a rigorous and transparent method of analysis, yielding results that can be
readily traced back to assumptions;
●
easy links to regularly updated, authoritative forecasts at the national and
regional levels;
●
an explicit way to introduce local knowledge and views;
●
substantial sectoral detail, so that projections can be identified closely with
major local firms;
●
labour market detail (sector, gender, occupation);
●
the ability to implement scenarios and sensitivity testing;
●
the ability to carry out impact studies (e.g. opening/closure of a large
establishment);
●
easy access to results for evaluation, plotting and file transfer to other
software for report writing, presentations, etc.
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
193
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
In LEFM employment data play a central role, since it is on this variable
that the most detailed and reliable information is available at local level. Most
of the economic indicators at local level are derived by combining information
on the corresponding regional level indicator and the local area’s share of
regional employment or population.
The model’s basic structure (simplified) shares a number of features in
common with REMI (see Figure 7.1). It is based around a detailed Leontieff input
output table. It provides a dynamic solution using annual historical data from
1981 with prospectus forward to 2015. However, in contrast to REMI, LEFM is
focussed primarily upon the demand side. LEFM distinguishes 50 employing
activities and 6 status/gender types. In addition, the occupational results
provide a breakdown by 2 occupational categories, for males and females
separately. It is comprised of a series of behavioural and technical relationships
at local level which mirror those at national and regional level in terms of
parameter values, etc. The main elements are:
●
Consumer demand.
●
Investment demand.
Figure 7.1. LEFM – A local economy forecasting model
Rest of the
world
Outside the
local area
Activity, prices
Government
Inside the
local area
National activity
and prices
Exports
Imports
Total demand
Consumers
expenditure
194
Investment,
inputs to
Incomes
Employment
Unemployment
Occupations
Output
Working
population
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
●
Government expenditure.
●
Exports and imports.
●
Intermediate demand.
●
Total commodity demand (Q).
●
Industrial gross output (Y).
●
Value o.
●
Employment ondustry (YEO).
●
Employment by gender and type.
●
Employment by occupation.
●
Labour supply, etc.
●
Unemployment.
●
Incomes.
LEFM is calibrated so that employment in the local area follows the
observed historical patterns, while the forecast moves in line with regional or
national trends. This is based on a simple econometric analysis. Calibration is
achieved by altering local gross output levels. This in turn is achieved by
amending the export share ratio. This ensures that, over the historical period,
the model tracks the observed employment data and that the initial forecast
is “sensible”. The adjustments to the “net commuting” residual also ensures
that the model traces the observed unemployment data and that the initial
base forecast changes in line with more general regional trends. Complete
details of the model can be found in Wilson et al. (1995).
Initial discussion: statistical significance and validation
The statistical significant and robustness of the results from REMI were
queried and questions were also raised about the applicability of US
parameters to European contexts. This can be seen as part of a more general
debate about the applicability of model results based on particular historical
experiences to other contexts and periods.
Dr Treyz emphasised that considerable care was taken to ensure that
estimated parameters were statistically robust, but recognised that, when
looking at forecast results, it is much more difficult to make precise
statements about statistical accuracy. Rather, the emphasis should be on
producing reasonable or sensible outcomes and focusing on the difference
between policy “on” and “off” or differences between different policies.
There was a lengthy discussion on the problems of applying REMI in a
European context, given the different social and economic context. As well as
problems of data classification and consistency, this assumes that the
performance and behaviour are common across international boundaries.
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
195
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
While this may be true in a very general sense, different institutional and legal
frameworks undoubtedly affect the way people behave, the incentives they
face, etc.
The social context of a given situation, social cohesion and similar factors
are not readily incorporated into a quantitative model framework. The general
parameters should be regarded as a starting point subject to modification to
reflect local circumstances. This reinforces the importance of getting local
involvement when applying these tools, so that such local factors can be taken
into account. Models such as REMI and LEFM enable local evaluation analysts
and policy makers to gain insights based on general findings from economic
analysis and from other disciplines.
There was some discussion of the complexities of using such models,
which does undoubtedly require a certain minimum level of expertise. The
results from applying such approaches can be quite sensitive to how they are
operated and the input assumptions used. As always there is the danger of
“garbage in garbage out”.
A further concern was raised about the timeliness of some key data
inputs, especially elements such as input output tables. However, such values
do tend to change quite slowly over time, so this was not felt to be a major
issue. This again emphasises the importance of focusing on relative effects (of
one policy compared to another, policy on compared to policy off) rather than
absolute changes.
Usefulness of macro models
Earlier sessions in the conference, particularly the contribution by Bartik,
emphasised the importance, when evaluating particular policies and
programmes, of moving beyond the immediate impact on individuals or firms.
In particular, a comprehensive evaluation requires an assessment of the
overall effect on revenues and total employment levels. While the various
approaches to evaluation advocated in previous sessions focused on issues
such as deadweight and displacement at a micro level, they generally provide
no mechanism for reaching a macroeconomic overview of the effects. Equally,
while these other methods provide various means by which the past impact of
such possible interventions may be assessed they provide no insight into
possible future effects.
In order to obtain these additional insights a macroeconomic forecasting
model is needed. This provides the mechanism by which the macroeconomic
consequences of individual effects can be summed (building in multiplier and
input-output linkage effects) and also providing a useful counterfactual of
what might have happened in the absence of the policy intervention. The key
conclusion from the discussion is that such models, while not a panacea, and
196
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
while providing just one leg of a complete evaluation approach, can deliver
important insights into the evaluation process. They help to remind us of all
the complex interactions that should be taken into account but which are
sometimes forgotten.
Other papers in the conference (e.g. the contribution by Wong) emphasised
the need to take into account geographical displacement effects. Macro models
cannot by themselves answer this question but they provide an important
framework within which it can be addressed.
Bartik also emphasised that rigorous evaluation is possible through a
variety of methods but including the need to link in regional econometrics
with fiscal impact and local labour market models. A mixture of techniques,
quantitative and qualitative, is needed with a focus on improving programmes
rather than “killing” poor ones.
Smith also raised the importance of synergy and macroeconomic effects
which might otherwise be neglected in a purely microeconomic evaluation.
Oldsman emphasised the importance of establishing robust baseline
data. A key aspect of this is providing a counter-factual – what would have
happened anyway, even if the policy had not been introduced/changed?
Eberts and O’Leary made the important point that social programmes
that are intended to affect local and regional economies are especially difficult
to evaluate. Their scope is much broader than simply looking at impacts on
individual actors or organisations. On the other hand, the scale of interventions
are often so small as to have indiscernible effects. A macro modelling tool
provides a means to at least get a handle on these kinds of outcomes.
Walker emphasised the need to collect longitudinal data to facilitate model
building and the development of better, more sophisticated forecasting tools.
Finally, another important issue is that polices and subsequent
evaluations may interact and evolve, changing behaviour and generating
additional capacity (Stern). This requires quite careful handling in the model
framework, in which the parameters are based on previous patterns of
behaviour and institutional and related frameworks.
Wong reminded us that intervention in the social sciences is rarely a
matter of repeating a controlled experiment. Often the carrying out of the
experiment, as well as any evaluation of it, will influence the outcomes and
behaviours of those involved. This applies with special force when using
macro models of the REMI or LEFM type. Changing behaviour may result in the
need to reassess the explicit assumption of fixed parameters based on
historical experience. Robson emphasised a similar point, citing the
importance of action orientated research, perceptual indicators and softer,
qualitative, rather than hard quantitative approaches.
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
197
7. A COMMENTARY ON FREDERICK AND GEORGE TREYZ’S PAPER AND THE WORKSHOP “ANALYSIS POLICIES...
Reference
WILSON, R., A. ASSEFA and J. BEARD (1995), “A Local Economy Forecasting Model
(LEFM) for the UK Economy”. Paper presented at the European Symposium of
Labour Market Developments, University of Warwick, 18th-19th May, 1995.
198
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
ISBN 92-64-01708-9
Evaluating Local Economic and Employment Development
How to Assess What Works among Programmes and Policies
© OECD 2004
Chapter 8
Area-based Policy Evaluation
by
Brian Robson,
Centre for Urban Policy Studies,
University of Manchester,
United Kingdom
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
199
8.
AREA-BASED POLICY EVALUATION
The policy context
Formal urban policies in Britain date from the late 1960s and, over the last
three decades, policy has placed great faith in area-based initiatives (ABIs) as
a mechanism to tackle the problems associated with dereliction and
deprivation in the large ex-industrial towns and cities of Britain (for example,
Hall and Nevin, 1999).
Since the 1980s there has been a bewildering array of different types of
policy instrument, amongst which have been (or are):
●
Action for Cities (AfC) in the 1980s which focused additional resources on
57 local authority districts (LADs).
●
City Challenge (CC) which operated between 1992 and 1998 and targeted
resources at 31 sub-district areas.
●
the Single Regeneration Budget (SRB) which rolled together the expenditures
from 21 earlier programmes, went through 6 rounds of allocation during the
1990s, and was largely targeted at small geographically defined areas which
received funding for periods of up to seven years.
●
New Deal for Communities (NDC) which currently supports 39 sub-district
areas with populations of up to 4 000 over a period of ten years.
●
Urban Regeneration Companies (URC), the first of which were created in
1999, with 11 having now been established to develop master plans mainly
for city centre areas, and with running costs funded through partnerships
between Regional Development Agencies (RDAs), LADs and English
Partnerships (EP).
Each of these programmes has been funded by the government department
with principal responsibility for urban regeneration – successively named the
Department of Environment (DoE), Department of Environment Transport and
the Regions (DETR), Department of Transport, Local Government and the Regions
(DTLR), and now the Office of the Deputy Prime Minister (ODPM).
In addition, in the last five years, there has been an increasing number of
complementary area-based initiatives from other central government
departments. Examples include the Crime Reduction Programme, Sure Start,
Education Action Zones, Employment Zones, Sports Action Zones, and Health
Action Zones – from government departments such as the Home Office,
Employment, Education, Sport and Culture, and Health.
200
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
8.
AREA-BASED POLICY EVALUATION
Such area-based approaches are predicated on the belief either that there
are area-related processes that compound the problems that are faced by
deprived individuals, or that there are efficiencies associated with the delivery
of policy within defined targeted areas. While there has been much agonising
in the academic literature about whether or not an “area effect” can be
demonstrated (Dorling et al., 2001; Atkinson and Kintrea, 2001), there seems to
be general agreement that there are administrative benefits associated with
spatial targeting – not least the potential synergy that can be achieved across
different policy domains, and the efficiency of deploying personnel and
resources within a limited number of areas. The case for area-targeting is that,
given the wide disparities in deprivation, the neighbourhood is the most
appropriate scale for fostering community identity and involvement and that
resources concentrated at small areas over a number of years can achieve a
step improvement in the circumstances of deprived areas (Lawless et al., 2000).
Over time, the focus of ABIs has changed. At the outset, in the 1970s, most
programmes broadly covered economic, social and environmental objectives,
but the additional targeted resources were generally small. In the 1980s, the
majority of funding was directed to the physical improvement of derelict areas
in an attempt to revive the working of property markets. Typical interventions
included programmes such as Enterprise Zones which offered financial
incentives to firms to locate in decayed urban sub-areas, or the Urban
Development Corporations which were run by private-sector boards with the
aim of re-furbishing the infrastructure of derelict ex-industrial sites. In the
1990s, City Challenge and the SRB programmes brought local authorities back
into the frame and broadened the focus to encompass social and economic
issues as well as environmental and property-related aims. With this change of
emphasis, two principal features have come to dominate recent approaches to
urban regeneration; a stress on partnership working and, related to this, the aim
of developing better co-ordination across policy domains.
The recent Urban White Paper (DETR, 2000) fundamentally changed the
main thrust of policy. It moved away from the previous almost exclusive
dependence on area-based initiatives and espoused the aim of “mainstreaming”
as a way of better tackling urban problems. Mainstreaming can be thought of
in three distinct ways:
●
the attempt to bend resources from main spending programmes (such as
education, social services, housing) to target areas of especial need or to
improve the quality of service delivery to such areas;
●
the attempt to learn lessons from what works in specific programmes and
projects and apply them more generally to other areas; and
●
the attempt to incorporate into mainstream services the policy lessons that
arise from specific initiatives.
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004
201
8.
AREA-BASED POLICY EVALUATION
While mainstreaming has long been a mantra within urban policy, the new
policy framework is the first to put it centre stage. Local Strategic Partnerships are
now in process of being established in local authorities and they are charged with
developing Community Strategies that consciously use mainstream as well as
specific resources from funds such as the Neighbourhood Renewal Fund (which is
targeted at 88 local authorities) to reverse the fortunes of deprived areas.
So, even though many area-based initiatives remain – not least the NDCs,
URCs and some of the later rounds of SRB funding – English urban policy has
now drawn a line under its almost exclusive dependence on area-based
initiatives.
The growth of monitoring and evaluation
From the outset of this 30-year history of urban policy, there has been a
continuous development of monitoring and evaluation under the auspices of
central government. The total resources channelled at monitoring and
evaluation have grown considerably. On the face of it, evaluation has now
become an integral part of the policy environment. As well as the evaluation
projects sponsored by funding departments of central government, generic
policy reviews are undertaken by bodies such as the Audit Commission (for
example, Audit Commission, 1989; 2002) and the National Audit Office; and
the newly-established co-ordinating units established by central government
such as the Social Exclusion Unit and the Regional Co-ordination Unit also
draw on research-based evidence with the aim of steering the direction of
policy. A recent example is the Regional Co-ordination Unit’s review of areabased initiatives (RCU, 2002) which makes recommendations about merging
and mainstreaming several of the existing separately funded regeneration
programmes from across a range of government departments. In addition to
such government-sponsored evaluation, there have been programmes
evaluating area-based policies both by research charities – especially the Area
Regeneration Programme of the Joseph Rowntree Foundation (see, for
example, Maclennan, 2000) – and by the Economic and Social Research
Council through its Cities Programme (see, for example, Begg, 2002). In this
paper, however, the focus is restricted more narrowly to governmentsponsored research.
Much of the research supported by the Department of the Environment in
the 1970s and 1980s was relatively haphazard. Most of the evaluations were ex
post and researchers had some difficulty in assembling data that could
identify the initial conditions when programmes started and in recreating
data on outputs during the lifespan of initiatives. However, since the 1990s
(and especially since City Challenge and the SRB programmes), most funding
of policy interventions has usually been contingent on continuous monitoring
202
EVALUATING LOCAL ECONOMIC AND EMPLOYMENT DEVELOPMENT – ISBN 92-64-01708-9 – © OECD 2004