Tải bản đầy đủ - 0 (trang)
14-3. Persisting Destination Database Index Metadata

14-3. Persisting Destination Database Index Metadata

Tải bản đầy đủ - 0trang

Chapter 14 ■ etL proCess aCCeLeration



is_primary_key BIT NULL,

is_unique_constraint BIT NULL,

fill_factor tinyINT NULL,

is_padded BIT NULL,

is_disabled BIT NULL,

allow_row_locks BIT NULL,

allow_page_locks BIT NULL,

has_filter BIT NULL,

filter_definition NVARCHAR(max) NULL,

is_included_column BIT NULL,

is_descending_key BIT NULL,

FileGroup NVARCHAR(128) NULL,

TableObjectID INT NULL,

IsNoRecompute BIT NULL,

IndexDepth INT NULL,

IsAutoStatistics BIT NULL,

IsClustered BIT NULL,

IsFulltextKey BIT NULL,

DataSpace NVARCHAR(128) NULL

) ;

GO

2.



Run the following code to gather the index metadata (C:\SQL2012DIRecipes\CH14\

GatherIndexMetadata.Sql):

DECLARE @SERVER_NAME NVARCHAR(128) = @@SERVERNAME

DECLARE @DATABASE_NAME NVARCHAR(128) = DB_NAME()

INSERT INTO MetaData_Indexes

(

SERVER_NAME

,DATABASE_NAME

,SCHEMA_NAME

,TABLE_NAME

,INDEX_NAME

,name

,index_column_id

,key_ordinal

,type_desc

,is_unique

,ignore_dup_key

,is_primary_key

,is_unique_constraint

,fill_factor

,is_padded

,is_disabled

,allow_row_locks

,allow_page_locks

,has_filter

,filter_definition

,is_included_column

,is_descending_key



813

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



,FileGroup

,TableObjectID

,IsNoRecompute

,IndexDepth

,IsAutoStatistics

,IsClustered

,IsFulltextKey

,DataSpace

)



SELECT DISTINCT TOP (100) PERCENT

@SERVER_NAME

,@DATABASE_NAME

,SCH.name AS SCHEMA_NAME

,TBL.name AS TABLE_NAME

,SIX.name AS INDEX_NAME

,COL.name

,SIC.index_column_id

,SIC.key_ordinal

,SIX.type_desc

,SIX.is_unique

,SIX.ignore_dup_key

,SIX.is_primary_key

,SIX.is_unique_constraint

,SIX.fill_factor

,SIX.is_padded

,SIX.is_disabled

,SIX.allow_row_locks

,SIX.allow_page_locks

,SIX.has_filter

,SIX.filter_definition

,SIC.is_included_column

,SIC.is_descending_key

,CAST(NULL AS VARCHAR(128))

,TBL.object_id

,CAST(NULL AS BIT)

,INDEXPROPERTY(TBL.object_id, SIX.name,'IndexDepth') AS IndexDepth

,INDEXPROPERTY(TBL.object_id, SIX.name,'IsAutoStatistics') AS IsAutoStatistics

,INDEXPROPERTY(TBL.object_id, SIX.name,'IsClustered') AS IsClustered

,INDEXPROPERTY(TBL.object_id, SIX.name,'IsFulltextKey') AS IsFulltextKey

,DSP.name AS DataSpace



FROM

sys.data_spaces DSP

INNER JOIN sys.indexes SIX

ON DSP.data_space_id = SIX.data_space_id

INNER JOIN sys.tables TBL

ON SIX.object_id = TBL.object_id

INNER JOIN sys.schemas SCH

ON TBL.schema_id = SCH.schema_id

INNER JOIN sys.index_columns SIC

ON SIX.index_id = SIC.index_id



814

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



AND SIX.object_id = SIC.object_id

INNER JOIN sys.columns COL

ON SIC.column_id = COL.column_id

AND TBL.object_id = COL.object_id

LEFT OUTER JOIN

sys.xml_indexes XMI

ON SIX.name = XMI.name

AND SIX.object_id = XMI.object_id



WHERE



TBL.is_ms_shipped = 0

AND XMI.name IS NULL





ORDER BY SCHEMA_NAME, TABLE_NAME, INDEX_NAME, SIC.key_ordinal



-- Add FileGroup, NoRecompute, note that online & drop existing are not stored in

-- metadata



IF OBJECT_ID('tempdb..#Tmp_IndexFileGroups') IS NOT NULL

DROP TABLE tempdb..#Tmp_IndexFileGroups;



SELECT DISTINCT

DSP.name AS DataSpace

,TBL.name AS TABLE_NAME

,TBL.object_id AS TableObjectID

,SIX.name

,SIX.type_desc

,STT.no_recompute



INTO #Tmp_IndexFileGroups



FROM

sys.data_spaces DSP

INNER JOIN sys.indexes SIX

ON DSP.data_space_id = SIX.data_space_id

INNER JOIN sys.tables TBL

ON SIX.object_id = TBL.object_id

INNER JOIN sys.stats STT

ON STT.object_id = TBL.object_id

AND STT.name = SIX.name



WHERE

SIX.name IS NOT NULL



-- Update Filegroup





UPDATE

D

SET

D.FileGroup = Tmp.DataSpace

,D.IsNoRecompute = TMP.no_recompute

FROM

MetaData_Indexes D

INNER JOIN #Tmp_IndexFileGroups TMP

ON D.TableObjectID = Tmp.TableObjectID



815

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



How It Works

Dropping and re-creating indexes—as opposed to disabling and rebuilding them, which we saw in Recipe 14-2—

requires you to be able to store DROP and CREATE index scripts. One way to obtain these is to use SQL Server

Management Studio to extract index scripts either individually or for a selection of tables. These can then be

stored in text files or stored procedures that you call. However, it can become very wearing to maintain a

script-based solution like this, and so I have another suggestion, which is to gather and store, independently, the

index metadata from the system views in a denormalized form, which then allows you to drop and create any of

your indexes whenever you want. Although more work, this solution does have several advantages:





It is easy to regenerate the metadata as indexes are added, deleted, and updated in the

database.







You do not have to handle a parallel set of scripts.







It suits ETL tasks where your tests show that it is faster to load the data into heaps and add

clustered indexes after the data load—possibly because the source data is not sorted and

adding sort transforms in SSIS creates too much memory pressure.



In this recipe, then, is a script that gathers and stores the metadata needed to drop and create (nearly) all the

indexes in a staging database and store the index metadata in persisted tables on disk.

Once you have learned how to use this script, it becomes part of a more dynamic ETL process. This is done by





Creating and executing the DROP script before the data load.







Creating and executing the CREATE after the data load.



However, these extensions of the process are the subject of Recipes 14-4 and 14-5, respectively.

The fields that are stored and used to drop and re-create the indexes need some explaining, as provided in

Table 14-2.

Table 14-2.  Fields Used to Store Index Metadata



Field Name



Source Table



Description



SERVER_NAME



Gets the server name from the current environment.



DATABASE_NAME



Gets the database name from the current environment.



SCHEMA_NAME



sys.tables



The schema name.



TABLE_NAME



sys.tables



The table name.



INDEX_NAME



sys.indexes



The index name.



name



sys.columns



The column name.



index_column_id



sys.index_columns



The internal ID of the index column.



key_ordinal



sys.index_columns



The position of the index column.



type_desc



sys.indexes



Full description of the type of index.



is_unique



sys.indexes



Is the index unique?



ignore_dup_key



sys.indexes



Is a duplicate key to be ignored?



is_primary_key



sys.indexes



Is this column part of a primary key?

(continued)



816

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



Table 14-2.  (continued)



Field Name



Source Table



Description



is_unique_constraint



sys.indexes



Is this column part of a unique constraint?



fill_factor



sys.indexes



The fill factor for the index.



is_padded



sys.indexes



Is the index padded?



is_disabled



sys.indexes



Is the index disabled?



allow_row_locks



sys.indexes



Does the index allow row locking?



allow_page_locks



sys.indexes



Does the index allow page locking?



has_filter



sys.indexes



Does the index have a filter (i.e., is it a filtered index)?



filter_definition



sys.indexes



The definition of any filter, if there is one.



is_included_column



sys.indexes



The column is part of a covering index.



is_descending_key



sys.indexes



The sort key is DESC.



FileGroup



sys.tables



The file group used for storage.



TableObjectID



sys.stats



The internal object ID.



IsNoRecompute



sys.stats



Indicates that a computed column is not to be recalculated.



IndexDepth



sys.stats



The index level.



IsAutoStatistics



sys.stats



Index statistics are created automatically.



IsClustered



sys.stats



The index is a clustered index.



IsFulltextKey



sys.stats



This is the key for a full text index.



DataSpace



sys.dataspaces



The dataspace for storage.



■■Note Stored index metadata can become a core element in ETL process index maintenance. Instead of

truncating the MetaData_Indexes table at the start of each process, you can merely update the table with new

index metadata—and delete unused index metadata. You can then add a flag column to indicate index priority and

sequencing. This allows you to classify essential indexes as priority one, for example, and process them early in the

ETL cycle. You continue in this way until any remaining indexes are processed at the end of the cycle.



Hints, Tips, and Traps





Always script your staging database and make a backup copy before dropping indexes.







The script assumes no need for quoted identifiers in the metadata. If this is not the case

with your database naming convention, then you need to handle quoted identifiers.







You can tweak the script to select all the tables that follow a specific naming convention,

or all the objects in a specific schema by judicious tweaks of the WHERE clause of the initial

SELECT statement.



817

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration







No spatial index management is handled in this script.







No full-text index management is handledin this script—this approach is designed for

staging databases.







No column indexes are managed in this script—these have to be dropped and re-created

in any case.







The scripts used to store the index metadata DROP and subsequently CREATE the indexes

can be turned into stored procedures and called from SSIS Execute SQL tasks in an SSIS

package. Recipe 14-2 is an example of this.







The index metadata can be stored in a separate database or a separate schema, if you

prefer.







No IF EXISTS trapping has been added to the script since we are dealing with staging

databases, where indexes are regularly dropped. You may prefer to add this to the script.







I have no qualms about using cursors in these kinds of script. The overhead is minimal

given the small number of records processed. In any case the cursor overhead is

infinitesimal compared to that used by an indexing process.



14-4. Scripting and Executing DROP Statements for

Destination Database Indexes

Problem

You want to drop all existing indexes in a staging database before running a data load.



Solution

To remove all the existing indexes, use the index metadata that you obtained using the technique described in

Recipe 14-3 to generate and run the DROP scripts. The following is the script to do this

(C:\SQL2012DIRecipes\CH14\DropIndexes.Sql):

-- Create table to hold script elements



IF OBJECT_ID('tempdb..#ScriptElements') IS NOT NULL

DROP TABLE tempdb..#ScriptElements;



CREATE TABLE #ScriptElements (ID INT IDENTITY(1,1), ScriptElement NVARCHAR(MAX))



-- Non-Clustered Indexes



INSERT INTO #ScriptElements (ScriptElement)



SELECT DISTINCT

'DROP INDEX '

+ INDEX_NAME

+ ' ON '

+ DATABASE_NAME + '.' + SCHEMA_NAME + '.' + TABLE_NAME



FROM

MetaData_Indexes





818



www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



WHERE



type_desc = 'NONCLUSTERED'

AND is_primary_key = 0

AND is_unique_constraint = 0



-- Unique constraints



INSERT INTO #ScriptElements (ScriptElement)



SELECT DISTINCT

'ALTER TABLE '

+ DATABASE_NAME + '.' + SCHEMA_NAME + '.' + TABLE_NAME

+ ' DROP CONSTRAINT '

+ INDEX_NAME



FROM

MetaData_Indexes

WHERE

is_unique_constraint = 1





-- Clustered Indexes



INSERT INTO #ScriptElements (ScriptElement)



SELECT DISTINCT

'DROP INDEX '

+ INDEX_NAME

+ ' ON '

+ DATABASE_NAME + '.' + SCHEMA_NAME + '.' + TABLE_NAME



FROM

MetaData_Indexes

WHERE

type_desc = 'CLUSTERED'

AND is_primary_key = 0



-- Primary Key Indexes



INSERT INTO #ScriptElements (ScriptElement)



SELECT DISTINCT

'ALTER TABLE '

+ DATABASE_NAME + '.' + SCHEMA_NAME + '.' + TABLE_NAME

+ ' DROP CONSTRAINT '

+ INDEX_NAME



FROM

MetaData_Indexes

WHERE

is_primary_key = 1



-- Create and execute DROP scripts



DECLARE @DropIndex NVARCHAR(MAX)



DECLARE DropIndex_CUR CURSOR





819

www.it-ebooks.info



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

14-3. Persisting Destination Database Index Metadata

Tải bản đầy đủ ngay(0 tr)

×