Tải bản đầy đủ - 0 (trang)
14-2. Disabling and Rebuilding Nonclustered Indexes in a Destination Table

14-2. Disabling and Rebuilding Nonclustered Indexes in a Destination Table

Tải bản đầy đủ - 0trang

Chapter 14 ■ ETL Process Acceleration



CREATE PROCEDURE dbo.pr_IndexesToDisable

AS



TRUNCATE TABLE dbo.IndexList;



INSERT INTO

dbo.IndexList (TableName, SchemaName , IndexScript)



SELECT

,SSC.name

,SOB.name

,'ALTER INDEX ' + SIX.name + ' ON ' + SSC.name + '.' + SOB.name + '

DISABLE'



FROM

sys.indexes SIX

INNER JOIN sys.objects SOB

ON SIX.object_id = SOB.object_id

INNER JOIN sys.schemas AS SSC

ON SOB.schema_id = SSC.schema_id



WHERE

SOB.is_ms_shipped = 0

AND SIX.type_desc = 'NONCLUSTERED'

AND SOB.name = 'Clients' -- Enter the table name here



ORDER BY

SIX.type_desc, SOB.name, SIX.name ;

GO

3.



Execute the following code to create a stored procedure that will disable the indexes

in the selected database (C:\SQL2012DIRecipes\CH14\pr_DisableIndexes.Sql):



USE CarSales_Staging;

GO



CREATE PROCEDURE dbo.pr_DisableIndexes

AS



DECLARE @TableName NVARCHAR(128), @SchemaName NVARCHAR(128), @DisableIndex NVARCHAR(4000)



DECLARE DisableIndex_CUR CURSOR

FOR

SELECT DISTINCT TableName, SchemaName FROM dbo.IndexList



OPEN DisableIndex_CUR



FETCH NEXT FROM DisableIndex_CUR INTO @TableName, @SchemaName



WHILE @@FETCH_STATUS <> −1

BEGIN



SET @DisableIndex = 'ALTER INDEX ALL ON' + @SchemaName + '.' + @TableName + 'DISABLE'





808

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



EXEC (@DisableIndex)



FETCH NEXT FROM DisableIndex_CUR INTO @TableName, @SchemaName



END;

CLOSE DisableIndex_CUR;

DEALLOCATE DisableIndex_CUR;

4.



Run the following script to create a stored procedure that rebuilds all disabled indexes

(C:\SQL2012DIRecipes\CH14\pr_RebuildIndexes.Sql):



USE CarSales_Staging;

GO



CREATE PROCEDURE dbo.pr_RebuildIndexes

AS



DECLARE @TableName NVARCHAR(128), @SchemaName NVARCHAR(128), @RebuildIndex NVARCHAR(4000)



DECLARE RebuildIndex_CUR CURSOR



FOR

SELECT DISTINCT TableName, SchemaName FROM dbo.IndexList



OPEN RebuildIndex_CUR



FETCH NEXT FROM RebuildIndex_CUR INTO @TableName, @SchemaName



WHILE @@FETCH_STATUS <> −1

BEGIN



SET @RebuildIndex = 'ALTER INDEX ALL ON ' + @SchemaName + '.' + @TableName + ' REBUILD'



EXEC (@RebuildIndex)



FETCH NEXT FROM RebuildIndex_CUR INTO @TableName, @SchemaName



END;



CLOSE RebuildIndex_CUR;

DEALLOCATE RebuildIndex_CUR;

5.



Create a new SSIS package.



6.



Add two OLEDB connection managers. The first, named CarSales_OLEDB, connects

to the CarSales database; and the second, named CarSales_Staging_OLEDB,

connects to the CarSales_Staging database.



7.



Add a new ADO.NET connection manager named CarSales_Staging_ADONET that

you configure to connect to the CarSales_Staging database.



809

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



8.



Add an Execute SQL task named Create Index Metadata. Double-click to edit.



9.



Configure the Execute SQL task to use the ADO.NET connection manager CarSales_

Staging_ADONET. Set IsQueryStoredProcedure to True, and the SQL Statement to

dbo.pr_IndexesToDisable.



10.



Add an Execute SQL task named Disable Indexes. Connect the preceding Execute

SQL task to it. Double-click to edit.



11.



Configure the Execute SQL task to use the ADO.NET connection manager CarSales_

Staging_ADONET. Set “IsQueryStoredProcedure” to True, and the SQL Statement to

dbo.pr_DisableIndexes.



12.



Add a dataflow task. Connect the preceding Execute SQL task to it. Double-click

to edit.



13.



Add an OLEDB source task. Configure it to use CarSales_OLEDB the connection

manager and the Clients source table.



14.



Add an OLEDB destination task. Configure it to use the CarSales_OLEDB_Staging

connection manager and the Clients destination table.



15.



Click Columns and ensure that the columns are mapped correctly.



16.



Click OK to finish configuring the data flow. Return to the Data Flow pane.



17.



Add an Execute SQL task named Rebuild Indexes. Connect the preceding Data Flow

task to it. Double-click to edit.



18.



Configure the Execute SQL task to use the CarSales_Staging_ADONET connection

manager. Set IsQueryStoredProcedure to True, and the SQL Statement to dbo.pr_

RebuildIndexes. The data flow should look like Figure 14-4.



810

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



Figure 14-4.  The data flow when disabling and rebuilding nonclustered indexes

Having disabled any nonclustered indexes first, you can now run the package to load the data, and rebuild

them after the load.



How It Works

When loading large amounts of data, it is frequently much faster to disable any nonclustered indexes (especially

if there are several of them) and rebuild them after the data load. This is due to the following:





The overhead required to maintain multiple indexes is considerable.







SQL Server cannot use the Bulk Load API if there are nonclustered indexes on a table

(in most cases anyway, but more of that in Recipe 14-10).



Consequently, you could find yourself needing a technique that detects and disables any nonclustered

indexes before running a data load, and then rebuilds the indexes on the tables once the load phase has

completed. This is what happens in the current recipe. We use a persisted table to store the code required to

disable any indexes, as well as the table(s) and schema(s) to which they belong. This is what the

dbo.pr_IndexesToDisable stored procedure does. After that, a simple cursor in the dbo.pr_DisableIndexes

stored procedure loops over the code that disables indexes and applies it. Once the load has finished, the same

table provides the information (table(s) and schema(s)) needed to REBUILD the indexes—using the stored

procedure dbo.pr_RebuildIndexes.



811

www.it-ebooks.info



Chapter 14 ■ ETL Process Acceleration



This technique is ideal when you wish to keep clustered indexes (which means that your data will have to

be loaded presorted according to the index key)—or there are no clustered indexes. I realize that this approach

may seem convoluted, but it avoids hard-coding index names into an SSIS package, and so allows the package to

manage changes to indexes without any input from the developer.



Hints, Tips, and Traps





Never disable a clustered index! You will render the table inaccessible.







Remember that the stored procedure sp_foreachtable can be used as a quick way to

rebuild all the indexes in a database, too.







By judicious extension of the WHERE clause, you can perform this operation for multiple

tables. You will have to alter the ALTER INDEX...REBUILD statement to take account of this.







You can REORGANIZE instead of REBUILD indexes if the table fragmentation is minimal, and

you are adding or replacing very small amounts of data. However, you cannot REORGANIZE

indexes where ALLOW_PAGE_LOCKS is set to OFF.







You must disable ColumnStore indexes before modifying the data in the underlying table.

You can identify ColumnStore indexes in the sys.indexes system view since they have the

type_desc 'NONCLUSTERED COLUMNSTORE'.



14-3. Persisting Destination Database Index Metadata

Problem

You want to gather relevant index metadata about your destination database and store it independent of the SQL

Server system tables. This way, you can drop indexes with impunity and re-create them after a data load.



Solution

Query SQL Server metadata to gather the index metadata and store it in a database table. The following script

shows one way of doing this.

1.



Create a table in the destination database to store the index metadata using the

following DDL (C:\SQL2012DIRecipes\CH14\tblMetaData_Indexes.Sql):

IF OBJECT_ID('dbo.MetaData_Indexes') IS NOT NULL

DROP TABLE dbo.MetaData_Indexes;

CREATE TABLE CarSales_Staging.dbo.MetaData_Indexes

(

SERVER_NAME NVARCHAR(128) NULL,

DATABASE_NAME NVARCHAR(128) NULL,

SCHEMA_NAME NVARCHAR(128) NULL,

TABLE_NAME NVARCHAR(128) NULL,

INDEX_NAME NVARCHAR(128) NULL,

name NVARCHAR(128) NULL,

index_column_id INT NULL,

key_ordinal INT NULL,

type_desc NVARCHAR(60) NULL,

is_unique BIT NULL,

ignore_dup_key BIT NULL,



812

www.it-ebooks.info



Chapter 14 ■ etL proCess aCCeLeration



is_primary_key BIT NULL,

is_unique_constraint BIT NULL,

fill_factor tinyINT NULL,

is_padded BIT NULL,

is_disabled BIT NULL,

allow_row_locks BIT NULL,

allow_page_locks BIT NULL,

has_filter BIT NULL,

filter_definition NVARCHAR(max) NULL,

is_included_column BIT NULL,

is_descending_key BIT NULL,

FileGroup NVARCHAR(128) NULL,

TableObjectID INT NULL,

IsNoRecompute BIT NULL,

IndexDepth INT NULL,

IsAutoStatistics BIT NULL,

IsClustered BIT NULL,

IsFulltextKey BIT NULL,

DataSpace NVARCHAR(128) NULL

) ;

GO

2.



Run the following code to gather the index metadata (C:\SQL2012DIRecipes\CH14\

GatherIndexMetadata.Sql):

DECLARE @SERVER_NAME NVARCHAR(128) = @@SERVERNAME

DECLARE @DATABASE_NAME NVARCHAR(128) = DB_NAME()

INSERT INTO MetaData_Indexes

(

SERVER_NAME

,DATABASE_NAME

,SCHEMA_NAME

,TABLE_NAME

,INDEX_NAME

,name

,index_column_id

,key_ordinal

,type_desc

,is_unique

,ignore_dup_key

,is_primary_key

,is_unique_constraint

,fill_factor

,is_padded

,is_disabled

,allow_row_locks

,allow_page_locks

,has_filter

,filter_definition

,is_included_column

,is_descending_key



813

www.it-ebooks.info



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

14-2. Disabling and Rebuilding Nonclustered Indexes in a Destination Table

Tải bản đầy đủ ngay(0 tr)

×