Tải bản đầy đủ - 0 (trang)
12-2. Pulling Changes into a Destination Table with Change Tracking

12-2. Pulling Changes into a Destination Table with Change Tracking

Tải bản đầy đủ - 0trang

Chapter 12 ■ Change Tracking and Change Data Capture



-- Get all data for INSERTS/UPDATES/DELETES into temp tables

-- Deletes

DECLARE @DeleteSQL VARCHAR(8000) =

'SELECT ID

FROM OPENQUERY(R2, ''SELECT

ID

FROM

CHANGETABLE(CHANGES Carsales.dbo.Client,' + CAST(@LAST_SYNC_VERSION AS VARCHAR(20)) + ')

AS CT

WHERE CT.SYS_CHANGE_OPERATION = ''''D'''''')'

IF OBJECT_ID('tempdb..#ClientDeletes') IS NOT NULL

DROP TABLE tempdb..#ClientDeletes

CREATE TABLE #ClientDeletes (ID INT)

INSERT INTO #ClientDeletes EXEC (@DeleteSQL)

-- Inserts

DECLARE @InsertsSQL VARCHAR(8000) =

'SELECT ID,ClientName,Country,Town,County,Address1,Address2,ClientType,ClientSize

FROM OPENQUERY(R2,

''SELECT SRC.ID,ClientName,Country,Town,County,Address1,Address2,

ClientType,ClientSize

FROM

Carsales.dbo.Client SRC

INNER JOIN CHANGETABLE(

CHANGES Carsales.dbo.Client, ' +

CAST(@LAST_SYNC_VERSION

AS VARCHAR(20)) + ') AS CT

ON SRC.ID = CT.ID

WHERE

CT.SYS_CHANGE_OPERATION = ''''I'''''')'

IF OBJECT_ID('tempdb..#ClientInserts') IS NOT NULL DROP TABLE tempdb..#ClientInserts

CREATE TABLE #ClientInserts

(

ID INT NOT NULL,

ClientName VARCHAR(150) NULL,

Country VARCHAR(50) NULL,

Town VARCHAR(50) NULL,

County VARCHAR(50) NULL,

Address1 VARCHAR(50) NULL,

Address2 VARCHAR(50) NULL,

ClientType VARCHAR(20) NULL,

ClientSize VARCHAR(10) NULL

)



689

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



INSERT INTO #ClientInserts EXEC (@InsertsSQL)

-- Updates

DECLARE @UpdatesSQL VARCHAR(8000) =

'SELECT

ID,ClientName,Country,Town,County,Address1,Address2,

ClientType,ClientSize

FROM

OPENQUERY(R2, ''SELECT

SRC.ID,ClientName,Country,Town,

County,Address1,Address2,

ClientType,ClientSize

FROM

Carsales.dbo.Client SRC

INNER JOIN

CHANGETABLE(CHANGES

Carsales.dbo.Client,'

+ CAST(@LAST_SYNC_VERSION AS VARCHAR(20))

+ ') AS CT

ON SRC.ID = CT.ID

WHERE

CT.SYS_CHANGE_OPERATION = ''''U'''''')' ;

IF OBJECT_ID('tempdb..#ClientUpdates') IS NOT NULL DROP TABLE tempdb..#ClientUpdates;

CREATE TABLE #ClientUpdates

(

ID INT NOT NULL,

ClientName VARCHAR(150) NULL,

Country VARCHAR(50) NULL,

Town VARCHAR(50) NULL,

County VARCHAR(50) NULL,

Address1 VARCHAR(50) NULL,

Address2 VARCHAR(50) NULL,

ClientType VARCHAR(20) NULL,

ClientSize VARCHAR(10) NULL

) ;

INSERT INTO

#ClientUpdates

(ID,ClientName,Country,Town,County,Address1,Address2,ClientType,ClientSize) EXEC

(@UpdatesSQL) ;

-- Carry out INSERTS/UPDATES/DELETES

-- Inserts

INSERT INTO



SELECT

FROM



CarSales_Staging.dbo.Client

(ID, ClientName, Country, Town, County, Address1, Address2,

ClientType, ClientSize)

ID, ClientName, Country,Town, County, Address1, Address2,

ClientType, ClientSize

#ClientInserts



690

www.it-ebooks.info



Chapter 12 ■ Change traCking and Change data Capture



-- Updates

UPDATE

SET



DST

DST.ClientName = UPD.ClientName

,DST.Country = UPD.Country

,DST.Town = UPD.Town

,DST.County = UPD.County

,DST.Address1 = UPD.Address1

,DST.Address2 = UPD.Address2

,DST.ClientType = UPD.ClientType

,DST.ClientSize = UPD.ClientSize

CarSales_Staging.dbo.Client DST

#ClientUpdates UPD

ON DST.ID = UPD.ID



FROM

INNER JOIN



-- Deletes

DELETE FROM

FROM

INNER JOIN



DST

CarSales_Staging.dbo.Client DST

#ClientDeletes DLT

ONDLT.ID = DST.ID



-- To set the new LAST_SYNC_VERSION

EXEC R2.CarSales.sys.sp_updateextendedproperty

@level0type = N'SCHEMA'

,@level0name = dbo

,@level1type = N'TABLE'

,@level1name = Client

,@name = LAST_SYNC_VERSION

,@value = @CURRENT_VERSION

;

END



How It Works

You can perform “push” data synchronization over linked servers using Change Tracking, but can it be done

using a “pull” approach? The answer is a guarded “yes.” Guarded not because there is any inherent difficulty,

but because you will need to apply a series of workarounds to the limitations of the Change Tracking functions,

which are supplied out of the box, and which we saw in the Recipe 12-1.

So what are the potential problems?





The CHANGETABLE function will not work using a linked (four-part notation) server.







The CHANGE_TRACKING_MIN_VALID_VERSION function will not work over a linked server.







Wrapping the CHANGETABLE function in a table-valued user-defined function on the

source server will not work, because UDFs also are problematic over linked servers.



691

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture







Wrapping the CHANGETABLE function in a stored procedure on the source server can

work—but extracting the data requires MSDTC (the Distributed Transaction Coordinator)

to be running on the source server, because it will create an implicit transaction. Also,

you need to configure the linked server for RPC (you will be executing stored procedures

against a remote server). These two limitations can be a deal-breaker for some DBAs—as

well as a considerable hassle. So for these reasons, we will avoid this approach.



This leaves us with OPENQUERY and dynamic SQL as a solution. It is not perfect, but it allows us to return

all the required data (minimum change version, maximum change version, as well as all the data pertaining to

Inserts, Updates, and Deletes) without any hassle as far as the server-level configuration is concerned.



Hints, Tips, and Traps





You can use table variables instead of temporary tables if you prefer. For larger datasets

temporary tables may be preferable, as they can be indexed, which can be useful in some

situations.







Here too you can use specific column tracking using the COLUMNPROPERTY function, as

described in the “How It Works” section of Recipe 12-1.



12-3. Using Change Tracking as Part of a Structured

ETL Process

Problem

You want to use Change Tracking as part of a structured ETL process.



Solution

Use Change Tracking information as part of an SSIS package to ensure that data modifications in a source table

are applied to a destination table. Since this package is a little complex, I suggest that before leaping in to create

it, you first read it through to get an idea of where it is going. Having said that, the following steps illustrate how it

is done.

1.



Run steps 1 through 4 in Recipe 12-1 to enable Change Tracking on the source server

(R2), and place the destination table on the local server (unless you have already done

this, of course).



2.



Create a new SSIS package, and add the three following connection managers:

Source_OLEDB:



An OLEDB connection to the source database.



Source_ADONET:



An ADO.NET connection to the source database.



Destination_OLEDB:



An OLEDB connection to the destination database.

Set the RetainSameConnection property to False.



692

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



3.



Add the following package-scoped variables:



VariableName



Type



Value



CurrentVersion



Int64



0



DeleteTable



String



TMP_Deletes



LastSynchVersion



Int64



0



MinValidVersion



Int64



0



SQLDelete



String



SELECT ID FROM dbo.Client



SQLInsert



String



SELECT ID, ClientName, Country, Town, County,

Address1, Address2, ClientType, ClientSize

FROM dbo.Client



SQLUpdate



String



SELECT ID, ClientName, Country, Town, County,

Address1, Address2, ClientType, ClientSize

FROM dbo.Client



UpdateTable

4.



String



TMP_Updates



Add an Execute SQL task, configured as follows:

Name:



Get Parameters



Connection Type:



Source_ADONET



SQL Statement:



SELECT @CurrentVersion = CHANGE_TRACKING_CURRENT_VERSION();

SELECT @MinValidVersion =

CHANGE_TRACKING_MIN_VALID_VERSION(OBJECT_ID('dbo.Client'));

SELECT @LastSynchVersion = CAST(value AS BIGINT)



5.



6.



FROM



sys.extended_properties



WHERE



major_id = OBJECT_ID('dbo.Client')



AND



name = N'LAST_SYNC_VERSION';



Map the following parameters:



Variable Name



Direction



Data Type



Parameter Name



User::CurrentVersion



Output



Int64



@ CurrentVersion



User::MinValidVersion



Output



Int64



@MinValidVersion



User::LastSynchVersion



Output



Int64



@LastSynchVersion



Confirm your alterations.



693

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



7.



8.



9.



Add a Script task, name it Define SQL Variables, and connect the task Get Parameters

to it. Double-click the precedence constraint and set to:

Evaluation Operation:



Expression and Constraint



Value:



Success



Expression:



@LastSynchVersion >= @MinValidVersion



Add the following variables:

Read-only:



LastSynchVersion



Read-Write:



SQLDelete,SQLInsert,SQLUpdate



Add the following script, which will set the SQL SELECT required to return only the

delta data (C:\SQL2012DIRecipes\CH11\ChangeTrackingSSIS.vb):



Public Sub Main()

Dim SQLInsertText As String = "SELECT SRC. ID, ClientName, Country, Town, County, 

Address1, Address2, ClientType, ClientSize" _

& " FROM dbo.Client SRC" _

& " INNER JOIN CHANGETABLE(CHANGES Client, " 

 

& Dts.Variables("LastSynchVersion").Value & ") AS CT" _

& " ON SRC.ID = CT.ID" _

& " WHERE CT.SYS_CHANGE_OPERATION = 'I'"

Dts.Variables("SQLInsert").Value = SQLInsertText

Dim SQLDeleteText As String = "SELECT ID FROM CHANGETABLE(CHANGES Client, " 

& Dts.Variables("LastSynchVersion").Value & ") AS DEL" _

& " WHERE SYS_CHANGE_OPERATION = 'D'"

Dts.Variables("SQLDelete").Value = SQLDeleteText

Dim SQLUpdateText As String = "SELECT SRC. ID, ClientName, Country, Town, 

County, Address1, Address2, ClientType, ClientSize " _

& " FROM dbo.Client SRC" _

& " INNER JOIN CHANGETABLE(CHANGES Client, " 

& Dts.Variables("LastSynchVersion").Value & ") AS CT" _

& " ON SRC.ID = CT.ID" _

& " WHERE

CT.SYS_CHANGE_OPERATION = 'U'"

Dts.Variables("SQLUpdate").Value = SQLUpdateText

Dts.TaskResult = ScriptResults.Success

End Sub



694

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



10.



Confirm your changes.



11.



Add an Execute SQL task, name it Prepare temporary tables, connect the Define

SQL Variables task to it and configure as follows (C:\SQL2012DIRecipes\CH11\

PrepareTemporaryTables.Sql):

Name:



Get Parameters



Connection Type:



OLEDB



Connection:



Source_OLEDB



SQL Statement:



CREATE TABLE ##Tmp_Deletes (ID INT);

CREATE TABLE ##Tmp_Updates

(

ID INT NOT NULL,

ClientName VARCHAR(150) NULL,

Country VARCHAR(50) NULL,

Town VARCHAR(50) NULL,

County VARCHAR(50) NULL,

Address1 VARCHAR(50) NULL,

Address2 VARCHAR(50) NULL,

ClientType VARCHAR(20) NULL,

ClientSize VARCHAR(10) NULL

)



12.



Confirm your alterations.



13.



Add a Sequence container, which you name Upsert and Delete Deltas, and connect

the “Prepare temporary tables” task to it.



695

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



14.



Inside the Sequence container, add three Data Flow tasks named Inserts, Updates, and

Deletes. Set DelayValidation for Deletes and Updates to True. Configure as follows:

Inserts

Source (OLEDB)

OLEDB Connection Manager:



Source_OLEDB



Data Access Mode:



SQLCommandFromVariable



Variable Name:



SQLInsert



Destination (OLEDB)

OLEDB Connection Manager:



Destination_OLEDB



Data Access Mode:



Table or View–Fast Load



Name of Table or View:



dbo.Clients



Keep Identity:



True



Updates

Source (OLEDB)

OLEDB Connection Manager:



Source_OLEDB



Data Access Mode:



SQLCommandFromVariable



Variable Name:



SQLUpdate



Destination (OLEDB)

OLEDB Connection Manager:



Destination_OLEDB



Data Access Mode:



Table or View name from variable



Variable Name:



UpdateTable



Validate External Metadata (property):



False



Deletes

Source (OLEDB)

OLEDB Connection Manager:



Source_OLEDB



Data Access Mode:



SQLCommandFromVariable



Variable Name:



SQLDelete



Destination (OLEDB)

OLEDB Connection Manager:



Destination_OLEDB



Data Access Mode:



Table or View name from variable



Variable Name:



DeleteTable



Validate External Metadata (property):



False



696

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



15.



Ensure that all source to destination mappings for all three destinations are correct,

and confirm your changes.



16.



Add an Execute SQL task, name it Delete Data, and connect the “Upsert and Delete

Deltas” Sequence container to it. Configure as follows:

Connection Type:



OLEDB



Connection:



Destination_OLEDB



SQL Statement:



DELETE



DST



FROM



dbo.Client DST



INNER JOIN



Tmp_Deletes DL



ON



DST.ID = DL.ID



17.



Confirm your changes.



18.



Add an Execute SQL task, name it Update Data, and connect the “Delete Data”

Execute SQL task to it. Configure as follows

(C:\SQL2012DIRecipes\CH11\UpdateData.Sql):

Connection Type:



OLEDB



Connection:



Destination_OLEDB



SQL Statement:



UPDATE DST

SET

DST.ClientName = SRC.ClientName

,DST.Country = SRC.Country

,DST.Town = SRC.Town

,DST.County = SRC.County

,DST.Address1 = SRC.Address1

,DST.Address2 = SRC.Address2

,DST.ClientType = SRC.ClientType

,DST.ClientSize = SRC.ClientSize

FROM



dbo.Client DST



INNER JOIN



Tmp_Updates SRC

ON DST.ID = SRC.ID



19.



Confirm your changes.



20.



Add an Execute SQL task, name it Set latest version number, and connect the

“Update Data” Execute SQL task to it.



697

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



21.



Configure the Execute SQL task as follows

(C:\SQL2012DIRecipes\CH11\UpdateExtendedPropertySSIS.Sql):

Connection Type:



ADO.NET



Connection:



Source_ADONET



SQL Statement:



EXEC sys.sp_updateextendedproperty

@level0type = N'SCHEMA'

,@level0name = dbo

,@level1type = N'TABLE'

,@level1name = Client

,@name = LAST_SYNC_VERSION

,@value = @CurrentVersion

;



22.



23.



Add the following parameter:



Name



Direction



Type



Value



User::CurrentVersion



Input



Int64



@CurrentVersion



Confirm your changes with OK. The package should look like Figure 12-1.



698

www.it-ebooks.info



Chapter 12 ■ Change Tracking and Change Data Capture



Figure 12-1.  Change Tracking process flow in SSIS



How It Works

You can also use Change Tracking information in an SSIS package to ensure that only delta data changes are

applied to a destination database. This presumes that Change Tracking has been enabled and configured for

a source table (or tables). Unfortunately, you have to convince the source system DBA to configure Change

Tracking, which might not be easy.

I want to draw your attention to the fact that this recipe uses two “scaffolding” techniques that will help

create the package more easily, but which will be altered during (or just before) processing:





One, I use variables to define the source data that refer directly to the source

tables—without using Change Tracking. These variables will be replaced during

execution by the code required to select delta data only.







Two, I start by using permanent “scratch” tables in the destination database for the delta

deletes and updates, and finish by switching (once everything is tested and ready for

deployment) to temporary tables.



699

www.it-ebooks.info



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

12-2. Pulling Changes into a Destination Table with Change Tracking

Tải bản đầy đủ ngay(0 tr)

×