Tải bản đầy đủ - 0 (trang)
6-6. Importing Files into SQL Server on a Regular Basis

6-6. Importing Files into SQL Server on a Regular Basis

Tải bản đầy đủ - 0trang

Chapter 6 ■ Miscellaneous Data Sources



Variable



Type



DocName



String



DocFullName



String



DocPath



String



DocType



String



5.



Add a Foreach Loop container to the Control Flow pane. Name it Collect All

Documents in Directory. Double-click to edit it. Click Collection on the left.



6.



Select the Foreach File Enumerator as the Foreach Loop Editor type. Browse to or

enter the folder containing the documents that you wish to load, as well as the file

filter (*.Docx in this example). Ensure that you select Fully Qualified as the file type.

You should end up with a dialog box that looks like Figure 6-6.



Figure 6-6.  Configuring a Foreach Loop to load multiple files



298

www.it-ebooks.info



Chapter 6 ■ Miscellaneous Data Sources



7.



Click Variable Mappings on the left, and select the User::DocFullName variable as the

variable that SSIS will use to hold the file name when loading the file into SQL Server.

The dialog box should look like Figure 6-7.



Figure 6-7.  Setting the variable for multiple file loads

8.



Click OK to confirm the parameters of the Foreach Loop container.



9.



Add a Script task inside the Foreach Loop container. Name the Script task Get File

and Path Elements. Double-click to edit and click Script on the left. In the pane on

the right, enter the following:



Variable Type



Variable



ReadOnlyVariables



DocFullName



ReadWriteVariables



DocName



ReadWriteVariables



DocPath



ReadWriteVariables



DocType



299

www.it-ebooks.info



Chapter 6 ■ MisCellaneous Data sourCes



You should see something like Figure 6-8.



Figure 6-8. The Script task for multiple file loads

10.



Select Microsoft Visual Basic 2010 as the Script Language. Click Edit Script.



11.



Add the following line to the Imports region:

Imports System.IO



12.



Replace the Main method with the following code

(C:\SQL2012DIRecipes\CH06\DocuBlobs.vb):

Public Sub Main()

Dim sDocName As String

sDocName = Dts.Variables("DocFullName").Value.ToString



300

www.it-ebooks.info



Chapter 6 ■ Miscellaneous Data Sources



Dts.Variables("DocType").Value = Right(Path.GetExtension(sDocName), 3)

Dts.Variables("DocPath").Value = Path.GetDirectoryName(sDocName)

Dts.Variables("DocName").Value = Path.GetFileName(sDocName)

Dts.TaskResult = ScriptResults.Success

End Sub

13.



Close the Script editor. Click OK to close the Script task.



14.



Add an Execute SQL task inside the Foreach Loop container, under the Script task,

and connect the Script task to the SQL task.



15.



Double-click the SQL task to edit it. Select ADO.NET as the connection type. Create

or select a .NET connection to the database where you have created the stored

procedure (CarSales_Staging) and select dbo.DocumentList as the destination table.



16.



Click Parameter Mapping on the left. Add four parameters, configured as follows:



VariableName



Direction



DataType



ParameterName



User::DocFullName



Input



String



@DocFullName



User::DocType



Input



String



@DocType



User::DocPath



Input



String



@DocPath



User::DocName



Input



String



@DocName



This will give you the dialog box shown in Figure 6-9.



301

www.it-ebooks.info



Chapter 6 ■ Miscellaneous Data Sources



Figure 6-9.  Setting the parameters for the pr_LoadBLOBsFromSSIS stored procedure

17.



Set IsQueryStoredProcedure to True.



18.



 dd the following to the SQLStatement: pr_LoadBLOBsFromSSIS. Click OK to

A

confirm. The resulting SSIS package should look more or less like in Figure 6-10.



302

www.it-ebooks.info



Chapter 6 ■ Miscellaneous Data Sources



Figure 6-10.  The completed SSIS package

You can now run the SSIS package, which will import all the files of the specified type into the SQL Server table.



How It Works

Using SSIS as part of a structured ETL process to load files into a database can bring several useful features to the

party, including:





Cycling through directories to find files







File name manipulation (if required) using .NET string functions



Given the requirement to use dynamic SQL and the multiple variables that are needed, I find it easier to

present the T-SQL to SSIS as a stored procedure, and call the stored procedure from SSIS that passes in the

required variables (the document name, type, path, and file to be loaded). This is an extension of the code used

in the previous recipe.

The Script task harnesses the power of the System.IO class in .NET to separate out the file name, extension,

and path, which are then passed to the stored procedure. They are also stored in the destination table.

Admittedly, this path parsing could be done using T-SQL—but it is so much easier using a script component

and the ready-made .Net IO class. The stored procedure pr_LoadBLOBsFromSSIS then performs the actual load

as many times as required by the iterations in the Foreach Loop container. This approach can be extended very

easily to handle multiple source directories or file types—but I will leave that up to you.



Hints, Tips, and Traps





Creating the variables before adding any other objects to the SSIS package avoids

potential scope bugs. You can create the variables at any time, of course.







Remember to use Imports System.IO in the script, or you will not have access to the

extremely useful built-in .NET functions for parsing the file and path data.







Don’t forget that SSIS variables are case-sensitive!



303

www.it-ebooks.info



Chapter 6 ■ Miscellaneous Data Sources



6-7. Importing Files with Their Attributes into SQL Server

Problem

You want to import files for storage in SQL Server tables using SSIS along with their attributes as a step toward

efficient file management.



Solution

Use SSIS to load the files by configuring an Import Column task to do this, and then add a Script task to add the

required file metadata. You can do it as follows.

1.



Create the following destination table (C:\SQL2012DIRecipes\CH06\tblCarPhotos.Sql):

CREATE TABLE CarSales_Staging.dbo.CarPhotos

(

ID int IDENTITY(1,1) NOT NULL,

FileName varchar(50) NULL,

FileExtension varchar(10) NULL,

DirectoryName varchar(150) NULL,

FileSize bigint NULL,

DateFileCreated datetime2(7) NULL,

DateFileModified datetime2(7) NULL,

ItemImage varbinary(max) NULL,

UNIQUE NONCLUSTERED (ID ASC )

);

GO



2.



Create a new SSIS package, and add a new Data Flow task. Click the Data Flow tab to

edit the Data Flow task.



3.



Add a new OLEDB connection manager for the destination database (CarSales_

Staging), name it CarSales_Staging_OLEDB.



4.



Add a Script Component to the Data Flow pane. When prompted, set it to be a

“source”. Name it Script Component to collect source files.



5.



Double-click the Script Component to edit it. Set the Script Language to Microsoft

Visual Basic 2010.



6.



Click Inputs and Outputs on the left. Select Output 0 in the Input Output columns list

view and rename it to SourceOutput.



7.



Expand SourceOutput in the Input Output columns list view and expand

OutputColumns. Add the following columns:



304

www.it-ebooks.info



Chapter 6 ■ Miscellaneous Data Sources



Column Name



Column Type



FileExtension



String - 10



FileName



String - 50



DirectoryName



String - 150



FileSize



8-byte signed integer



DateModified



Date



DateCreated



Date



FullPath



String - 150



The result should look something like Figure 6-11.



Figure 6-11.  Source column definition



305

www.it-ebooks.info



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

6-6. Importing Files into SQL Server on a Regular Basis

Tải bản đầy đủ ngay(0 tr)

×