Tải bản đầy đủ - 0 (trang)
7-32. Exporting Data from SQL Server Azure

7-32. Exporting Data from SQL Server Azure

Tải bản đầy đủ - 0trang

Chapter 7 ■ Exporting Data from SQL Server



Using SSIS

When using SSIS to connect to an SQL Server Azure data source, you can use either an OLEDB or an ADO.NET

data source. Just remember to use SQL Server authentication and to specify the fully qualified domain name of

the SQL Server Azure database (and append @Database to the username if you are using the OLEDB data source).



Using BCP

Providing that you are using the version of BCP that comes with SQL Server 2008 R2 or above, then the following

command will export data from SQL Server Azure:

C:\Users\Adam > BCP Carsales.dbo.client OUT C:\SQL2012DIRecipes\CH07\Azure.BCP -U 

MeForPrimeMinister@ETLCookbook -PGubbins –SETLCookbook.database.windows.net -N



Using the Import/Export Data Wizard

Merely use the .NET Framework data provider for SQL Server as the data source. Configure it as described

in Chapter 5.



How It Works

For all intents and purposes, SQL Server Azure is just another SQL Server database. So as long as you are using

SQL Server 2008 R2 and above, you can use all the tools that you are used to using to export data from Microsoft’s

cloud-based RDBMS. The essential thing to remember is that you must specify the fully qualified domain name

of the SQL Server Azure database. If you are using the OLEDB data source, you must also append @Database to

the username.



Hints, Tips, and Traps





It is essential to add @ServerName to the usernamewhen using BCP.







You must use SQL Server authenticationwhen using BCP.







You must use the fully qualified DNS name (as it is given in the Azure Management

Portal) as the server name.







You do not have to use SQL native format with BCP. You can otherwise treat this as a

standard BCP command.







You can use SQL Select statements or stored procedures to select the data that is

returned.



Summary

This chapter reviewed many ways of exporting data using SQL Server. To recap these methods, Table 7-10

summarizes my take on the various methods.



423

www.it-ebooks.info



Chapter 7 ■ Exporting Data from SQL Server



Table 7-10.  Advantages and Disadvantages of Approaches Used in This Chapter



Technique



Advantages



Disadvantages



SSIS



Intuitive, efficient and powerful.



Something of a learning curve.



BCP



Ruthlessly efficient.



Clunky and definitely not intuitive.



OPENROWSET



Good for ad hoc exports.



Requires a destination file or table to exist.



OPENDATASOURCE



Good for ad hoc exports.



Requires a destination file or table to exist.



Linked Server



Once up and running is easy to use.



Can be tricky to configure, slow.



Import/Export Wizard



Easy to use.



Needs a little learning.



XML using SSIS



Quick to set up.



Requires lots of memory.



XML using BCP



Fast.



Can require source data to be “chunked.”



Clearly, the technique that you decide to apply when exporting data will depend entirely on the

requirements of your environment. The T-SQL-based solutions (OPENROWSET and OPENQUERY) are easy to deploy,

but are limited when it comes to debugging and logging. Linked servers can be notoriously slow and tricky to

set up, yet once in production tend to be very reliable. SSIS is the tool of choice for scheduled and regular export

processes, despite the development (and deployment) efforts that it can require. BCP can leave you wondering

why the command line was ever invented—except as a form of torture—until the sheer speed of data export using

it begins to impress. But nothing beats the Export Wizard when you have the requirement to export a few tables,

or even subsets of data from a few tables “off the cuff.”



424

www.it-ebooks.info



Chapter 8



Metadata

Metadata is data about data. Specifically, in the context of data ingestion processes, it means discovering and

knowing all that is necessary to know about the structures of your source data. This means having, at a minimum,

all the details of the following:





Tables used







Columns available







Data types







Length of data in columns







Column nullability



For certain sources, you should also know:





Primary and unique keys







Indexes







Triggers



Exactly how much of this information is essential to the process you are building will depend on each

individual project. Some of it will be merely useful, but other aspects could be key to a successful data load process.

Now, while it is possible to create and operate highly successful ETL processes without knowing very much

about the source metadata, having a clear definition of how your source data is structured and defined can be

very useful for several reasons, including:





When you are defining data flows, if you can specify the correct data types and lengths

from the start, you will make your process more robust and avoid unexpected package

failures when your code moves into production.







When you are mapping a source data type to a destination data type, you can have greater

confidence in their compatibility.



I am sure that you can find other reasons to complement this short list. So, having assumed that obtaining

and understanding the metadata that underpins data flows is the key to creating and maintaining robust and

efficient data ingestion processes, this chapter will show you how to





Obtain metadata.







Use metadata to debug data ingestion issues.







Store metadata and use it to detect changes in source data structures, such as structural

modifications (new or removed columns) and type changes.



425

www.it-ebooks.info



Chapter 8 ■ Metadata



We will look at the following primary ways that metadata can be obtained:





Metadata from linked server connections.







Metadata repositories—and the requisite techniques for accessing them—in the major

relational databases that we have used for source data so far in this book.







Using .NET and its GetSchema method, which can be used with database connection

objects to get schema data for SQL Server, OLEDB, and ODBC connections.



Essentially, then, this chapter is all about becoming forearmed through being forewarned. You may be

able to manage quite cheerfully without digging for source system metadata for years—or simply by using the

metadata provided by SSIS or T-SQL when linking to source data tables. Yet there are bound to be occasions

when digging into the source data to learn how it is defined is a prerequisite for a successful ETL process. The

following recipes are designed to help you in this process of discovery.

If you have downloaded the accompanying files from the book’s companion web site, you will find the

examples for this chapter in C:\SQL2012DIRecipes\CH08\.



8-1. Listing Available Tables from a Linked Server

Problem

You want to discover the tables available over a linked server connection.



Solution

Use the sp_tables_ex system stored procedure to list all the tables that you can query using a linked server.

This stored procedure is as easy as it is effective. At its simplest, it only requires the linked server name, as the

following code snippet (using an Oracle linked server) shows:

EXECUTE master.dbo.sp_tables_ex 'MyOracleDatabase'

Running this code snippet will return the list of all tables that are visible to the user on the

MyOracleDatabase linked server.



How It Works

If you have a linked server connection to an external database, then obtaining the metadata from these is really

easy, as SQL Server provides the stored procedure sp_tables_ex, which returns a list of the tables that you can

query across a linked server. The only required parameter is the linked server name. The linked server can be

any correctly configured linked server, be it an Excel spreadsheet, an SQL Server database, or any data source

that SQL Server allows to be defined as a linked server. Also, if you wish, you can restrict the data returned by

this stored procedure by adding the schema and table type. Here again, I am showing how this can be done

for an Oracle linked server—though this is common to all linked server sources (C:\SQL2012DIRecipes\CH08\

LinkedServerTables.sql):

EXECUTE master.dbo.sp_tables_ex @table_server = 'MyOracleDatabase', @table_schema = 'HR',

@table_type = 'VIEW'

The data that is returned is given in Table 8-1.



426

www.it-ebooks.info



Chapter 8 ■ Metadata



Table 8-1.  Metadata Returned from the sp_tables_ex System Stored Procedure



Column Name



Data Type



Description



TABLE_CAT



sysname



The database name.



TABLE_SCHEM



sysname



The table schema.



TABLE_NAME



sysname



The table name.



TABLE_TYPE



VARCHAR(32)



The table type—user table, system table, or view.



REMARKS



VARCHAR(254)



SQL Server leaves this blank.



The sp_tables_ex stored procedure uses the parameters shown in Table 8-2. These parameters allow you to

filter the information returned by the stored procedure.

Table 8-2.  sp_tables_ex Parameters



Parameter



Comments



@table_server



The linked server name.



@table_catalog



The database name.



@table_schema



The table schema name.



@table_name



The table name.



@table_type



The three really useful table types are SYSTEM TABLE, TABLE, and VIEW.



■■Note The sp_tables_ex stored procedure returns an empty result set if the OLEDB provider of the linked server

does not support the TABLES rowset of the IDBSchemaRowset interface.



8-2. Listing the Columns Available When Using a Linked Server

Problem

You want to list all the columns that you can query using a linked server.



Solution

Use the sp_columns_ex system stored procedure to display the columns in the tables accessible over a linked

server connection. Here is a code snippet to illustrate this:

EXECUTE master.dbo.sp_columns_ex @table_server = 'MyOracleDatabase'



427

www.it-ebooks.info



Tài liệu bạn tìm kiếm đã sẵn sàng tải về

7-32. Exporting Data from SQL Server Azure

Tải bản đầy đủ ngay(0 tr)

×