How to use Lookup Transformation in SSIS

Description:-

There might be times when developing a SQL Server Integration Services (SSIS) package that you want to perform a lookup in order to supplement or validate the data in your data flow. A lookup lets you access data related to your current dataset without having to create a special structure to support that access.

In this article we are going to explain, How to perform Lookup Transformation in SSIS using OLE DB Connection Manager with example. In this example we are going to use OLE DB Connection to connect with SQL Server.

The Lookup in SSIS is one of most useful transformation which is used to compare the source data to the existing data from the reference table (or Lookup Table) and find matching ones. For the non-matching rows we can further do some more work.

The Lookup transformation lets you access a referenced dataset either through an OLE DB connection manager or through a Cache connection manager. The Cache connection manager accesses the dataset held in an in-memory cache store throughout the duration of the package execution. You can also persist the cache to a cache file (.caw) so it can be available to multiple packages or be deployed to several computers.

The component can join only two data sets at a time, so in order to join three or more data sets, you would need to chain multiple Lookup Transformations together, using an output from one Lookup Transformation as an input for another. Compare this to relational join semantics, whereby in a similar fashion you join two tables at a time and compose multiple such operations to join three or more tables.

Case Sensitivity in Lookup Transformation: Lookup in SSIS is Case-Sensitive, Kana-Sensitive, Accent-Sensitive and so on. Lookup Transformation doesn’t matter even your source and destinations databases setting are case-insensitive. So, if your data is not in proper case, it is advisable to convert your lookup table and source data to Uppercase or Lowercase.

Before we start creating the SSIS package, Let us see the Lookup table and its data before we start creating the package.

Now let’s see the source data as well.

STEP 1: Drag and drop the Data Flow Task from toolbox to control flow region. Next rename it as Lookup Transformation DFT. Double click on it and it will open the data flow tab.

STEP 2: Drag and drop OLE DB Source from toolbox to data flow region. Double click on OLE DB source in the data flow region will open the connection manager settings. Configure the source connection with the table “CountryWiseData”.

STEP 3: Click on columns tab to verify the columns. In this tab we can uncheck the unwanted columns also.

Click ok and Drag the Lookup Transformation from toolbox to the Data Flow tab.

STEP 4: Double click on the Lookup Transformation will open the Lookup Transformation Editor. Use this to configure the lookup table (reference table).

In the General Tab, under “Specify how to handle rows with no matching entries”. Change the default Fail Component to redirect rows to no match Output option. By changing this option, unmatched rows will be redirected to No match output otherwise, Package will fail.

Under the connection type section we selected the OLE DB Connection manager. Because we are using SQL Server table as reference table. If you are using any cache file (.caw) then use Cache Connection Manager.

STEP 5: If you already created the OLE DB connection Manager then select from the list otherwise, Click New button and configure it.

For this example, we are using [LookupRef_Mst] table as the lookup table or reference table. Source data will be compared with the data present in this table.

TIP: We should not use whole table as reference table. Perhaps you should use SQL command to select the required column name you want to reference.

STEP 6: Click on the Available Input columns. And drag to the available lookup columns to create a join between the 2 data sets.

Country column in the source data will check against the [Country Name] column in the lookup table.

·         If each row of the Country column in the source data match with any one row of the [Country Name] column in the lookup table, then that row will be passed to Matched Output.

·         If each row of the Country column in the source data does not match with any one row of the [Country Name] column in the lookup table, then that row will be passed to No Match Output.

Sometimes we may have some extra columns in the lookup table which we want to pass to the matched output. In these situations we have check mark that column in Available lookup Columns like we did for Cid column.

Drag and drop two Flat file Destinations on to the data flow region, and then drag the Output Arrow to this new Flat file Destination. A pop up window Input Output Selection Window will appear prompting you to select the Input Output type: either Lookup Match Output or Lookup No Match Output. Select Lookup Match Output and click OK.

When you drag another Output Arrow to OLE DB Destination 1, it will automatically allocate Lookup No Match Output to it.

STEP 9: Click on Mappings tab to check whether the source columns are exactly mapped to the destination columns.

Right click on the Lookup Transformation in SSIS Package in the Solution Explorer and select Execute Package.

From the above screenshot you can observe that our Lookup Transformation in SSIS Package has executed successfully. Let’s open the Destination file to verify the output.

Author : Umesh Patel

Related Posts