Processing Excel files in SSIS 2008 – Lessons Learned
I have recently been working on a project where we need to load and transform data held in Excel 2007 into a SQL Server database. The Excel sheets were fairly complex and had different structures. Fortunately, SSIS 2008 provides some good tools to handle this situation.
Lessons learned
1. It is possible to read the Excel metadata using the mechanism listed in this knowledge base article, https://support.microsoft.com/kb/318452 HOW TO: Retrieve Meta Data from Excel by Using GetOleDbSchemaTable in Visual C# .NET. You may be wondering why I want to do this. Handling Excel sheets with different structures can be tricky so reading the metadata can help determine control flow processing i.e. which data flow to use to process the file.
2. Remember, if you are testing on x64 then your package will not execute if you are using the Excel source since it is not supported on 64-bit so you need to disable the Run64BitRuntime property as below
3. The Script component is really great when it comes to data manipulation in the data flow. This can be used as either a source, transformation or destination and allows you to manipulate the rows in the pipeline using either VB.NET or C#.
4. As mentioned, Excel files can also be read using the Execute SQL task in the control flow, which is a nice feature e.g. SELECT * FROM $PivotData
5. The File System Task can make file handling a lot easier. Combine this with the use of variables and property expressions, dynamic file manipulation became a whole lot easier. For example, after processing I either move the file to another folder based on the outcome e.g. success or failure.
Comments
- Anonymous
February 05, 2012
Thansk for the information. It helped me to resolve one of the SSIS issue I was facing. Reading Excel Meta data is complex specially if you do not know how to read the Excel schema. I came across one of the nice post similar to what you have writen here.www.singhvikash.in/.../ssis-how-to-read-excel-meta-data.html