Dynamic editing using foreach loop "Variable does not contain valid data object" and a dynamic query data flow error

Blazer22 0 Reputation points
2025-01-13T08:42:33.5466667+00:00

Hi! I'm currently trying to create an SSIS package that dynamically acquire columns from an azure SQL database and transform it. This is my current setup:User's image

Populate column list has the following query:
SELECT COLUMN_NAME

FROM INFORMATION_SCHEMA.COLUMNS

WHERE TABLE_NAME = 'PerformanceRating'

AND COLUMN_NAME LIKE 'Performance_Rating%'

AND COLUMN_NAME NOT LIKE '%20XX%';

It takes the headers that contain performance rating data in their column and stores it in a object variable called ColumnList (Below is variables lista nd output of columnList
User's image

User's image

Transform Obj to Str transforms ColumnList from an object class into a string so that DynamicSQLQuery variable can be used:
User's image

This is the dynamic SQL query:
"SELECT Employee_Number, " + @[Performance_Rating::ColumnListStr] + " FROM PerformanceRating WHERE " +

REPLACE(REPLACE(@[Performance_Rating::ColumnListStr], ",", " IS NOT NULL OR "), " IS NOT NULL OR ", " IS NOT NULL")

And this is the output which is correct:
User's image

The foreach loop is contains a data flow component that houses an OLE DB Source and Derived column (for now) OLE DB Source uses the DynamicSQLQuery to acquire all rows for the columns with rating data and the derived column has substring to make 2 new columns for each rating column and splits the number and letter (e.g. a performance rating of A2 gets split to 2 columns, 1 containing A and the other the number 2.) I currently have 2 issues. The first is the foreach loop, where this error is shown:User's image

These are my current configurations: User's image

User's image

The second issue is to do with the Data flow, where using the DynamicSQLQuery looks like this: User's image

and I'm unable to run the foreach loop because of this error (Even though the DynamicSQLQuery is only completed when I begin running the package)

I have tried many solutions with these 2 issues to no avail, any ideas on how to solve them?

SQL Server Integration Services
SQL Server Integration Services
A Microsoft platform for building enterprise-level data integration and data transformations solutions.
2,632 questions
0 comments No comments
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.