How to implement IQueryable (Part 2)
This is Part 2 of my 2 part series on “How to Implement IQueryable”. Please see the first post for additional resource links and full source code download (https://blogs.msdn.com/kevin_halverson/archive/2007/07/10/how-to-implement-iqueryable.aspx).
GetEnumerator
In many Linq providers, it will probably make sense to combine the expressions from the various query methods (Where, Select, etc) into a single set of instructions and allow the “server” to do all of the data processing (returning the final results of the query). For some underlying data APIs, however, it may not be possible or may not make sense to try to translate the rich set of expressions available in the client language. In the interest of providing the most flexible end-user scenario (greatest functionality), it may be useful to process part of the query on the “client” side for these cases (as if you were using Linq on an ordinary CLR collection). In my example, this is how I chose to handle “SELECT” expressions. As was detailed in my previous post, we will generate an SQL string to query the “server” (Windows Desktop Search index) for “WHERE” expressions. This query will return a collection of FileInfo objects that we can in turn project into the appropriate form with a little help from the expression tree compiler. To do this processing, I created a new IQueryable object called WDSQueryObjectProjector. Inside of CreateQuery1, I construct one of these and pass along the “SELECT” expression:
Public Function CreateQuery1(Of TElement)(ByVal expression As Expression) As IQueryable(Of TElement) Implements IQueryProvider.CreateQuery
…
Select Case nodeType
Case ExpressionType.Call
…
Select Case methodName
Case "Select"
querySource = New WDSQueryObjectProjector(Of TElement)(expression)
Case "Where"
…
Case Else
…
End Select
Case Else
…
End Select
Return querySource
End Function
Let’s take a look at the expression we get when CreateQuery1 is called for the “SELECT” case.
As before (with the Where method), the outer node in the expression is the query method that is calling us (in this case, Select). The first argument is the instance of the WDSQueryObject returned by CreateQuery1 when we processed the “WHERE” expression. Also as before, the second argument is a quoted lambda. In this case, it represents the projection expression ‘file.FullName’ (reference the original query in the previous post for details). When we attempt to enumerate the results of the query, the GetEnumerator method will be called on the WDSQueryObjectProjector instance. It will in turn enumerate over the items in the WDSQueryObject instance (first argument above) and apply the projection expression (returning the results of the projection). To make that a bit more clear, here’s the implementation of WDSQueryObjectProjector.GetEnumerator:
Public Function GetEnumerator() As IEnumerator(Of TResult) Implements IEnumerable(Of TResult).GetEnumerator
Dim m As MethodCallExpression = m_expression
Dim qo As WDSQueryObject = CType(m.Arguments(0), ConstantExpression).Value
Dim quote As UnaryExpression = m.Arguments(1)
Dim projector As Expression(Of Func(Of FileInfo, TResult)) = quote.Operand
Dim func = projector.Compile()
Dim tuples As New List(Of TResult)()
For Each obj In qo
tuples.Add(func(obj))
Next
Return tuples.GetEnumerator()
End Function
The interesting part of the code above is when we call LambdaExpression.Compile on the Expression (Of Func(Of FileInfo, TResult)) . The expression tree compiler is doing a lot of work for us to translate the “SELECT” expression into a delegate function that can take a FileInfo object and transform it into the type of our result. In our case (file.FullName), it will simply return a String, but the expression tree compiler is a very powerful tool and supports every kind of expression supported by the expression tree API (something far outside the scope of my little project). So for example, if you had an expression like:
Select file.FullName, file.CreationTime
Then the delegate will correctly return an anonymous type with properties for FullName (as String) and CreationTime (as Date). Pretty cool, huh? Since the expression tree compiler is doing all the heavy lifting here, we simply need to loop through the results of the Where method and apply the projection. Speaking of which, how we get the results from the Where (‘qo’ in the code above)? Well, when we iterate over ‘qo’, WDSQueryObject.GetEnumerator will be called, and we’ll finally execute the query string that we built up when processing the “WHERE” expression. Here’s the code:
Public Function GetEnumerator() As IEnumerator(Of FileInfo) Implements IEnumerable(Of FileInfo).GetEnumerator
Dim tuples As New List(Of FileInfo)()
EvaluateFunclets()
Dim connection As New OleDbConnection("Provider=Search.CollatorDSO;Extended Properties='Application=Windows';")
connection.Open()
Dim command As New OleDbCommand()
command.Connection = connection
command.CommandText = m_query.ToString()
Dim reader = command.ExecuteReader()
While reader.Read()
Dim col = reader(0)
If Not IsDBNull(col) Then
tuples.Add(New FileInfo(col))
End If
End While
reader.Close()
connection.Close()
Return tuples.GetEnumerator()
End Function
This is pretty straight forward. We open a connection to the WDS OLE DB provider, execute the query and gather the results (doing some null checking). The thing to note is the call to EvaluateFunclets. In my previous post, I talked a little about lambdas and how we captured the information necessary to access the variables used in our query expression (but didn’t actually capture the values). Since this is the point when the query is executed, now is the time to evaluate all of the variables and plug their values into our query. I accomplish this by simply iterating through all the funclets and inserting the result of their invocation into the query string.
Private Sub EvaluateFunclets()
For Each pair In m_funclets
m_query.Replace(pair.Key, pair.Value.Invoke())
Next
End Sub
And that’s it—happy Linqing!