StreamInsight: Obscure LINQ error - The field 'Key' of type 'IKey`1' either contains a nested member, or is a type that is not supported as event type field

This is a pretty obscure one, and a nice short snippet of a blog post.  I ran into this error yesterday while building out some StreamInsight applications for real time web analytics, and was very briefly stumped.  Let’s have a quick peek at how I ran into it.

Starting with a working query, I needed a different pivot/aggregation on the same data stream.  My working query was:

Code Snippet

1. // Create the "average" quality metrics for each video from raw video quality
2. // metrics (i.e. not quantized)
3. var videoQualityByVideo = from e in this.videoQualityStream
4. group e by new { e.ApplicationId, e.VideoId } into videoGroups
5. from win in videoGroups.TumblingWindow(defaultWindowSize,
6.     HoppingWindowOutputPolicy.ClipToWindowEnd)
7. select new
8. {
9.     ApplicationId = videoGroups.Key.ApplicationId,
10.     VideoId = videoGroups.Key.VideoId,
11.     AverageBitrate = win.Avg(e => e.BitRate)
12. };

Pretty basic stuff – now, I wanted to group by another field (in this case a field called EdgeServerName).  So, I copy and paste, change the grouping field name and away we go:

Code Snippet

1. // Create the "average" quality metrics for each video from raw video quality
2. // metrics (i.e. not quantized)
3. var videoQualityByEdgeServer = from e in this.videoQualityStream
4. group e by new { e.EdgeServerName } into videoGroups
5. from win in videoGroups.TumblingWindow(defaultWindowSize,
6.     HoppingWindowOutputPolicy.ClipToWindowEnd)
7. select new
8. {
9.     EdgeServerName = videoGroups.Key,
10.     AverageBitrate = win.Avg(e => e.BitRate)
11. };

Should be fine – right?  Those of you familiar with LINQ are probably already laughing at the minor syntax mistake I made, but for the rest of us here’s what happened when I tried to run the query:

Microsoft.ComplexEventProcessing.Linq.QueryGenerationException was unhandled Message=The field 'Key' of type 'IKey`1' either contains a nested member, or is a type that is not supported as event type field. Source=Microsoft.ComplexEventProcessing.Diagnostics StackTrace:Microsoft.ComplexEventProcessing.Diagnostics.Exceptions.Throw(Exception exception) at Microsoft.ComplexEventProcessing.Linq.GroupUnionSelectRewriter.VisitMemberAccess(MemberExpression memberExpr)...

Ouch.  What did I do wrong?  A composite group key must have more than one field!

Right	Wrong!	Right
Code Snippet group e by new { e.ApplicationId, e.VideoId } into videoGroups	Code Snippet group e by new { e.ApplicationId } into videoGroups	Code Snippet group e by e.ApplicationId into videoGroups

Removing the new {} syntax for a single grouping field worked perfectly.  Another bit of LINQ learned!

Code Snippet

1. // Create the "average" quality metrics for each video from raw video quality
2. // metrics (i.e. not quantized)
3. var videoQualityByEdgeServer = from e in this.videoQualityStream
4. group e by e.EdgeServerName into videoGroups
5. from win in videoGroups.TumblingWindow(defaultWindowSize,
6.     HoppingWindowOutputPolicy.ClipToWindowEnd)
7. select new
8. {
9.     EdgeServerName = videoGroups.Key,
10.     AverageBitrate = win.Avg(e => e.BitRate)
11. };

But wait… there’s more!  Through the course of the same project, I also ran into another way to throw this error, in a slightly different fashion.

Code Snippet

1. var videoQualityByEdgeServer = from e in this.videoQualityStream
2.     group e by new { e.ApplicationId, e.EdgeServerName } into edgeGroups
3.     from win in edgeGroups.TumblingWindow(videoQualityEventWindowSize, windowAlignment,
4.         HoppingWindowOutputPolicy.ClipToWindowEnd)
5.     select new
6.     {
7.         ApplicationId = edgeGroups.Key,
8.         EdgeServerName = edgeGroups.Key.EdgeServerName,
9.  
10.         // Buffering Milliseconds -> Seconds
11.         // Divide by sampling frequency seconds
12.         // Multiply by 100 (1 / 1000 * 100 = 1 / 10)
13.         AvgBitrate = win.Avg(e => e.BitRate)
14.     };

If you have a composite key, and don’t use the individual properties of the grouping key, the error is thrown.  Easy in retrospect, right?