Layering enumerators

Now that I've touched upon XmlReader and how it can be used to stream data while allowing clean layering at the same time, I want to touch on the layer-able interface by excellence in the .NET Framework: IEnumerable<T>.

You may have also seen the term 'composing' enumerables, but I tend to think of composing them as being able to do things with multiple enumerables at once, like joining to enumerations, and I want to stick to the simpler, one-in-one-out layering for a bit longer.

LINQ-enabled languages provide great support for layering through the use of Enumerable and Queryable classes, although their functionality is typically accessed through LINQ keywords (from, where, select, etc) or as extension methods (where customers.Select(c => c.Name) actually translates to Enumerable.Select(customers, c => c.Name)).

So let's say we have something like the following.

string[] values = new string[] { "Mary", "had", "a", "little", "lamb };
var shortWordsInUpperCase = values.Where(v => v.Length < 4).Select(v => v.ToUpper());

What we end up with is an enumerator that will do the Select on the results on the enumerator that applies a Where on an enumerator that reads from an array - a chained set of layered enumerators. The good thing is that all of these will never be processing more than one of these at a time, so if the original source where streaming them from disk for example, we only need enough resources for the item we're currently processing.

C# also has great support by allowing you to write methods that use the yield keyword to build methods that can enumerate their results element by element.

It's often useful to think of XmlReader as a very specific enumerator that walks a document node-by-node; many of the same layering considerations that apply to enumerators will apply to XmlReader. There is less syntactic sugar in the language to deal with XmlReader, and typically there is more work in producing valid results so things still work like consumers will expect, with element starts balanced with element ends; but by and large, it's very straightforward to apply the design learnings from one realm to another.

Next week, a note on streaming layered components that break streaming (yes, it may sound weird, but it's actually quite common).

Enjoy!