Future Breaking Changes, Part Two

Last time I mentioned that one of the subtleties of programming language design is weighing the benefit of adding a feature against the pain it will cause you in the future. This is a specific subset of a more general set of problems. Languages run into the same problem that other large, multiply-versioned software products run into; if you have n existing features and want to add m new features, then you have at least m x n possible interactions to consider. In an ideal world the features would all be orthogonal to each other, but in reality of course new features modify the behaviour of existing features.

Just as a random example, what on earth do partial methods have to do with expression trees? Both are new features for C# 3.0, but other than that, they seem to have nothing to do with each other. But what happens when you have an expression tree lambda which contains a call to a partial method which is going to be removed? This interaction has to be carefully defined. Every interaction between every feature has to be carefully defined, and the larger the area of the feature interaction is, the more likely it is that there will be unfortunate consequences of this nonorthogonality, such as breaking changes.

I said last time that implicitly typed lambdas are just such a feature -- big surface area of nonorthogonal interactions which will lead to future breaking changes. I'd like to explore that in more detail, because it might not be immediately obvious why that is the case.

The rule for determining whether an implicitly typed lambda is convertible to a given delegate type is (somewhat simplified) “If possible, infer the return type and the types of the lambda parameters from the delegate type. Try to bind the lambda with those types. If the inference and binding succeeds with no errors then the lambda expression is convertible to that delegate type.”

For example, if we had a lambda expression like x=>x - 123.4, that would be convertible to Func<double, double> but not to Func<string, double> (because the subtraction cannot be legally bound) or Func<double, int> (because the conversion to the return type cannot be bound).

Notice how the overlap between this new feature – implicitly typed lambda convertibility to a delegate type – overlaps with almost every other feature in the entire C# language. Consider the impact of this design decision on the potential for breaking changes in the future. Every single time we make a change to the rules for how the body of a method is bound, we are potentially changing whether a given lambda expression is convertible to a given type.

Now, is that really so bad? Generally when we make changes to the method body binding rules, we do so in as non-breaking a manner as possible. We should never be making an existing lambda that does convert suddenly stop converting, because then we would also be breaking compilation of the equivalent nominal method body. Really we should only be causing presently-erroneous code to suddenly start being non-erroneous, and as we’ve already discussed, that’s not a breaking change.

Or is it?

Actually, now in a lot of cases it could be.

Consider a silly example. Suppose we decide that in some hypothetical C# 4.0(‡) language it should be legal to subtract a double from a string. That’s not entirely farfetched – that’s perfectly legal in JScript. You just convert the string to a number and subtract. We might reason that this new feature is not a breaking change, because no program that ever subtracted a double from a string ever compiled before. But it is a breaking change, because now this program stops compiling:

using System;
class Program {
void M(Func<double, int> f){}
void M(Func<string, double> f){}
void M(Func<double, double> f){}
static void Main() {
M(x=>x-123.4);
}
}

Before the change, the program unambiguously chooses the third overload. But with our change to addition semantics, now the second and third overloads would both work. Neither would be clearly better than the other. The compiler would then give an ambiguity error. Thus, this is a breaking change.

It gets even worse; with some cleverness we could come up with more subtle breaking changes, where the compiler would not produce an error but instead would choose a different overload than before. At least a program which fails to compile calls attention to the problem; recompiling and having the behaviour change subtly might go unnoticed for a long time.

This is a general problem with overload resolution: it depends upon conversion semantics. If we make the conversion semantics more strict then it is possible to go from having exactly one overload which works to zero, which is a breaking change. If we make the conversion semantics less strict then it is possible to go from having exactly one overload which works to two, which is also (usually) a breaking change.

Since lambda convertibility depends on every other language rule for binding an expression, any change to any of those rules is a potential change in convertibility, and hence a potential breaking change for overload resolution. Now, hopefully few real-world codebases will end up in the above situation, where a method is overloaded solely on a bunch of different delegate types, but still, I worry. We think implicitly typed lambdas are worth it, but this was a tough call that we agonized over for a long time.

Next time on FAIC: More on subtle language design issues involving breaking changes. Have a pleasant Labour Day weekend, Canadian and American readers!

---

(‡) I call out that this is the hypothetical C# 4.0 compiler because of course I do not discuss the feature set of unannounced and non-existing products on public blogs.

Comments

• Anonymous
  August 31, 2007
  Heck yea they're worth it. Without it, I don't even want to imagine the syntax :. While you're on the topic, can you explain why method groups don't get the same inferrence?

• Anonymous
  August 31, 2007
  The comment has been removed

• Anonymous
  August 31, 2007
  Michael: Method groups do get the same inference. Or, at least, very similar inference.  Can you clarify the question?

• Anonymous
  August 31, 2007
  Eric, Out of interest are the C# team keeping abreast of the Java language features being proposed? In particular, I thought you and other readers might be interested in this blog: http://blogs.sun.com/abuckley/ The two most recent entries talk about compatibility and the cost of new features, which are of course closely related to the topic of this blog post. I have to say, it's great that both communities are blogging about language design issues. The difficulties involved have been somewhat shrouded in mystery in the past... Jon

• Anonymous
  August 31, 2007
  For example: static void executor<T>(Action<T> f) { f(default(T)); } static void Main(string[] args) {    executor(Main); } Generates error CS0411: The type arguments for method 'Program.executor<T>(System.Action<T>)' cannot be inferred from the usage. Try specifying the type arguments explicitly. It'd also be nice if method groups could automatically be treated as delegates. The rest of my inference desires are summed up here: http://www.atrevido.net/blog/2007/08/02/C+Frustration.aspx

• Anonymous
  September 04, 2007
  MichaelGiagnocavo: Thanks for clarifying. There is a good reason why we do not do that.  I'll leave a comment on your blog explaining why.

• Anonymous
  September 06, 2007
  As I said earlier , we hate causing breaking changes in our product, the C# compiler, because they cause

• Anonymous
  June 12, 2008
  NOTE: If you haven't read the first post in this series, I would encourage you do to that first , or

• Anonymous
  February 11, 2009
  NOTE: If you haven&#39;t read the first post in this series, I would encourage you do to that first