Dynamic in C# VI: What dynamic does NOT do

Article
12/15/2008

As I mentioned last time, there are a few gotchas that we'll need to look at in order to get a full understanding of the dynamic feature and its capabilities. Today we'll take a look at some of those limitations. As we go along, I'll try to shed some insights as to how the decision making process came about, and why we feel these calls are the right ones.

Mutating values using dynamic

Consider the following code:

 static void Main()
{
    dynamic d = 10;
    d++;
}

What should happen here?

Intuitively, we'd expect d to contain the value 11. However, recall that under the covers, dynamic is really object. That means that the first line will generate a boxing conversion from int to object. The local variable d contains a boxed copy of the integral value specified. The second line, then, is really an application of the ++ operator on the local variable d. At runtime, calling operator ++ on d will result in an unbox of the object d into an integer, and a call to the operator ++ on the unboxed value. But this value is not copied back inside the box!

Turns out that what you expect to happen isn't really what would happen if we naively implemented this in the runtime binder. Good thing your trusty C# compiler team isn't naive!!

The solution to this little problem then, is essentially to pass things by ref to the runtime binder so that the runtime binder can write back into the value. This gets us half-way there - we still have the problem of boxed structs. Luckily for us, the architecture of the runtime binder is such that we return expression trees to the DLR (for more information on this, read my post that talks about this). There is an expression tree that performs an unbox and modifies the boxed value, and puts that value back into the box, allowing you to mutate the boxed values of these things.

Nested struct mutation

If we take our above example to the next level however, things start getting a bit trickier. Because of the nature of our architecture, dotted expressions get broken up into parts and bound in segments. So that means that for the expression A.B.C.D, the compiler will encode a site for A.B, use that result as the receiver for a second site for .C, and use the result of that as the receiver for the third site for .D.

That seems like a sensible architecture, doesn't it? Indeed, it is the same architecture that the compiler uses when it does it's binding. However, the runtime architecture has the limitation that it does not have the ability to return values by ref. (Well, this really isn't a limitation in the CLR, as they already have the support for this. This is more a limitation in the .NET languages as none of them provide the ability to have ref returns).

This means that if any of the dotted expressions were to bind to a value type, that value will be boxed (and hence a copy would be made), and further dots into it would be made on the copy of the value, and not the initial value as would be expected. Consider the following code:

 public struct S
{
    public int i;
}

public class D
{
    public S s;
    public static void Main()
    {
        dynamic d = new D();
        d.s = default(S);
        d.s.i = 10;
        Console.WriteLine(d.s.i);
    }
}

We would intuitively expect the value '10' to be printed in the console. However, the value '0' is printed instead. We're currently working on determining the best way to fix this issue, and are also debating whether or not this is a critical enough of a scenario to fix.

The rule of thumb? Remember that dynamic is like object, and so boxing happens!

Base calls

There is a restriction in the CLR that prevents the compiler from generating non-virtual calls on virtual methods. This means that there is no way to call a base overload dynamically. This means that one cannot call a base call with any dynamically typed arguments, as it will trigger a dynamic binding.

The possible solution (which we have chosen not to implement) would be somewhat akin to the solution we performed for lambdas. Recall that if you had the lambda: x => base.M(x), the compiler will generate a private method that performs the call to the base access, and will have the lambda body call the generated method. The down side, however, is that for lambdas, we knew exactly which call the user was trying to make. In the dynamic scenario, we would be doing overload resolution at runtime, and so we would have to generate a base call stub for each possible overload. This solution is quite ugly, and since we currently lack an extremely compelling scenario, we have opted not to do this and simply give a compile time error when the user attempts to make a base call with any dynamic arguments.

Explicitly implemented interface methods

As one avid reader commented in one of my previous posts, explicitly implemented interfaces kinda get the shaft again here. Because interfaces are really compile time constructs, and have no runtime representation, explicitly implemented interface members get the short end of the stick at runtime. Consider the following:

 interface IFoo
{
    void M();
}

class C : IFoo
{
    void IFoo.M() { }
}

Because of the way the compiler implements explicitly implemented interfaces, C.M gets its name removed (making it uncallable via a C pointer). Now this is fine at compile time, because the compiler can see when a receiver is known to be an IFoo pointer. However, at runtime, there is no notion of interfaces, and so there is no IFoo available for the runtime binder to use to dispatch methods. Combined with the fact that C.M's name has been removed, this makes the method entirely uncallable dynamically.

Accessibility

This last topic isn't really a limitation yet. We are still working on drawing the line between doing the pragmatic thing and doing the most consistent thing on this issue. The CTP implementation of dynamic currently performs accessibility checks only on the member that you are accessing. This means that the runtime binder checks to verify that any member you're trying to use is public.

Namely, we do not do accessibility checks on the type itself (ie if you really ought to be able to access the type of the object in your current context, or should it just look like an opaque object to you), and do not allow any non-public members to be used dynamically.

The down side of this scenario is that you could make a call with a static receiver to a private method that you know you can access from your context, but because a dynamic argument is given, the runtime binder will prevent you from calling the method. Below is an example:

 public class C
{
    private void M(int x) { }

    static void Main()
    {
        dynamic d = 10;
        C c = new C();
        c.M(d);
    }
}

When the compiler encounters this expression at compile time, it will do the verification and know that C.M is accessible from the calling context, but because the argument is dynamic, the call gets resolved at runtime. Because of the public-only policy of the current binder, overload resolution will not bind to the method.

Conclusions?

As always, I love getting your feedback, whether positive or negative. But this post in particular I would love to get your thoughts on! The design is not set in stone, so any of your thoughts will definitely be personally brought to the design team by yours truly. Thanks for your comments in advance, and as always, happy coding!

Comments

Anonymous
December 15, 2008
You've been kicked (a good thing) - Trackback from DotNetKicks.com
Anonymous
December 15, 2008
The second example is wrong - I think you mean: dynamic d = new D(); (By-the-way, anonymous comments are disabled)
Anonymous
December 15, 2008
Whoops, yes you're right, that was supposed to be dynamic. Anonymous comments are disabled by design.. :)
Anonymous
December 15, 2008
The comment has been removed
Anonymous
December 16, 2008
Hey int19h, I kind of agree with you, though the current thought is that when you've got a statically known receiver, you can guarantee that your arbitrary string will fail at runtime, since your receiver doesn't implement IDynamicObject. We're currently still in the trenches about the question of how much static time information we should use up front, and what we should delay load. For instance, should we check the arity of the methods? Should we check names? Accessibility? Should we do type inference where possible? How far should we go when we have a dynamic argument, since we know that most likely the call will be dispatched dynamically?
Anonymous
December 16, 2008
Sam, What a great entry this is on the current by-design limitations of the DLR; much thanks for sharing your insight. On the matter of accessibility, I think that it would be valuable to make dynamic code behave in the same way as regular C# code wherever possible, particularly with respect to accessibility and type inference which both lend themselves to intuitive coding. I think that moving forward it would be preferable to avoid things like.... IEnumerable<object> someObjects = enumerableInstances; // Inexplicably fail for an entire major version of the .NET Framework. Consider this, for example: A self-contained factory model class has a public, static method that consumes dynamic objects returned by a COM server, instantiates a concrete .NET type and returns a public interface that the .NET type implements. In this scenario, which happens to be something I encounter every day when working with legacy machine control systems, it makes perfect sense for a public, static member to hand off a dynamic object to an instance of itself through a private member or constructor. Could this scenario be reworked to be compatible with a DLR implementation that does not allow for this type of accessibility with dynamic arguments? Sure, it could. But that would place more of the unboxing and handling of the dynamic instance in the static portion of the type, ostensibly increasing the amount of time spent in a locked/critical state. Well, at least this is my take on it, but you are the low level CLR expert -- not me. Maybe my concerns/wants are misguided in some way. Thanks again for the great articles, Sam. Rob
Anonymous
December 19, 2008
I am glad that this article tries to somewhat dispell the thoughts of dynamically typed languages such as P cough H cough P. I was getting a little worried. First MVC then dynamic, I felt as if we were going back 10 years in time. I'd be perfectly OK with that however I am still in my 30s! Seriously though, thanks for pointing out that dynamic is not much more then an evolved object. Keep up the great work over there!
Anonymous
December 22, 2008
The comment has been removed
Anonymous
December 31, 2008
One of the things that kills me about C++ is the language is incredibly complex. Paraphrasing Scott Myers, there's an exception to every rule, and an exception to most exceptions. While I think supporting inetrop with dynamic languages is an important feature, some of the behaviors you bring up here really concern me. I think that the issue with struct mutation and accessibility really need to be addressed. If these issues are not addressed we could end up with C# evolving into a language that is as complex or nearly as complex as C++.
Anonymous
January 01, 2009
Whew, sorry I haven't responded to comments in a while! Been on Christmas vacation :) Okay, lets try to address these: Rob - Thanks for your kind words! These comments definitely help make it much easier to keep blogging. I appreciate your thoughts! I entirely agree with you - we've recently taken the design change to make this type of scenario work. Dynamic will now consider the same accessibility rules that the static language considers. This means that each of the callsites will encode their calling context (ie the containing type that the callsite lives in) so that the runtime binder can check the accessibility from the calling context location. This will then enable the scenario you described, and will give runtime binding errors if the method is not accessible from the calling context location. Kornblum - thanks for your comments! We are definitely trying very hard to make sure that the designs that we come up with have been vetted with the community, and are trying to take all the feedback that we can! Pop.Catalin - the scenario you described works identically in both dynamic and static code. The thing that does NOT work is the nested struct scenario. This means that you cannot prototype with dynamic and then convert back to struct - this makes sense, since dynamic is a reference type and structs are value types. Edgar - as I mentioned above, we're addressing the accessibility issue. I entirely agree with you that these types of differences are bad, and so we're trying to work out all the kinks in the implementation. The nested struct problem is one that we're trying to figure out the philosophy on - since dynamic is a reference type, does it make sense to mutate structs? Combine that with costs and benefits, and you've got the factors that we're considering when making our design decisions. Thanks for all the great comments guys!
Anonymous
January 04, 2009
please examples of class in c# FORWARD in id "Baverd_mohammad@hotmail.com" Thank you FAR
Anonymous
January 14, 2009
Welcome to the 48th Community Convergence. The C# team continues to work hard to get out the next version
Anonymous
January 14, 2009
"There is a restriction in the CLR that prevents the compiler from generating non-virtual calls on virtual methods." This is obviously not true, since "base.ToString()" in C# generates "call instance string [mscorlib]System.Object::ToString()", which is a non-virtual call to a virtual method. Can you clarify what you mean by this? I would very strongly agree with the previous posters that EVERY effort needs to be made to preserve intuitive behavior in dynamic dispatch, even to the point of limiting the feature if it is necessary to maintain that intuitive behavior. As Edgar said, the last thing we need is for C# to become C++.
Anonymous
January 14, 2009
The comment has been removed
Anonymous
February 03, 2009
The comment has been removed

Partager via