다음을 통해 공유


How to implement IAsyncResult using compiler extensibility

In previous posts I gave an introduction to Boo and the extensibility of the compiler. In this post we are now able to look at how compiler extensibility can be used to improve the creation of asynchronous operations.

Initially, I thought it would be straighs't forward to modify the model created earlier with text templates for use in compiler extensions. I did run into some interesting problems.

First, I realized that the model I had created was really two integrated models. One for capturing metadata about API exception behavior, and one for modeling the IAsyncResult design pattern. Since my goal was to model the IAsyncResult design pattern, I dropped the portions of the model related to exception handling.

Second, I quickly found that the syntax I wanted to use was not legal Boo syntax and had to change strategy. My first syntax attempt was something like this:

operation Send(host as string, port as int, buffer as (byte), offset as int, size as int): MyClass

 

MyClass is the class on which the asynchronous operation would be implemented. The problem is that this is a C# notation, and the parser was not happy.

The solution

The solution that seemed more compatible with Boo turned out to be as follows:

partial class MyClass:
    operation Send(host as string, port as int, buffer as (byte), offset as int, size as int):
        m_socket as Socket
        
        def Helper(something as bool):
            print something
            
        step Connect:
            m_socket = Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp)
            m_socket.BeginConnect(m_host, m_port)
            
        step Send:
            m_socket.BeginSend(m_buffer, m_offset, m_size, SocketFlags.None)
    
        step Disconnect:
            m_socket.BeginDisconnect(true)

 

By nesting the operation in the class it was to be applied, the macros are now able to find the class where to add the BeginXXX and EndXXX methods by navigating to its parent in the AST.

In addition any additional member variables or methods (e.g. m_socket and Helper) can be defined within the operation and copied to the class derived from AsyncResultNoResult.

You can see some other aspects of the syntax if you look closely:

  • The name of the operation appears as the method name after the operation declaration. Send in this example.
  • Arguments passed into the Begin method are specified as arguments to the operation
  • The userCallback and state parameters are not specified in the BeginXXX calls within the steps. These are auto generated method names and are inserted automatically for you.
  •  The step macro specifies the name of the Step, which is used to construct generated callback methods.

Step macro

Because macros are evaluated innermost to outermost, the step macro will be evaluated before the operation macro. Since the step macro can’t render all that it needs to when evaluated, it saves its information in the operation macro for when it is evaluated. Its implementation is as follows:

    macro step:
        nameExpression = step.Arguments[0] as ReferenceExpression
        
        stepName = nameExpression.Name
        
        d = operation["steps"] as List[of Step]
        if d == null:
            d = List[of Step]()
            operation["steps"] = d
             
        stepBody = step.Body
            
        d.Add(Step(stepName, stepBody))

        return null

As you can see it is very simple. It saves the name and body of the step in a collection of Steps on the operation macro for evaluation later. It then returns null to remove itself from the AST.

Operation macro

The operation macro first creates the BeginXXX and EndXXX methods on the class it is extending. Since the operation macro is nested within the class it is extending, navigation of the AST will get the class declaration it needs to extend:

macro operation:    
                    
    # Get the class definition and class name
    if operation.ParentNode != null:
        definition = operation.ParentNode.ParentNode as ClassDefinition
    else:
        definition = null

    if definition == null:
        # There must be a better way to fail the compilation than this.
        assert false, "operation must be used within class definition."
        return null
        
    className = definition.Name
    
    # Keep the module to add classes to
    module = definition.ParentNode as Module
    
    # Extract the name of the method and arguments.
    methodExpression = operation.Arguments[0] as MethodInvocationExpression
    assert methodExpression != null
    
    nameExpression = methodExpression.Target as ReferenceExpression
    assert nameExpression != null
    
    shortName = nameExpression.Name
    resultClassName = nameExpression.Name + "AsyncResult"

 

    # This prevents the method name from being 'quoted'. Probably not the right node type
    # but I believe it may get turned into a string and reparsed anyway.    
    methodReference = ReferenceExpression(resultClassName)
          
    # Build a partial class to hold Begin/End methods
    
    extendedClass = [|
        partial internal class $className:
            
            def    $("Begin" + shortName)(asyncCallback as AsyncCallback, state as object):                
                result = $methodReference(asyncCallback, state, self, $(shortName))
                result.Process()
                return result
                
            def $("End" + shortName)(result as IAsyncResult):
                AsyncResultNoResult.End(result, self, $(shortName))
    |]

 

At this point, we have the class name and a prototype class with BeginXXX and EndXXX members. From the AST assigned to extendedClass, we can extract the members and add the arguments to the BeginXXX method. Note that the arguments are also passed to a prototype constructor of the AsyncResult derived class so it is done in the same loop. Here is the prototype for the asyncresult class:

    # Build a template for the derived AsyncResult class
    resultClass = [|
        partial internal class $resultClassName(AsyncResultNoResult):
            internal def constructor(
                asyncCallback as AsyncCallback, 
                state as object
                owner as object
                operationId as string):
                super(asyncCallback,state,owner,operationId)
                                
            internal override def Process():
                pass
    |]

 

Here’s the code that finds the constructor and the begin method, and appends the arguments to the prototypes of the methods:

    # Find Begin method which needs additional arguments
    beginMethod = extendedClass.Members[0] as Method
    assert beginMethod != null
    
    # Find the constructor call on right side of assignment statement
    es = beginMethod.Body.Statements[0] as ExpressionStatement
    assert es != null
    
    be = es.Expression as BinaryExpression
    assert be != null
    mie = be.Right as MethodInvocationExpression
    assert mie != null
    
    # Find the constructor which needs additional arguments
    ctor = resultClass.Members[0] as Constructor
    assert ctor != null
    
    for a in methodExpression.Arguments:
        # Add parameter to begin method
        beginMethod.Parameters.Insert(-2, ArgumentToParameterDeclaration(a))
        
        # Pass the parameter through to the constructor
        mie.Arguments.Insert(-4, ArgumentToPassedArgument(a))
        
        # Add parameter to ctor
        ctor.Parameters.Insert(-4, ArgumentToParameterDeclaration(a))
        
        # Add field to result class
        resultClass.Members.Insert(-2, ArgumentToField(a))
        
        # Add field assignment to constructor
        ctor.Body.Statements.Add(ArgumentToAssignFieldStatement(a))

 

In addition to adding the arguments, it also adds the fields and assignment statements for the parameters of the constructor to the fields of the result class in the constructor body.

Building the steps of the AsyncResult

Building the sequence of steps necessary to do the asynchronous operation is similar to how it was done using text templates in my earlier post. The first step is started in method called Process on the AsyncResultNoResult derived class, and no exception handling is added.

Each subsequent step starts the following step when its completed handler is called. Each completed handler needs exception handling to make sure that Complete() is called on the result class on failure. The very last step also has to call Complete() on success.

The following code achieves this:

    first = true
    processMethod = resultClass.Members["Process"] as Method
    previous as Step = null
    method as Method = null
    
    steps = operation["steps"] as List[of Step]
    for step in steps:        
        if  first:
            first = false
            for s in step.Body.Statements:
                processMethod.Body.Statements.Add(s.CloneNode())                
        else:
            ts = method.Body.Statements[1] as TryStatement
            assert ts != null
            
            for s in step.Body.Statements:
                ts.ProtectedBlock.Statements.Add(s)
                
            resultClass.Members.Add(method)
            
        previous = step
        method = step.GetCompletedMethod(false)

    method = previous.GetCompletedMethod(true)
    resultClass.Members.Add(method)

 

Note that steps beyond the first have a try/ensure/finally inside the handler where the statements are added. The first step does not have exception handling so the body statements are added directly to the Process() method.

Copying member and field declarations

In our DSL we allow developers to declare fields and methods that should be included on the result class. In this case it was m_socket and a method called Helper(). After the step macros were expanded, they returned null to remove themselves from the operation body. That means everything that remains should be copied to the result class as is. The following code does just that:

    # Add member statements to resultClass
    for s in operation.Body.Statements:
        decl = s as DeclarationStatement
        
        assert decl != null, "Only declarations are supported in operation body."
        
        c = [|
                class temp:
                    $(decl)
            |]
            
        forin c.Members:
            resultClass.Members.Add(m)

 

Since both fields and methods both are DeclarationStatements, the code copies only DeclarationStatements to the result class for simplicity.

Finishing up

Once the result class has all its fields, methods and the arguments are added to the constructor, the class needs to be added to the model. The Begin and End methods need to be added to the hosting class, and we need to remove the operation macro from the AST by returning null. This is done by the few lines shown below:

    # Copy the Begin/End methods to the class the operation state was nested in.
    forin extendedClass.Members:
        definition.Members.Add(m.CloneNode())
        
    # Add the result class to the module
    module.Members.Add(resultClass)
    
    # Everything has been added to the module or class
    # so remove the node.
    return null

 

Summary

In this post, I finally got to show you how you can use some macros in Boo to extend the compiler to support asynchronous results. The solution takes a little while to understand but once you have extended the compiler, the compiler supports a pretty simplified syntax for implementing asynchronous operations. Making things simple should improve productivity.

Previously we evaluated using text templates with the code DOM for generating code. In the next post, I will review how text templates compare with compiler extensibility as a mechanism for generating repetitive code.

Series

Start of series previous next

20110711_CompilerExtensibilityIAsyncResult.zip