Jaa

Building Scriptable Applications by hosting JScript

  • Artikkeli

The kind of food I should have, but I don't

If you have played around with large applications, I'm sure you have been intrigued how they have been build to be extendable. The are multiple options

  1. Develop your own extension mechanism where you pick up extension binaries and execute them.
    One managed code example is here, where the application loads dlls (assemblies) from a folder and runs specific types from them. A similar unmanaged approach is allow registration of guids and use COM to load types that implement those interfaces
  2. Roll out your own scripting mechanism:
    One managed example is here where on the fly compilation is used. With DLR hosting mechanism coming up this will be very easy going forward
  3. Support standard scripting mechanism:
    This involves hosting JScript/VBScript inside the application and exposing a document object model (DOM) to it. So anyone can just write standard JScript to extend the application very much like how JScript in a webpage can extend/program the HTML DOM.

Obviously the 3rd is the best choice if you are developing a native (unmanaged) solution. The advantages are many because of low learning curve (any JScript programmer can write extensions), built in security, low-cost.

In this post I'll try to cover how you go about doing exactly that. I found little online documentation and took help of Kaushik from the JScript team to hack up some code to do this.

The Host Interface

To host JScript you need to implement the IActiveScriptSite. The code below shows how we do that stripping out the details we do not want to discuss here (no fear :) all the code is present in the download pointed at the end of the post). The code below is in the file ashost.h

 class IActiveScriptHost : public IUnknown 
{
public:
    // IUnknown
    virtual ULONG __stdcall AddRef(void) = 0;
    virtual ULONG __stdcall Release(void) = 0;
    virtual HRESULT __stdcall QueryInterface(REFIID iid,
                                        void **obj) = 0;

    // IActiveScriptHost
    virtual HRESULT __stdcall Eval(const WCHAR *source, 
                                         VARIANT *result) = 0;
    virtual HRESULT __stdcall Inject(const WCHAR *name, 
                                         IUnknown *unkn) = 0;
};

class ScriptHost : 
    public IActiveScriptHost, 
    public IActiveScriptSite 
{
private:
    LONG _ref;
    IActiveScript *_activeScript;
    IActiveScriptParse *_activeScriptParse;

    ScriptHost(...){}

    virtual ~ScriptHost(){}
public:
    // IUnknown
    virtual ULONG __stdcall AddRef(void);
    virtual ULONG __stdcall Release(void);
    virtual HRESULT __stdcall QueryInterface(REFIID iid, void **obj);

    // IActiveScriptSite
    virtual HRESULT __stdcall GetLCID(LCID *lcid);
    virtual HRESULT __stdcall GetItemInfo(LPCOLESTR name,
        DWORD returnMask, IUnknown **item, ITypeInfo **typeInfo);
    virtual HRESULT __stdcall GetDocVersionString(BSTR *versionString);
    virtual HRESULT __stdcall OnScriptTerminate(const VARIANT *result,
        const EXCEPINFO *exceptionInfo);
    virtual HRESULT __stdcall OnStateChange(SCRIPTSTATE state);
    virtual HRESULT __stdcall OnEnterScript(void);
    virtual HRESULT __stdcall OnLeaveScript(void);
    virtual HRESULT __stdcall OnScriptError(IActiveScriptError *error);

    // IActiveScriptHost
    virtual HRESULT __stdcall Eval(const WCHAR *source,
                                           VARIANT *result);
    virtual HRESULT __stdcall Inject(const WCHAR *name, 
                                           IUnknown *unkn);
public:

    static HRESULT Create(IActiveScriptHost **host)
    {
        ...
    }

};

Here we are defining an interface IActiveScriptHost. ScriptHost implements the IActiveScriptHost and also the required hosting interface IActiveScriptSite. IActiveScriptHost exposes 2 extra methods (in green) that will be used from outside to easily host js scripts.

In addition ScriptHost also implements a factory method Create. This create method does the heavy lifting of using COM querying to get the various interfaces its needs (IActiveScript, IActiveScriptParse) and stores them inside the corresponding pointers.

Instantiating the host

So the client of this host class creates the ScriptHosting instance by using the following (see ScriptHostBase.cpp)

 IActiveScriptHost *activeScriptHost = NULL;
HRESULT hr = S_OK;
HRESULT hrInit = S_OK;

hrInit = CoInitializeEx(NULL, COINIT_APARTMENTTHREADED);
if(FAILED(hr)) throw L"Failed to initialize";

hr = ScriptHost::Create(&activeScriptHost);
if(FAILED(hr)) throw L"Failed to create ScriptHost";

With this the script host is available through activeScriptHost pointer and we already have JScript engine hosted in our application

Evaluating Scripts

Post hosting we need to make it do something interesting.This is where the IActiveScriptHost::Eval method comes in.

 HRESULT __stdcall ScriptHost::Eval(const WCHAR *source, 
                                   VARIANT *result)
{
    assert(source != NULL);

    if (source == NULL)
        return E_POINTER;

    return _activeScriptParse->ParseScriptText(source, NULL, 
                                  NULL, NULL, 0, 1, 
                                  SCRIPTTEXT_ISEXPRESSION, 
                                  result, NULL);
}

Eval accepts a text of the script, makes it execute using IActiveScriptParse::ParseScriptText and returns the result.

So effectively we can accept input from the console and evaluate it (or read a file and interpret the complete script in it.

 while (true) 
{
    wcout << L">> ";
    getline(wcin, input);
    if (quitStr.compare(input) == 0) break;

    if (FAILED(activeScriptHost->Eval(input.c_str(), &result)))
    {
        throw L"Script Error";
    }
    if (result.vt == 3)
        wcout << result.lVal << endl;
}

So all this is fine and at the end you can run the app (which BTW is a console app) and this what you can do.

 JScript sample Host
q! to quit

>> Hello = 7
7
>> World = 6
6
>> Hello * World
42
>> q!
Press any key to continue . . .

So you have extended your app to do maths for you or rather run basic scripts which even though exciting but is not of much value.

Extending your app

Once we are past hosting the engine and running scripts inside the application we need to go ahead with actually building the application's DOM and injecting it into the hosting engine so that JScript can extend it.

If you already have a native application which is build on COM (IDispatch) then you have nothing more to do. But lets pretend that we actually have nothing and need to build the DOM.

To build the DOM you need to create IDispatch based DOM tree. There can be more than one roots. In this post I'm not trying to cover how to build IDispatch based COM objects (which you'd do using ATL or some such other means). However, for simplicity we will roll out a hand written implementation which implements an interface as below.

 class IDomRoot : public IDispatch 
{
    // IUnknown
    virtual ULONG __stdcall AddRef(void) = 0;
    virtual ULONG __stdcall Release(void) = 0;
    virtual HRESULT __stdcall QueryInterface(REFIID iid, 
                                             void **obj) = 0;

    // IDispatch
    virtual HRESULT __stdcall GetTypeInfoCount( UINT *pctinfo) = 0;
    virtual HRESULT __stdcall GetTypeInfo( UINT iTInfo, LCID lcid,
                                           ITypeInfo **ppTInfo) = 0;
    virtual HRESULT __stdcall GetIDsOfNames( REFIID riid, 
                                      LPOLESTR *rgszNames,
                                      UINT cNames, LCID lcid,  
                                      DISPID *rgDispId) = 0;

    virtual HRESULT __stdcall Invoke( DISPID dispIdMember, REFIID riid, 
                                      LCID lcid, WORD wFlags, 
                                      DISPPARAMS *pDispParams, 
                                      VARIANT *pVarResult, 
                                      EXCEPINFO *pExcepInfo, 
                                      UINT *puArgErr) = 0;

    // IDomRoot
    virtual HRESULT __stdcall Print(BSTR str) = 0;
    virtual HRESULT __stdcall get_Val(LONG* pVal) = 0;
    virtual HRESULT __stdcall put_Val(LONG pVal) = 0;
};

At the top we have the standard IUnknown and IDispatch methods and at the end we have our DOM Root's methods (in blue). It implements a Print method that prints a string and a property called Val (with a set and get method for that property).

The class DomRoot implements this method and an additional method named Create which is the factory to create it. Once we are done with creating this we will inject this object inside the JScript scripting engine. So our final script host code looks as follows

 IActiveScriptHost *activeScriptHost = NULL;
IDomRoot *domRoot = NULL;
HRESULT hr = S_OK;
HRESULT hrInit = S_OK;

hrInit = CoInitializeEx(NULL, COINIT_APARTMENTTHREADED);
if(FAILED(hr)) throw L"Failed to initialize";

// Create the host
hr = ScriptHost::Create(&activeScriptHost);
if(FAILED(hr)) throw L"Failed to create ScriptHost";

// create the DOM Root
hr = DomRoot::Create(&domRoot);
if(FAILED(hr)) throw L"Failed to create DomRoot";

// Inject the created DOM Root into the scripting engine
activeScriptHost->Inject(L"DomRoot", (IUnknown*)domRoot);

What happens with the inject is as below

 map rootList;
typedef map::iterator MapIter;
typedef pair InjectPair;

HRESULT __stdcall ScriptHost::Inject(const WCHAR *name, 
                                     IUnknown *unkn)
{
    assert(name != NULL);

    if (name == NULL)
        return E_POINTER;

    _activeScript->AddNamedItem(name, SCRIPTITEM_GLOBALMEMBERS | 
                                      SCRIPTITEM_ISVISIBLE );      
    rootList.insert(InjectPair(std::wstring(name), unkn));

    return S_OK;
}

In inject we store the name of the object and the corresponding IUnknown in a map (hash table). Each time the script will encounter a object in its code it calls GetItemInfo with that objects name and we then de-reference into the hash table and return the corresponding IUnknown

 HRESULT __stdcall ScriptHost::GetItemInfo(LPCOLESTR name,
                                    DWORD returnMask,
                                    IUnknown **item,
                                    ITypeInfo **typeInfo)
{  
    MapIter iter = rootList.find(name);
    if (iter != rootList.end())
    {
        *item = (*iter).second;
        return S_OK;
    }
    else
        return E_NOTIMPL;
}

After that the script calls into that IDispatch to look for properties and methods and calls into them.

The Whole Flow

By now we have seen a whole bunch of code. Let's see how the whole thing works together. Let's assume we have a extension written in in JScript and it calls DomRoot.Val = 5; this is what happens to get the whole thing to work

  1. During initialization we had created the DomRoot object (DomRoot::Create) which implements IDomRoot and injected it in the script engine via AddNamedItem and stored it at our end in a rootList map.
  2. We call activeScriptHost->Eval(L"DomRoot.Val = 5;", ...) to evaluate the script. Evan calls _activeScriptParse->ParseScriptText.
  3. When the script parse engine sees the "DomRoot" name it figures out that the name is a valid name added with AddNamedItem and hence it calls its hosts ScriptHost::GetItemInfo("DomRoot");
  4. The host we have written looks up the same map filled during Inject and returns the IUnknown of it to the scripting engine. So at this point the scripting engine has a handle to our DOM root via an IUnknown to the DomRoot object
  5. The scripting engine does a QueryInterface on that IUnknown to get the IDispatch interface from it
  6. Then the engine calls the IDispatch::GetIDsOfNames with the name of the property "Val"
  7. Our DomRoots implementation of GetIDsOfNames returns the required Dispatch ID of the Val property (which is 2 in our case)
  8. The script engine calls IDispatch::Invoke with that dispatch id and a flag telling whether it wants the get or the set. In this case its set. Based on this the DomRoot re-directs the call to DomRoot::put_Val
  9. With this we have a full flow of the host to script back to the DOM

In action

 JScript sample Host
q! to quit

>> DomRoot.Val = 5;
5
>> DomRoot.Val = DomRoot.Val * 10
50
>> DomRoot.Val
50
>> DomRoot.Print("The answer is 42");
The answer is 42

 

Source Code

First of all the disclaimer. Let me get it off my chest by saying that the DomRoot code is a super simplified COM object. It commits nothing less than sacrilege. You shouldn't treat it as a sample code. I intentionally didn't do a full implementation so that you can step into it without the muck of IDispatchImpl or ATL coming into your way.

However, you can treat the script hosting part (ashost, ScriptHostBase) as sample code (that is the idea of the whole post :) )

The code organization is as follows

ashost.cpp, ashost.h - The Script host implementation
DomRoot.cpp, DomRoot.h - The DOM Root object injected into the scripting engine
ScriptHostBase.cpp - Driver

Note that in a real life example the driver should load jscript files from a given folder and execute it.

Download from here

Comments

  • Anonymous
    May 22, 2008
    Wow, That is awesome.  Great job.

  • Anonymous
    May 24, 2008
    DebugASP.NETTips:Whattogathertotroubleshoot-part6-RecycleduetomemoryLimitDebugDiag...

  • Anonymous
    May 24, 2008
    Debug ASP.NET Tips: What to gather to troubleshoot - part 6 - Recycle due to memoryLimit DebugDiag 1

  • Anonymous
    April 09, 2013
    cool! But isn't the refcount wrong if you insert IUnknown objects into std::map?

  • Anonymous
    September 01, 2014
    My understanding of the SCRIPTITEM_GLOBALMEMBERS flag is that members of the DomRoot object should be accessible without explicitly writing "DomRoot.".  However, it wasn't working this way for me until I moved the rootList.insert() call above the AddNamedItem() call.  I presume this is because AddNamedItem(L"DomRoot",...) was immediately calling GetItemInfo(L"DomRoot",...), which was failing because it wasn't in the list yet.