Behind the Scenes of Add-In Discovery in the Orcas System.AddIn [Jesse Kaplan]

Details about the System.AddIn’s Implementation of Add-In Discovery

 

Last time we discussed several common approaches to add-in discovery and some of the pros and cons of each. We reviewed type hierarchy, custom attribute, and xml manifest based discovery systems and primarily evaluated them on three high level criteria: development complexity, performance, and add-in developer experience. This time I’d like to go into the details on what we developed and how it builds on the lessons we learned while evaluating previous solutions.

 

Implementation Overview

We had two high level goals in mind when we developed our discovery system: minimize complexity for the add-in developer and make the solution fast enough to be used at runtime. We wanted the add-in developer experience to be as close to “inherit from an abstract base class and run with it” as possible and we wanted hosts to be able to use the UI thread to discover add-ins.

 Fundamentally the system we developed is mostly based on the custom attribute approach but with intelligent caching and updating that lets you avoid doing the work at runtime. We tried to capture the add-in developer experience of the custom attribute solutions with the performance of the manifest based approach.

At development time the add-in developers experience is as simple as with the custom attribute model: they inherit from the right abstract base class, apply the AddInAttribute, and then start writing the add-in.

As we noted earlier however, the custom attribute based approaches have the downside that computing this information requires loading and examining a bunch of assemblies and are too slow to do on application start up or on the UI thread. With this in mind we went a step further and built a system that caches this information to disk. This also led us to split our discovery into two phases: Update and Find.

AddInStore.Update(addinPath);

IList<AddInToken> tokens =

AddInStore.FindAddIns(typeof(AddInType), addinPath);

 

Update Details

The update phase is where we do the work to examine the add-in directory and build the store (or cache) of all the add-ins available there. For each sub directory at the provided path we look at each dll in the directory looking for a type that has [AddInAttribute] applied to it and record the information in the attribute and the base class of the add-in and store all the found information in a file called AddIns.store in the directory provided. One thing to note here is that we examine all of these assemblies in a different AppDomain and we use ReflectionOnly loading: this ensures that the add-in code is never executed until it is activated and that a call to update does not pollute the hosts AppDomain with these assemblies. Rather than recomputing all this information at each call to update we examine the timestamps of the various items in this directory and compare it to the existing store file (if there is one) and decide whether or not there is anything new we need to look at and if not we return quickly.

In addition to providing an update method that does this work we also ship the command tool AddInUtil.exe with the framework that was designed to be called with a custom action in an installer or via a build script.

By providing both a runtime Update method and the command line tool we leave an important decision up to the host: the host can choose to call AddInStore.Update from within the host app and thus make the add-in deployment experience truly an x-copy one, or it can require that add-ins run the tool themselves as part of their setup. In addition we’ve also ensured that the store file itself is x-copyable along with the add-in directory and so if, for example, a host ships with a few add-ins pre-installed it can build the store file when it’s building its setup and just deploy that store file along with the add-ins.

 

FindAddIns Details

In FindAddIns the host passes us the type it wants to use to talk to the add-ins, the directory it wants us to look in and it receives back a collection of AddInTokens representing all the add-ins we found that can be used through the provided abstract base class. This method was designed to be usable during application startup and on the UI thread and we go out of our way to make sure that no types are ever loaded during this call and that the only files we read are the store files at the specified location. If no one (either the host or the add-in) performed an update on that location then no add-ins will be found. Another interesting thing we do is that we keep the store file in memory so that subsequent calls to FindAddIns on the same directory will not require us loading the file again: we simply check to see if the file has been updated since we last loaded it from disk and only reload it if necessary.

The AddInToken objects returned from a FindAddIns call are the objects that the host can actually activate to get back an instance of the add-in. They also contain metadata about the add-in that the host can use to decide which add-ins it wants to load: the add-in always provides a friendly name for itself and can optionally specify a version, publisher, and a short description. All of this information was originally recorded by examining, but not loading, activating, or executing, the add-in during the Update call and was stored in the token object during FindAddIns: this means that the host can get specific information about the add-ins and make informed decisions about which ones to activate without having to every execute any add-in code.

 

Putting it Together

Let’s start with reviewing our three high level goals:

· Minimize complexity of the add-in developer

· Performance of discovery code at runtime

· Ease of host development

Fundamentally all an add-in developer has to do with our system is to inherit from the right abstract base class, apply the [AddInAttribute] to the add-in, and then copy the add-in to the directory the host looks. This was our pri-1 requirement and we’ve made sure that it’s just about as low overhead on the add-in developer as possible.

As far as performance is concerned we leave it up to the host to decide what is most important: can it handle the cost of updating the stores at runtime when new add-ins are installed and reduce the complexity of add-in deployment, or is it acceptable to require a custom action in the installer. If the add-in deployment is responsible for updating the store then our solution is faster than the xml based discovery – since we only need to look at one file and we don’t have to load an xml parser to do so – and if the host decides to do runtime Update then it only pays the cost of discovery when new things are installed and we handle caching that information to disk for future look ups.

Finally we come to the ease of host development and here we may have the best story of all: while our solution may be harder to develop than the solutions we reviewed last time, we’ve removed this complexity since will be shipping as part of the framework.

We really do let you install and find add-ins in two lines of code:

AddInStore.Update(addinPath);

IList<AddInToken> tokens =

AddInStore.FindAddIns(typeof(AddInType), addinPath);

 

At the beginning of the last article I mentioned some exotic discovery mechanisms such as in a database, embedded in a document, and over RSS. We’re actually working with partners who are building these mechanisms on top of our discovery system and down the line we’ll have a series of blog posts demonstrating some of these techniques.

In our next behind the scenes post we’ll pull back the curtains on the magical third line of code that activates your add-ins:

token.Activate<AddInType>(AddInSecurityLevel.Internet)

Comments

• Anonymous
  February 08, 2007
  How does it work if you're a limited user account and running an application located in your Program Files directory? Do you still write the store file out in the Program Files directory? Wouldn't that go against the recommendations about creating/modifying files there in a non-privileged account?Thanks!
• Anonymous
  February 09, 2007
  Thanks for the question Aaron, this is something I've been wanting to get to but haven't yet.We thought a lot about install locations, limited user accounts, and write permissions when we were building this and that led to, or solidified, a few key design points in our system.• First is that we always write the store in the parent directory of the add-ins.  • Second is that FindAddIns actually takes in a params [] of addin paths that makes it very easy for a host to search for add-ins in multiple locations.• Finally we only need to write to disk on Update if there are new add-ins installed (or previous ones were uninstalled).These three things together give hosts that are concerned with limited user accounts (LUA) a good story. Typically they will search for add-ins in two locations: one location will probably be a addins directory sitting next to their app, with the other being somewhere in the users directory. This way the per-user add-ins can be installed from a LUA account and there is still a way, with an elevation to admin, to install per-machine add-ins.Once the decision to look in two places is made the host has the same choice as normal as to whether it wants to require add-ins to update or it will update for them.If the add-in needs to install in the per-machine (typically “program files”) directory it will need to elevate admin rights anyways and will be able to update itself during the install phase. The advice from the Windows LUA team is that the install phase is the proper time to elevate if needed and this pattern fits in well with that. If the app decides to go in later and run an update on this directory it will only require an elevation if an add-in had previously installed/uninstalled but had not ran an update.If the add-in only needs to be per-user than it can install itself in the user directory location and run the update there or, if the host decides, just wait for the host to run update. Since this is user directory neither the host nor the add-in will need to elevate to run the update.--Jesse
• Anonymous
  February 24, 2007
  What about extensibility of this model?  I don't see anyway to customize the update/find process.The reason I ask, is that I have a need to classify different add-ins in a way that gives them different security rights.  This classification must be built upon something akin to a assertion/authentication mechanism, so simply using the type name is insufficient.Most likely this comes out to being digital signatures.  It would be possible to build this on top of the AddIn Store, (locate add-ins with the AddIn store, and then load them and test).The problem with this is that would require nearly all of the same work as the AddInStore is already doing since to make it secure that would need separate AppDomains and ReflectionOnly loading.  Once I've done all that, all your hard work won't have been much help at all.If however, you allowed for some type of interaction during Update which allowed me to accept/reject add-ins, or better yet to classify them into separate stores, or attach metadata to the AddInTokens, I could benefit from your efforts.So, any chance of that?
• Anonymous
  February 26, 2007
  The comment has been removed
• Anonymous
  February 27, 2007
  The comment has been removed
• Anonymous
  March 05, 2007
  Thanks for your comments everyone. We're finding that both in this blog and in our talks with internal partners the question about how to have a customized discovery system on top of ours bubbles to the top very quickly. Stay tuned this week for another post of on discovery.Thank you,Jesse Kaplan
• Anonymous
  November 16, 2007
  FIBER OPTICHAND HELD METERSPHOTO ELECTRIC SENSORPOWER SUPPLY & CONTROLLERSPRESSURE CONTROLLERSPRESSURE SENSOR