C#: Writing extendable applications using on-the-fly compilation
Sometime back I had posted about writing applications that can load plugins using late binding. Users can drop assemblies in a specific folders which is scanned by an host application at startup. Using reflection the host can locate classes that implement a specific interface and instantiate those classes as plugins. The drawback is that the host processes plugin assemblies and you need to store/maintain the corresponding sources elsewhere.
This can be extended using on the fly compilation. In this technique the user drops source files in a specific folder and the host-application either at startup or on user request compiles the sources in-memory and loads the generated assembly. The sources can be in any of the .NET languages like C#, VB.NET or J#. The whole of this can be trivially accomplished using the Microsoft.CSharp and the System.CodeDom.Compiler namespaces from System.dll.
We can call these sources as scripts as they are available as sources. However, in the truest sense they are not scripts because they are not interpreted but are compiled and executed. They are more of plugins where the step of compilation is done by the host application.
There are several security issues to be considered before using this technique and so you might want to read the drawbacks section at the end of this post before trying this out.
The Common Contract
The first and one of the most important things to decide is the Document Object Model or the DOM that the host-application exposes. The DOM is used by the script to manipulate the host-application. Similarly it's also important to decide the interface that the script exposes so that the host application can locate the script class in the script-source and instantiate the class. The DOM and the script interface together form the SW contract that both parties use to communicate with each other.
It's best to define the DOM and the script interface in a common extension dll and make both the host application and all scripts refer to it. The DOM is also defined in terms of an interface which the host application implements. For our purpose let's consider the simple DOM/script-interface combination.
namespace ScriptableApplication.Extensibility
{
public delegate void ClickEvent(int x, int y); public interface IHost { string Title { get; set; } Color BackgroundColor { get; set; } event ClickEvent Click; } public interface IScript { string Name {get;} string Description { get;} void Initialize(IHost host); void Close(); }}
Here the host exposes a simple DOM using which the scripts can modify the host-applications title text and background color. It also exposes an event which is fired each time user clicks on the host-form. Scripts can subscribe to the event to get click notification along with the coordinates of the click.
Each script needs to have at least one class that implements the IScript interface. The host application compiles the script on the fly and using reflection looks for any class that implements this interface and instantiates that class. It then calls the Initialize method passing it the IHost pointer which the script can use to manipulate the host DOM.
On the Fly Compilation
This is what makes the whole technique work. .NET framework exposes all the compilers and associated framework in a very well designed namespace. This can be used to locate any compiler registered with a given file extension and use it to compile the sources.
private CompilerResults LoadScript(string filepath)
{
string language = CSharpCodeProvider.GetLanguageFromExtension(
Path.GetExtension(filepath));
CodeDomProvider codeDomProvider =
CSharpCodeProvider.CreateProvider(language);
CompilerParameters compilerParams = new CompilerParameters();
compilerParams.GenerateExecutable = false;
compilerParams.GenerateInMemory = true;
compilerParams.IncludeDebugInformation = false;
string extAssembly = Path.Combine(
Path.GetDirectoryName(Application.ExecutablePath),
"Extensibility.dll");
compilerParams.ReferencedAssemblies.Add(extAssembly);
compilerParams.ReferencedAssemblies.Add("System.dll");
compilerParams.ReferencedAssemblies.Add("System.Drawing.dll");
compilerParams.ReferencedAssemblies.Add("System.Windows.Forms.dll"); return codeDomProvider.CompileAssemblyFromFile(compilerParams,
filepath);
}
In the first part we use the .NET framework to locate the relevant compiler based on the extension of the script file. In case of unsupported language System.Configuration.ConfigurationErrorsException is thrown.
After we get the code-dom provider we create the compiler parameters. The GenerateInMemory flag is set to indicate the assembly is not generated on disk but in-memory.
Since the script needs to refer to the extensibility dll and some bare minimal .NET framework dlls, they are added to the ReferencedAssemblies collection. After that the compilation is done and the result of the compilation is returned as a CompilerResults object.
In case compilation fails the failures are available and are enumerated as follows
CompilerResults result = LoadScript(filePath);
if (result.Errors.HasErrors)
{
StringBuilder errors = new StringBuilder();
string filename = Path.GetFileName(filePath);
foreach (CompilerError err in result.Errors)
{
errors.Append(string.Format("\r\n{0}({1},{2}): {3}: {4}",
filename, err.Line, err.Column,
err.ErrorNumber, err.ErrorText));
}
string str = "Error loading script\r\n" + errors.ToString();
throw new ApplicationException(str);
}
This gives detailed failure messages along with rom/column numbers of the failure.
Loading the plugins
If there are no compilation errors then the compiled in-memory assembly is available in result.CompiledAssembly. The host uses reflection to search through all the types in the assembly that implements IScript interface and loads that type and calls IScript methods.
GetPlugins(result.CompiledAssembly);
private void GetPlugins(Assembly assembly)
{
foreach (Type type in assembly.GetTypes())
{
if (!type.IsClass || type.IsNotPublic) continue;
Type[] interfaces = type.GetInterfaces();
if (((IList<Type>)interfaces).Contains(typeof(IScript)))
{
IScript iScript = (IScript)Activator.CreateInstance(type);
iScript.Initialize(m_host);
// add the script details to a collection
ScriptDetails.Add(string.Format("{0} ({1})\r\n",
iScript.Name, iScript.Description));
}
}
}
With this we are done initializing the plugins
Implementing the scripts
Implementing the scripts is simple and can be done in any .NET language. The following example is a script in C#.
using System;
using ScriptableApplication.Extensibility;
using System.Drawing;
public class Script : IScript
{
int i = 0;
IHost host = null;
public string Name {get {return "CoolScript";}}
public string Description { get {return "Coolest script";}}
public void Initialize(IHost host)
{
this.host = host;
host.Click +=
delegate(int x, int y)
{
host.Title = string.Format("Clicked on {0}, {1}",
x, y);
Random autoRand = new Random(); host.BackgroundColor =
Color.FromArgb(autoRand.Next(0, 255),
autoRand.Next(0, 255),
autoRand.Next(0, 255));
}; }
public void Close()
{
}
}
The script stores the IHost reference and subscribes to the IHost.Click event. In the event handler it sets the host dialogs title to the position of the click and changes the background-color to some random generated color.
Similarly scripts can be written in VB.NET as follows
Imports Microsoft.VisualBasic
Imports System
Imports System.Drawing
Imports ScriptableApplication.Extensibility
PublicClass Class1
Implements IScript
Public Sub Initialize(ByVal host As IHost)
Implements IScript.Initialize
host.Title = "VB Script loaded"
End Sub
Public Sub Close() Implements IScript.Close
End Sub
Public ReadOnly Property Description() As String
Implements IScript.Description
Get
Description = "I'm a nice VB script"
End Get
End Property
Public ReadOnly Property Name() As String
Implements IScript.Name
Get
Name = "VB Script"
End Get
End Property
End Class
Sample project
You can download the complete sample from here. It contains the following
Extensibility project: This contains only one source file Interfaces.cs which contains definition for the IScript and IHost interfaces. This project generates the Extensibility.dll.
ScriptableApplication project: This contains ScriptHandler.cs which has the complete implementation of on the fly compilation and loading of the dlls. MainForm.cs which implements the host application.
Scripts: Contains two sample scripts script.cs and script.vb
Drawbacks
There are two huge drawbacks to this technique, both of which related to security. If any user with lesser privilege has access to the plugins folder or can direct the host-application to pick up a script from any folder then the user can execute any code he wants using the credentials of the host application. In my case I used this to create motion detection algorithm plugins. This was a personal use test code and I ensured that only I and other admins of the box had write permission to the plugins folder, and so it is kind-of safe for use. However, if you want to use this in a program that's distributed outside you need to plug this security-flaw.
I had initially assumed that I can apply assembly level security restrictions on the generated assembly to disallow file and network access using something like [assembly: FileIOPermission(SecurityAction.RequestRefuse, Unrestricted = true)] and I'll be done. I am not an expert on security related issues and so I asked around on what is the best option in securing this scenario. Shawn Farkas suggested that the best option would be to use sand-boxed AppDomains. However since the plugin will be in an AppDomain different from the AppDomain in which the host application exists, there'll be performance cost due to cross appdomain calls that needs marshalling.
The other issue is error conditions in the script. If there is an unhandled exception in the script it'll bring the whole application down.
Comments
- Anonymous
February 08, 2006
The comment has been removed - Anonymous
February 08, 2006
I don't think that attribute is the right way to go here. As you pointed out there is no way to enforce it. Moreover, the contract as in the interface should be able to complete tell the script writer what he is suppsed to use. Doing things differently for name/description does not add value and no one reads a comment which says he has to apply some attribute.
The solution you suggested of throwing exception is just suicidal. I added a script without an attribute and the application I targetted crashed with a unhandled exception. Specially if the scripts are loaded on-demand as opposed to on-init then users unsaved work just gets lost for a faulty script.... - Anonymous
February 09, 2006
I know that the article ended asking about security, but this was a great demonstration of writing extendable applications using on-the-fly compilation! Kudos!
Keep up the great posts! - Anonymous
February 09, 2006
Why not just detect the folder change event and then process the assemblies then without user interaction or just at start up.
I hope you threat model this attack vector, and every giblet is treated as rogue. - Anonymous
February 09, 2006
Moo, as a demonstration I showed user interaction for loading plugins. You'd never want to do this in production code. Typically at startup plugins are loaded.
Using FileSystem watcher you can get notification when an assembly is dropped into the plugins folder. Using that doesn't make that much sense becuase plugins are installed rarely and its perfectly ok to ask users to restart application once plugins are installed. - Anonymous
February 26, 2006
I'm currently working on a personal project that needs to spit out code after parsing some XML file.... - Anonymous
July 18, 2006
The comment has been removed - Anonymous
July 18, 2006
You need to load the assembly in a seperate AppDomain and tear it down next time. This way the assembly get unloaded.
However, you pay marshaling perf penalties in case your assembly in that AppDomain needs to communitate across AppDomains - Anonymous
January 23, 2007
My friend Greg. He's learning from this code right now. He told me so. - Anonymous
April 23, 2007
readonly configuration. how to resolve this problem//snehal 23-4-2007 add connection string dynamically and culture wise. ConnectionStringSettings css = null; if (System.Globalization.CultureInfo.CurrentCulture.Name == "da-DK") { css = new ConnectionStringSettings("RecipiesGUI.Properties.Settings.ConnectionString", "data source="db\Danish\FoodAndDrink"", "System.Data.SQLite"); } else if(System.Globalization.CultureInfo.CurrentCulture.Name == "en-US") { css = new ConnectionStringSettings("RecipiesGUI.Properties.Settings.ConnectionString", "data source="db\English\FoodAndDrink"", "System.Data.SQLite"); } ConfigurationManager.ConnectionStrings.Add(css); //end snehal