How to unit test the Bot Framework
Today, let's test the Microsoft Bot Framework.
One of the first things I like to do when attempting to grasp a new framework (well, new to me) is to attempt to unit test the "Hello world" version of the framework. This can give a vague level of complexity to the overall framework, or at least the most basic version to get started, and is just a good exercise overall, I believe. You are, after all, unit testing your code, aren't you?
In today's exercise, we'll be unit testing the RootDialog object, and in particular, just the MessageReceivedAsync method, as it comes straight from the default template in Visual Studio. This should give us a flavor of the testability of the Microsoft Bot Framework overall. I'll be using v3.8.2, which is the latest version as of this writing. The Bot Framework was a little more complex than some to get started unit testing with, which made it an interesting challenge. As usual, mocking the right objects was the challenge. In unit testing, the easiest solution would be if there was no need to mock anything at all, just instantiate objects as would happen in the end to end application and be on your way. However, the Bot Framework, is heavily littered with interfaces, and I was not able to find concrete implementations of those interfaces with which to easily test (more about this later). So mocking it is - or so I thought at first.
The one good thing about a plethora of interfaces in a .NET library is that you have the feeling that it was built with testability in mind, or at least you hope so. There are, of course, other reasons to use interfaces, but the trend over the last ten years or so has been that we should be able to "mock all dependent objects" in our tests. This goes a bit overboard in my opinion, and tends to clutter our tests with (perhaps) unnecessary "plumbing code". If a dependent object is pretty simple and doesn't cross any process boundaries (i.e. it wouldn't make the test slow to a relative crawl), then just instantiate the dependent object and complete your test as quickly and simply as possible. I find the idea of this akin to when we first learned about database normalization - we immediately attempted to normalize our databases to the nth degree (literally) until we realized that it made querying them a slow process, and then learned to judiciously de-normalize the data to improve query performance. So it goes with mocking - the better we initially learned our mocking lessons, the more we intertwined our tests with mocks everywhere, to the detriment of the maintainability of our test code. The moral of this story is, "Don't just automatically mock-all-the-things."
However, with the Bot Framework, it appeared that we would be mocking pretty much every dependent object and parameter - interfaces everywhere:
[csharp]
using System;
using System.Threading.Tasks;
using Microsoft.Bot.Builder.Dialogs;
using Microsoft.Bot.Connector;
namespace MisthavenBot.Dialogs
{
[Serializable]
public class RootDialog : IDialog<object>;
{
public Task StartAsync(IDialogContext context)
{
context.Wait(MessageReceivedAsync);
return Task.CompletedTask;
}
public async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
{
var message = await result as Activity;
int length = (message.Text ?? string.Empty).Length;
await context.PostAsync($"You sent {message.Text} which was {length} characters");
context.Wait(MessageReceivedAsync);
}
}
}
[/csharp]
I initially chose Moq as my mocking framework because, well, I just like Moq. It takes a lot of the heavy lifting out of mocking and you can write more concise mocking code with it than others I've worked with. However, I quickly ran into a problem. I was having a problem mocking IDialogContext.PostAsync. After a frustratingly long amount of time, I came to realize that that method was actually an extension method (Remember, Doug, 'F1' is your friend). As you may or may not know, you can't simply mock an extension method. I'm not aware of any particular reason that it needs to be an extension method. The Bot Framework team owns both the IDialogContext interface (and presumably any of the standard implementers of this interface) and the extension code that includes PostAsync. Why not then merge the two and have a more testable result? Perhaps the thought was to have a clever way to give an interface actual implementation code for any derived class? That's what an extension method with an interface for the "this" object actually achieves. Then why not use an abstract class? Perhaps the answer to that is that the Bot Framework likes to use interfaces so much that it uses multiple interfaces per object - for interfaces themselves, even. For example, IDialogContext implements both IBotToUser and IDialogStack. It's a legitimate design. Since .NET does not support multiple inheritance, such ends are similarly achieved via multiple interfaces.
"But wait," you say. "The nice API docs say that IDialogContext is realized in the concrete DialogContext object. Why not just use that?"
That's true. But the DialogContext constructor takes an IBotToUser object, an IBotData object, an IDialogStack object, an IActivity object, and a CancellationToken.
"Well, is that insurmountable?" you ask.
Let's look at a reasonable implementation for the first object, IBotToUser, which to me is BotToUserTextWriter. The constructor for that takes another IBotToUser and a TextWriter object. Things are getting recursively difficult and it's time to cut bait and mock.
But as I said, extension methods can't be mocked. So what are we to do? We "mock" the impossible with the Microsoft Fakes Framework. In particular, we'll be using a combination of Stubs, for the things that CAN be mocked, with Shims, for the things that normally could not be mocked. Thus, our successful first test of our RootDialog object now looks like:
[csharp]
using Microsoft.Bot.Builder.Dialogs.Fakes;
using Microsoft.Bot.Builder.Dialogs.Internals;
using Microsoft.Bot.Builder.Internals.Fibers.Fakes;
using Microsoft.Bot.Connector;
using Microsoft.QualityTools.Testing.Fakes;
using MisthavenBot.Dialogs;
using System.Threading.Tasks;
using Xunit;
namespace MisthavenBot.Tests
{
public class RootDialogTests
{
string _actualText = null;
[Fact]
public async Task It_should_start_with_the_initial_text()
{
using (ShimsContext.Create())
{
// Arrange
var it = new RootDialog();
StubIDialogContext fakeContext = FakeTheContext();
StubIAwaitable<object> fakeWaitable = FakeTheWaitable();
// Act
await it.MessageReceivedAsync(fakeContext, fakeWaitable);
// Assert
Assert.Equal("You sent Message from User which was 17 characters", _actualText);
}
}
private StubIDialogContext FakeTheContext()
{
var fakeContext = new StubIDialogContext();
Microsoft.Bot.Builder.Dialogs.Fakes.ShimExtensions.PostAsyncIBotToUserStringStringCancellationToken =
(botToUser, text, locale, cancellationToken) =>
{
_actualText = text;
return Task.CompletedTask;
};
return fakeContext;
}
private static StubIAwaitable<object> FakeTheWaitable()
{
var fakeWaiter = new StubIAwaiter<object>
{
GetResult = () => new Activity { Text = "Message from User" },
IsCompletedGet = () => true
};
var fakeWaitable = new StubIAwaitable<object>
{
GetAwaiter = () => fakeWaiter
};
return fakeWaitable;
}
}
}
[/csharp]
This is far from simple for a "Hello World" level example, but it does get the job done.
Note that the use of Shims from the Microsoft Fakes framework means that we won't be able to use Visual Studio 2017 Live Unit Testing. Most likely the injected code from the Shim is more than Live Unit Testing can currently handle (see here for more info). Interestingly enough, it appears that NCrunch has solved this problem.