Testing Ruby Applications on Windows Azure - part 1: Overview and Challenges
Following up on Brian’s post about running tests in Windows Azure, I want to talk about some specific challenges to testing Ruby applications on Windows Azure. In subsequent posts I'll walk through examples of running tests against a Ruby application hosted on Windows Azure
Why Test on Azure?
You may be thinking “My tests pass fine on Windows, and I’ve even ran my tests in the Windows Azure Emulator. Windows Azure can’t be that different”. In many cases you’re right, but there are differences between even the emulator and the Windows Azure live environment (see https://msdn.microsoft.com/en-us/library/windowsazure/gg432968.aspx for more information.) Also, it’s always a good idea to test on the actual environment you’ll be running on.
Ways to Test
Brian mentions three ways to test; remote desktop (RDP), using a web front-end for running tests, and automating tests as part of the deployment. This brings up our first challenge; I don’t know of any Ruby testing frameworks that provide a web front-end for running tests. There’s a lot of Ruby testing frameworks though, so there might be one that I haven’t found yet that does this. If someone knows of one, leave a comment with a link to the framework and I'll check it out. For now though, I’m going to skip web front-end testing.
Remote Desktop
Windows Azure doesn’t provide SSH as a default. Even if you manually install it there’s no way to guarantee sticky sessions to a specific instance, so RDP is currently the only option if you want to get direct access to a console session. Unfortunately, RDP is configured as part of the deployment package. Since Microsoft doesn’t currently publish any deployment/packaging tools for Ruby, this means we need access to Visual Studio.
I’ve talked in the past about the ways to deploy a Ruby application to Windows Azure, and while some come with pre-built packages, only one comes with RDP enabled by default. AzureRunMe provides the option for RDP as a default, but the default configuration isn’t very secure as the certificate file, username, and password for the default configuration are published on the project page. To modify these defaults, or to create an RDP configuration for another deployment solution such as the SmarxRole project, you need Visual Studio.
Automated Testing
With the recent addition of startup tasks to Windows Azure, it’s relatively easy to add a task that runs your tests when a role instance starts. While there's no REST API to configure diagnostics dynamically from Ruby, you can include a diagnostics.wadcfg as part of your deployment.
Beyond the Application Instance
One thing that Brian didn't mention in his post was that at some point you need to get away from testing each instance and test the application as a whole. An application in Windows Azure generally has multiple instances, is stateless, and the instance you initially connect to from your client (web browser for instance,) may not be the same instance that subsequent requests go to. What you need then is functional testing at the overall application level.
This is where things like Watir or Selenium come in handy. Both provide browser automation, so you can script the actions a user would take when visiting your web application and verify that the application behaves as expected. These are equally applicable to Windows Azure and non-Azure web sites however, so I'm not planning on going into details on how to use them, but I thought it warranted mentioning that the testing within the Windows Azure hosted application isn't the be-all and end-all of application testing.
What’s next?
Next week I’ll post an example of enabling and using RDP to connect to a Ruby application hosted on Windows Azure, and then follow up with an example of automated testing.