Analytics and the Cloud

When we talk about Cloud with our peers in the Industry, the conversation immediately goes to the Data Center level, Infrastructure as a Service is the best understood part of a very complex and new environment. But recently more and more conversations go straight to the higher application level to talk about analytics, algorithms and complex models. There are hundreds of vendors, small and big, providing sophisticated models to price, analyze and evaluate every asset class in the market. But the fragmented nature of the offerings, makes selecting the best combination a complex game of finding the right, sometimes little known, vendors, and continue to create an internal patchwork of libraries, custom apps and Excel add-ins that after a while become a Frankenstein environment so fragile it can never be upgraded or changed because it can stop working completely if any piece is moved.

However, the “appstore” like idea of having all the models and libraries available through a cloud service where you can apply any combination of models and algorithms and pay per use via subscription or calculations performed sounded great but unrealistic given the sensitivity of the data that will need to be sent to the cloud service to take advantage of the models. The obvious advantages to having an always up to date version of the models in whatever platform they were developed was not enough to outweigh the security and privacy concerns around sending trade data to an external cloud provider.

We believe that the availability of hybrid models, that allow accessing and processing data that resides inside the firewall but taking advantage of a trusted external cloud provider to host and execute the models, opens the door for a new approach to building a best of breed model and algorithm combination without creating a management and maintenance nightmare inside the quant, risk or trading desks. There are several options for the trusted cloud providers: the exchanges changing business model could evolve to host more business apps for their members, specialized hosting companies that offer mainly data center space today could expand to compete with the exchanges on offering business services.

Some of our partners are starting to take advantage of Azure, Azure AppFabric and, soon, the Azure Appliance to move complex risk calculations and pricing models to the cloud. There are many other applications that could take advantage of this model, but analytic models and algorithms, with the complexity they create when installed and maintained on premise, can be the ideal first real application of the cloud for the capital markets business.

Comments

• Anonymous
  November 09, 2010
  You're right about Microsoft.  Not sure how the game will be changed, but I agree it will be changed.Jimwww.neocontext.com