IDAHO Department of Labor “Job Search” engine leveraging Windows Azure cloud platform

 

Idaho Department of Labor’s IT team has leveraged unique capability of Microsoft’ cloud platform Windows Azure, and it is about building “hybrid services and applications”. Hybrid application architecture implies some parts of the application are hosted and running on their on-premise infrastructure, but other modules/parts are hosted and running in the cloud. Hybrid application is another example of how government organizations can explore and adopt cloud platform on their own terms.

Job Search application engine used traditional n-layer architecture and was hosted on-premise, but the Idaho Department of Labor’s IT team was looking for creative ways drive to higher scalability and performance without investing heavily on their infrastructure. Team has come up with their next-gen version of Job Search engine with a new architecture whereby most applications and data are still hosted on their on-premise servers, but the job search engine service (performing the job search) is running in the cloud and listing/data for the open-jobs (non-sensitive data) is stored in cloud database SQL Azure.

Here’s high-level diagram of the architecture of their Job Search engine and service.

 

• Main ASP.NET application is running on their on-premise infrastructure, and Job Engine service is running in the cloud. Job Service engine was built with .NET Framework in C# programming language, and has both WCF (Windows Communication Foundation) and ASMX interfaces. WCF services not only performs basic function of returning job listings based on the keywords entered by the user, but it also does some interesting things like proximity checks (matching job-seekers location to the location where jobs are available). Results are also ranked by an internal numeric algorithm, distance and the job posting/listing date.
• Sync Framework 2.0 is being utilized to replicate and sync all open job listings from their on-premise database to SQL Azure database hosted in the cloud. This hybrid architecture has enabled them to leverage the cloud for what it offers (higher scalability, availability and performance on commodity servers managed by third-party).
• Open-jobs listings are retrieved from SQL Azure, and stored in the memory cache on each virtual instance for better search performance. Memory cache is updated based regularly on an algorithm. They are also looking at distributed caching capability of the up-coming version of Windows Azure AppFabric.
• Performance counters are currently stored in Azure Table Storage and search metrics are stored in an on premise MSSQL database. They have also written a monitoring application for internal use, which is built using ASP.NET and new ASP.NET 4.0 chart controls and consumes data from counters stored in Windows Azure Table Storage.
• Job Engine service is hosted in only 2 small virtual machine instances of Windows Azure, but they can dial-up or dial-down the number of instances based on schedule/time (peak days in month/week) or cpu-load. This helps them lower their cost and yet provide unlimited scalability on-demand.
• Management of the number of instances running is handled by their on-premise Windows Service (also written in the C# using .NET 4.0 Framework). It relies on the Windows Azure RESTful API and Microsoft.WindowsAzure.StorageClient library to get live statistics and manage the number of Virtual Machines.

Idaho Department of Labor’s IT team started their Windows Azure exploration early September, and deployed Job Search engine service in the Windows Azure (cloud platform) in October. Service is running successfully more than two months, and the department is now looking at other IT work-load scenarios to derive benefits of the Windows Azure cloud platform. Special thanks to Brett Richard (Development Manager) and Greg Gipson (Lead Architect and Developer) from Idaho Department of Labor for this great success story.

Hope this solution provides you with some ideas to adopt hybrid architecture with the cloud platform to drive scalability, efficiency and lower costs.

Here are some resources on learning Windows Azure and SQL Azure

• Windows Azure on MSDN: https://msdn.microsoft.com/windowsazure
• Windows Azure Training Kit: https://msdn.microsoft.com/en-us/wazplatformtrainingcourse.aspx
• Sync Framework: https://msdn.microsoft.com/en-us/sync/default.aspx

Cheers.

Comments

• Anonymous
  January 07, 2011
  Nice solution for a mixed environment.

• Anonymous
  January 10, 2011
  This is a question targeted at Microsoft, but Dept of Labor could answer... I am hearing you say that this Microsoft Azure is providing for better scalability, performance, and reliability.  Do you have any real data, or even just a better explanation, to support these claims? How should we know that Azure is so much better than what Labor was doing before? Better yet, how do we know that labor couldn't have upgraded premise services on their own, with the same or better results, without the Microsoft cloud? We all know that cloud technology is the flavor of the day.  But until we are offered some convincing proof, we'll most likely sum it up as just another game of musical chairs with I.T. services, of course with Microsoft always reaping the rewards. Thanks. Chris Christopher Eaton • Network Analyst Information Technology • Idaho State Tax Commission (208) 334-7568 e-mail: Chris.Eaton@tax.idaho.gov •  website: tax.idaho.gov