Dryad and DryadLINQ on HPC Server

Microsoft Research has an interesting project called Dryad which is investigating programming models for distributed data-parallel problems. If you've heard some of the hype around map/reduce programming or the Hadoop framework, you might be familiar with these type of problems. If you have gigabytes, terabytes, or even petabytes of data that you need to churn through, this is a powerful tool. On top of the Dryad framework is a programming model called DryadLinq which brings the power and familiarity of .NET's LINQ syntax to cluster computing. You can write your data analytics algorithm as a LINQ expression in your favorite .NET language and submit cluster jobs directly from Visual Studio.

I am delighted to announce that Microsoft Research has released a distribution of Dryad and DryadLINQ that runs on top of HPC Server 2008. You can download this release from here and contribute feedback and suggestions at the Microsoft Connect site here. If you want to learn more about Dryad and DryadLINQ, you can find more information on the Microsoft Research site. The source for DryadLINQ is also included in this release.

Releasing this has been a great collaboration between the HPC team and Microsoft Research. I am excited to get this work out there and see what people can do with it. So try it out and give us some feedback!

John Vert

Architect

Windows HPC Server

Update - Channel9 has a video interview with Erik Meijer and Roger Barga discussing this: https://channel9.msdn.com/posts/Charles/Expert-to-Expert-Erik-Roger-Barga-Introduction-to-Dryad-and-DryadLINQ/