Share via

MS Open Tech Open Sources Rx (Reactive Extensions) – a Cure for Asynchronous Data Streams in Cloud Programming

  • Article

From:
Erik Meijer, Architect, Microsoft Corp.
Claudio Caldato, Lead Program Manager, Microsoft Open Technologies, Inc.

Updated with a quote from Ferranti Computer Systems NV

Updated: added quotes from Netflix and BlueMountain Capital Management

If you are a developer that writes asynchronous code for composite applications in the cloud, you know what we are talking about, for everybody else Rx Extensions is a set of libraries that makes asynchronous programming a lot easier. As Dave Sexton describes it, “If asynchronous spaghetti code were a disease, Rx is the cure.”

Reactive Extensions (Rx) is a programming model that allows developers to glue together asynchronous data streams. This is particularly useful in cloud programming because helps create a common interface for writing applications that come from diverse data sources, e.g., stock quotes, Tweets, computer events, Web service requests.

Today, Microsoft Open Technologies, Inc., is open sourcing Rx. Its source code is now hosted on CodePlex to increase the community of developers seeking a more consistent interface to program against, and one that works across several development languages. The goal is to expand the number of frameworks and applications that use Rx in order to achieve better interoperability across devices and the cloud.

Rx was developed by Microsoft Corp. architect Erik Meijer and his team, and is currently used on products in various divisions at Microsoft. Microsoft decided to transfer the project to MS Open Tech in order to capitalize on MS Open Tech’s best practices with open development.

There are applications that you probably touch every day that are using Rx under the hood. A great example is GitHub for Windows.

According to Paul Betts at GitHub, "GitHub for Windows uses the Reactive Extensions for almost everything it does, including network requests, UI events, managing child processes (git.exe). Using Rx and ReactiveUI, we've written a fast, nearly 100% asynchronous, responsive application, while still having 100% deterministic, reliable unit tests. The desktop developers at GitHub loved Rx so much, that the Mac team created their own version of Rx and ReactiveUI, called ReactiveCocoa, and are now using it on the Mac to obtain similar benefits."

And Scott Weinstein with Lab49 adds, “Rx has proved to be a key technology in many of our projects. Providing a universal data access interface makes it possible to use the same LINQ compositional transforms over all data whether it’s UI based mouse movements, historical trade data, or streaming market data send over a web socket. And time based LINQ operators, with an abstracted notion of time make it quite easy to code and unit test complex logic.”

Netflix Senior Software Developer Jafar Husain explained why they like Rx. "Rx dramatically simplified our startup flow and introduced new opportunities for performance improvements. We were so impressed by its versatility and quality, we used it as the basis for our new data access platform. Today we're using both the Javascript and .NET versions of Rx in our clients and the technology is required learning for new members of the team."

And Howard Mansell, Quantitative Strategist with BlueMountain Capital Management added, “We are very pleased that Microsoft are Open-Sourcing the Reactive Extensions for .NET. This will allow users to better reason about performance and optimize their particular use cases, which is critical for performance and latency sensitive applications such as real-time financial analysis.”

From Belgium, Guido Van de Velde, Director  MECOMS Product Organisation for Ferranti Computer Systems NV, explains how Rx is important for their global company: “Ferranti uses Rx in its vertical solution for the utility market, MECOMS™, to process and manage all data and events from the Smart Grid. Its architecture allows the setup of data processing pipelines which can scale and deliver excellent performance. Performance testing together with Microsoft showed that this architectures supports up to hundreds of millions of smart meters and other sensors, running on  commodity hardware. Thanks to Rx we can focus on component functionalities and don’t have to worry about interfaces and connections between the different components saving significant development time.”

Part of the Rx development team will be on assignment with the MS Open Tech Hub engineering program to accelerate the open development of the Rx project and to collaborate with open source communities. Erik will continue to drive the strategic directions of the technology and leverage MS Open Tech Hub engineering resources to update and improve the Rx libraries. With the community contribution we want to see Rx be adopted by other platforms. Our goal is to build an open ecosystem of Rx-compliant libraries that will help developers tackle the complexity of asynchronous programming and improve interoperability.

We are also happy to see that our decision is welcome by open source developers.

“Open sourcing Rx just makes sense. My hope is that we’ll see a couple of virtuous side-effects of this decision. Most likely will be faster releases for bug fixes and performance improvements, but the ability to understand the inner workings of the Rx code should encourage the creation of additional tools and Rx providers to remote data sources,” said Lab 49’s Scott Weinstein.

According to Dave Sexton, http://davesexton.com/blog, “It’s a solid library built around core principles that hides much of the complexity of controlling and coordinating asynchrony within any kind of application. Opening it will help to lower the learning curve and increase the adoption rate of this amazing library, enabling developers to create complex asynchronous queries with relative ease and without any spaghetti code left over.”

Starting today, the following libraries are available on CodePlex:

  • Reactive Extensions
    • Rx.NET: The Reactive Extensions (Rx) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators.
    • RxJS: The Reactive Extensions for JavaScript (RxJS) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators in JavaScript which can target both the browser and Node.js.
    • Rx++: The Reactive Extensions for Native (RxC) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators in both C and C++.
  • Interactive Extensions
    • Ix: The Interactive Extensions (Ix) is a .NET library which extends LINQ to Objects to provide many of the operators available in Rx but targeted for IEnumerable<T>.
    • IxJS: An implementation of LINQ to Objects and the Interactive Extensions (Ix) in JavaScript.
    • Ix++: An implantation of LINQ for Native Developers in C++.
  • Bindings
    • Tx: a set of code samples showing how to use LINQ to events, such as real-time standing queries and queries on past history from trace and log files, which targets ETW, Windows Event Logs and SQL Server Extended Events.
    • LINQ2Charts: an example for Rx bindings. Similar to existing APIs like LINQ to XML, it allows developers to use LINQ to create/change/update charts in an easy way. We would love to see more Rx bindings like this one.
    • With these libraries we are giving developers open access to both push-based and pull-based data via LINQ in Microsoft’s three fundamental programming paradigms (native, JScript and Managed code).

We look forward to seeing you guys use the library, share your thoughts and contribute to the evolution of this fantastic technology built for all you developers.

Comments

  • Anonymous
    November 06, 2012
    Huge +1 guys. Best news I've heard in months. Now we can help you make this product even better, and help diagnose problems more quickly. Rx is destined to be as core a part of .Net as TPL.

  • Anonymous
    November 06, 2012
    Awesome news! :D

  • Anonymous
    November 06, 2012
    This is great news!

  • Anonymous
    November 07, 2012
    We encountered a problem loading "Ix%2b%2b%2fsrc%2fcpplinq%2flinq.hpp". If you continuously encounter this issue, please contact us.

  • Anonymous
    November 07, 2012
    @Dinasty: please open a bug on https://rx.codeplex.com

  • Anonymous
    November 07, 2012
    yes! yes! yes!....Best news ever

  • Anonymous
    November 07, 2012
    Very informative post indeed.. being enrolled in www.wiziq.com/.../6314-learn-c-programming-language-low-priced-student-edition was looking for such articles online to assist me.. and your post helped me a lot

  • Anonymous
    November 19, 2012
    The goal is to expand the number of frameworks and applications that use Rx in order to achieve better interoperability across devices and the cloud

  • Anonymous
    February 05, 2014
    Hi. I work in an industry that's extremely suspicious of open-source. (Personally I think this is reactionary, but I do not have the power to change it.) There's some confusion now within my organisation whether we can continue using Reactive Extensions. Please could you produce a statement confirming:

  • The Reactive Extensions SDK from Microsoft.com is distributed under the same Microsoft proprietary license as ever, and one can use it without agreeing to any open-source license. msdn.microsoft.com/.../gg577610

  • The Reactive Extensions Nuget packages are distributed under Microsoft proprietary licenses, and one can use them without agreeing to any open-source license. www.nuget.org/.../Rx-Main

  • It's only the Reactive Extensions source code on Codeplex that's now distributed under an open source license (viz. Apache 2) and this has no effect on the use of SDK. https://rx.codeplex.com/

  • The Reactive Extensions SDK will, for as long as it's available, be available under a Microsoft proprietary license. Thanks. I'd really appreciate this. Otherwise, we might have to remove RX from our app. Again, I'm personally fine with open-source and understand the difference between permissive licenses (eg. Apache, MIT, MPL) and copyleft licenses (eg. GPL). Nevertheless, it would help to see an official looking document explaining that the Reactive Extensions SDK can be used without agreeing to any open-source license.