Condividi tramite

Introducing the Microsoft Machine Learning Server 9.3 Release

  • Articolo

This post is authored by Nagesh Pabbisetty, Partner Director of Program Management at Microsoft.

Thank you for your overwhelming response to Microsoft Machine Learning Server and SQL Server Machine Learning Services. We are humbled to see the millions of installs since we first launched Microsoft R Server in 2016, and the pace is only accelerating! Also, a special thank you to all our customers who have published their case studies, a list that includes Acxiom, Attom (RealtyTrac), Blackwoods, eToro, Heartland Bank, InfoSys, Merck, OSISoft, SayInt, Worldsmart and more.

I am excited to announce our latest Microsoft Machine Learning Server 9.3 release which addresses important requests from our users. With the foundational capabilities already in place, starting with this release, we have been iterating rapidly with our customers on the finer details that will accelerate the adoption and usage of SQL Server Machine Learning Services and Machine Learning Server. Key areas of investment in 9.3 are set-up and configuration of Operationalization, platform upgrades, better-together with Azure ML, support for local Spark, improved revoscalepy library, Linux R-Client support for SQL Server compute context, more partnerships and solution templates to quickly get you started and become effective at building and managing intelligent applications. Look for exciting updates to Machine Learning Services in upcoming releases of SQL Server and Azure SQL Database with R Services (preview).

We continue to invest in strengthening the value proposition of Microsoft Machine Learning Server as the most inclusive enterprise platform that caters to the needs of all constituents – be it data engineers, data scientists, line-of-business programmers or IT professionals – with full support for Python and R. This flexible platform offers a choice of languages and features algorithmic innovation that brings the best of open source and proprietary worlds together. It enables best-in-class operationalization support for batch and real-time.

Microsoft Machine Learning Server includes:

  1. High-performance ML and AI wherever your data lives.
  2. The best AI innovation from Microsoft and open source.
  3. Simple, secure and high-scale operationalization and administration.
  4. A collaborative data science environment for intelligent application development.
  5. Deep ecosystem engagements, to deliver customer success with optimal TCO.

You can immediately download Microsoft Machine Learning Server 9.3 from Visual Studio Dev Essentials, or provision it via ML Server VMs in Azure. It comes packed with the power of the open source R and Python engines, making both R and Python ready for enterprise-class ML and advanced analytics. Also check out the R Client for Windows, R Client for Linux, and Visual Studio 2017 with R and Python Tools.

Platform Upgrades & Improved Operationalization

With every new release of CRAN R, we have been making a new release of the enhanced and more performant Microsoft R Open within 6 weeks. True to that tradition, during this release period, we have made two releases of Microsoft R Open, with the same release number as the CRAN R version on which it is based – MRO 3.4.2 and MRO 3.4.3 across all supported platforms. We curate what's new with each release and make this available through MRAN Spotlight.

Machine Learning Server 9.3 is now based on the Microsoft R Open 3.4.3 release. We have also enabled support for Cloudera's CDH 5.12 release. Check out the comprehensive list of supported platforms here.

We have also enabled Microsoft ML Server for Hadoop/Spark to work on a local Spark standalone installation, allowing our customers to develop locally with ML Server for Hadoop/Spark. This will allow data scientists to build models using our platform in a development environment without requiring availability of a Spark cluster. Check out the installation instructions here.

We have now introduced Command Line Interface (CLI) for Machine Learning Server operationalization, making configuration both easy and powerful. These CLIs are similar to Azure CLIs and offer full parity with the Admin Utility in earlier releases. Operationalization refers to the process of deploying R and Python models to Machine Learning Server in the form of web services and the subsequent consumption of these services within client applications to deliver business results. With 1-click Azure Resource Manager (ARM) templates, you can set up an enterprise grade operationalization environment with auto-scaling, high availability and monitoring.

In this release, we further optimized web service request-response time to make it significantly faster. You can construct a dedicated session pool for a specific web service to preload models and code; this will greatly reduce the web service request-response time especially when the models are big.

Azure VMs (Windows, Linux)
Microsoft Machine Learning Server VMs, available on Azure, have been extremely popular, enabling data scientists and application developers to quickly use ML Server on-demand at scale. We offer ML Server across both Windows and multiple versions of Linux – Ubuntu, CentOS and Red Hat Enterprise. We have packed the Linux VMs with common packages that data scientists need, including packages that provide connectivity to Azure data stores making it very easy to use. You can get started easily through the Azure Marketplace. Starting today, you will be able to use ML Server 9.3 VMs and in a few weeks you will be able to use this on all editions of Data Science Virtual Machines.
Azure Stack (Windows, Linux)

One of the biggest differentiators for Microsoft in the field of data, analytics and machine learning is that we recognize the reality of IT organizations world over, in that you have data and applications spread across on-premises and cloud. This means, you need your analytics and AI infrastructure to also reflect and support that reality. Azure Stack enables this hybrid scenario. With Azure Stack, customers get the ease of infrastructure management in edge and disconnected solutions, ability to meet regulatory and governance needs, and modernization of on-premises applications using cloud technologies.

Microsoft Machine Learning Server has been available on Azure Stack since its launch and starting today you can use Machine Learning 9.3 (Windows and Linux) on Azure Stack right away from here.

Azure ML – Machine Learning Server Better-Together

Microsoft announced the preview of Azure Machine Learning, a cloud solution bringing AI to everyone with an end-to-end, scalable, trusted platform. We are now bringing ML Server and Azure ML closer together, and this integration will be enhanced further in successive releases.

  • The high-performance ML packages from ML Server, such as revoscalepy and microsoftml, work seamlessly with AML Services. We offered these packages as standalone Python packages that can be downloaded (PIP installed with a dedicated URL) and used as any of other Python packages in AML services, for experimentation and model management. Please refer to the samples about how to use these packages with AML services.
  • For the models you built with AML, you can use the azureml-model-management-sdk to deploy them to on-premises environment via ML Server operationalization.

Enhanced SQL Server ML Services

In ML Server 9.3, we have added support for SQL compute context in ML Server and in R Client running on Linux platforms, so data scientists who work on Linux workstations can directly use in-database analytics with SQL Server compute context. Additionally, the SQLRUtils package can now be used to package the R scripts into T-SQL stored procedures and run them from R environment on Linux clients.

An interesting scenario enabled by the addition of SQL Server Compute context in ML Server running on Linux is that organizations can now provide a browser-based interface for accessing SQL Server compute context with R Studio Server and ML Server running on a Linux machine connecting to SQL Server.

Since introducing revoscalepy library in the last release of ML Server and SQL Server 2017, we have shipped several additions and improvements in the Python APIs as part of CU releases of SQL Server 2017. We have added APIs like rx_create_col_info, rx_get_var_info etc. that make it easier to get column information, esp. with large number of columns. We added rx_serialize_model for easy model serialization. We have also improved performance when working with string data in different scenarios.

If you want to upgrade your SQL Server in-database R to the latest MRO 3.4.3, you can bind your SQL Server instance to ML Server 9.3. Stay tuned for enhanced in-database ML functionality in SQL Server v.Next. We are also working on bringing all the in-database ML and AI in SQL Server to the cloud, in Azure SQL Database.

Building a Vibrant Ecosystem

During the Microsoft Machine Learning Server 9.2 release period, we announced several marquee partnerships with Alteryx that integrates with SQL Server and Microsoft ML Server, KNIME, and Microstrategy. Since then, Tableau, one of the most popular Business Intelligence software, has been integrated with SQL Server Machine Learning Services to make it easy to leverage SQL in-database analytics from within Tableau. SayInt conversational analytics solution has integrated with SQL Server to derive insights from conversations using Machine Learning Services.

Concurrent with Microsoft Machine Learning Server 9.2 release, Microsoft announced a broad-ranging partnership with Anaconda, provider of the most-popular Python distribution. Although Anaconda is famous for Python, the distribution also includes R packages. We are now unveiling another part of our partnership – Anaconda distribution will now include Microsoft R Open as the default R, enabling data scientists to get the Microsoft's enhanced distribution of R using the conda installer. Anaconda's R distribution will also include Microsoft R Client packages including RevoScaleR and MicrosoftML libraries. Anaconda's graphical package manager also now creates MRO environments by default.

Making It Easy to Adopt & Use Machine Learning Services

Solution Templates

We are continuously refreshing our Solution Templates to work with latest releases of our servers and clients, and also adding new ones. Solution templates bring intelligent applications to life with a few clicks using a guided experience. We have made the source code available for you to use and customize for your own needs. This release includes the following updates:

  • Overview of ML Server solutions via Channel 9 video, and a webinar on Hospital LOS.
  • SQL Server templates refreshed to work with SQL Server 2017.
  • All solution templates now support version ML Sever 9.2 and will be refreshed to 9.3 soon.
  • Offline support for all SQL Server solution templates.
  • Python version of Predicting Hospital Length of Stay template using SQL.

As always, we welcome feedback via the ML solutions Feedback channel.

Microsoft R Archive Network (MRAN)
MRAN site has gone through an infrastructure refresh and a face lift in early December to reflect feedback from our open source and enterprise users. We are glad to see the R-Community leveraging the snapshot feature which is key to reproducibility, and the catalog of R packages that we update daily from the CRAN-R mirror. MRAN continues to be the one-stop shop for all things R, collating a lot of useful resources and launch points for great content, relevant to R data scientists world over. In coming months, we plan to release an updated visualization of package dependencies that makes it easier for our users to traverse packages.

In Summary

With the launch of Microsoft Machine Learning Server 9.3, we are proud to bring enterprises worldwide, an inclusive platform for machine learning and advanced analytics, developed in close collaboration with our customers. We continue to take an inclusive approach by embracing multiple ML/AI toolkits, multiple languages, multiple ways of operationalizing and making applications intelligent, and bringing this value to multiple compute and data contexts. Expect us to take a cloud-first and better-together approach to enabling enterprise AI development life-cycle, and, bringing all of this ML innovation to work well with Azure ML in the cloud. Enterprises can look to Microsoft to bring the best combination of services, infrastructure and tools to make building and deploying intelligent applications as streamlined as can be, for data engineers, data scientists, application developers and IT.

Nagesh