Freigeben über

Microsoft Updates its Deep Learning Toolkit

  • Artikel

This post is by Chris Basoglu, Partner Engineering Manager in the AI & Research group at Microsoft.

We are delighted to announce that Microsoft has brought Microsoft Cognitive Toolkit version 2.0 out of beta and is making the first release candidate available today. The toolkit, previously known as CNTK, is a system for deep learning used to speed advances in areas such as speech and image recognition and search relevance on CPUs and NVIDIA® GPUs. Cognitive Toolkit can be used on-premises or in the cloud with Azure GPUs.

Cognitive Toolkit is being used extensively by a wide variety of Microsoft products, by companies worldwide with a need to deploy deep learning at scale, and by students interested in the very latest algorithms and techniques. The latest version of the toolkit is available on GitHub via an open source license. Since releasing the beta in October 2016, more than 10 beta releases have been deployed with hundreds of new features, performance improvements and fixes.

Cognitive Toolkit as a Library

Cognitive Toolkit has added more bindings beyond BrainScript. Version 2.0 enables using Cognitive Toolkit as a software library with the following supported bindings:

Python Examples and Tutorials (Jupyter Notebooks)

Recognizing the importance of Python in deep learning, we have prepared a set of Python examples and tutorials (the latter are implemented as Jupyter Notebooks). You will find relevant information at these locations:

You may also use Cognitive Toolkit Docker Containers to run Jupyter Notebooks tutorials.

Layers

The Layers library is significantly improved.  A large number of common “layers” are predefined – making it very easy to write simple networks that consist of standard layers layered on top of each other. 

New Evaluation Library

The new Cognitive Toolkit evaluation library has also been significantly enhanced, with improved performance and ease of use. The evaluation library can be used on Windows and Linux utilizing C++, Python, as well as C# and other .NET languages.

Learn more about the Cognitive Toolkit Evaluation Library here.

New Features

We have introduced several new features in the version 2.0 beta, including:

  • The ability to extend Cognitive Toolkit functions, learners, trainers and optimizers with your own algorithms in Python, C++.
  • Enhanced, built-in distributed readers for speech, image, and text deep learning tasks.
  • The ability to use TensorBoard visualizations from Cognitive toolkit! Read more here.
  • Pretrained models available for use.
  • Performance improvements.
  • Support of distributed scenarios in Python API. See more in the sections on distributed scenarios in the ConvNet and ResNet examples.
  • Support of Asynchronous Stochastic Gradient Descent (ASGD)/Hogwild! training parallelization support using Microsoft's Parameter Server (Project Multiverso).
  • Support for training on one-hot and sparse arrays via NumPy.
  • Support of object recognition using Fast R-CNN algorithm.
  • Integration with NVIDIA NCCL, a stand-alone library of standard collective communication routines, such as all-gather, reduce, broadcast, etc., that have been optimized to achieve high bandwidth over PCIe. See how to enable NCCL in the Cognitive Toolkit Wiki.
  • Lambda rank and NDCG at 1 are accessible from Python for real this time.
  • Performance Profiler for BrainScript and Python.
  • Support in training session for cross validation and preservation of all checkpoints.

New Automated Installation Procedures

We have enabled different ways to install CNTK in an automated manner:

  • Installation scripts for binary distribution.
  • Installation scripts for the users who would like to work with CNTK source code.
  • Installation via pip install procedure.

Read more about the different ways to install Cognitive Toolkit here.

Docker Hub and Self-Built Docker Images

CNTK is now available as Docker Images at Docker Hub.

You may also build your own Docker Images using pre-configured Docker files from the CNTK Code base. See this Wiki page on using Cognitive Toolkit as Docker Images and Containers.

NuGet Packages

Windows developers are welcome to use multiple CNTK NuGet packages supporting both v.1 and v.2 model formats. See more on Cognitive Toolkit NuGet packages here.

Contact Us

You can always work with us through the community pages on GitHub issues, Stack Overflow and @mscntk.

Chris

Comments

  • Anonymous
    April 04, 2017
    I basically don't want to learn/install/use python. Is there way to use CTNK with these constraints ?
    • Anonymous
      April 04, 2017
      Yes. For training, you can use BrainScript or C++ instead. For hosting pre-trained models, you can use C#.
      • Anonymous
        April 07, 2017
        The comment has been removed
  • Anonymous
    April 06, 2017
    Awesome development! We need it, keep up the progress!