Deep learning frameworks

One of the primary reasons for the widespread popularity and adoption of deep learning is the Python deep learning ecosystem, which consists of easy-to-use open source deep learning frameworks. However, the deep learning landscape is rapidly changing, considering how new frameworks keep getting launched and older ones reach the end of their life. Deep learning enthusiasts might know that Theano was the first and most popular deep learning framework, created by MILA (https://mila.quebec/), which was headed by Yoshua Bengio. Unfortunately, it was recently announced that further development and support for Theano is ending after the launch of its latest version (1.0) in 2017. Hence, it is of paramount importance to understand what frameworks are out there that can be leveraged to implement and solve deep learning. Another point to remember here is that several organizations themselves are building, acquiring, and launching these frameworks (often trying to compete with each other in terms of better features, faster execution, and so on) for everyone's benefit. The following figure showcases some of the most popular deep learning frameworks, as of 2018:

You can also check out the source article of this excellent depiction by Indra den Bakkar at Towards Data Science (https://towardsdatascience.com/battle-of-the-deep-learning-frameworks-part-i-cff0e3841750) for more details. Let's briefly look at some of the most popular deep learning frameworks:

  • Theano: By default, Theano is a low-level framework enabling efficient numeric computations on multi-dimensional arrays, popularly known now as tensors. Theano is quite stable and the syntax is very similar to TensorFlow. It does have GPU support, but is quite limited, especially if we want to use multiple GPUs. Due to its development and support ceasing after 1.0, care should be taken if you intend to use theano for your deep learning implementations.
  • TensorFlow: This is perhaps the most (or at least one of the most) popular deep learning framework. Created by Google Brain and open sourced in 2015, it has quickly gathered the attention of ML, deep learning researchers, engineers, and data scientists. While initial releases had problems with performance, it is under active development and keeps getting better with each release. TensorFlow has support for multi-CPU and GPU-based executions, as well as supporting multiple languages, including C++, Java, R, and Python. It only used to support the symbolic programming style to build deep learning models, which is slightly more complex, but given the widespread adoption since v1.5, it has started supporting the more popular and easy-to-use imperative programming style (also called eager execution). TensorFlow in general is a low-level library, similar to Theano, but also has capabilities for leveraging high-level APIs for rapid prototyping and development. An important part of TensorFlow also includes the tf.contrib module, which contains various experimental features, including the Keras API itself! 
  • Keras: If you find yourself getting perplexed with leveraging low-level deep learning frameworks to solve problems, you can always rely on Keras! This framework is used widely by people with varied skill sets, including scientists who may not be hardcore developers. This is because Keras provides a simple, clean, and easy-to-use high-level API for building efficient deep learning models with minimal code. The beauty of this is that it can be configured to work on top of multiple low-level deep learning frameworks (called backends), including theano and TensorFlow. The Keras documentation is accessible at https://keras.io/ and is quite detailed.
  • Caffe: This is also one of the first and relatively older deep learning frameworks developed in C++ (including Python bindings) by the Berkeley Vision and Learning Center. The best part about Caffe is that it offers many pretrained deep learning models as a part of the Caffe Model Zoo. Facebook recently open sourced Caffe2, which is built upon Caffe with more enhancements and features and is easier to use than its predecessor.
  • PyTorch: The Torch framework is written in Lua and is extremely flexible and fast, often giving huge performance gains. PyTorch is a Python-based framework for building deep learning models, that draws inspiration from Torch. It is not just an extension or a Python wrapper over Torch but a complete framework in itself, improving various aspects of the Torch framework architecture. This includes getting rid of containers, leveraging modules, and also performance improvements such as memory optimizations. 
  • CNTK: The Cognitive Toolkit framework has been open sourced by Microsoft and supports both Python and C++. The syntax is very similar to Keras, and it supports a wide variety of model architectures. Though not immensely popular, this is the framework that Microsoft uses internally for several of their cognitive-intelligence capabilities.
  • MXNet: This has been developed by the Distributed Machine Learning Community (DMLC), the creators of the very popular XGBoost package. This is now an official Apache Incubator project. MXNet was one of the first deep learning frameworks to support diverse languages, including C++, Python, R, and Julia, as well as multiple operating systems, including Windows, which is often neglected by other deep learning frameworks. The framework is extremely efficient and scalable, with support for multi-GPUs. Hence, it has become Amazon's deep learning framework of choice for which it has also developed a high-level interface, called Gluon.
  • Gluon: This is a high-level deep learning framework, or rather interface, that can be leveraged on top of MXNet as well as CNTK. Developed jointly by Amazon AWS and Microsoft, Gluon is very similar to Keras and can be thought of as a direct competitor. It claims, however, that it will support more low-level deep learning frameworks with time, with a vision of democratizing artificial intelligence (AI). Gluon provides a very simple, clean, and concise API that anyone can use to easily build deep learning architectures with minimal code.
  • BigDL: Think of BigDL as deep learning on big data at scale! Developed by Intel, this framework can be leveraged on top of Apache Spark to build and run deep learning models as Spark programs in a distributed fashion over Hadoop clusters. It also leverages the very popular Intel Math Kernel Library (MKL) for efficient and improved performance.

The preceding list of frameworks is definitely not an exhaustive list of deep learning frameworks, but should give you a good idea of what is out there in the deep learning landscape. Feel free to explore these frameworks and choose any of them based on what suits you best.

Always remember that some of the frameworks have steep learning curves, so do not get disheartened if it takes time to learn and leverage them. While each of these frameworks has their own advantages and disadvantages, you should always focus more on the problem to be solved and then leverage the framework most suitable for solving the problem.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset