In this section, we will learn about two new variants of gradient descent, called momentum and Nesterov accelerated gradient.
Momentum-based gradient descent
by Sudharsan Ravichandiran
Hands-On Deep Learning Algorithms with Python
- Title Page
- Copyright and Credits
- Dedication
- About Packt
- Contributors
- Preface
- Section 1: Getting Started with Deep Learning
- Introduction to Deep Learning
- Getting to Know TensorFlow
- What is TensorFlow?
- Understanding computational graphs and sessions
- Variables, constants, and placeholders
- Introducing TensorBoard
- Handwritten digit classification using TensorFlow
- Introducing eager execution
- Math operations in TensorFlow
- TensorFlow 2.0 and Keras
- Should we use Keras or TensorFlow?
- Summary
- Questions
- Further reading
- Section 2: Fundamental Deep Learning Algorithms
- Gradient Descent and Its Variants
- Generating Song Lyrics Using RNN
- Improvements to the RNN
- LSTM to the rescue
- Gated recurrent units
- Bidirectional RNN
- Going deep with deep RNN
- Language translation using the seq2seq model
- Summary
- Questions
- Further reading
- Demystifying Convolutional Networks
- Learning Text Representations
- Understanding the word2vec model
- Building the word2vec model using gensim
- Visualizing word embeddings in TensorBoard
- Doc2vec
- Understanding skip-thoughts algorithm
- Quick-thoughts for sentence embeddings
- Summary
- Questions
- Further reading
- Section 3: Advanced Deep Learning Algorithms
- Generating Images Using GANs
- Differences between discriminative and generative models
- Say hello to GANs!
- DCGAN – Adding convolution to a GAN
- Least squares GAN
- GANs with Wasserstein distance
- Summary
- Questions
- Further reading
- Learning More about GANs
- Reconstructing Inputs Using Autoencoders
- Exploring Few-Shot Learning Algorithms
- Assessments
- Chapter 1 - Introduction to Deep Learning
- Chapter 2 - Getting to Know TensorFlow
- Chapter 3 - Gradient Descent and Its Variants
- Chapter 4 - Generating Song Lyrics Using an RNN 
- Chapter 5 - Improvements to the RNN
- Chapter 6 - Demystifying Convolutional Networks
- Chapter 7 - Learning Text Representations
- Chapter 8 - Generating Images Using GANs
- Chapter 9 - Learning More about GANs
- Chapter 10 - Reconstructing Inputs Using Autoencoders
- Chapter 11 - Exploring Few-Shot Learning Algorithms
- Other Books You May Enjoy
Momentum-based gradient descent
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.