In the preceding methods the same learning rate is applied to all parameter updates. Having sparse data, we may instead want to update the parameters in different extent. Adaptive gradient descent algorithms, such as AdaGrad, AdaDelta, RMSprop, and Adam, provide an alternative to classical SGD by keeping per parameter learning rates.
Adaptive learning rate – separate for each connection
by Tamoghna Ghosh, Raghav Bali, Dipanjan Sarkar
Hands-On Transfer Learning with Python
- Title Page
- Copyright and Credits
- Dedication
- Packt Upsell
- Foreword
- Contributors
- Preface
- Machine Learning Fundamentals
- Deep Learning Essentials
- What is deep learning?
- Deep learning frameworks
- Setting up a cloud-based deep learning environment with GPU support
- Setting up a robust, on-premise deep learning environment with GPU support
- Neural network basics
- A simple linear neuron
- Gradient-based optimization
- The Jacobian and Hessian matrices
- Chain rule of derivatives
- Stochastic Gradient Descent
- Non-linear neural units
- Learning a simple non-linear unit – logistic unit
- Loss functions
- Data representations
- Multilayered neural networks
- Backprop – training deep neural networks
- Challenges in neural network learning
- Initialization of model parameters
- Improvements of SGD
- Overfitting and underfitting in neural networks
- Hyperparameters of the neural network
- Summary
- Understanding Deep Learning Architectures
- Neural network architecture
- Various architectures
- Summary
- Transfer Learning Fundamentals
- Unleashing the Power of Transfer Learning
- Image Recognition and Classification
- Text Document Categorization
- Text categorization
- Word representations
- CNN document model
- Building a review sentiment classifier
- What has embedding changed most?
- Transfer learning – application to the IMDB dataset
- Training on the full IMDB dataset with Word2vec embeddings
- Creating document summaries with CNN model
- Multiclass classification with the CNN model
- Visualizing document embeddings
- Summary
- Audio Event Identification and Classification
- DeepDream
- Style Transfer
- Automated Image Caption Generator
- Understanding image captioning
- Formulating our objective
- Understanding the data
- Approach to automated image captioning
- Image feature extraction with transfer learning
- Building a vocabulary for our captions
- Building an image caption dataset generator
- Building our image language encoder-decoder deep learning model
- Training our image captioning deep learning model
- Evaluating our image captioning deep learning model
- Automated image captioning in action!
- Summary
- Image Colorization
- Other Books You May Enjoy
Adaptive learning rate – separate for each connection
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.