Title Page Copyright and Credits Hands-On Neural Network Programming with C# Dedication Packt Upsell Why subscribe? Packt.com Contributors About the author About the reviewers Packt is searching for authors like you Preface Who this book is for What this book covers To get the most out of this book Download the example code files Download the color images Code in Action Conventions used Get in touch Reviews A Quick Refresher Technical requirements Neural network overview Neural network training A visual guide to neural networks The role of neural networks in today's enterprises Types of learning Supervised learning Unsupervised learning Reinforcement learning Understanding perceptrons Is this useful? Understanding activation functions Visual activation function plotting Function plotting Understanding back propagation Forward and back propagation differences Summary References Building Our First Neural Network Together Technical requirements Our neural network Neural network training Synapses Neurons Forward propagation Sigmoid function Backward propagation Calculating errors Calculating a gradient Updating weights Calculating values Neural network functions Creating a new network Importing an existing network Importing datasets Testing the network Exporting the network Training the network Testing the network Computing forward propagation Exporting the network Exporting a dataset The neural network Neuron connection Examples Training to a minimum Training to a maximum Summary Decision Trees and Random Forests Technical requirements Decision trees Decision tree advantages Decision tree disadvantages When should we use a decision tree? Random forests Random forest advantages Random forest disadvantages When should we use a random forest? SharpLearning Terminology Loading and saving models Example code and applications Saving a model Mean squared error regression metric F1 score Optimizations Sample application 1 The code Sample application 2 – wine quality The code Summary References Face and Motion Detection Technical requirements Facial detection Motion detection Code Summary Training CNNs Using ConvNetSharp Technical requirements Getting acquainted  Filters Creating a network Example 1 – a simple example Example 2 – another simple example Example 3 – our final simple example Using the Fluent API GPU Fluent training with the MNIST database Training the network Testing the data Predicting data Computational graphs Summary References Training Autoencoders Using RNNSharp Technical requirements What is an autoencoder? Different types of autoencoder Standard autoencoder Variational autoencoders De-noising autoencoders Sparse autoencoders Creating your own autoencoder Summary References Replacing Back Propagation with PSO Technical requirements Basic theory Swarm intelligence Particle Swarm Optimization Types of Particle Swarm Optimizations Original Particle Swarm Optimization strategy Particle Swarm Optimization search strategy Particle Swarm Optimization search strategy pseudo-code Parameter effects on optimization Replacing back propagation with Particle Swarm Optimization Summary Function Optimizations: How and Why Technical requirements Getting started Function minimization and maximization What is a particle? Swarm initialization Chart initialization State initialization Controlling randomness Updating the swarm position Updating the swarm speed Main program initialization Running Particle Swarm Optimization Our user interface Run button Rewind button Back button Play button Pause button Forward button Hyperparameters and tuning Function Strategy Dim size Upper bound Lower bound Upper bound speed Lower bound speed Decimal places Swarm size Max iterations Inertia Social weight Cognitive weight Inertia weight Understanding visualizations Understanding two-dimensional visualizations Understanding three-dimensional visualizations Plotting results Playing back results Updating the information tree Adding new optimization functions The purpose of functions Adding new functions Let's add a new function Summary Finding Optimal Parameters Technical requirements Optimization What is a fitness function? Maximization Gradient-based optimization Heuristic optimization Constraints Boundaries Penalty functions General constraints Constrained optimization phases Constrained optimization difficulties Implementation Meta-optimization Fitness normalization Fitness weights for multiple problems Advice Constraints and meta-optimization Meta-meta-optimization Optimization methods Choosing an optimizer Gradient descent (GD) How it works Drawbacks Pattern Search (PS) How it works Local Unimodal Sampling (LUS) How it works Differential Evolution (DE) How it works Particle Swarm Optimization (PSO) How it works Many Optimizing Liaisons (MOL) Mesh (MESH) Parallelism Parallelizing the optimization problem  Parallel optimization methods Necessary parameter tuning And finally, the code Performing meta-optimization Computing fitness Testing custom problems Base problem Creating a custom problem Our Custom Problem Summary References Object Detection with TensorFlowSharp Technical requirements Working with Tensors TensorFlowSharp Developing your own TensorFlow application Detecting images Minimum score for object highlighting Summary References Time Series Prediction and LSTM Using CNTK Technical requirements Long short-term memory LSTM variants Applications of LSTM CNTK terminology Our example Coding our application Loading data and graphs Loading training data Populating the graphs Splitting data Running the application Training the network Creating a model Getting the next data batch Creating a batch of data How well do LSTMs perform? Summary References GRUs Compared to LSTMs, RNNs, and Feedforward networks Technical requirements QuickNN Understanding GRUs Differences between LSTM and GRU Using a GRU versus a LSTM Coding different networks Coding an LSTM Coding a GRU Comparing LSTM, GRU, Feedforward, and RNN operations Network differences Summary Activation Function Timings Function Optimization Reference The Currin Exponential function Description Input domain Modifications and alternative forms The Webster function Description Input distributions The Oakley & O'Hagan function Description Input domain The Grammacy function Description Input fomain Franke's function Description Input domain The Lim function Description Input domain The Ackley function Description Input domain Global minimum The Bukin function N6 Description Input domain Global minimum The Cross-In-Tray function Description Input domain Global minima The Drop-Wave function Description Input domain Global minimum The Eggholder function Description Input domain Global minimum The Holder Table function Description Input domain Global minimum The Levy function Description Input domain Global minimum The Levy function N13 Description Input domain Global minimum The Rastrigin function Description Input domain Global minimum The Schaffer function N.2 Description Input domain Global minimum The Schaffer function N.4 Description Input domain The Shubert function Description Input domain Global minimum The Rotated Hyper-Ellipsoid function Description Input domain Global minimum The Sum Squares function Description Input domain Global minimum The Booth function Description Input domain Global minimum The Mccormick function Description Input domain Global minimum The Power Sum function Description Input domain The Three-Hump Camel function Description Input domain Global minimum The Easom function Description Input domain Global minimum The Michalewicz function Description Input domain Global minima The Beale function Description Input domain Global minimum The Goldstein-Price function Description Input domain Global minimum The Perm function Description Input domain Global minimum The Griewank function Description Input domain Global minimum The Bohachevsky function Description Input domain Global minimum The Sphere function Description Input domain Global minimum The Rosenbrock function Description Input domain Global minimum The Styblinski-Tang function Description Input domain Global minimum Summary Keep reading Other Books You May Enjoy Leave a review - let other readers know what you think