The best way to make a neural model have better generalizations or test performance is by training it with more data. In practice, we have very limited training data. The following are a few popular strategies used to get more training data:
- Synthetically generate some training samples: It's not always easy to generate fake training data. But, for some types of data, such as image/video/speech, transformations can be applied to the original data to generate new data. For example, images can be translated, rotated, or scaled to generate new image samples.
- Training with noise: Adding controlled random noise to the training data is another popular data-augmentation strategy. Noise can also be added to the hidden layers of the neural network.