Developing deep learning neural network models requires the variables to have a certain format. Independent variables may come with a varying scale, with some variable values in decimals and some other variables in thousands. Using such varying scales of variables is not very efficient when training a network. Before developing deep learning networks, we make changes such that the variables have similar scales. The process used for achieving this is called normalization.
Two commonly used methods for normalization are z-score normalization and min-max normalization. In z-score normalization, we subtract the mean from each value and divide it by the standard deviation. This transformation results in values that lie between -3 and +3 with a mean of 0 and a standard deviation of 1. For a min-max normalization, we subtract the minimum value from each data point, and then divide it by the range. This transformation converts data to having values between zero and one.
As an example, see the following plots, where we have obtained 10,000 data points randomly from a normal distribution with a mean of 35 and a standard deviation of 5:
From the preceding plots, we can observe that after z-score normalization, the data points mostly lie between -3 and +3. Similarly, after min-max normalization, the range of values changes to data points between 0 and 1. However, the overall pattern seen in the original data is retained after both types of normalization.
Another important step in preparing data when using a categorical response variable is to carry out one-hot encoding. One-hot encoding converts a categorical variable to a new binary format that has values containing either 0 or 1. This is achieved very easily by using the to_categorical() function available in Keras.
Typically data processing steps for unstructured data, such as image or text, are more involved compared with a situation where we are dealing with structured data. In addition, the nature of data preparation steps can vary from one type of data to another. For example, the way we prepare image data for developing a deep learning classification model is likely to be very different from the way we prepare text data for developing a movie review sentiment classification model. However, one important thing to note is that before we can develop deep learning models from unstructured data, they need to be first converted into a structured format. An example of converting unstructured image data into a structured format is shown in the following screenshot, using a picture of the handwritten digit five:
As can be observed from the preceding screenshot, when we read an image file containing a black and white handwritten digit five with 28 x 28 dimensions in R, it gets converted to numbers in rows and columns, giving it a structured format. The right-hand side of the screenshot shows data with 28 rows and 28 columns. The numbers in the body of the table are pixel values that range from 0 to 255, where a value of zero represents the black color and 255 represents the white color in the picture. When developing deep learning models, we make use of some forms of such structured data that are derived from image data.
Once the data for developing the model is prepared in the required format, we can then develop the model architecture.