This chapter presents one well-known application in daily life to which neural networks can perfectly be applied: forecasting weather. We are going to walk through the entire process of designing a neural network solution to this problem: how to choose the neural architecture and the number of neurons, as well as selecting and preprocessing data. Then the reader will be presented with techniques to handle time series datasets, from which our neural network is going to make predictions on weather variables using the Java programming language. The topics covered in this chapter are as follows:
- Neural networks for regression problems
- Loading/selecting data
- Input/output variables
- Choosing inputs
- Preprocessing
- Normalization
- Empirical design of neural networks
So far, the reader has been presented with a number of neural network implementations and architectures, so now it is time to get into more complex cases. The power of neural networks in predictions is really astonishing since they can perform learning from historical data in such a way that neural connections are adapted to produce the same results according to some input data. For example, for a given situation (cause), there is a consequence (result) and this is represented as data; neural networks are capable of learning the nonlinear function that maps the situation to the consequence (or the cause to the result).
Prediction and regression problems are an interesting category to apply neural networks to. Let's take a look at a sample table containing weather data:
|
Date |
Avg. Temperature |
Pressure |
Humidity |
Precipitation |
Wind Speed |
|---|---|---|---|---|---|
|
July 31st |
23 ºC |
880 mbar |
66% |
16 mm |
5 m/s |
|
August 1st |
22 ºC |
881 mbar |
78% |
3 mm |
3 m/s |
|
August 2nd |
25 ºC |
884 mbar |
65% |
0 mm |
4 m/s |
|
August 3rd |
27 ºC |
882 mbar |
53% |
0 mm |
3 m/s |
|
… | |||||
|
December 11th |
32 ºC |
890 mbar |
64% |
0 mm |
2 m/s |
The table above depicts five variables containing hypothetical values of weather data collected from a hypothetical city, only for the purpose of this example. Now let's suppose that each of the variables contains a list of values sequentially taken over time. We can think of each list as a time series. On a time series chart, one can see how they evolve with time:
The relationship between these time series denotes a dynamic representation of weather in a certain city, as depicted in the chart above. We want the neural network to learn this dynamic representation; however, we need to structure this data the in a way neural networks can process, that is, identifying which data series (variables) are the cause and which are the effect. Dynamic systems have variables whose value depends on past values, so neural network applications can rely not only on the present situation, but also on the past. This is very important because historical events influence the present and future.
Only after structuring data can we structure the neural network, that is, the number of inputs, outputs, and hidden nodes. However, there are many other architectures that may be suitable for prediction problems, such as radial basis functions and feedback networks, among others. In this chapter, we are dealing with a feedforward multilayer perceptron with the Backpropagation learning algorithm, to demonstrate how this architecture can be simply exploited to predict weather variables; also, this architecture presents very good generalized results with good selected data and there is little complexity involved in the design process.
The overall process for designing neural networks for prediction processes is depicted in the figure below:
If the neural network fails to be validated (step 5), usually a new structure (step 3) is defined, although sometimes steps 1 and step 2 may be repeated. Each of the steps in the figure will be addressed in the next sections in this chapter.