38 4. SAFETY VALIDATION OF NEURAL NETWORKS
formation can then be used to create heat maps which visualize the parts of the input image
that contribute most to the neural network’s output. Similar heat maps can also be produced
by occluding a region of the image and observing the change in the classification output. By
repeating this process iteratively over all regions of the input image, the contribution of each
image region to the output can be plotted [211].
4.1.6 BLACK BOX TESTING
Black box testing refers to test methods where the internal structure, design, and implementation
of the software is unknown to the tester. e software is therefore only being tested for its
outputs to specified inputs and the general usability of the system. e aim of black box testing
is to investigate interface errors, initialization and termination errors, as well as performance.
is has the advantage that the testers do not need to be familiar with the internal structure of
the software, therefore the test can be carried out by an independent body from the developers,
removing developer-bias from testing. Due to the inherent opaqueness of neural networks, black
box testing is likely to play a critical role in the safety validation of these systems [141].
For neural network controllers in automotive applications, these black box testing tech-
niques will include targeted testing of the learned function. is will include testing the vehicle
controller in a number of critical scenarios (e.g., emergency braking at high speeds, avoiding
pedestrians on the road, etc.) with a variety of parameters (e.g., distance to lead vehicle at the
beginning of emergency braking). Using black box testing with a wide variety of test scenarios
will allow for a better understanding of its performance and generalization capabilities. How-
ever, it is unlikely to be viable to carry out all of this testing through field testing alone. erefore,
simulation as well as manipulated real data will play a critical role in the black box testing. A
detailed analysis of the impact of synthetic data sources should be carried out to understand the
reliability of the results. erefore, the black box testing for learned autonomous vehicles will
likely include evidence from field tests, simulation, and statistical extrapolation. Using a com-
bination of these sources, the aim should be to obtain the best possible coverage of the desired
operational environment, in order to ensure the results are representative of the system’s actual
level of performance [141].
Grey box testing is also a promising validation framework for deep learning systems. Grey
box testing combines partial knowledge of the internal mechanisms of a software system with
black box testing methods [212]. e benefit of grey box testing is that the testing still involves
investigating the inputs and outputs using the straightforward and unbiased nature of black box
testing methods, however introducing partial knowledge of the system’s internal mechanisms
allows the designer to develop test methods better suited to the specific system. For instance, a
grey box testing framework for deep neural network-driven autonomous vehicles was investi-
gated by Tian et al. [213]. By combining black box testing methods and the measure of neuron
coverage suggested by Pei et al. [198], Tian et al. introduced DeepTest, an automated testing
tool for deep neural network-driven vehicles. DeepTest generates synthesized inputs using nine
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.