1
C H A P T E R 1
Introduction
Autonomous vehicles have the potential to transform our transportation systems in terms of
safety and efficiency. e steady increase in the number of vehicles on the road has led to in-
creased pressure to solve issues such as traffic congestion, pollution, and road safety. e leading
answer to resolving these issues among the research community is self-driving cars [1–3]. For
instance, according to the World Health Organization, an estimated 1.3 million people die in
road accidents yearly [4]. Meanwhile, up to 90% of all car accidents are estimated to be caused by
human errors [5], therefore autonomous vehicles can provide significant safety improvements by
eliminating driver errors. Further benefits provided by autonomous vehicles include better fuel
economy, reduced pollution, car sharing, increased productivity, and improved traffic flow [6–9].
Autonomous vehicles generally consist of the five functional components shown in
Fig. 1.1: Perception, Localization, Planning, Control, and System Management [10]. Percep-
tion observes the environment around the vehicle and identifies important objects such as traffic
signals and obstacles. Localization maps the surrounding environment and identifies the loca-
tion of the vehicle in absolute position. Planning uses the input from perception and localization
to determine the high-level actions the vehicle will take in terms of routes, lane changes, and
desired velocity. Control module oversees carrying out low-level actions indicated by the plan-
ning, such as steering, accelerating, and braking for the vehicle. System management oversees
the operation of all the modules and provides the Human-Machine Interface.
Among the earliest autonomous vehicles are the projects presented by Carnegie Mel-
lon University for driving in structured environments [11] and University of Bundeswehr Mu-
nich for highway driving [12] in the 1980s. Since then, projects such as DARPA Grand Chal-
lenges [13, 14] have continued to drive forward research in autonomous vehicles. Besides aca-
demic research, car manufacturers and tech companies have also begun developing their own au-
tonomous vehicle capabilities. ese early steps toward autonomy have led to multiple Advanced
Driver Assistance Systems (ADAS) such as Adaptive Cruise Control (ACC), Lane Keeping
Assistance (LKA), and Lane Departure Warning (LDW) technologies, which provide modern
vehicles with level 1-2 autonomy (see Fig. 1.2) [15]. While these technologies have increased
the safety of modern vehicles and made driving easier, the end goal in autonomous vehicle re-
search is to achieve level 5 autonomy, with a fully autonomous vehicle which does not require
human intervention. erefore, these partially autonomous systems pave the way for the future
autonomous vehicles.
2 1. INTRODUCTION
Autonomous
Vehicles
LocalizationPerception
System
Management
Human-
Machine
Interface
Planning Control
Scene
Understanding
System
Diagnosis
Fault
Management
Semantic
Segmentation
Obstacle
Recognition
Mapping Path Planning Steering
Relative
Positioning
Driving
Maneuvers
Acceleration
Lane Selection Braking
Global
Positioning
Figure 1.1: Autonomous vehicle functional units.
0
No Automation
Zero autonomy.
Driver is
responsible for
all driving tasks.
1
Driver
Assistance
Driver is in
control of the
vehicle, but some
driver assist
features are
used.
2
Partial
Automation
Vehicle has
combined
autonomous
functions, but
driver still needs
to stay in control.
3
Conditional
Automation
Vehicle can be
fully autonomous
in certain
situations, but
driver must be
ready to take
control of the
vehicle.
4
High Automation
Vehicle can be
fully autonomous
under certain
situations,
without the
driver monitoring
the environment
at all times.
5
Full Automation
Vehicle can
be fully
autonomous
at all times. No
driver required.
Figure 1.2: e six levels of autonomy in road vehicles, as defined by the Society of Automotive
Engineers (SAE).
Early autonomous vehicles used a perception-planning-control framework, with each
function achieved separately [16]. Perception relied on accurate sensor data, with multi-sensor
setups and sensor fusion used to capture the driving environment accurately. Planning and con-
trol were achieved through rule-based systems and classical controllers were often based on lin-
earized (or otherwise simplified) vehicle models [17–19]. e downside of this approach is that
the control parameters required hand-tuning based on simulation and field test results, which
was very time intensive and resulted in systems which had difficulty generalizing to a wide variety
1. INTRODUCTION 3
of scenarios [20, 21]. Moreover, linearized vehicle models can cause significant inaccuracies in
the highly nonlinear regions of driving, which meant that these control methods were infeasible
or did not scale well in certain scenarios [22, 23]. e shortcomings of early autonomous vehi-
cle systems led researchers to look for alternative solutions. Recently, the rise of deep learning
has drastically changed research in several areas of Artificial Intelligence (AI), and significantly
improved the state-of-the-art in fields such as image classification and speech recognition [24–
26]. e powerful function approximation capabilities of Deep Neural Networks (DNNs) has
also motivated the use of deep learning in several autonomous vehicle applications, including
planning and decision making [27–29], vehicle-to-vehicle communications [30, 31], percep-
tion [32–34], as well as mapping and localization [35–37].
Deep learning has gained significant interest as a promising solution to autonomous ve-
hicle control. Deep learning not only provides excellent performance in control applications but
can also provide the capability to generalize its previously learned rules to new scenarios [38–42].
Rather than requiring a formal specification of the exact behavior as you would in a rule-based
system, deep learning enables the system to learn a general behavior from examples either via
demonstration or interaction with the environment. e powerful representational power of
DNNs and generalization capability given by deep learning makes these systems well suited for
complex and dynamic tasks and operational environments [43–45], such as those seen by an
autonomous vehicle. However, the disadvantage of deep learning is its complex and opaque na-
ture. is poses a challenge for the safety validation and verification of deep learning algorithms.
Due to the lack of interpretability in terms of what the neural network has learned and how it
makes its decisions, there are currently no known methods for guaranteeing the safety of such
a system to a certifiable level. erefore, new methods for validating the safety of deep learning
systems in autonomous vehicles are required.
is book offers a comprehensive view of the current state of autonomous vehicle control
techniques through deep learning. A sampling of the most recent research works is presented
in this field, and different approaches are compared and analysed. Relevant research challenges
and future research directions are also discussed. e contents of the remainder of the book
are as follows. Chapter 2 gives the reader a brief overview of deep learning theory. Although a
comprehensive description of the whole deep learning field is out of the scope of this book, the
chapter serves as a good introduction to deep learning for beginners in this field, and provides
a summary of key deep learning concepts discussed later in the book. Further reading for a
more comprehensive look into deep learning concepts are also recommended. Chapter 3 reviews
current work done in deep learning vehicle control, giving the reader a comprehensive view of the
state-of-the-art approaches in the field. e strengths and limitations of different approaches are
described through comparative analysis. Furthermore, research challenges and future research
directions are discussed. Chapter 4 discusses the safety validation of deep neural networks in
autonomous vehicles. e difficulty of validating the safety of these opaque systems is discussed
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.