Index
Note: Page numbers followed by f indicate figures, t indicate tables and n indicate notes.
A
auditorium seat probability,
29
in optical communication system,
39–40
in statistical independence,
30
auditorium seat probability,
29
in optical communication system,
40
in statistical independence,
30
Amplitude, of stochastic signals,
Aperiodic Markov chain,
396
Arcsine distribution,
124
MATLAB rand command and,
15
Auditorium seat probability, Bayes’s theorem and,
29
autocovariance function and,
347
average normalized power,
356
of continuous time processes,
344–345
of discrete-time processes,
344–347
sinusoidal random processes,
352
windowing function for,
443
Fourier transform pair of PSD and,
433
of Gaussian random processes,
358–359
of Poisson processes,
363
for sinusoidal random processes,
346
Autocovariance function,
347
of Poisson processes,
363
Autoregressive moving average process (ARMA), for PSD estimation,
449
Autoregressive process (AR)
autocorrelation function of,
450
Average normalized power,
356
Average power, in random process,
431
B
Backward Kolmogorov equations,
403
Bandpass filters (BPFs), noise equivalent bandwidth of,
480
Bandpass random process,
494
Bandwidth
of random telegraph process,
441
auditorium seat probability,
29
for estimation and detection problems,
266–267
in optical communication system,
39–40
Bernoulli,
Bernoulli random variable,
35,574
Bernoulli trial
binomial random variable and,
36
Rician random variables and,
83
Best linear unbiased estimator (BLUE),
291
for mean IID random variables,
291–292
Beta random variable,
566
Bilinear transformation,
530
Binary data, in optical communication system,
38–41,38f
Binary symmetric channel (BSC),
224,224f
binomial random variable and,
36
in failure probability,
314
Poisson random variable for,
37
probability-generating functions of,
137–138
Binomial random variables, characteristic function of,
132
Bridge, partitioning in,
25–26
C
Cartesian coordinates
transformation into polar coordinates,
216–217
Cartesian space, for Bernoulli trial,
35
Cauchy random variables,
84,566
Central moments
of random variables, exercises for,
162–163
of single random variables,
117–121
Channel coding theorem,
225
continuous time version of,
402
Characteristic functions
of chi-squared random variables,
259
of Laplace random variables,
260
in quadratic transformations of Gaussian random vectors,
258–260
of single random variables,
130–136
transformations of pairs of random variables using,
211–212,214
Chebyshev’s inequality,
142
in law of large numbers,
304
characteristic function of,
259
Circular Gaussian density function,
220
Classical approach
of assigning probabilities,
14
relative frequency approach to,
16
unsatisfactory results with,
15–16
for joint probability,
18
for Gaussian random variables,
135
for Gaussian random variables,
135
Coin flipping
probability assignment for,
14
sample space and outcomes of,
8–9
English
v. mathematical usage of,
24
principle of counting,
21
Communicating states,
395
Erlang random variable in,
82
Rayleigh random variables in,
82
Rician distribution in,
84
statistical independence and,
32
Complementary error function integral, for CDF of Gaussian random variable,
74
Complex envelopes
Complex numbers, Rayleigh distribution and,
82
Computer communication network, Markov processes describing,
413–415,415f
Computer generation
nonbinary pseudorandom number generators,
521–522
from specified distribution,
521–522
Conditional cumulative distribution function,
86–87
for pairs of random variables,
188–191
Conditional expected values
of functions of single random variables,
122
of pairs of random variables,
196–197
of single random variables,
121–122
Conditional probability,
18–20
deck of playing cards and,
19–20
joint probability compared with,
19
in optical communication system,
40–41
Conditional probability density function,
87–89,88f
for multiple random variables,
245–247
for pairs of random variables,
188–191
transformations of pairs of random variables using,
213–214
Conditional probability mass functions
for multiple random variables,
245–247
for pairs of random variables,
188–191
constants used to calculate,
311t
for failure probability,
314
for IID random variables,
312
Student’s t-distribution,
313
Continuous random variables,
565–574
Continuous sample space,
10
Continuous time and discrete amplitude signals,
335
Continuous time linear systems
Continuous time processes
autocorrelation function of,
344–345
Convergence almost everywhere
strong law of large numbers and,
305
Convergence everywhere
Convergence in distribution
of central limit theorem,
308
Convergence in mean square sense
Convergence in probability
weak law of large numbers and,
305
Convergence modes
convergence in mean square sense,
302,303t
Convolution, transformations of pairs of random variables using,
211
Coordinate systems
transformation of pairs of random variables for changes of,
214–217
Corollary of probability,
11–12
Correlation
of complex random variables,
221
for Gaussian random variables in multiple dimensions,
251
of pairs of random variables,
193–195
Correlation coefficient
for jointly Gaussian random variables,
206
of pairs of random variables,
194–195
Correlation matrix
of multiple random variables,
247–249
Covariance of
complex random variables,
221
of pairs of random variables,
193–195
Covariance function, of random walk,
360
Covariance matrix, of multiple random variables,
247–249
Cross spectral density
Cross-correlation function,
347–348
Cross-covariance function,
348
of Cauchy random variables,
84
of chi-squared (χ
2) random variables,
81–82
of discrete random variables,
68
of Erlang random variable,
81–82
estimating of IID random variables,
297–298
of exponential random variables,
79–80,79
of gamma random variables,
81
in Q-function terms,
75–76
joint
of Laplace random variables,
80,80f
of Rayleigh random variables,
82,83f
of standard normal random variables,
74
transformations of pairs of random variables using,
210,212–214
Customer arrivals, exponential random variables in,
80
D
Deck of playing cards
conditional probability and,
19–20
joint probability and,
18–20
Detection probability
sequential detection and,
320
Deterministic signal,
continuous, Fourier transform for,
429
noise compared with,
Dice rolling
probability assignment of,
15
sample space and outcomes of,
statistical independence and,
30–31
Die rolling
probability assignment of,
14
sample space and outcomes of,
Discrete Fourier transform, for discrete-time processes,
429
characteristic function of,
131–132
conditional expected values of,
122
expected value of functions of,
113
joint probability-generating function for,
208–209
probability-generating functions of,
136–138
staircase transformations of continuous random variables into,
128,128f
Discrete time and continuous amplitude signals,
335
Discrete time and discrete amplitude signals,
335
Discrete-time Fourier transform (DTFT),
560–561
for discrete-time linear systems,
477–478
Discrete-time linear systems
Discrete-time processes
autocorrelation function of,
344–347
autocovariance function of,
347
discrete Fourier transform for,
429
Distribution
Distribution of infinitesimal increments, of Poisson processes,
361
E
Energy, of random process,
430
Ensemble averaged power,
430
Ensemble of functions,
336
Entropy
Ergodic in autocorrelation, sinusoidal random processes,
351
Ergodicity
autocorrelation function
of sinusoidal random processes,
352
strict sense stationary and,
351–352
two limited forms of,
351
moment-generating functions of,
140
Error function integral, for CDF of Gaussian random variable,
74
Errors, in optical communication system,
40–41,42f
definition of,
probability as function of,
11
of random number generator,
10
Expected values.See also Mean conditional
of functions of single random variables,
122
of pairs of random variables,
196–197
of single random variables,
121–122
of multiple random variables,
247–249
for pairs of random variables,
192–197
conditional expected values,
196–197
of single random variables,
111–113
definition of,
gamma random variable and,
81
memoryless property of,
93
F
Factorial function, gamma function and,
81
Fading
Rayleigh random variables in,
82
Rician distribution in,
84
Failure rate function,
92
exponential distribution of,
93
of Rayleigh distribution,
93
reliability function and,
92
of system
parallel interconnection,
94–95
series interconnection,
93–95
False alarm probability
sequential detection and,
320
First central moment,
118
Fokker–Planck equation,
411
Forward Kolmogorov equations,
403
of cross-correlation function,
438–439
for deterministic continuous signal,
429
autocorrelation function and PSD as,
433
Fourth central moment,
118
Frequency domain techniques,
for random processes,
429
moment-generating
of random sum of Gaussian random variables,
317
of random sum of IID random variables,
316
of random variables
conditional expected value of,
122
transformations with
Fundamental frequency,
559
G
Galileo,
Gambling, probability and,
7–8
Gamma distribution, random telegraph signal,
341
Gaussian distribution
autocorrelation function of,
346–347
Box-Muller transformation for generation of,
218
in Q-function terms,
75–76
characteristic function of,
132,135
continuous time and discrete amplitude signals as,
341
pairs of, conditional PMF, PDF, and CDF for,
190–191
shorthand notation for,
73
Gaussian random vectors, quadratic transformations of,
257–260
Gaussian white noise
in discrete-time linear systems,
478–479
Gaussian-Multivariate random variables,
570
Genetic models, Markov chains in,
388
probability-generating functions of,
138
Global balance equations,
403
H
Heavy-tailed distribution, Cauchy PDF as,
84
Histogram, for MATLAB random number generation,
71,72f
Homogenous Markov process,
402
I
I components
Impulse invariance transformation,
530
communication network and,
32
for Gaussian random variables in multiple dimensions,
251
mutual exclusiveness compared with,
31
of set of
N random variables,
247
Independent and identically distributed (IID) random variables,
289–298
confidence intervals for,
312
likelihood ratio test for,
319
Independent increments, of Poisson processes,
361
Information
Integrals
Interference, accounting for,
Interpolation problems,
264
CDF and random variable in,
64,69
of exponential random variables,
79,79
Inverse transform, for deterministic continuous signal,
429
Irreducible Markov chain,
395
J
Joint CDFs
for multiple random variables,
245–247
Joint characteristic functions, for pairs of random variables,
206–209
Joint factorial moments,
208
Joint moment-generating functions,
209
Joint PDFs
for Gaussian random variables in multiple dimensions,
250–252
for multiple random variables,
245–247
Joint PMFs
for multiple random variables,
245–247
for pairs of random variables,
186–188
conditional probability compared with,
19
deck of playing cards and,
18–20
Joint probability-generating functions,
208–209
joint characteristic functions for,
207–208
p-n Junction diode
K
of padlock combinations,
23
L
Lagrange multiplier techniques, for IID random variables,
291
Laplace,
characteristic function of,
260
License plates, principle of counting and,
22
Likelihood ratio test
for IID Gaussian random variables,
319–321
Linear minimum mean square error (LMMSE) estimator,
270–272
Linear prediction
Linear systems, random processes in,
473–504
Linear time-invariant (LTI) system autocorrelation function of,
473–475
cross-correlation function of,
475–476
passage of signals through,
561–563
white Gaussian noise in,
476
Linear transformations
of Gaussian random variables,
253–257
of multiple random variables,
253–257
of pairs of random variables,
218–219
Log-likelihood ratio test, for radar system,
321–322
Log-normal random variables,
570–571
Lottery game, combinations in,
24
Lowpass filter (LPF)
Gaussian white noise in,
476
Lowpass random process,
494
M
(
m, n)th joint moment of two random variables,
195–196
transition and state probability calculations in,
388–393
continuous time and continuous amplitude,
409–411
definitions and examples of,
383–388
transition and state probability calculations in,
388–393
Matched filter
Mathematical tools random telegraph process,
340–341
autocovariance function,
347
cross-correlation function,
347–348
cross-covariance function,
348
MATLAB,
for bilinear approximation of random processes,
530–532,531f
binomial coefficient exercise for,
62
central moment exercise for,
120–121
dice rolling exercise for,
62
for ergodicity and autocorrelation function estimation,
352–354,353f
for estimating CDF of IID random variables,
297–298
for Gaussian random variable,
109
Markov chain transition and state probability calculation exercise for,
393–394,394f
Markov process exercises for,
427–428
mean function of sinusoidal random processes,
344,345f
multiple random variables exercises for,
287–288
multiple random variables transformation exercise for,
256–257
pairs of random variables exercises for,
242–243
for periodogram estimate of PSD in random telegraph process,
447–448,448f
histogram construction for,
71,72f
of continuous time and discrete amplitude signals,
338–339,339f
of discrete time and continuous amplitude signals,
339–340,340f
relative frequency approach simulation,
16–17,16t
for Rician random variable,
110
simulation technique exercises,
546
sine of uniform random variables,
85,85f
single random variable operation exercises for,
174–175
single random variable transformation exercise for,
129,130f
for uniform random variables,
109
for variance of IID random variable estimation,
296–297
Maximal length linear feedback shift register (MLLFSR),
520–521,521f
Maximum a posteriori (MAP),
39
in estimation and detection problems,
265–267
Maximum likelihood (ML)
for mean of IID random variable estimation,
292–294
for variance of IID random variable estimation,
295–296
Mean
of Bernoulli random variable,
297
of complex random variables,
220
conditional
of functions of single random variables,
122
of single random variables,
121–122
estimating of IID random variables,
290–295
of failure probability,
314
in law of large numbers,
304
of multiple random variables,
247–249
for pairs of random variables,
192–197
probability-generating functions and,
137–138
of random sum of Gaussian random variables,
317
of random sum of IID random variables,
315–316
of random variables, estimation of,
304
of Rician random variables,
296–297
of single random variables,
111–113
characteristic functions and,
133
tail probability evaluations using,
141–142
of WSS random processes,
351
Mean function
autocovariance function and,
347
of Gaussian random processes,
358–359
of random telegraph process,
343
of sinusoidal random processes,
343–344
Mean square error
for IID random variables,
291
Mean square (MS) sense, random sequence convergence in,
302,303f
Mean vector, for IID random variables,
291
Median, order statistics and,
261–262
Memoryless property, of exponential random variables,
93
Minimum mean square error (MMSE), for estimation problems,
268–272
Mixed random variables,
67
Modulation efficiency,
503
Moment-generating functions
of single random variables,
139–140
of single random variables,
115–117
characteristic functions and,
133–135
probability-generating functions and,
136–138
Monotonically decreasing function transformations,
123f,124–125
Monotonically increasing function transformations,
122–124,123f
Monty Hall problem,
61–62
Moving average process (MA), for PSD estimation,
449
Multiple random variables,
245
conditional PMF and PDF for,
245–247
in estimation and detection problems,
264
joint PMF, CDF, and PDF for,
245–247
independence compared with,
31
of PMF of binomial random variable,
36
Mutual information exercises for,
239–240
N
Nakagami random variable,
571
accounting for,
autocovariance function and,
347
deterministic signal or function compared with,
Poisson random variable for,
37
Nonbinary pseudorandom number generators,
521–522
Noncoherent systems
Rayleigh random variables in,
82
Rician distribution in,
84
Non-stationary
Wiener–Khintchine–Einstein theorem for,
435
Normal random variables,
73
nth moment, of single random variables,
115
nth order PDF, of random processes,
342
Numerical routines, for CDF of Gaussian random variable,
74
Nyquist’s theorem, thermal noise and,
453
O
Bayes’s theorem for,
39–40
conditional probability for,
40–41
probability of error for,
41,42f
Orthogonal variables,
193
of coin flipping,
definition of,
of dice rolling,
of die rolling,
of random number generator,
10
total number of possible,
21–22
P
Padlock combinations,
k-permutations of,
23
conditional expected values,
196–197
joint characteristic functions for,
206–209
joint moment-generating functions for,
209
joint probability-generating functions for,
208–209
joint characteristic functions for,
207–208
Parallel interconnection, system, reliability function and failure rate function of,
94–95
Parseval’s theorem, in PSD,
430
Pascal,
Periodogram
Phase, of stochastic signals,
Φ-function
for CDF of Gaussian random variable,
74
Q-function relation with,
75,75f
Photodetector, in optical communication system,
38–39
Photoemissive surface, in optical communication system,
39
Points,
of Poisson counting processes,
364
autocorrelation function of,
363
Poisson distribution, random telegraph signal,
341
Poisson processes
autocovariance function of,
363
Poisson random variable,
37,575
in optical communication system,
39
Polar coordinates, Cartesian coordinate transformation into,
216–217
Population modeling, with birth– death processes,
407–409
Power, in random process,
431
Power spectral density (PSD),
430
for complex envelope,
500
Fourier transform for,
457
Fourier transform in,
430
Fourier transform pair of autocorrelation function and,
433
Parseval’s theorem in,
430
of sinusoidal random processes,
432
Power transfer function,
475
Principle of counting,
21
classical approach to,
14
MATLAB rand command and,
15
relative frequency approach for,
16
relative frequency interpretation of,
297
Probability densities, of randn function,
76–78,77f
Probability density function (PDF),
69–71,69f
of Cauchy random variables,
84
of chi-squared (
x2) random variables,
81–82
transformations of pairs of random variables using,
213–214
of continuous random variables, for CDF,
289
of continuous time and discrete amplitude signals,
341
of Erlang random variable,
81–82
of exponential random variables,
79–80,79
of gamma random variables,
81
of Gaussian random processes,
357–359
of IID random variables,
289
joint
for Gaussian random variables in multiple dimensions,
250–252
of Laplace random variables,
80,80f
of random variables,
69,76
characteristic functions and,
130–131
of Rayleigh random variables,
82,83f
Probability mass function (PMF),
33
binomial random variable,
35–37
conditional
of continuous random variables,
63
of discrete random variables, for CDF,
289
geometric random variable,
37–38
joint
for pairs of random variables,
186–188
in optical communication system,
39
Poisson random variable,
37
of random telegraph process,
342f
Probability of error, for optical communication system,
40–41,42f
Probability ratio test, for radar system,
318
Probability theory,
conditional probability,
18–20
experiments, samples spaces, and events,
7–10
statistical independence,
29–32
Probability-generating functions
of single random variables,
136–138
Pulse amplitude modulation (PAM),
455–456
autocorrelation function of,
456–457
Fourier transform for,
457
Q
Q components complex envelopes for,
499–500
for CDF of Gaussian random variable,
74
Φ-function relation with,
75,75f
probabilities in terms of,
76
Quadratic transformations, of multiple random variables,
257–260
Quantization
exponential random variables in,
80
gamma random variables in,
81
R
false alarm and detection probabilities for,
318–319
probability ratio test for,
318
sequential detection,
320
signal-to-noise ratio for,
323
system performance for,
323
Radio frequency, phase of,
79
rand, MATLAB, probability assignment and,
15
histogram construction for,
71,72f
Random number generator
histogram creation for,
71,72f
sample space, events, and outcomes of,
10
average normalized power,
356
of continuous time and discrete amplitude signals,
338–339,339f
cross spectral density between,
438–439
of discrete time and continuous amplitude signals,
339–340,340f
ergodic in autocorrelation,
351
frequency domain techniques for,
429
autocorrelation function of,
346–347
autocovariance function,
347
cross-correlation function,
347–348
cross-covariance function,
348
PSD for
autocorrelation function for,
346
strict sense stationary,
349
spectral characteristics of,
429–430
convergence in mean square sense,
302,303t
false alarm and detection probabilities for,
318–319
probability ratio test for,
318
sequential detection,
320
system performance for,
323
Random sums
of Gaussian random variables,
317
Random telegraph process
Random variables.See also specific types
characteristic functions of,
130–136
nonbinary pseudorandom number generators,
521–522
from specified distribution,
521–522
conditional expected values of,
121–122
conditional expected value of,
122
in interval, CDF and,
64,69
mean of, estimation of,
304
moment-generating functions of,
139–140
characteristic functions and,
133–135
probability-generating functions and,
136–138
characteristic functions and,
130–131
probability-generating functions of,
136–138
autocorrelation function for,
346
strict sense stationary,
349
Rare events, simulation of,
534–538
Rayleigh distribution,
82
reliability function and failure rate function of,
93
Rician random variables and,
83
Realization, of random process,
336
Relative frequency approach
to assigning probabilities,
16
for joint probability,
18
Relative frequency interpretation of probability,
297
exponential distribution of,
93
failure rate function and,
92
of Rayleigh distribution,
93
of system
parallel interconnection,
94–95
series interconnection,
93–95
Root-mean-square (RMS)
S
confidence intervals and,
312
of IID random variables,
299
in law of large numbers,
304
of Rician random variables,
296–297
definition of,
of dice rolling,
of die rolling,
of random number generator,
10
of Rician random variables,
296–297
Satellite communication channels, Rician distribution in,
84
Second-order PDF, of random processes,
342
Sensor, voltage of, noise and,
289–290
Sequential detection
Series interconnection, system, reliability function and failure rate function of,
93–95
autocorrelation function of,
366–367
Poisson counting processes,
365
Poisson impulse processes,
365
as strict sense stationary,
368
Signal-to-noise ratio, of radar system,
323
Signal-to-noise ratios (SNR)
computer generation of random processes,
525–534
computer generation of random variables,
517–524
nonbinary pseudorandom number generators,
521–522
from specified distribution,
521–522
MATLAB exercises for,
546
autocorrelation function for,
346
strict sense stationary,
349
Wiener–Khintchine–Einstein theorem for,
435–436
Smoothing function, for periodogram,
446
Spectral estimation exercises for,
466–468
Speech recognition system,
2–4,3f
Speech signal amplitude, Laplace random variables in,
80
Spherical coordinates, Cartesian coordinate transformation into,
262–263,263f
Staircase function transformation,
128,128f
Standard deviation
estimating of IID random variables,
295–297
second central moment of random variables and,
118
Standard normal random variables,
73
State transition probability matrix, calculation of,
388–393
Stationary
strict sense, random processes,
348–349
wide sense, random processes,
349–351
Stationary increments, of Poisson processes,
361
for continuous time Markov processes,
403–405
Stochastic signals,
Strict sense stationary ergodicity and,
351–352
shot noise processes as,
368
sinusoidal random processes,
349
Strong law of large numbers,
304–305
convergence almost everywhere and,
305
T
Telephone exchange, Markov processes describing,
416–417,417f
Telephone traffic, Erlang random variable in,
82
unconditional cumulative distribution function and,
89–90
Third central moment,
118
Time, continuous function of,
335
Time difference variable, autocorrelation function and,
345
Time domain techniques,
Time-averaged power, of random process,
430
Transformations
of multiple random variables,
252–253
of pairs of random variables,
210–219
of single random variables,
122
on uniform random variables,
79
Transition probability matrix,
384–385
for continuous time Markov processes,
402
Trigonometric identities,
577
U
Unconditional cumulative distribution function,
89–90
Uncorrelated random variables,
193
histogram of MATLAB generation of,
71,72f
V
Variability, accounting for,
of complex random variables,
220–221
estimating of IID random variables,
295–297
of failure probability,
314
of IID random variables,
292
in law of large numbers,
304
probability-generating functions and,
137–138
of random sum of Gaussian random variables,
317
of random sum of IID random variables,
315–316
of Rician random variables,
296–297
tail probability evaluations using,
142
Voice conversation duration, exponential random variables in,
80
W
Wald’s inequalities, for radar system,
321–323
convergence in probability and,
305
Wide sense stationary (WSS)
shot noise processes as,
367
impulse response with,
489
to minimize mean square error,
487–489
for sinusoidal random processes,
435–436
Windowing function, for autocorrelation function estimation,
443
Wireless communication channels, Rayleigh random variables in,
82
Wireline telecommunication networks, Erlang random variable in,
82
Y
Yule-Walker equations,
488
Z
Zero-mean Gaussian distribution,
82
Zeroth central moment,
118
for discrete-time linear systems,
477–478
for discrete-time processes,
429