Index

Note: Page numbers followed by f indicate figures, t indicate tables and n indicate notes.

A

a posteriori probability, 28,28n2
auditorium seat probability, 29
in optical communication system, 39–40
in statistical independence, 30
a priori probability, 28,28n1
auditorium seat probability, 29
in optical communication system, 40
for radar system, 318
in statistical independence, 30
Absorbing state, 385–386,386f
Absorption probability, 407–409
Accessible state, 395
Amplitude, of stochastic signals, 7
Amplitude modulation (AM), 500–503
MATLAB for, 503–504,504f
Analog communication system, 500–504,500f,501f,504f
Aperiodic Markov chain, 396
Arcsine distribution, 124
Arcsine random variable, 565–566
Arrival, 360
Arrival state, 403,416
Atomic outcomes, 13–14
of coin flipping, 14
of dice rolling, 15
of die rolling, 14
exercises for, 46–47
MATLAB rand command and, 15
probability, 14
Auditorium seat probability, Bayes’s theorem and, 29
Autocorrelation function, 344–345
of AR process, 450
autocovariance function and, 347
average normalized power, 356
for complex envelope, 499–500
of continuous time processes, 344–345
of discrete-time processes, 344–347
ergodic in, 351
estimation of, 352–354,353f
influence of, 354–355
sinusoidal random processes, 352
random telegraph process, 442–444,442f,444f
windowing function for, 443
exercises for, 375
Fourier transform pair of PSD and, 433
of Gaussian random processes, 358–359
of LTI system, 473–475
of PAM, 456–457
of Poisson processes, 363
properties of, 356–357
for PSD spectral estimation, correlation method, 441–445,442f,444f,445f
of shot noise processes, 366–367
of signal-to-noise ratios, 480–481
for sinusoidal random processes, 346
for thermal noise, 454
wide sense stationary and, 349–351,356–357
Autocovariance function, 347
of Poisson processes, 363
of shot noise processes, 366–367
Autoregressive moving average process (ARMA), for PSD estimation, 449
Autoregressive process (AR)
autocorrelation function of, 450
for PSD estimation, 449–450
random telegraph process, 451–452,452f
Available power, 453
Average., See Expected values
Average information, 156
Average normalized power, 356
Average power, in random process, 431
Axioms of probability, 10–13,13f
exercises for, 44–45

B

Backward Kolmogorov equations, 403
Bandlimited random processes, 494–498,495f,496f,497f,498f
Bandpass filters (BPFs), noise equivalent bandwidth of, 480
Bandpass process, 439
bandwidth of, 439,439f,440f
Bandpass random process, 494
Bandwidth
of random processes, 439–441,439f,440f
exercises for, 465–466
of random telegraph process, 441
Bayes’s theorem, 27–29,28f
a posteriori probability, 28,28n2
a priori probability, 28,28n1
auditorium seat probability, 29
conditional, 90
for estimation and detection problems, 266–267
exercises for, 52–54
in optical communication system, 39–40
theorem of total probability, 27–28,28f
Bernoulli, 8
Bernoulli random variable, 35,574
CDF of, 310,310f
mean of, 297
PMF of, 310
random telegraph signal, 340–341
Bernoulli trial
binomial random variable and, 36
Cartesian space for, 35
exercises for, 56
Bernoulli trials, 35
Bessel function, 565
Rician random variables and, 83
Best linear unbiased estimator (BLUE), 291
for mean IID random variables, 291–292
Beta function, 565
Beta random variable, 566
Bilinear transformation, 530
Binary data, in optical communication system, 38–41,38f
Binary entropy function, 156,157f
Binary pseudorandom number generators, 517–521,500f,519f,520t,521f
Binary symmetric channel (BSC), 224,224f
Binomial coefficient, 23–24., See also Combinations
binomial random variable and, 36
exercises for, 48–49
MATLAB exercise for, 62
Binomial random variable, 35–37,574
central moments of, 118
in failure probability, 314
moments of, 116
PMF of, 36
Poisson random variable for, 37
probability-generating functions of, 137–138
Binomial random variables, characteristic function of, 132
Birth processes, 360–361
Birth–death processes, 401–411
MATLAB exercise for, 406–407,406f,412–413,412f
M/M/∞ queue, 405
M/M/1 queue, 405–407,406f
population modeling with, 407–409
telephone exchange described by, 416–417,417f
Bounds, on tail probabilities, 140–147,143f,147f
Box-Muller transformation, 218,524
BPFs., See Bandpass filters
Branching process, 388
Bridge, partitioning in, 25–26

C

Cartesian coordinates
transformation into polar coordinates, 216–217
transformation into spherical coordinates, 262–263,263f
Cartesian space, for Bernoulli trial, 35
Cauchy distribution, 213
Cauchy random variables, 84,566
Central limit theorem, 306–310
exercises for, 329
Gaussian distribution and, 308–310
proof of, 307–308
Central moments
of random variables, exercises for, 162–163
of single random variables, 117–121
MATLAB exercise for, 120–121
Channel capacity, 221–225,223f,224f,225f
exercises for, 239–240
Channel coding, 221–225,223f,224f,225f
exercises for, 239–240
Channel coding theorem, 225
Chapman–Kolmogorov equation, 389,409–410
continuous time version of, 402
Characteristic functions
of chi-squared random variables, 259
joint, for pairs of random variables, 206–209,235–236
of Laplace random variables, 260
in quadratic transformations of Gaussian random vectors, 258–260
of single random variables, 130–136
exercises for, 167–168
moments and, 133–135
natural logarithm of, 135–136
PDFs and, 130–131
transformations of pairs of random variables using, 211–212,214
Chebyshev’s inequality, 142
in law of large numbers, 304
Chernoff bound, 142–144,143f
Chi-squared (χ2) random variables, 81–82,567
characteristic function of, 259
Circular Gaussian density function, 220
Classical approach
of assigning probabilities, 14
exercises for, 46–47
relative frequency approach to, 16
unsatisfactory results with, 15–16
for joint probability, 18
Coded digital communication system, simulation of, 539–541,539f,541f
Coefficient of kurtosis, 119–120
for Gaussian random variables, 135
Coefficient of skewness, 119–120
for Gaussian random variables, 135
Coin flipping
MATLAB simulation of, 15
PMF for, 33
probability assignment for, 14
random process of, 336–338,336f,338f
sample space and outcomes of, 8–9
Column vector, 551
Combinations, 23–24,26t
English v. mathematical usage of, 24
in lottery game, 24
Combinatorics, 20–26,26t
combinations, 23–24
exercises for, 48–52
k-permutations, 23
partitions, 24–26
permutations, 22
principle of counting, 21
Combined experiment, 21–22,26t
Communicating states, 395
Communication system, 4–5,5f., See also Digital modulation formats
Erlang random variable in, 82
matched filter for SNR for, 484–485,485f
Rayleigh random variables in, 82
Rician distribution in, 84
simulation of coded digital, 539–541,539f,541f
sinusoid in, 79
statistical independence and, 32
Complement, 545,546f
Complementary error function integral, for CDF of Gaussian random variable, 74
Complex envelopes
exercises for, 514
for I and Q components, 499–500
Complex numbers, Rayleigh distribution and, 82
Complex random variables, 219–221
exercises for, 238–239
Computer communication network, Markov processes describing, 413–415,415f
Computer generation
of random processes, 525–534
exercises for, 544
frequency domain approach to, 525–528,526f,528f
of Gaussian white noise, 533–534,534f
time domain approach, 529–532,529f,531f,533f
of random variables, 517–524
binary pseudorandom number generators, 517–521,518f,519f,520t,521f
correlated, 524
exercises for, 543–544
nonbinary pseudorandom number generators, 521–522
from specified distribution, 521–522
Conditional cumulative distribution function, 86–87
exercises for, 104–106
for pairs of random variables, 188–191
exercises for, 231
Conditional entropy, 221–222
Conditional expected values
exercises for, 163–164
of functions of single random variables, 122
of pairs of random variables, 196–197
of single random variables, 121–122
Conditional probability, 18–20
deck of playing cards and, 19–20
definition of, 18–19
exercises for, 47–48
independence and, 31
joint probability compared with, 19
in optical communication system, 40–41
Conditional probability density function, 87–89,88f
exercises for, 104–106
for multiple random variables, 245–247
exercises for, 277–278
for pairs of random variables, 188–191
exercises for, 231
properties of, 87–88
transformations of pairs of random variables using, 213–214
Conditional probability mass functions
for multiple random variables, 245–247
exercises for, 277–278
for pairs of random variables, 188–191
exercises for, 231
Confidence intervals, 310–315
constants used to calculate, 311t
exercises for, 330–331
for failure probability, 314
for IID random variables, 312
sample mean and, 312
Student’s t-distribution, 313
Confidence level, 311,311t
Continuous random variables, 565–574
PDF for, 289
conditional, 90
PDF of, 69
PMF and, 63
staircase transformations of, 128,128f
Continuous sample space, 10
events on, 10
mix with discrete, 10
Continuous time and discrete amplitude signals, 335
PDF of, 341
random process of, 338–339,339f
Continuous time linear systems
exercises for, 505–507
random processes in, 473–477
Continuous time Markov processes, 401–411,409–413
exercises for, 426–427
MATLAB exercise for, 406–407,406f,412–413,412f
M/M/∞ queue, 405
M/M/1 queue, 405–407,406f
population modeling with, 407–409
Continuous time processes
autocorrelation function of, 344–345
mean function of, 343
strict sense stationary, 348–349
Continuous time signals, 335,581
Convergence almost everywhere
exercises for, 326–327
of random sequences, 301,303t
strong law of large numbers and, 305
Convergence everywhere
exercises for, 326–327
of random sequences, 300–301,303t
Convergence in distribution
of central limit theorem, 308
exercises for, 326–327
of random sequences, 302–303,303t
Convergence in mean square sense
exercises for, 326–327
of random sequences, 302,303t
Convergence in probability
exercises for, 326–327
of random sequences, 301–302,303t
weak law of large numbers and, 305
Convergence modes
law of large numbers and, 304–305
of random sequences, 298–303,303t
convergence almost everywhere, 301,303t
convergence everywhere, 300–301,303t
convergence in distribution, 302–303,303t
convergence in mean square sense, 302,303t
convergence in probability, 301–302,303t
exercises for, 326–327
Convolution, transformations of pairs of random variables using, 211
Convolution function, MATLAB, 352–354,353f
Coordinate systems
three-dimensional transformations of, 262–263,263f
transformation of pairs of random variables for changes of, 214–217
Corollary of probability, 11–12
Correlation
of complex random variables, 221
independence and, 195,198–199
for Gaussian random variables in multiple dimensions, 251
for jointly Gaussian random variables, 205–206,206f
of pairs of random variables, 193–195
Correlation coefficient
independence and, 198
for jointly Gaussian random variables, 206
of pairs of random variables, 194–195
Correlation matrix
for IID random variables, 291–292
of multiple random variables, 247–249
Correlation method, for PSD spectral estimation, 441–445,442f,444f,445f
Counting processes, 362., See also Poisson counting processes
PMF of, 361–362
properties of, 360–361
Covariance of
complex random variables, 221
independence and, 198
of pairs of random variables, 193–195
Covariance function, of random walk, 360
Covariance matrix, of multiple random variables, 247–249
Cross spectral density
of I and Q components, 497–498
between random processes, 438–439
Cross-correlation function, 347–348
Fourier transform of, 438–439
of LTI system, 475–476
Cross-covariance function, 348
Cumulants, 135–136
Cumulative distribution function (CDF), 64–68,65f,66f
of Bernoulli random variable, 310,310f
of Cauchy random variables, 84
of chi-squared (χ2) random variables, 81–82
conditional, 86–87
exercises for, 104–106
for pairs of random variables, 188–191,231
of continuous random variables, 65–67,65f,66f
PDF for, 289
of discrete random variables, 68
PMF for, 289
of Erlang random variable, 81–82
estimating of IID random variables, 297–298
exercises for, 97–99
of exponential random variables, 79–80,79
of gamma random variables, 81
of Gaussian random variable, 69,73–76,75f
evaluation of, 75
in Q-function terms, 75–76
standard forms of, 74
transformation of, 74–75
joint
exercises for, 227–228
for multiple random variables, 245–247,277–278
for pairs of random variables, 178–180,179f
of Laplace random variables, 80,80f
PDF compared with, 69–71,69f
PMF and, 68
properties of, 64
of random processes, 340,342
of random variables, 65–66,65f,66f
conditional, 86–87
reliability function, 91
of Rayleigh random variables, 82,83f
of Rician random variables, 83–84,84f
of standard normal random variables, 74
transformations of pairs of random variables using, 210,212–214
unconditional, 89–90
of uniform random variables, 78–79,78f
Customer arrivals, exponential random variables in, 80

D

Dark current, 41
De Morgan’s laws, 549
Deck of playing cards
conditional probability and, 19–20
joint probability and, 18–20
Definite integrals, 580
Departure state, 403,416
Detection, 264
exercises for, 283–284
MAP detection, 265–267
Detection probability
for radar system, 318–323
sequential detection and, 320
thresholds for, 319–320
Determinant of matrix, 554–555
Deterministic signal, 7
continuous, Fourier transform for, 429
noise compared with, 7
Diagonal matrix, 552
Dice rolling
MATLAB exercise for, 62
probability assignment of, 15
relative frequency approach for, 16–17,16t
sample space and outcomes of, 9
statistical independence and, 30–31
Die rolling
probability assignment of, 14
sample space and outcomes of, 9
Difference set, 545–546,546f
Differential pulse code modulation (DPCM), 272–275,273f,274f,275f,276f
Diffusion modeling, 411–413,412f
Digital communication systems, mutual information of, 222–225,223f,224f,225f
Digital modulation formats, PSDs of, 455–461,458f,459f,460f,461f
Discrete Fourier transform, for discrete-time processes, 429
Discrete random variables, 32–38,34f,574–575
Bernoulli, 35
binomial, 35–37
CDF of, 68
PMF for, 289
central moments of, 117
characteristic function of, 131–132
for coin flipping, 33
conditional expected values of, 122
conditional PMF for, 188
exercises for, 55–59
expected value of, 112
expected value of functions of, 113
expected values for, 192
geometric, 37–38
joint PMFs for, 186–188
joint probability-generating function for, 208–209
moments of, 115–116
PMF of, 33
Poisson, 37
probability-generating functions of, 136–138
staircase transformations of continuous random variables into, 128,128f
Discrete time and continuous amplitude signals, 335
random process of, 339–340,340f
Discrete time and discrete amplitude signals, 335
Discrete-time Fourier transform (DTFT), 560–561
for discrete-time linear systems, 477–478
Discrete-time linear systems
exercises for, 477–479,508–509
Gaussian white noise in, 478–479
random processes in, 477–479
Discrete-time processes
autocorrelation function of, 344–347
autocovariance function of, 347
discrete Fourier transform for, 429
mean function of, 343
Discrete-valued Markov processes., See Markov chains
Disjoint, 546
Distortion, signal quantization causing, 148–153,149f,152f
Distribution
arcsine, 124
Cauchy, 213
Laplace, quantization of, 151–154,152f
random sequence convergence in, 302–303,303t
Distribution of infinitesimal increments, of Poisson processes, 361
Domain, of functions, 32
Drift, of Markov chain, 414–415,415f

E

Efficient estimator, 290
Eigenvalue, 555–557
Eigenvector, 555–557
Empty set, 545,546f
Energy, of random process, 430
Ensemble averaged power, 430
Ensemble of functions, 336
Entropy
conditional, 221–222
of single random variables, 155–158,157f,158t
exercises for, 173
Envelope detector, for AM, 501–502
Ergodic in autocorrelation, sinusoidal random processes, 351
Ergodic in mean, 351
Ergodicity
autocorrelation function
and estimation of, 352–354,353f
influence of, 354–355
of random processes, 351–356
of sinusoidal random processes, 352
strict sense stationary and, 351–352
two limited forms of, 351
of WSS, 351–356
Erlang random variable, 81–82,567–568
moment-generating functions of, 140
Erlang-B formula, 416,417f
Error function integral, for CDF of Gaussian random variable, 74
Errors, in optical communication system, 40–41,42f
Estimation, 264
exercises for, 283–284
MMSE, 268–272
Events, 7–10
of CDF, 64
on continuous space, 10
definition of, 8
exercises for, 43–44
intersection of, 17
probability as function of, 11
of random number generator, 10
Exam permutations, 22
Expected values.See also Mean conditional
ent
exercises for, 163–164
of functions of single random variables, 122
of pairs of random variables, 196–197
of single random variables, 121–122
of functions of random variables, 113–114,115t
exercises for, 160–161
of multiple random variables, 247–249
exercises for, 279–280
for pairs of random variables, 192–197
conditional expected values, 196–197
correlation, 193–195
covariance, 193–195
exercises for, 231–233
(m, n)th joint moment, 195–196
of single random variables, 111–113
exercises for, 159–160
Experiments, 7–10
combined, 21–22,26t
definition of, 8
exercises for, 43–44
function domains of, 32
sample spaces for, 10
Exponential random variables, 79–80,79f,568
central moments of, 119
characteristic function of, 131,133–134
expected value of, 112
failure rate and, 93
gamma random variable and, 81
memoryless property of, 93
transformation of, 124,129
Extinction probability, 407–409

F

F random variable, 568–569
Factorial function, gamma function and, 81
Factorial moments, 137–138
joint, 208
Fading
Rayleigh random variables in, 82
Rician distribution in, 84
Failure probability, MATLAB example for, 314–315,314f
Failure rates, 91–95
changes in, 92
Failure rate function, 92
exponential distribution of, 93
of Rayleigh distribution, 93
reliability function and, 92
of system
parallel interconnection, 94–95
series interconnection, 93–95
False alarm probability
for radar system, 318–323
sequential detection and, 320
thresholds for, 319–320
Filtering problems, 264
First central moment, 118
First moment, 115–117., See also Expected values; Mean
First return probability, 396–397
Fokker–Planck equation, 411
Forward Kolmogorov equations, 403
Fourier Series, 559–560
Fourier Transform., See Discrete Fourier transform
of cross-correlation function, 438–439
for deterministic continuous signal, 429
for PAM, 457
in PSD, 430
review of, 560–561
Fourier Transform pairs, 560,581–582t
autocorrelation function and PSD as, 433
Fourth central moment, 118
Frequency domain techniques, 7
for computer generation of random processes, 525–528,526f,528f
MATLAB example for, 527–528,528f
for random processes, 429
Frequency shift keying modem, 337–338,338f
ensemble of, 336
moment-generating
of single random variables, 139–140,169–170
tail probability evaluations using, 142–147,143f,147f
of outcomes, 32
probability-generating, of single random variables, 136–138,168–169
of random sum of Gaussian random variables, 317
of random sum of IID random variables, 316
of random variables
conditional expected value of, 122
exercises for, 160–161
expected value of, 113–114,115t
transformations with
monotonically decreasing functions, 123f,124–125
monotonically increasing functions, 122–124,123f
nonmonotonic functions, 125–129,125f,128f,130f
staircase functions, 128,128f
Fundamental frequency, 559

G

Galileo, 7
Gambler’s ruin problem, 386,386f,392
Gambling, probability and, 7–8
Gamma distribution, random telegraph signal, 341
Gamma function, 81,565
exercise for, 104
Gamma random variables, 81,127,569
Gaussian distribution
central limit theorem and, 308–310
zero-mean, 82
Gaussian random processes, 342,357–360,359f
autocorrelation function of, 346–347
definition of, 357
exercises for, 375–376
PDF of, 357–359
random walk, 359–360,359f
simulation of, 525–527,526f
MATLAB example for, 527–528,528f
Gaussian random variables, 71–78,73f,75f,569., See also Jointly Gaussian random variables
Box-Muller transformation for generation of, 218
evaluation of, 75
in Q-function terms, 75–76
standard forms of, 74
transformation of, 74–75
characteristic function of, 132,135
continuous time and discrete amplitude signals as, 341
exercises for, 101–103
generation of, 523–524
correlated, 524
likelihood ratio test for, 319–321
linear transformations of, 253–257
MATLAB exercise for, 109
in multiple dimensions, 249–251
exercises for, 280–281
MATLAB exercise for, 251–252
pairs of, conditional PMF, PDF, and CDF for, 190–191
PDF of, 72–73,73f
for radar system, 317–318
random sum of, 317
shorthand notation for, 73
transformation of, 124,127
transformations of pairs of, 212–213,216–219
Gaussian random vectors, quadratic transformations of, 257–260
Gaussian white noise
computer generation of, 533–534,534f
in discrete-time linear systems, 478–479
in LTI system, 476
Gaussian-Multivariate random variables, 570
Genetic models, Markov chains in, 388
Geometric random variable, 37–38,129,574–575
PMF of, 37
probability-generating functions of, 138
Global balance equations, 403

H

Half-power bandwidth, 440,440f
Harmonic frequency, 559
Heavy-tailed distribution, Cauchy PDF as, 84
Hermitian symmetry, 439,551, 557
Histogram, for MATLAB random number generation, 71,72f
Homogenous Markov process, 402

I

I components
complex envelopes for, 499–500
cross spectral density of, 497–498
extraction of, 496,496f
Identity matrix, 552
Importance sampling, 537–538
for simulation of coded digital communication system, 539–541,539f,541f
Impulse invariance transformation, 530
Indefinite integrals, 578–580
Independence, 29–32
communication network and, 32
correlation and, 195,198–199
for Gaussian random variables in multiple dimensions, 251
for jointly Gaussian random variables, 205–206,206f
dice rolling and, 30–31
exercises for, 54–55
mutual exclusiveness compared with, 31
of random variables, 197–199
exercises for, 233–234
MATLAB exercise for, 200–201,201f
of set of N random variables, 247
test for, 30
Independent and identically distributed (IID) random variables, 289–298
in central limit theorem, 306–308
confidence intervals for, 312
estimating CDF of, 297–298
estimating mean of, 290–295
BLUE for, 291–292
confidence intervals for, 310–315,311t
ML for, 292–294
estimating variance of, 295–297
exercises for, 325–326
likelihood ratio test for, 319
PDF of, 289
random sum of, 315–317
function of, 316
mean and variance of, 315–316
sample mean of, 299
Independent increments, of Poisson processes, 361
Information
exercises for, 239–240
quantitative definition of, 155–158,157f,158t
Integrals
definite, 580
indefinite, 578–580
Interference, accounting for, 7
Interpolation problems, 264
Intersection, 545,546f
Interval., See also Confidence intervals
CDF and random variable in, 64,69
of exponential random variables, 79,79
of uniform random variables, 78–79,78f
Inverse of matrix, 553–554
Inverse transform, for deterministic continuous signal, 429
Irreducible Markov chain, 395

J

Joint CDFs
for multiple random variables, 245–247
exercises for, 277–278
for pairs of random variables, 178–180,179f
exercises for, 227–228
Joint characteristic functions, for pairs of random variables, 206–209
exercises for, 235–236
Joint factorial moments, 208
Joint moments, 192–197,208–209
Joint moment-generating functions, 209
Joint PDFs
exercises for, 228–229
for Gaussian random variables in multiple dimensions, 250–252
for multiple random variables, 245–247
exercises for, 277–278
for pairs of random variables, 180–185,184f
MATLAB exercise for, 185–186
Joint PMFs
for multiple random variables, 245–247
exercises for, 277–278
for pairs of random variables, 186–188
exercises for, 229–230
Joint probability, 17–20
computation of, 17–18
conditional probability compared with, 19
deck of playing cards and, 18–20
exercises for, 47–48
independence and, 31
of three events, 19
Joint probability-generating functions, 208–209
Jointly Gaussian random variables, 202–206,203–204f,205f,206f
exercises for, 234
joint characteristic functions for, 207–208
linear transformation of, 218–219,254
MAP estimator for, 265
p-n Junction diode
operation of, 365
shot noise processes in, 365–370
with current pulse shape, 369–370,370f
exercises for, 378–380
with square wave, 368–369,368f,369f

K

k-permutations, 23,26t
of padlock combinations, 23
Kurtosis, 118., See also Coefficient of kurtosis

L

Lagrange multiplier techniques, for IID random variables, 291
Laplace, 7
Laplace distribution, signals with, 151–153,152f
Laplace random variables, 80,80f,570
central moments of, 120
characteristic function of, 260
exercise for, 104
Laplace transforms, 584t
Law of large numbers, 304–306
exercises for, 327–328
strong, 304–305
weak, 304–305
Level of significance, 311,311t
License plates, principle of counting and, 22
Likelihood ratio test
for IID Gaussian random variables, 319–321
for radar system, 318,321–322
Linear algebra, 551–557
Linear estimator, 290–292
Linear feedback shift register (LFSR), for binary pseudorandom number generators, 518–521,518f,519f,520t
Linear minimum mean square error (LMMSE) estimator, 270–272
Linear prediction
of PSD, 451
Linear systems, random processes in, 473–504
analog communication system, 500–504,500f,501f,504f
bandlimited and narrowband, 494–498,495f,496f,497f,498f
complex envelopes, 499–500
exercises for, 514
continuous time, 473–477
exercises for, 505–507
discrete-time, 477–479
exercises for, 508–509
matched filter, 482–486,482f,485f,486f
MATLAB exercises for, 514–515
noise equivalent bandwidth, 479–480,479f,509–510
exercises for, 509–510
signal-to-noise ratios, 480–482
exercises for, 480–482,510–511
Linear time-invariant (LTI) system autocorrelation function of, 473–475
cross-correlation function of, 475–476
mean function of, 473–475
passage of signals through, 561–563
random processes in, 473–477
white Gaussian noise in, 476
white noise conversion, 476–477,477f
Linear transformations
of Gaussian random variables, 253–257
of jointly Gaussian random variables, 218–219,254
of multiple random variables, 253–257
of pairs of random variables, 218–219
of vector random variable, 248–249
Log-likelihood ratio test, for radar system, 321–322
Log-normal random variables, 570–571
Lottery game, combinations in, 24
Lower triangular, 552
Lowpass filter (LPF)
Gaussian white noise in, 476
noise equivalent bandwidth of, 479–480,479f
SNR and, 481–482
white noise conversion in, 476–477,477f
Lowpass process, 439
bandwidth of, 439,439f,440f
Lowpass random process, 494

M

(m, n)th joint moment of two random variables, 195–196
marcumq, MATLAB, 110
Marcum’s Q-function, 83–84,84f,565
Marginal CDFs, 178,180,246
Marginal PDFs, 182–183,246
Marginal PMFs, 187–188,246
Markov chains, 384
characterization of, 394–401
exercises for, 424–426
computer communication network described by, 413–415,415f
definitions and examples of, 384–385,385f,390–391
gambler’s ruin problem, 386,386f,392
genetic models, 388
queuing system, 387,393–394,394f
random walks, 387,396,399–401
transition and state probability calculations in, 388–393
Markov processes, 383
computer communication network described by, 413–415,415f
continuous time, 401–411
exercises for, 426–427
MATLAB exercise for, 406–407,406f,412–413,412f
M/M/∞ queue, 405
M/M/1 queue, 405–407,406f
population modeling with, 407–409
continuous time and continuous amplitude, 409–411
definitions and examples of, 383–388
branching process, 388
exercises for, 419–421
gambler’s ruin problem, 386,386f,392
genetic models, 388
Poisson counting process, 383–384
queuing system, 387,393–394,394f
random walks, 387,396,399–401
telephone exchange described by, 416–417,417f
transition and state probability calculations in, 388–393
exercises for, 421–424
MATLAB exercise for, 393–394,394f
Markov’s inequality, 141
Matched filter
for binary PAM signal, 485–486,486f
for communication system, 484–485,485f
Mathematical tools random telegraph process, 340–341
for studying random processes, 340–348,342f,345f
autocorrelation function, 344–345
autocovariance function, 347
cross-correlation function, 347–348
cross-covariance function, 348
exercises for, 371–372
MATLAB, 2
for amplitude modulation, 503–504,504f
for bilinear approximation of random processes, 530–532,531f
binomial coefficient exercise for, 62
central moment exercise for, 120–121
continuous time Markov process exercise for, 406–407,406f,412–413,412f
convolution function, 352–354,353f
dice rolling exercise for, 62
for ergodicity and autocorrelation function estimation, 352–354,353f
for estimating CDF of IID random variables, 297–298
for failure probability, 314–315,314f
for Gaussian random processes simulation, 527–528,528f
for Gaussian random variable, 109
in multiple dimensions, 251–252
help, 15
histogram of randn, 76–78,77f
independent random variable exercise for, 200–201,201f
joint PDF exercise for, 185–186
marcumq function, 110
Markov chain transition and state probability calculation exercise for, 393–394,394f
Markov process exercises for, 427–428
mean function of sinusoidal random processes, 344,345f
multiple random variables exercises for, 287–288
multiple random variables transformation exercise for, 256–257
pairs of random variables exercises for, 242–243
for periodogram estimate of PSD in random telegraph process, 447–448,448f
for PMF, 34,34f
PSD exercises, 469–471
for Q-function, 110
random generation, 15
exercises for, 61–62
histogram construction for, 71,72f
random process exercises, 381–382
of continuous time and discrete amplitude signals, 338–339,339f
of discrete time and continuous amplitude signals, 339–340,340f
in linear systems, 514–515
relative frequency approach simulation, 16–17,16t
for Rician random variable, 110
for sample mean convergence, 305–306,305f
scalar quantization exercise for, 153–154,154f
of shot noise processes, 369–370,370f
simulation technique exercises, 546
sine of uniform random variables, 85,85f
single random variable operation exercises for, 174–175
single random variable transformation exercise for, 129,130f
for uniform random variables, 109
for variance of IID random variable estimation, 296–297
Matrices, 551–557
Maximal length linear feedback shift register (MLLFSR), 520–521,521f
Maximum a posteriori (MAP), 39
in estimation and detection problems, 265–267
Maximum likelihood (ML)
for estimation problems, 267–268
for mean of IID random variable estimation, 292–294
for variance of IID random variable estimation, 295–296
Mean
of Bernoulli random variable, 297
of complex random variables, 220
conditional
exercises for, 163–164
of functions of single random variables, 122
of single random variables, 121–122
ergodic in, 351
estimating of IID random variables, 290–295
BLUE for, 291–292
confidence intervals for, 310–315,311t
ML for, 292–294
of failure probability, 314
of functions of random variables, 113–114,115t
exercises for, 160–161
of Gaussian random variables, 72–73,72n1,73f,317–318
in law of large numbers, 304
of multiple random variables, 247–249
order statistics and, 261–262
for pairs of random variables, 192–197
probability-generating functions and, 137–138
of random processes, 343
of random sum of Gaussian random variables, 317
of random sum of IID random variables, 315–316
of random variables, estimation of, 304
of Rician random variables, 296–297
sample, 292
of single random variables, 111–113
characteristic functions and, 133
exercises for, 159–160
tail probability evaluations using, 141–142
of WSS random processes, 351
Mean function
autocovariance function and, 347
of Gaussian random processes, 358–359
of LTI system, 473–475
of random processes, 343
of random telegraph process, 343
of shot noise processes, 365–366
of sinusoidal random processes, 343–344
MATLAB for, 344,345f
wide sense stationary and, 349–351
Mean square error
for IID random variables, 291
for PSD estimation, 451
Wiener filter to minimize, 487–489
Mean square (MS) sense, random sequence convergence in, 302,303f
Mean time to first return, 400–401
Mean vector, for IID random variables, 291
Median, order statistics and, 261–262
Memoryless property, of exponential random variables, 93
Minimum mean square error (MMSE), for estimation problems, 268–272
Mixed random variables, 67
transformation of, 127
M/M/∞ queue, 405
M/M/1 queue, 405–407,404f
Modulation efficiency, 503
Moment-generating functions
joint, 209
of single random variables, 139–140
exercises for, 169–170
tail probability evaluations using, 142–147,143f,147f
Moments., See also Central moments
of single random variables, 115–117
characteristic functions and, 133–135
exercises for, 161
probability-generating functions and, 136–138
Monotonically decreasing function transformations, 123f,124–125
Monotonically increasing function transformations, 122–124,123f
Monte Carlo simulations, 535–537
for simulation of coded digital communication system, 539–541,539f,541f
Monty Hall problem, 61–62
Moving average process (MA), for PSD estimation, 449
Multinomial coefficient, 25., See also Partitions
exercises for, 48
Multiple random variables, 245
conditional PMF and PDF for, 245–247
exercises for, 277–278
in estimation and detection problems, 264
exercises for, 283–284
MAP estimation, 265–267
ML estimation, 267–268
MMSE estimation, 268–272
expected values of, 247–249
exercises for, 279–280
Gaussian, 249–251
exercises for, 280–281
MATLAB exercise for, 251–252
independence of, 247
joint PMF, CDF, and PDF for, 245–247
exercises for, 277–278
in linear prediction of speech, 272–275,273f,274f,275f,276f
transformations involving, 252–253
coordinate systems in three dimensions, 262–263,263f
exercises for, 281–283
linear, 253–257
MATLAB exercise for, 256–257
order statistics, 260–262
quadratic, 257–260
Mutual exclusiveness, 31
independence compared with, 31
of PMF of binomial random variable, 36
Mutual information exercises for, 239–240
of pairs of random variables, 221–225,223f,224f,225f
Mutually exclusive, 546

N

Nakagami random variable, 571
Narrowband random processes, 494–498,495f,496f,497f,498f
exercises for, 514
n-bit quantizer, 149–153,149f,152f
Negative binomial random variable., See Pascal random variable
Negative definite, 556–557
accounting for, 7
autocovariance function and, 347
deterministic signal or function compared with, 7
Poisson random variable for, 37
sensor voltage and, 289–290
thermal, 452–454
variance of, 290
white, 454
Noise equivalent bandwidth, 479–480,479f
exercises for, 509–510
Nonbinary pseudorandom number generators, 521–522
Noncoherent systems
Rayleigh random variables in, 82
Rician distribution in, 84
Nonmonotonic function transformations, 125–129,125f,128f,130f
Non-singular matrix, 553–554
Non-stationary
Wiener–Khintchine–Einstein theorem for, 435
WSS and, 349
Normal equations, 488
Normal random variables, 73
n-step transition probability, 389–391,396–397
nth central moment, 117
nth moment, of single random variables, 115
nth order PDF, of random processes, 342
Null recurrent, 400–401
Numerical routines, for CDF of Gaussian random variable, 74
Nyquist’s theorem, thermal noise and, 453

O

Optical communication system, 38–41,38f,42f
Bayes’s theorem for, 39–40
conditional probability for, 40–41
overview of, 38–39,38f
PMFs of, 39
probability of error for, 41,42f
Order statistics, 260–262
Orthogonal variables, 193
complex, 221
Orthogonal vectors, 552
Orthogonality principle, 270,488
Outcomes., See also Atomic outcomes
of coin flipping, 8
definition of, 8
of dice rolling, 9
of die rolling, 9
exercises for, 43–44
functions of, 32
of random number generator, 10
total number of possible, 21–22

P

Padlock combinations, k-permutations of, 23
Pairs of random variables, 177–178
channel capacity and channel coding, 221–225,223f,224f,225f
exercises for, 239–240
complex random variables, 219–221
exercises for, 238–239
conditional CDFs for, 188–191
exercises for, 231
conditional PDFs for, 188–191
exercises for, 231
conditional PMFs for, 188–191
exercises for, 231
expected values for, 192–197
conditional expected values, 196–197
correlation, 193–195
covariance, 193–195
exercises for, 231–233
(m, n)th joint moment, 195–196
independence of, 197–199
exercises for, 233–234
MATLAB exercise for, 200–201,201f
joint CDFs for, 178–180,179f
exercises for, 227–228
joint characteristic functions for, 206–209
exercises for, 235–236
joint moment-generating functions for, 209
joint PDFs for, 180–185,184f
exercises for, 228–229
MATLAB exercise for, 185–186
joint PMFs for, 186–188
exercises for, 229–230
joint probability-generating functions for, 208–209
exercises for, 234
joint characteristic functions for, 207–208
linear transformation of, 218–219,254
MAP estimator for, 265
mutual information of, 221–225,223f,224f,225f
exercises for, 239–240
transformations of, 210–219
exercises for, 236–238
Parallel interconnection, system, reliability function and failure rate function of, 94–95
Parametric estimation, of PSD, 448–452,452f
Parseval’s theorem, in PSD, 430
Partitions, 24–26,26t
of bridge, 25–26
three groups, 25
two groups, 24–25
Pascal, 8
Pascal random variable, 38,575., See also Geometric random variable
Period of states, 396
Periodogram
for PSD spectral estimation, 441,445–448,448f
random telegraph process, 447–448,448f
Permutations, 22,26t
k-permutations, 23
Phase, of stochastic signals, 7
Φ-function
for CDF of Gaussian random variable, 74
evaluation of, 75
Q-function relation with, 75,75f
Photodetector, in optical communication system, 38–39
Photoemissive surface, in optical communication system, 39
Points,
of Poisson counting processes, 364
Poisson counting processes, 360–364,364f,383–384
autocorrelation function of, 363
PMF of, 360,363
points of, 364
Poisson distribution, random telegraph signal, 341
Poisson impulse processes, 364,364f
Poisson processes
autocovariance function of, 363
exercises for, 376–378
Poisson random variable, 37,575
exercises for, 56–58
expected value of, 112
in optical communication system, 39
Polar coordinates, Cartesian coordinate transformation into, 216–217
Population modeling, with birth– death processes, 407–409
Positive definite, 556–557
Positive recurrent, 400–401
Power, in random process, 431
Power residue method, 521–522
Power spectral density (PSD), 430
AR process for, 449–450
random telegraph process, 451–452,452f
of bandpass random process, 494–495,495f
bandwidth of random processes, 439–441,439f,440f
exercises for, 465–466
for complex envelope, 500
cross spectral density, 438–439
definition of, 430–433
exercises for, 463
of digital modulation formats, 455–461,458f,459f,460f,461f
autocorrelation of PAM, 456–457
Fourier transform for, 457
with IID sequence of random variables, 457–459,458f,459f
MATLAB example for, 460–461,461f
with memory, 459–460,460f
Fourier transform in, 430
Fourier transform pair of autocorrelation function and, 433
of I and Q components, 496–497,497f
of lowpass random process, 494–495,495f
MATLAB exercises for, 469–471
Parseval’s theorem in, 430
properties of, 431
of sinusoidal random processes, 432
spectral estimation, 441–452
exercises for, 466–468
parametric, 448–452,452f
thermal noise, 452–455,453f,454f
exercises for, 468–469
Wiener–Khintchine–Einstein theorem, 433–439,434f,436f,437f
exercises for, 463–465
of WSS, 433
Power transfer function, 475
of LPF, 479,479f
Prediction problems, 264
Principle of counting, 21
a posteriori, 28,28n2
a priori, 28,28n1
applications of, 1–5,3f,4f,5f
assignment of, 13–17,16t
classical approach to, 14
of coin flipping, 14
of dice rolling, 15
of die rolling, 14
exercises for, 46–47
MATLAB rand command and, 15
relative frequency approach for, 16
of atomic outcomes, 14
axioms of, 10–13,13f
exercises for, 44–45
definition of, 10–11
gambling and, 7–8
random sequence convergence in, 301–302,303t
relative frequency interpretation of, 297
theorem of total probability, 27–28,28f
Probability densities, of randn function, 76–78,77f
Probability density function (PDF), 69–71,69f
of Cauchy random variables, 84
CDF compared with, 69–71,69f
of chi-squared (x2) random variables, 81–82
conditional, 87–89,88f
exercises for, 104–106
for multiple random variables, 245–247,277–278
for pairs of random variables, 188–191,231
properties of, 87–88
transformations of pairs of random variables using, 213–214
of continuous random variables, for CDF, 289
of continuous time and discrete amplitude signals, 341
of Erlang random variable, 81–82
of exponential random variables, 79–80,79
of gamma random variables, 81
of Gaussian random processes, 357–359
of Gaussian random variables, 72–73,73f
histogram of randn function, 76–78,77f
of IID random variables, 289
joint
exercises for, 228–229
for Gaussian random variables in multiple dimensions, 250–252
for multiple random variables, 245–247,277–278
for pairs of random variables, 180–186,184f
of Laplace random variables, 80,80f
properties of, 70
of random processes, 340,342
nth order, 342
second-order, 342
of random variables, 69,76
characteristic functions and, 130–131
conditional, 87–89,88f
exercises for, 99–101
reliability function, 91
of Rayleigh random variables, 82,83f
of Rician random variables, 83–84,84f
of shot noise processes, 367–368
of uniform random variables, 78–79,78f,309,309f
verify validity of, 70
Probability mass function (PMF), 33
of Bernoulli random variable, 35,310,310f
binomial random variable, 35–37
CDF and, 68
for coin flipping, 33
conditional
for multiple random variables, 245–247,277–278
for pairs of random variables, 188–191,231
of continuous random variables, 63
on counting processes, 361–362
of discrete random variables, for CDF, 289
exercises for, 55–59
geometric random variable, 37–38
joint
exercises for, 229–230
for multiple random variables, 245–247,277–278
for pairs of random variables, 186–188
MATLAB for, 34,34f
in optical communication system, 39
of Poisson processes, 360,363
Poisson random variable, 37
problems with, 63–64
of random processes, 340,342,342f
of random telegraph process, 342f
Probability of error, for optical communication system, 40–41,42f
Probability ratio test, for radar system, 318
Probability theory, 7
assigning probabilities, 13–17,16t
axioms of probability, 10–13,13f
basic combinatorics, 20–26,26t
Bayes’s theorem, 27–29,28f
conditional probability, 18–20
development of, 11
discrete random variables, 32–38,34f
exercises for, 43–62
experiments, samples spaces, and events, 7–10
joint probability, 17–20
statistical independence, 29–32
Probability-generating functions
joint, 208–209
of single random variables, 136–138
exercises for, 168–169
Pseudorandom number generators binary, 517–521,518f,519f,520t,521f
nonbinary, 521–522
Pulse amplitude modulation (PAM), 455–456
autocorrelation function of, 456–457
Fourier transform for, 457
with IID sequence of random variables, 457–459,458f,459f
matched filter for SNR in, 485–486,486f
MATLAB example for, 460–461,461f
with memory, 459–460,460f
Pulse code modulation (PCM), differential, 272–275,273f,274f,275f,276f

Q

Q components complex envelopes for, 499–500
cross spectral density of, 497–498
extraction of, 496,496f
PSD of, 497
Q-function, 565
for CDF of Gaussian random variable, 74
evaluation of, 74
expression of, 75–76
Marcum’s, 83–84
MATLAB exercise for, 110
numerical methods for evaluating, 587–591,588f,589f
Φ-function relation with, 75,75f
probabilities in terms of, 76
values of, 584–585,585t
Quadratic transformations, of multiple random variables, 257–260
Quantization
of single random variables, 148–153,149f,152f
exercises for, 172–173
MATLAB exercise for, 153–154,154f
Queuing systems, 387,393–394,394f
conditional PDF in, 89
exponential random variables in, 80
gamma random variables in, 81
M/M/∞ queue, 405
M/M/1 queue, 405–407,406f
Queuing theory, 402–403

R

Radar system, 4,4f,317–323
approaches to, 318
false alarm and detection probabilities for, 318–319
probability ratio test for, 318
sequential detection, 320
signal-to-noise ratio for, 323
system performance for, 323
thresholds for, 319–320
Wald’s inequalities for, 321–322
Radio frequency, phase of, 79
rand, MATLAB, probability assignment and, 15
exercises for, 61–62
histogram construction for, 71,72f
randn, MATLAB, histogram of, 76–78,77f
Random number generator
with MATLAB, 15
exercises for, 61–62
histogram creation for, 71,72f
PMF and, 63
sample space, events, and outcomes of, 10
Random processes, 335–370
average normalized power, 356
bandpass, 494
bandwidth of, 439–441,439f,440f
exercises for, 465–466
CDF of, 340,342
of coin flipping, 336–338,336f,338f
computer generation of, 525–534
exercises for, 544
frequency domain approach to, 525–528,526f,528f
of Gaussian white noise, 533–534,534f
time domain approach, 529–532,529f,531f,533f
of continuous time and discrete amplitude signals, 338–339,339f
cross spectral density between, 438–439
definition of, 336
of discrete time and continuous amplitude signals, 339–340,340f
ergodic in autocorrelation, 351
ergodic in mean, 351
ergodicity of, 351–356
frequency domain techniques for, 429
autocorrelation function of, 346–347
definition of, 357
exercises for, 375–376
PDF of, 357–359
in linear systems, 473–504
analog communication system, 500–504,500f,501f,504f
bandlimited and narrowband, 494–498,495f,496f,497f,498f
complex envelopes, 499–500,514
continuous time, 473–477,505–507
discrete-time, 477–479,508–509
MATLAB exercises for, 514–515
noise equivalent bandwidth, 479–480,479f
signal-to-noise ratios, 480–482,510–511
lowpass, 494
mathematical tools for studying, 340–348,342f,345f
autocorrelation function, 344–345
autocovariance function, 347
cross-correlation function, 347–348
cross-covariance function, 348
exercises for, 371–372
MATLAB exercises for, 381–382
mean function of, 343
mean of, 343
PDF of, 340,342
nth order, 342–343
second-order, 342
PMF of, 340,342,342f
Poisson processes, 360–364,364f
exercises for, 376–378
PSD for
definition of, 430–433
Wiener–Khintchine–Einstein theorem, 433–439,434f,436f,437f
of random variable, 337,337f
autocorrelation function for, 346
ergodicity of, 352
MATLAB for, 344,345f
mean function of, 343–344
strict sense stationary, 349
realization of, 336
shot noise processes, 365–370,368f,369f,370f
spectral characteristics of, 429–430
strict sense stationary, 348–349
exercises for, 372–375
wide sense stationary, 349–351
exercises for, 372–375
Wiener process, 360
Random sequences convergence modes of, 298–303,303t
convergence almost everywhere, 301,303t
convergence everywhere, 300–301,303t
convergence in distribution, 302–303,303t
convergence in mean square sense, 302,303t
convergence in probability, 301–302,303t
exercises for, 326–327
IID random variables, 289–298
estimating CDF of, 297–298
estimating mean of, 290–295
estimating variance of, 295–297
PDF of, 289
sample mean of, 299
radar system, 317–323
approaches to, 318
false alarm and detection probabilities for, 318–319
probability ratio test for, 318
sequential detection, 320
system performance for, 323
thresholds for, 319–320
Wald’s inequalities, 321–322
Random sums
central limit theorem, 306–310
Bernoulli random variables, 310,310f
exercises for, 329
proof of, 307–308
uniform random variables, 309,309f
of Gaussian random variables, 317
law of large numbers, 304–306
exercises for, 327–328
of random variables, 315–317
exercises for, 331–332
function of, 316
mean and variance of, 315–316
Random telegraph process
autocorrelation estimate for, 442,442f
with windowing function, 443–445,444f,445f
bandwidth of, 441
mathematical study of, 340–341
mean function of, 343
periodogram estimate of PSD for, 447–448,448f
PMF of, 342f
Wiener–Khintchine–Einstein theorem for, 436–438,436f,437f
Random variables.See also specific types
conditional, 86–87
exercises for, 97–99
reliability function, 91
central moments of, 117–121
exercises for, 162–163
MATLAB exercise for, 120–121
characteristic functions of, 130–136
exercises for, 167–168
moments and, 133–135
natural logarithm of, 135–136
PDFs and, 130–131
for coin flipping, 33
computer generation of, 517–524
binary pseudorandom number generators, 517–521,518f,519f,520t,521f
correlated, 524
exercises for, 543–544
nonbinary pseudorandom number generators, 521–522
from specified distribution, 521–522
conditional expected values of, 121–122
exercises for, 163–164
entropy associated with, 155–158,157f,158t
exercises for, 173
expected value of, 111–113
exercises for, 159–160
functions of, 113–114,115t
functions of, 85,85f
conditional expected value of, 122
exercises for, 160–161
expected value of, 113–114,115t
independence of, 197–199
exercises for, 233–234
MATLAB exercise for, 200–201,201f
in interval, CDF and, 64,69
list of common, 565–575
MATLAB histogram of randn, 76–78,77f
mean of, estimation of, 304
moment-generating functions of, 139–140
exercises for, 169–170
moments of, 115–117
characteristic functions and, 133–135
exercises for, 161
probability-generating functions and, 136–138
PDF of, 69,76
characteristic functions and, 130–131
conditional, 87–89,88f
exercises for, 99–101
reliability function, 91
PMF of, 33
probability-generating functions of, 136–138
exercises for, 168–169
random process of, 337,337f
autocorrelation function for, 346
ergodicity of, 352
MATLAB for, 344,345f
mean function of, 343–344
strict sense stationary, 349
random sums of, 315–317
exercises for, 331–332
function of, 316
mean and variance of, 315–316
scalar quantization of, 148–153,149f,152f
exercises for, 172–173
MATLAB exercise for, 153–154,154f
tail probability evaluations, 140–147,143f,147f
exercises for, 170–172
tangent of, 84
transformations of, 122
exercises for, 164–167
MATLAB exercise for, 129,130f
monotonically decreasing functions, 123f,124–125
monotonically increasing functions, 122–124,123f
nonmonotonic functions, 125–129,125f,128f,130f
Rare events, simulation of, 534–538
exercises for, 545–546
importance sampling, 537–538
Monte Carlo simulations, 535–537
Rayleigh distribution, 82
reliability function and failure rate function of, 93
Rayleigh random variables, 82,83f,571–572
exercise for, 104
expected value of, 113
Rician random variables and, 83
Realization, of random process, 336
Recurrent state, 398–401
Relative frequency approach
to assigning probabilities, 16
for dice rolling, 16–17,16t
for joint probability, 18
Relative frequency interpretation of probability, 297
Reliability function, 91
derivative of, 91
exponential distribution of, 93
failure rate function and, 92
of Rayleigh distribution, 93
of system
parallel interconnection, 94–95
series interconnection, 93–95
Reliability rates, 91–95
Rician distribution, 84
Rician random variables, 83–84,84f,572
central moments of, 120–121
MATLAB exercise for, 110
mean of, 296–297
variance of, 296–297
RMS bandwidth., See Root-mean-square
Root-mean-square (RMS)
bandwidth, 440
Row vector, 551

S

Saddle point approximation, 145–147,147f
Sample mean, 292
confidence intervals and, 312
of IID random variables, 299
in law of large numbers, 304
MATLAB example for, 305–306,305f
of Rician random variables, 296–297
Sample spaces, 7–10
choice of, 10
of coin flipping, 8–9
continuous, 10
mix with discrete, 10
definition of, 8
of dice rolling, 9
of die rolling, 9
exercises for, 43–44
of random number generator, 10
random variables of, 32
Sample variance, 296
MATLAB example for, 305–306,305f
of Rician random variables, 296–297
Satellite communication channels, Rician distribution in, 84
Scalar, 551
Scalar quantization, of single random variables, 148–153,149f,152f
exercises for, 172–173
MATLAB exercise for, 153–154,154f
Second central moment, 118., See also Variance
Second moment, 115–117
Second-order PDF, of random processes, 342
Sensor, voltage of, noise and, 289–290
Sequential detection
performance of, 320–322
for radar system, 320
Series expansions, 577–578
Series interconnection, system, reliability function and failure rate function of, 93–95
Set theory, 545–549,548f
Shannon entropy, 156
Shot noise processes, 365–370,368f,369f,370f
autocorrelation function of, 366–367
in p-n junction diode, 365–370
with current pulse shape, 369–370,370f
exercises for, 378–380
with square wave, 368–369,368f,369f
mean function of, 365–366
PDF of, 367–368
Poisson counting processes, 365
Poisson impulse processes, 365
as strict sense stationary, 368
as WSS, 367
Signals review of, 559–563
scalar quantization of, 148–153,149f,152f
sinusoidal, 219
types of, 335
Signal-to-noise ratio, of radar system, 323
Signal-to-noise ratios (SNR)
for AM, 502–503
exercises for, 480–482,510–511
in linear systems, 480–482
LPF and, 481–482
matched filter for, 482–486,482f,485f,486f
for binary PAM signal, 485–486,486f
for communication system, 484–485,485f
of radar system, 323
Signal-to-quantization-noise power ratio (SQNR), 148–150,153–154,154f
for DPCM speech coders, 275,276f
Simulation techniques, 517–541
for coded digital communication system, 539–541,539f,541f
computer generation of random processes, 525–534
exercises for, 544
frequency domain approach to, 525–528,526f,528f
of Gaussian white noise, 533–534,534f
time domain approach, 529–532,529f,531f,533f
computer generation of random variables, 517–524
binary pseudorandom number generators, 517–521,518f,519f,520t,521f
correlated, 524
exercises for, 543–544
nonbinary pseudorandom number generators, 521–522
from specified distribution, 521–522
MATLAB exercises for, 546
of rare events, 534–538
exercises for, 545–546
importance sampling, 537–538
Monte Carlo simulations, 535–537
Single random variables., See Random variables
Singular matrix, 553–554
Sinusoidal random processes, 337,337f
autocorrelation function for, 346
ergodicity of, 352
mean function of, 343–344
MATLAB for, 344,345f
PSD of, 432
strict sense stationary, 349
Wiener–Khintchine–Einstein theorem for, 435–436
Sinusoidal signals, 219
6 dB rule, 150
Skewness, 118., See also Coefficient of skewness
Slotted Aloha, 413–415,415f
Smoothing function, for periodogram, 446
Source coding, entropy and, 155–158,157f,158t
ent
exercises for, 173
Spectral estimation exercises for, 466–468
of PSD, 441–452
parametric, 448–452,452f
Speech, linear prediction of, 272–275,273f,274f,275f,276f
Speech recognition system, 2–4,3f
Speech signal amplitude, Laplace random variables in, 80
Spherical coordinates, Cartesian coordinate transformation into, 262–263,263f
Staircase function transformation, 128,128f
Standard deviation
estimating of IID random variables, 295–297
of Gaussian random variables, 72–73,72n1,73f
second central moment of random variables and, 118
Standard normal random variables, 73
CDF of, 74
central limit theorem and, 307–308
State transition probability matrix, calculation of, 388–393
exercises for, 421–424
MATLAB exercise for, 393–394,394f
Stationary
strict sense, random processes, 348–349
wide sense, random processes, 349–351
Stationary increments, of Poisson processes, 361
Statistical independence., See Independence
Steady-state distribution, 391,393,401
for continuous time Markov processes, 403–405
Stochastic signals, 7
Strict sense stationary ergodicity and, 351–352
exercises for, 372–375
of random processes, 348–349
shot noise processes as, 368
sinusoidal random processes, 349
Strong law of large numbers, 304–305
convergence almost everywhere and, 305
Student’s t-distribution, 313,315n5,572–573
Subset, 545,548f
Symmetric matrix, 551
Systems, review of, 559–563

T

Tail probabilities, evaluation of, 140–147,143f,147f
exercises for, 170–172
Taylor series expansion of Q-function, 588–589,588f
Telephone exchange, Markov processes describing, 416–417,417f
Telephone traffic, Erlang random variable in, 82
Theorem of Total Probability, 27–28,28f
unconditional cumulative distribution function and, 89–90
Thermal noise, 452–454
exercises for, 468–469
Thevenin equivalent circuit, 453,453f
Third central moment, 118
Three dimensions., See Multiple random variables
3 dB bandwidth, 440,440f
Time, continuous function of, 335
Time difference variable, autocorrelation function and, 345
Time domain techniques, 7
for computer generation of random processes, 529–532,529f,531f,533f
MATLAB example for, 530–532,531f
Time-averaged power, of random process, 430
Total probability, theorem of, 27–28,28f
Transfer function, 562
Transformations
of multiple random variables, 252–253
coordinate systems in three dimensions, 262–263,263f
exercises for, 281–283
linear, 253–257
MATLAB exercise for, 256–257
order statistics, 260–262
quadratic, 257–260
of pairs of random variables, 210–219
exercises for, 236–238
of single random variables, 122
exercises for, 164–167
MATLAB exercise for, 129,130f
monotonically decreasing functions, 123f,124–125
monotonically increasing functions, 122–124,123f
nonmonotonic functions, 125–129,125f,128f,130f
on uniform random variables, 79
of vector random variable, 248–249
Transient state, 398–401
Transition probability matrix, 384–385
calculation of, 388–393
exercises for, 421–424
MATLAB exercise for, 393–394,394f
for continuous time Markov processes, 402
Transpose, 551
Trigonometric identities, 577
Two dimensions., See Pairs of random variables

U

Unbiased estimate, 262,290
Unconditional cumulative distribution function, 89–90
Uncorrelated random variables, 193
complex, 221
independence and, 195,199,251
Uniform quantizer, 149–150,149f
Uniform random variables, 78–79,78f,573
central moments of, 118
histogram of MATLAB generation of, 71,72f
MATLAB exercise for, 109
moments of, 117
PDF of, 309,309f
sine of, 85,85f
transformation on, 79
WSS and, 350
Union, 545,546f
Universal set, 545,546f
Upper triangular, 552

V

Variability, accounting for, 7
Variance, 118
of complex random variables, 220–221
estimating of IID random variables, 295–297
MATLAB example for, 296–297
ML for, 295–296
of failure probability, 314
of Gaussian random variables, 72–73,72n1,317–318
of IID random variables, 292
in law of large numbers, 304
of noise, 290
probability-generating functions and, 137–138
of random sum of Gaussian random variables, 317
of random sum of IID random variables, 315–316
of Rician random variables, 296–297
sample, 296
tail probability evaluations using, 142
Vector random variable., See Multiple random variables
Voice conversation duration, exponential random variables in, 80

W

Wald’s inequalities, for radar system, 321–323
Weak law of large numbers, 304–305
convergence in probability and, 305
Weibull random variable, 573–574
White noise, 454
LTI system conversion of, 476–477,477f
Whitening, 256
Whitening filter, 490–491,490f
Wide sense stationary (WSS)
autocorrelation function and, 349–351,356–357
ergodicity of, 351–356
exercises for, 372–375
Gaussian random processes, 358–359
PSD of, 433
random processes, 349–351
shot noise processes as, 367
impulse response with, 489
to minimize mean square error, 487–489
for prediction, 492–494,494f
with two filters, 490–492
Wiener process, 360
Wiener–Hopf equations, 488–489
for prediction, 492–494,494f
for two filters, 490–491
Wiener–Khintchine–Einstein theorem, 433–439,434f,436f,437f
exercises for, 463–465
for random telegraph process, 436–438,436f,437f
for sinusoidal random processes, 435–436
Windowing function, for autocorrelation function estimation, 443
random telegraph process, 443–445,444f,445f
Wireless communication channels, Rayleigh random variables in, 82
Wireline telecommunication networks, Erlang random variable in, 82

Y

Yule-Walker equations, 488

Z

Zero-mean Gaussian distribution, 82
Zeroth central moment, 118
Zeroth moment, 115
Z-transform, 561,582–583t
for discrete-time linear systems, 477–478
for discrete-time processes, 429
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset