Symbols

__call__ method 231, 235, 251

__dict__ method 630

__eq__ method 631, 634

__mul__ method 631

__repr__ method 208, 371, 373, 631

__rmul__ method 631

_repr_latex_ method 373

- operator 365, 445, 607

@abstractmethod decorator 211

* operator 208, 236, 365, 607

** operator 608, 621

/ operator 607

% (modulus operator) 607, 620

+ operator 208, 365, 445, 478, 607, 611

== operator 208, 631

≈ (approximate equality) 54

π(pi) symbol 56

Numerics

2D plane 22

2D vectors. See vectors

3D animation 166-168

3D objects 635-644

change perspective 639-641

exercises 643-644

octahedron 638

rendering in 2D 114-119

defining 3D object with vectors 114-116

exercises 119

orienting faces and shading 116-118

projecting to 2D 116

transformations of 123

composing 126-129

drawing transformed objects 124-126

exercises 134-137

inventing geometric transformations 131-133

rotating objects about an axis 129-131

translating in 4D world 196-199

Utah teapot 641-643

3D space 77

3D vectors. See vectors

A

ABC (abstract base class) 211, 633

abc module 211

abstract methods 633

abstractions, building with functions 17

acceleration 342-344

activations 560

calculating 572-574

calculating activations of each subsequent layer based on activations in preceding layer 572

each activation in next layer as function of all of activations in input layer 570-571

in matrix notation 574-576

return vector whose entries are activations of output layer 572

setting input layer activations to entries of input vector 570

addition

of 2D vectors 32-35

of 3D vectors 83-84

proving projection preserves vector sums 182-183

amplitude 469

Anaconda 596-597

angles

converting from components to 57-59

converting to components 52-55

measuring with dot products 97-100

of 3D vectors 87-88

antiderivatives, integrals as 387-388

append method 615

Apply class 369, 384

approximate equality (≈) 54

approximate gradient 450

arange function 311

arctangent function 66

arguments 619

arithmetic

2D vectors

addition 32-35

components and lengths 35-37

exercises 51

multiplying by numbers 37-38

subtraction 39-41

3D vectors 83-92

addition 83-84

angles and directions 87-88

exercises 89-92

lengths and distances 86-87

scalar multiplication 85

scalar subtraction 85

subtraction 86

arithmetic operators 365, 607

Arrow class 26, 72

Arrow3D class 80-81, 89-90

artificial neural networks 559

AssertionError 214

asteroid arcade game 258

accelerating spaceship 343-344

constant velocity motion

adding velocities to asteroids 339

adding velocities to objects 339

keeping asteroids on screen 340-342

updating game engine to move asteroids 339-340

deciding whether laser hits an asteroid 270

modeling game 259

rendering game 260-261

shooting laser 261-262

astype method 476

average_flow_rate function 305-306

axes 24

axn expression 382

B

backpropagation 562-592

calculating partial derivatives for last layer weights using chain rule 590-591

exercises 591-592

finding cost in terms of last layer weights 590

basis 243

bias 573

bit depth 467

BlackHole class 396, 399

bounds of integration 334

C

calculus 9, 301, 303

callable classes 225

Cartesian coordinates 51-52, 54-55, 57, 68, 88, 129

chain rule 383-384, 590-591

classes 632-634

classification 561

closed subspace 239

coefficients 511

column vectors 160, 178

combinators 359

combining terms 370

commutative operations 170

components, of 2D vectors 35-37

converting angles to 52-55

converting to angles 57-59

composition of functions 127

computer algebra 371

constant velocity motion 338-342

adding velocities to objects 339

exercises 342

keeping objects on screen 340-342

updating game engine to move objects 339-340

constructive interference 480

constructors 629

converging values 333

CoordinateVector class 216-217

cost functions 501

cross products 92, 103-113

computing 109-110

exercises 110-113

finding direction of 106-108

finding length of 108-109

overview 103-106

currying 129-130

D

Darcy’s law 10

data classification

with logistic regression 526-558

exploring possible logistic functions 544-557

finding best logistic function 551-557

framing classification as regression problem 536-544

picturing decision boundary 532-535

testing classification functions 528-530

with neural networks 561-568

building random digit classifier 565-566

exercises 567-568

measuring performance of digit classifier 566

data collections 610

lists 610

indexing and slicing 611-612

iterating over 612

decision boundaries 532-535

drawing better decision boundary 533

exercises 535

implementing classification function 534-535

definite integral 325

degrees of freedom 285

denominator 363

derivative method 384

derivative of a function at the point 318

derivative operators 321

derivatives 303, 315, 320

finding 374-381

exercises 379-381

of powers 374-375

of products and compositions 378-379

of special functions 377-378

of transformed functions 375-376

finding with computer algebra system 355-356

taking automatically 381-386

chain rule 383-384

derivative method for expressions 382-383

exercises 386

power rule 384-386

product rule 383-384

destructive interference 480

df/dx notation 412

Difference combinator 363

differential equations 9

dimensions 206, 243

directions

between 3D vectors 87-88

between cross products 106-108

displacement 40, 85

distance

of 2D vectors 39-41

of 3D vectors 86-87

distinct_variables function 366

distributive property 369

does_collide function 277

does_intersect function 263, 270-271

dot products 92-103

computing 95-97

examples 97

exercises 100-103

measuring angles with 97-100

picturing 93-95

drag 448

drawing 2D 24

dt value 329, 331, 349

E

Euler’s method 344-353

building projectile simulation with 426-427

carrying out by hand 344-346

exercises 349-353

implementing algorithm in Python 346-347

running with smaller time steps 348-353

evaluate function 367-368

expand function 370-371

exponential decay 520

Expression class 365, 367, 370, 373, 382

expression trees

building 359-360

translating to Python 360-362

expressions.py file 360

F

faces, of 3D objects, orienting 116-118

false positive 530

feedforward method 581-582

first-class values 621

flow rate

approximating change in volume 323-328

breaking up time into smaller intervals 324-325

exercises 328

for short time interval 323-324

picturing on flow rate graph 325-327

approximating instantaneous 315-322

building function 318-319

currying and plotting function 320-321

exercises 322

finding slope of small secant lines 315-318

flow rate (continued)

calculating average 305-309

average_flow_rate function 305-306

negative rates of change 308-309

picturing with secant line 306-308

plotting average over time 310-314

exercises 313

finding average in different time intervals 310-311

plotting interval flow rates 311-313

force field 392

four-dimensional vector 5

Fourier series, decomposing sound waves into 465-466, 486-495

defining inner products for periodic functions 488-490

exercises 494-495

finding vector components with inner products 487-488

Fourier coefficients for other waveforms 492-494

Fourier coefficients for square wave 491-492

Fourier coefficients, writing function to find 490-491

frequency 464

function application 359

Function class 233

Function(Vector) class 231

Function2(Vector) class 233

functional programming paradigm 621

functional programming techniques 128

functions

building abstractions with 17

derivatives of

finding 374-381

taking automatically 381-386

integrating symbolically 387-389

exercises 389

integrals as antiderivatives 387-388

SymPy library 388-389

treating as vectors 223-226

viewing square and non-square matrices as 180-181

G

gcf (get current figure) method 628

global maximum value 438

global minimum value 438

gradient ascent 424, 451, 453

gradient operator 11

gradients 393, 414

calculating with backpropagation 588-592

calculating partial derivatives for last layer weights using chain rule 590-591

exercises 591-592

finding cost in terms of last layer weights 589-590

optimizing projectile range using gradient ascent 449-461

exercises 457-461

finding uphill direction with gradient 451-453

gradient of range function 450-451

implementing gradient ascent 453-457

plotting range vs. launch parameters 449-450

training neural networks using gradient descent 582-588

automatic training with scikit-learn 585-586

calculating gradients with backpropagation 584

exercises 586-588

framing training as minimization problem 582-584

gravitational fields 393

H

hertz (Hz) 471

hidden layers 572

hypotenuse 36, 112

hypothesis 500

I

identity matrices 171

identity transformation 150, 170

images

manipulating with vector operations 227-230

vector subspaces of 245-248

ImageVector class 228-229, 236, 254

immutable tuples 615

independent system 285

infinite dimensional 226

inheritance 206

inner products

defining for periodic functions 488-490

finding vector components with 487-488

input vectors 190

instantaneous speed 313

instantaneous value 313

integral of the flow rate 333

integration 323

choosing right formula for line 263-264

finding standard form equation for line 265-267

linear equations in matrix notation 267-268

solving linear equations with NumPy 268-270

inverse trigonometric function 58

items method 618

J

Jupyter notebooks 600-606

L

length

of 2D vectors 35-37

of 3D vectors 86-87

of cross products 108-109

line of best fit 501

linear algebra 5, 199, 205

linear combination 142

linear equations, systems of 257

arcade game 258

intersection points of lines 270

modeling game 259-260

rendering game 260-261

shooting laser 261-262

choosing right formula for line 263-264

finding standard form equation for line 265-267

linear equations in matrix notation 267-268

solving linear equations with NumPy 268-270

linear functions 163, 181, 244, 501

linear maps

composing 184-185

projection as 181-184

proving projection preserves scalar multiples 183-184

proving projection preserves vector sums 182-183

linear regression 3, 501

linear transformation 138

linear transformations 155, 163, 380

computing 146-149

exercises 149-155

picturing 140-142

preserving vector arithmetic 138-140

reasons for using 142-146

representing with matrices 159-170

3D animation with matrix transformations 166-168

composing linear transformations by matrix multiplication 163-166

exercises 169-170

implementing matrix multiplication 166

multiplying matrices with vector 161, 163

writing vectors and linear transformations as matrices 159-160

LinearFunction class 244, 251-252

linearly dependent 243

linearly independent 242-243

lists 610

indexing and slicing 611-612

iterating over 612

local maximum value 438

local minimum value 438

logistic functions 527, 540

logistic regression 526, 540-558

exploring possible logistic functions 544-557

exercises 549-551

measuring quality of fit 546-547

parameterizing logistic functions 545-546

testing 548

finding best logistic function 551-557

exercises 555-557

gradient descent in 3D 551-552

testing an understanding best logistic classifier 554-555

using gradient descent 552-553

framing classification as regression problem 536-544

exercises 543-544

measuring likelihood 538-539

scaling raw data 536-538

sigmoid function 540-542

picturing decision boundary 532-535

drawing better decision boundary 533-534

exercises 535

implementing classification function 534-535

testing classification functions 528-530

exercises 530

loading car data 529

testing classification function 529-530

M

m-by-n matrix 181, 188, 578

m -dimensional space 227

m/s/s (meters per second per second) 343

math anxiety 2

math module 16, 608-609

math, learning through software and code 1-17

abstractions, building with functions 17

real-world use cases 2

comparing deals 5

Mathematica 355

Matplotlib

drawing 73

drawing 2D vectors in 72

matrices 158-204

interpreting matrices of different shapes 175-191

column vectors as matrices 176-178

composing linear maps 184-185

determining which pairs of matrices can be multiplied 178-180

exercises 186-191

projection as linear maps 181-184

viewing square and non-square matrices as vector functions 180-181

linear equations in matrix notation 267-268

representing linear transformations with 159-170

3D animation with matrix transformations 166-168

composing linear transformations by matrix multiplication 163-166

exercises 169-170

implementing matrix multiplication 166

multiplying matrices with vector 161-163

writing vectors and linear transformations as matrices 159-160

translating vectors with 191-203

combining translation with other linear transformations 195-196

exercises 199-203

finding 3D matrix for 2D translation 194-195

making plane translations linear 191-194

translating 3D objects in 4D world 196-199

treating as vectors 226-227

Matrix class 227, 234

matrix dimensions 179

matrix multiplication 161

matrix notation 159

maxima, identifying 437-438

measuring functions 511

meters per second per second (m/s/s) 343

minima, identifying 437-438

MLP (multilayer perceptron) 568

evaluating 580-581

implementing MLP class 578-580

testing performance of 581-582

MLP class 577, 585, 588

MLPClassifier class 588

modulus operator (%) 607, 620

motion simulation 337-353

acceleration 342-344

constant velocity motion 338-342

adding velocities to objects 339

exercises 342

updating game engine to move objects 339-340

Euler’s method 344-353

carrying out by hand 344-346

exercises 349-353

implementing algorithm in Python 346-347

running with smaller time steps 348-353

move method 340-341, 344, 401, 404

Mul constructor 388

multilayer perceptron. See MLP (multilayer perceptron)

multiplication

matrix

composing linear transformations by 163-166

implementing 166

when to use 178-180

of 2D vectors by numbers 37-38

scalar

of 3D vectors 85

proving projection preserves scalar multiples 183-184

N

n -dimensional space 227

n -dimensional vectors 181

Negative combinator 364

negative gradient 415

negative slope 308

neural networks 559-593

building in Python 577-582

evaluating MLP 580-581

exercises 582

implementing MLP class 578-580

testing performance of MLP 581-582

calculating gradients with backpropagation 588-592

calculating partial derivatives for last layer weights using chain rule 590-591

exercises 591-592

finding cost in terms of last layer weights 589-590

classifying images of handwritten digits 561-568

building 64-dimensional image vectors 563-565

building random digit classifier 565-566

exercises 567-568

measuring performance of digit classifier 566

designing 568-577

calculating activations 572, 576

data flow 569-572

exercises 576-577

organizing neurons and connections 568-569

training using gradient descent 582-588

automatic training with scikit-learn 585-586

calculating gradients with backpropagation 584

exercises 586-588

framing training as minimization problem 582-584

neurons 560

noise 467

Number class 386

numerator 363

NumPy library 268-270

O

octahedrons 114-115, 118, 635-638

OOP (object-oriented programming) 206-634

class methods 632

inheritance and abstract classes 632-634

OpenGL 635-644

change perspective 639-641

exercises 643-644

octahedron 635-638

Utah teapot 641-643

operator overloading 208

origin 7, 22

outliers 500

output layer 572

P

parabola 440

periodic functions 471

pi (π) symbol 56

pitch 471

pixels 227

planes 22

play() method 470

Points class 26, 72

Points3D class 80

polar coordinates 51-52, 68, 129

Polygon class 26, 72

PolygonModel class 259, 262, 270, 276, 340

Polynomial class 233

Power class 385

power rule 384-386

preserve vector sums 139

Product constructor 388

product rule 378, 383-384

projectile simulation 422-462

building with Euler’s method 426-427

calculating optimal range 432-440

as function of launch angle 432-435

exercises 439-440

identifying maxima and minima 437-438

solving for maximum range 435-437

enhancing 440-447

adding another dimension 441-442

exercises 447

modeling terrain around cannon 442-443

solving for range of projectile in 3D 443-446

optimizing using gradient ascent 449-461

exercises 457-461

finding uphill direction with gradient 451-453

gradient of range function 450-451

implementing gradient ascent 453-457

plotting range vs. launch parameters 449-450

testing 425-426

exercises 429-432

exploring different launch angles 428-429

measuring properties of trajectory 427-428

proofs, writing 214

properties 629

property-based testing 152

pure functions 16

PyGame 635-644

change perspective 639-641

exercises 643-644

octahedron 635-638

Utah teapot 641-643

pyplot 624

Pythagorean theorem 36, 57, 86

Python 595-634

building neural networks in 577-582

evaluating MLP 580-581

exercises 582

implementing MLP class 578-580

testing classification performance of MLP 581-582

checking for existing installation 595-596

data collections 610

Python (continued)

downloading and installing Anaconda 596-597

drawing 2D vectors in 26-29

drawing 3D vectors in 80-81

numbers and math 607-610

math module 608-609

random numbers 609-610

object-oriented programming 634

class methods 632

inheritance and abstract classes 632-634

symbolic algebra in 356-358

translating expression trees to 360-362

trigonometry in 56-57

using in interactive mode 597-606

creating and running script file 598-600

using Jupyter notebooks 600-606

Q

quadratic formula 434

quadratic functions 245

QuadraticFunction(Vector) class 252

quants 2

Quotient combinator 363

R

radians 56-57

random module 609

random numbers 609-610

range of projectiles

calculating optimal 432-440

as function of launch angle 432-435

exercises 439-440

identifying maxima and minima 437-438

solving for maximum range 435-437

optimizing using gradient ascent 449-461

exercises 457-461

finding uphill direction with gradient 451-453

gradient of range function 450-451

implementing gradient ascent 453-457

plotting range vs. launch parameters 449-450

solving for in 3D 443-446

ranges 613

rates of change 303-337

approximating change in volume 323-328

breaking up time into smaller intervals 324-325

exercises 328

for short time interval 323-324

picturing on flow rate graph 325-327

approximating instantaneous flow rates 315-322

building function 318-319

currying and plotting function 320-321

exercises 322

finding slope of small secant lines 315-318

calculating average flow rate 305-309

average_flow_rate function 305-306

exercises 309

negative rates of change 308-309

picturing with secant line 306-308

plotting average flow rate over time 310-314

exercises 313-314

finding average in different time intervals 310-311

plotting interval flow rates 311-313

plotting volume over time 328-335

definite and indefinite integrals 334-335

finding volume over time 328-329

improving approximation 332-333

picturing Riemann sums for volume function 329-331

Rectangle class 629-630, 632-634

reflection 152

regression 499

REPL (read-evaluate-print loop) 597

reverse() method 619

RGB (red, green, and blue) 227

Riemann sum 327, 488

right-hand rule 106

rotating objects 129-131

row vector 177

S

saddle point 451

sampling 465, 473

scalar field 394

scalar multiplication 37

scalars 37

scale function 211

scaling 37

scikit-learn library 562

secant lines

finding slope of small 315, 318

picturing average flow rate 306, 308

second derivatives 344

Segment class 26, 28, 72

segments() method 277

set_size_inches method 628

shading, of 3D objects 116-118

Shape class 633-634

Ship class 261

sigmoid function 540-542

sinusoidal functions 466

sinusoidal waves 471-477

building familiar function with 483-486

building linear combination of 481-482

changing frequency of 473-475

exercises 477

making audio from 471-473

sampling and playing sound waves 475-476

sort() method 619

sound wave analysis 463-496

combining sound waves 465-466, 478-486

adding sampled sound waves to build chords 478-479

building familiar function with sinusoids 483-486

building linear combination of sinusoids 481-482

exercises 486

picturing sum of two sound waves 479-481

decomposing sound waves into Fourier series 465-466, 486-495

defining inner products for periodic functions 488-490

exercises 494-495

finding vector components with inner products 487-488

Fourier coefficients for other waveforms 492-494

Fourier coefficients for square wave 491-492

Fourier coefficients, writing function to find 490-491

playing sound waves in Python 466, 471

exercises 471

playing musical notes 469-471

producing sound 467-469

turning sinusoidal wave into sound 471-477

changing frequency of sinusoid 473-475

exercises 477

making audio from sinusoidal functions 471-473

sampling and playing sound wave 475-476

spacetime 197

spans 240-243

spherical coordinates 88

Square class 632

square wave 470

standard form 264

steepest ascent 415

steepest descent 415

steepness 408

straight-line path 26

subscripts 569

subtraction

of 2D vectors 39-41

of 3D vectors 85-86

Sum constructor 388

summation symbol 282

superscripts 569

Symbol constructor 388

symbolic expressions 354-391

evaluating expressions 366-369

exercises 372-374

expanding expressions 369-371

finding all variables in expressions 365-366

finding derivatives 374-381

exercises 379-381

of powers 374-375

of products and compositions 378-379

of special functions 377-378

of transformed functions 375-376

with computer algebra system 355-356

integrating functions symbolically 387-389

exercises 389

integrals as antiderivatives 387-388

SymPy library 388-389

modeling algebraic expressions 358-365

breaking expressions into pieces 358-359

building expression trees 359-360

exercises 362-365

translating expression trees to Python 360-362

symbolic algebra in Python 356-358

taking derivatives automatically 381-386

chain rule 383-384

derivative method for expressions 382-383

exercises 386

power rule 384-386

product rule 383-384

symbolic programming 356, 371

SymPy library 388-389

systems of linear equations 257

arcade game 258

intersection points of lines 270

modeling game 259-260

rendering game 260-261

shooting laser 261-262

choosing right formula for line 263-264

finding standard form equation for line 265-267

linear equations in matrix notation 267-268

solving linear equations with NumPy 268-270

T

t values 473

t%1 value 491

taking a derivative 320

tangent of the angle 54

theorems 13

tick() method 640

timbre 464, 476

tip-to-tail addition 33, 43

tolerance 453

transformations 67-72, 121-157

combining 69-70

exercises 71-72

linear transformations 138-155

computing 146-149

exercises 149-155

picturing 140-142

preserving vector arithmetic 138-140

reasons for using 142-146

representing with matrices 159-170

of 3D objects 123-137

composing vector transformations 126-129

drawing transformed objects 124-126

exercises 134-137

inventing geometric transformations 131-133

rotating objects about an axis 129-131

translating by a vector 125

translating vectors with matrices

combining translation with other linear transformations 195-196

exercises 199-203

finding 3D matrix for 2D translation 194-195

making plane translations linear 191-194

translating 3D objects in 4D world 196-199

translations 191-203

transposition 188

Triangle class 634

trigonometric function 54

trigonometry 51-67

converting from angles to components 52-55

converting from components to angles 57-59

exercises 60-67

in Python 56-57

true negative 530

true positive 530

Turing complete 16

U

uniform 609

unit testing 214-215

updating game engine to move objects 339

Utah teapot 641-643

V

variable bindings 367

Variable class 368

Variable constructor 388

Vec0 class 220

Vec1 class 219-220

Vec2 class 207-208, 210-213, 215, 218, 251

creating 207-208

improving 208-209

Vec3 class 209-213, 218

vector addition 32

Vector base class 210-212

Vector class 211-212, 214-215, 219, 221, 226, 228, 230, 234-235, 244, 367

vector mathematics 22

vector operations, image manipulation with 227-230

vector products 88

vector spaces 206, 213, 219-237

defined 212-213

enumerating all coodinate vector space 219-221

exercises 230-237

identifying 221-223

manipulating images with vector operations 227-230

treating functions as vectors 223-226

treating matrices as vectors 226-227

unit testing vector space classes 214-215

vector subspaces 237-250

dimensions 243-244

exercises 248-250

identifying 238-240

of images 245-248

of vector space of functions 244-245

spans 240-243

starting with single vector 240

vector sums 33, 154

vector_drawing module 27

vectors 5, 21, 24-120

arithmetic for 2D

addition 32-35

components and lengths 35-37

displacement 39-41

distance 39-41

exercises 42-51

multiplying by numbers 38

subtraction 39-41

arithmetic for 3D 83-92

addition 83-84

angles and directions 87-88

exercises 89-92

lengths and distances 86-87

scalar multiplication 85

subtraction 85-86

column vectors as matrices 176-178

defining 3D objects with 114-116

multiplying matrices with 161-163

overview 22-24

transformations 67-72

combining 69-70

exercises 71-72

treating functions as 223-226

treating matrices as 226-227

viewing square and non-square matrices as vector functions 180-181

writing as matrices 159-160

volume

approximating change in 323-328

W

weights 573

white noise 469

Z

zero method 217-218

zero vector 230

zero-dimensional subspace 240

zero-dimensional vector 220

zero-indexed 610