INDEX  

acceptance testing Fig. 2.2, 16–17, 42, 46, 51–52, 150

ad hoc testing 85, 126

Agile development methodologies 18, 44, 146

and fundamental test process 24–25

and test tools 187, 196

alpha testing 52

analytical approaches 151

audit trails 64

see also traceability

authors 67

baselining 153–154

beta testing 52

big-bang integration 47

black-box techniques see specification-based techniques

bottom-up integration Fig. 2.5, 48–49

boundary value analysis 89–91

builds 186

burn-down charts 25

business rules 91–92

Capability Maturity Model Integration (CMMI) assessment 212

capture playback tools 193–194

Certified Tester Foundation Level (CTFL)

examination structure 223–224

examination technique 227

levels of understanding 2–3

purpose 1–2

question types 224–226

revision 227–228

sample paper 227

syllabus 2–4

CFGs see control flow graphs (CFGs)

changes

maintenance testing 54–55

regression testing 19, 22–23, 25, 44, 54, 55

retesting 15, 22–23, 54, 55

traceability 44, 185

checklists 67, 69

cloud storage 178

code

control flow graphs Figs. 4.8–4.9, 107–109

flow charts Figs. 4.4–4.7, Fig. 4.10, 103–105, 111

instrumentation 198

program structures 101–103

pseudo 100

reviewing 62

static analysis 71–73

see also structure-based techniques; work-products

code of ethics 27–28

communication 27, 63

comparators Tab. 6.5, 177, 192–193

compilers 72–73, 189

completion criteria

coverage measures as 83

defining 13, 21

see also exit criteria

component integration testing 49

component testing Fig. 2.2, 42, 46, 150

configuration management 148, 168

configuration management tools Fig. 6.3, Tab. 6.1, Tab. 6.5, 186–187, 188

confirmation testing see retesting

consultative approaches 151

contract acceptance testing 52

control flow graphs (CFGs) Figs. 4.8–4.9, 107–109

simplified Figs. 4.15–4.18, 100, 121–125

see also hybrid flow graphs

control flow models 53

control structures Fig. 2.4, 47

see also program structures

conversion testing 55

cost escalation model Fig. 1.3, Tab. 1.1, 17, 41

costs

of errors Fig. 1.3, Tab. 1.1, 17

reducing 63, 178, 195

test tools Fig. 6.1, Fig. 6.4, 177, 196, 203, 214–215

coverage see test coverage

coverage measurement tools Tab. 6.5, 198–199

CTFL see Certified Tester Foundation Level (CTFL)

cyclical development models see iterative development models

dashboards Fig. 5.3, 160

data-driven testing 194

data quality assessment tools 204

databases

in test tools 178, 183, 184

use in testing 189, 191

debugging 14

debugging tools 46, 189

decision coverage 117–121

decision table testing 91–94

decision testing Fig. 4.14, 117–121

decisions see iteration structures; selection structures

defect lists 127

defect management tools see incident management tools

defects 15, 72

causes Fig. 1.1, 9–10

clustering 18

early identification 17, 62–63, 71, 189

finding by performance testing 202

finding by reviews 63

finding by static analysis 72, 189

and quality 12–13

raising constructively 27

see also errors

design see test design

developers

modelling tools Tab. 6.5, 189–190

pair programming 69

as testers Tab. 5.1, 14, 26, 43–44, 46, 145

use of test tools Tab. 6.5

documentation

baselining 153–154

test plans Fig. 5.2, Tab. 5.2, 152–155

see also reviews; specifications; work-products

drivers 49, 196

dynamic analysis tools Tab. 6.5, 200–201

dynamic approaches 151

Dynamic Systems Development Methodology (DSDM) 44

dynamic testing 14

early testing 16–18

entry criteria

reviews 65

test activities 156–157

equivalence partitioning 87–89

‘error–defect–failure’ cycle 17

error guessing 126–127

errors

absence of 19

clustering 18

costs of Fig. 1.3, Tab. 1.1, 17

effects Fig. 1.1, 9–10

see also defects

‘errors of migration’ 17

estimation, test 157–159

ethics 27–28

events 95

examination

question types 224–226

revision 227–228

sample paper 227

structure 223–224

technique 227

executable statements 101, 105

testing Figs. 4.10–4.13, 110–117

execution postconditions 80

execution preconditions 80

execution, test see test execution

exhaustive testing 11, 16

exit criteria

evaluating 23

reviews 65, 67

test activities 157

see also completion criteria

experience-based techniques 84, 85, 126–127

expert-based estimation 157, 158

exploratory testing 127, 151, 194

factory acceptance testing 51

failure lists 127

failure, software 8–10

fault attack 127

field testing 52

flow charts Figs. 4.4–4.7, Fig. 4.10, 103–105, 111

formal reviews Fig. 3.1, 64–67, 69–70

Foundation Certificate see Certified Tester Foundation Level (CTFL)

FTP see fundamental test process (FTP)

functional requirements 50

functional specifications Figs. 2.1–2.2, 42, 49–50

functional testing 53, 127

fundamental test process (FTP) Figs. 1.4–1.5, 20–25

heuristic approaches 151

high-complexity measures 71

hybrid flow graphs Figs. 4.10–4.13, 111, 113

impact analysis 55

implementation

of test tools Fig. 6.6, 212–219

see also test execution

incident management 22, 165–167

incident management tools Fig. 6.3, Tab. 6.5, 177, 183–185

incident reports Fig. 5.4, Tab. 5.4, 166–167

incremental development models see iterative development models

independent testing Fig. 5.1, Tab. 5.1, 145–146

informal reviews 68–69

inspections 66, 69–70

tools for 187–188

instrumentation code 198

integration strategies

big-bang integration 47

bottom-up integration Fig. 2.5, 48–49

top-down integration Fig. 2.4, 47–48

integration testing Fig. 2.2, 42, 46–49, 150

interfaces

between tools 179

see also integration testing

International Software Testing Qualifications Board (ISTQB) 1

interoperability testing 53

‘invalid’ transitions 95

ISO/IEC/IEEE 29119 Software Testing 3–4

iteration structures 101, 102–103

decision testing 117–121

visual representations Fig. 4.6, Fig. 4.16, 107

iterative development models Fig. 2.3, 43–44

keyword-driven testing 194

lessons learned 24, 216

limited entry decision table 92

linear development models

V-model Fig. 2.2, 41–43

waterfall model Fig. 2.1, 39–40

load generators 202

load testing tools Tab. 6.5, 202–203

loops 101, 102–103

decision testing 117–121

visual representations Fig. 4.6, Fig. 4.16, 107

maintenance testing 54–55

management information 178, 183, 184

managers

review 67

test 28, 143, 148–149

master test plan Fig. 5.2, Tab. 5.2, 152, 153–155

maturity, organisational 212

menu structure models 53

methodical approaches 151

metrics

coverage 83

incident reports Fig. 5.4, Tab. 5.4, 166–167

reviews 66–67

static analysis 71

test progress Figs. 5.3–5.4, 160

test summary reports Tab. 5.3, 23, 163–164

from test tools 178, 183, 185

metrics-based estimation 157, 158

migration 55

model-based approaches 151

modelling tools Tab. 6.5, 189–190

moderators 65, 67, 69

monitoring see test progress monitoring

monitoring tools 203–204

non-executable statements 101, 105

non-functional failures 12

non-functional requirements 50

non-functional testing 53

non-valid inputs 83, 87–88

‘null’ transitions 95

open source tools 178, 213

operational acceptance testing 51–52, 150

organisational maturity 212

pair programming 69

Pareto principle 18

partitions 87–89

payback models, test tools Fig. 6.1, Fig. 6.4, 177, 196

performance models 53

performance testing tools Tab. 6.5, 202–203

personnel

reviews 64–65, 67, 69, 70

test managers 28, 143, 148–149

see also developers; testers

pilot projects 215

plain language specifications 53

planning see test planning

plans, test Fig. 5.2, Tab. 5.2, 152–155

preventative test approach 150

prioritisation 13

process-compliant approaches 151

process flows 53

process improvement 164, 166, 184

product risks 143–144

program specifications Figs. 2.1–2.2, 42, 46

program structures 101–103

control flow graphs Figs. 4.8–4.9, 107–109

decision testing Fig. 4.14, 117–121

flow charts Figs. 4.4–4.7, Fig. 4.10, 103–105, 111

see also structure-based techniques

progress data 160

progress monitoring Figs. 5.3–5.4, 159–160

project management 143, 148

project risks 142–143

project test plan see master test plan

proofs of concept 213–214

prototyping 44

pseudo code 100

psychology of testing 26–27

quality Fig. 1.2, 12

Rapid Application Development (RAD) 44

Rational Unified Process (RUP) 44

reactive test approach 150

record tools 193–194

regression-averse approaches 151

regression testing 19, 22–23, 25, 44, 54, 55

test tools for 193, 195, 196

regulation acceptance testing 52

reports

incident Fig. 5.4, Tab. 5.4, 166–167

test summary Tab. 5.3, 23, 163–164

from test tools 183

requirement specifications Figs. 2.1–2.2, 16–17, 42, 51

requirements

changes to 185

functional 50

non-functional 50

of test tools 212–213

requirements management tools Fig. 6.3, Tab. 6.5, 185–186

resources, triangle of Fig. 1.2, 12

response times 202

retesting 15, 22–23, 54, 55

review leaders 65, 67, 69

review process support tools Tab. 6.5, 187–188

reviewers 64–65, 67, 69, 70

reviews 17, 60, 62–63

formal Fig. 3.1, 64–67, 69–70

formality level Fig. 3.2, 63–64

objectives 64

process 63–67

roles and responsibilities 64–65, 67, 69, 70

success factors 70–71

tools for Tab. 6.5, 187–188

types Fig. 3.2, 68–70

rework 17, 66

risk 11, 19, 142

product risks 143–144

project risks 142–143

of test tools 177, 179

risk-based testing 144

risk management 144–145

root cause analysis 72

safety-critical systems 81, 126, 146, 152

test tools for 191, 198, 201

scribes 67, 69

scripts, test 80, 193–195

see also test design

Scrum 44

SDLC see software development life cycle (SDLC)

security testing 19

security testing tools Tab. 6.5, 199–200

security threat models 53

selection structures 101, 102

decision testing Fig. 4.14, 117–121

visual representations Fig. 4.5, Fig. 4.7, 107

sequence structures Fig. 4.4, 101–102

sequential development models see linear development models

simplified control flow graphs Figs. 4.15–4.18, 100, 121–125

site acceptance testing 51

software compilers 72–73, 189

software development life cycle (SDLC) 16–17, 37

costs of errors during Fig. 1.3, Tab. 1.1, 17

iterative models Fig. 2.3, 43–44

maintenance testing 54–55

test types 52–54

V-model Fig. 2.2, 41–43, 45

waterfall model Fig. 2.1, 39–40

see also test levels

software failure 8–10

software models 71

source code see code

specification-based techniques 84–86

boundary value analysis 89–91

decision table testing 91–94

equivalence partitioning 87–89

state transition testing Figs. 4.1–4.2, 94–98

use case testing Fig. 4.3, 98–99

specifications

functional Figs. 2.1–2.2, 42, 49–50

plain language 53

program Figs. 2.1–2.2, 42, 46

requirement Figs. 2.1–2.2, 16–17, 42, 51

technical Figs. 2.1–2.2, 42

test procedure 80

see also reviews

stand-up meetings 25

standard-compliant approaches 151

standards, testing 3–4

state tables (ST) Tab. 4.1, 95, 96–97

state transition models 53

state transition testing Figs. 4.1–4.2, 94–98

statement coverage 110–117

statement testing Figs. 4.10–4.13, 110–117

static analysis 71–73

static analysis tools Tab. 6.5, 72–73, 188–189, 191

static testing 14, 60

static analysis 71–73

test tools for Tab. 6.5, 72–73, 187–189, 191

see also reviews

stochastic testing 151

strategies

test 150–152

see also integration strategies

stress testing tools Tab. 6.5, 202–203

structural testing 53

structure-based techniques 84, 100, 126, 127

decision testing Fig. 4.14, 117–121

simplified control flow graphs Figs. 4.15–4.18, 100, 121–125

statement testing Figs. 4.10–4.13, 110–117

stubs 48, 196

system integration testing 49

system operators 150

system testing Fig. 2.2, 42, 46, 49–50, 150

technical reviews 69

technical specifications Figs. 2.1–2.2, 42

test analysis 21–22, 25

test analysts see testers

test approaches 150–152

test automation 54

test basis 45, 85, 190

test cases 21, 80, 190–191

see also test design

test charters 127

test closure activities 24, 25

test comparators Tab. 6.5, 177, 192–193

test conditions 21, 80

see also test design

test control 21, 24–25, 164–165

test coverage 83, 126, 127

decision 117–121

measurement tools Tab. 6.5, 198–199

statement 110–117

test data preparation tools Tab. 6.5, 178, 191–192

test design 21–22, 45, 77

in Agile development 25

choosing techniques 127–128

experience-based techniques 84, 85, 126–127

technique categories 84–85

test development process 79–83

see also specification-based techniques; structure-based techniques

test design tools Tab. 6.5, 190–191

test-driven development 43, 46

test environments

setting up 149

and test tools 179

test estimation 157–159

test execution 14, 20, 22–23

test execution schedules 83

test execution tools Fig. 6.3, Fig. 6.4, Tab. 6.5, 193–196

test executors see testers

test frames 190

test harnesses Fig. 6.5, Tab. 6.5, 178, 196–198

test implementation see test execution

test leaders 28, 143, 148–149

test-level plans Fig. 5.2, Tab. 5.2, 152–155

test levels 45–46

acceptance testing Fig. 2.2, 16–17, 42, 46, 51–52, 150

integration testing Fig. 2.2, 42, 46–49, 150

system testing Fig. 2.2, 42, 46, 49–50, 150

unit testing Fig. 2.2, 42, 46, 150

test management information 178, 183, 184

test management tools Fig. 6.3, Tab. 6.5, 182–183

test managers 28, 143, 148–149

test oracles Tab. 6.5, 178, 191

test planning Fig. 2.2, 21, 42, 152–157

activities 155–156

in Agile development 24–25

entry criteria 156–157

exit criteria 157

test plans Fig. 5.2, Tab. 5.2, 152–155

test procedure specifications 80

test process improvement 164, 166, 184

Test Process Improvement (TPI) assessment 212

test progress monitoring Figs. 5.3–5.4, 159–160

test reporting Tab. 5.3, 23, 163–164

test scripts 80, 193–195

see also test design

test strategies 150–152

test summary reports Tab. 5.3, 23, 163–164

test tasks 147–150

test tools 175

benefits 177, 178, 195–196

classification of 181–182

configuration management Fig. 6.3, Tab. 6.1, Tab. 6.5, 186–187, 188

costs Fig. 6.1, Fig. 6.4, 177, 196, 203, 214–215

coverage measurement Tab. 6.5, 198–199

data quality assessment 204

definition 177

dynamic analysis Tab. 6.5, 200–201

evaluating 213–214

implementation process Fig. 6.6, 212–219

incident management Fig. 6.3, Tab. 6.5, 177, 183–185

interfaces between 179

maintenance 179

modelling Tab. 6.5, 189–190

monitoring 203–204

open source 178, 213

other software tools 205–206

payback models Fig. 6.1, Fig. 6.4, 177, 196

performance testing Tab. 6.5, 202–203

proofs of concept 213–214

requirements management Fig. 6.3, Tab. 6.5, 185–186

review process support Tab. 6.5, 187–188

risks 177, 179

security testing Tab. 6.5, 199–200

static analysis Tab. 6.5, 72–73, 188–189, 191

success factors 215–216

test comparators Tab. 6.5, 177, 192–193

test data preparation Tab. 6.5, 178, 191–192

test design Tab. 6.5, 190–191

test execution Fig. 6.3, Fig. 6.4, Tab. 6.5, 193–196

test harnesses Fig. 6.5, Tab. 6.5, 178, 196–198

test management Fig. 6.3, Tab. 6.5, 182–183

test oracles Tab. 6.5, 178, 191

usability test 204–205

vendors 179, 213, 214–215

testers

code of ethics 27–28

developers as Tab. 5.1, 14, 26, 43–44, 46, 145

experience-based techniques 84, 85, 126–127

independent Fig. 5.1, Tab. 5.1, 145–146

tasks 149–150

use of test tools Tab. 6.5

see also reviewers; test managers

testing

early 16–18

exhaustive 11, 16

nature and purpose 14–15

principles 15–19

psychology of 26–27

stopping 13, 83

types 52–54

see also individual testing types

testing standards 3–4

testware, traceability 148, 168, 187

time-boxes 43

timescales see test estimation

tools see test tools

top-down integration Fig. 2.4, 47–48

traceability

configuration management Tab. 6.1, 148, 168, 187

in iterative development 44

requirements changes 185

in reviews 64, 187

of test cases 81

testware 148, 168, 187

transaction times 202

transitions 94–95

Unified Modeling Language (UML) 71, 189

unit test framework tools see test harnesses

unit test frameworks 46

unit testing Fig. 2.2, 42, 46, 150

usability models 53

usability test tools 204–205

usage patterns 202

use case testing Fig. 4.3, 98–99

user acceptance testing 51

user representatives 43, 51

user stories 44, 62

V-model Fig. 2.2, 41–43, 45

see also test levels

valid inputs 83, 87–88

validation 41

vendors, test tools 179, 213, 214–215

verification 41

walkthroughs 69

waterfall model Fig. 2.1, 39–40

web-based tools 178, 187

white-box techniques see structure-based techniques

Wide Band Delphi approach see expert-based estimation

work-products 16, 37, 40–41, 42

see also reviews

XML (Extensible Markup Language) 179