Index

A note on the digital index

A link in an index entry is displayed as the section title in which that entry appears. Because some sections have multiple index markers, it is not unusual for an entry to have several links to the same section. Clicking on any link will take you directly to the place in the text in which the marker appears.

A

Abort result (automated tests), Analysis
access (external) to bug tracking system, Attributes of a Bug Tracking System
accessibility testing, Eating Our Dogfood
(see also )
Accessible Event Watcher (AccEvent) tool, Testing for Accessibility
Accessible Explorer tool, Testing for Accessibility
accountability for quality, The Match Game
accuracy of test automation, To Automate or Not to Automate, That Is the Question
Act phase (PDCA cycle), Process Improvement
action simulation, in UI automation, User Interface Automation
active (bug status), Anatomy of a Bug Report
Active Accessibility software development kit, Testing for Accessibility
activities of code reviews, monitoring, Other Considerations
ad hoc approach, combinatorial testing, Combinatorial Analysis
ADDIE model, Process Improvement
“Adopt an App” program, Compatibility Testing
AFA (automatic failure analysis), Solving Tomorrow’s Problems Today
Agile methodologies, Spiral Model, Milestones
all states path (graph theory), Graph Theory and MBT
all transitions path (graph theory), Graph Theory and MBT
alpha release, defined, War, What Is It Good For?
analysis (test pattern attribute), Using Test Patterns
analysis phase (SEARCH test automation), What’s in a Test?, Execution
analysis tools (see )
analytical problem solving competency, I Need More Testers and I Need Them Now!
anticipating (see )
API testing with models, Models for API Testing, Grammar Models
application compatibility testing, Compatibility Testing
application libraries, Compatibility Testing, Tools, Tools, Everywhere
application usage data, collecting, Customers to the Rescue
Application Verifier tool, Application Libraries
approving big fixes (see )
asking questions, How Do You Measure Performance?
performance measurement, How Do You Measure Performance?
test design, Starting with Testing
assignment of bugs, Attributes of a Bug Tracking System
automated notification of, Attributes of a Bug Tracking System
limits on, How Not to Use the Data: Bugs as Performance Metrics
specified in bug reports, Anatomy of a Bug Report
assumptions about test case readers, The Value of a Test Case
audio features, accessibility of, Testing for Accessibility
authentication with WLID, Loosely Coupled vs. Tightly Coupled Services
Automate Everything attribute (BVTs), The Daily Build
automated service deployment, Release Frequency and Naming
automated tests, Anatomy of a Test Case
(see also )
as invalid testing solution, What’s in a Test?
test code analysis, Test Code Analysis
tracking changes in, Code Churn
automatic failure analysis (AFA), Solving Tomorrow’s Problems Today
automating models, Building a Finite State Model
automation (test case attribute), Anatomy of a Test Case
AutomationElement elements, User Interface Automation
awareness, interpersonal (competency), I Need More Testers and I Need Them Now!

B

backlog of bugs, Milestones
Ball, Tom, The IC Tester
Ballmer, Steve, Software Engineering at Microsoft
base choice (BC) approach, combinatorial testing, Combinatorial Testing Approaches
insufficiency of, Combinatorial Analysis in Practice
baseline performance, establishing, How Do You Measure Performance?
basic control flow diagrams (CFDs), Block Testing
basis path testing, Structural Testing Techniques
BATs (build acceptance tests), The Daily Build
Bayesian Graphical Modeling (BGM), Grammar Models
BC (base choice) approach, combinatorial testing, Combinatorial Testing Approaches
insufficiency of, Combinatorial Analysis in Practice
behavioral testing (see )
Beizer, Boris, The Need for Functional Testing
Bergman, Mark, What’s in a Test?
best guess approach, combinatorial testing, Combinatorial Analysis
beta release, War, What Is It Good For?
defined, War, What Is It Good For?
identifying product as, Stateful Services
BGM (Bayesian Graphical Modeling), Grammar Models
bias in white box testing, Structural Testing Techniques
Big Challenges (company value), The Microsoft Vision, Values and Why We “Love This Company!”
big company, Microsoft as, The Microsoft Vision, Values and Why We “Love This Company!”
working small in a big company, The Shared Team Model
big-picture performance considerations, How Do You Measure Performance?
Binder, Robert, I Need More Testers and I Need Them Now!, Practicing Good Software Design and Test Design
BIOS clocks, Analyzing Parameter Subsets
black box testing, Black Box, White Box, and Gray Box
Block result (automated tests), Analysis, Reporting
block testing, Structural Testing Techniques
Boehm, Barry, Waterfall Model
bottlenecks, anticipating, How Do You Measure Performance?
(see also )
boundary condition tests, Using the Data
boundary value analysis (BVA), Summary of Equivalence Class Partitioning
defining boundary values, Summary of Equivalence Class Partitioning
hidden boundaries, A New Formula for Boundary Value Analysis
using ECP tables with, The ECP Tests, Defining Boundary Tests
Boundary Value Analysis Test Pattern (example), Using Test Patterns
branching control flow, testing (see )
breadth of Microsoft portfolio, The Microsoft Vision, Values and Why We “Love This Company!”
breaking builds, The Daily Build
broken window theory, Native Code Analysis
browser-based service performance metrics, Production Data, so Tempting but Risky Too
brute-force UI automation, User Interface Automation
buckets for errors, Corporate Error Reporting, Continuous Quality Improvement Program
bug backlog, Milestones
bug bars (limits on bugs assigned), How Not to Use the Data: Bugs as Performance Metrics
bug lifecycle, Bug Tracking
bug metrics, Common Mistakes in Bug Reports
churn metrics and, Code Churn
bug morphing, Common Mistakes in Bug Reports
bug notification system, Attributes of a Bug Tracking System
bug reports, The Bug Workflow, Why Write a Bug Report?
examples of, A Bug’s Life
bug type (bug report field), Anatomy of a Bug Report
bug workflow, The Bug Workflow
bugs (see )
bugs, specific, Compatibility Testing
Friday the 13th bug, Analyzing Parameter Subsets
love bug, Bug Bars
milk carton bug, Bug Bars
tsunami effect, Common Bugs I’ve Seen Missed
USB cart of death, Stress Testing
“Adopt an App” program, Compatibility Testing
“disallow server cascading” failure, Common Bugs I’ve Seen Missed
bugs vs. features, Bug Triage
build acceptance tests (BATs), The Daily Build
build labs, The Daily Build
build process, A Home for Source Control
breaking builds, The Daily Build
testing daily builds, Test Machine Configuration Savings
build verification tests (BVTs), The Daily Build
building services on servers, Designing the Right S+S and Services Test Approach
built-in accessibility features, Eating Our Dogfood
BUMs (business unit managers), Working Small in a Big Company
Business Division (see )
business goals, in test time estimation, Starting with Testing
business unit managers (see BUMs)
BVA (boundary value analysis), Summary of Equivalence Class Partitioning
defining boundary values, Summary of Equivalence Class Partitioning
hidden boundaries, A New Formula for Boundary Value Analysis
using ECP tables with, The ECP Tests, Defining Boundary Tests
BVA Test Pattern (example), Using Test Patterns
BVTs (build verification tests), The Daily Build
by design (bug resolution value), Anatomy of a Bug Report, Bug Triage

C

C# development, Execution
calculating code complexity (see )
call center data, collecting, Continuous Quality Improvement Program
campus recruiting, I Need More Testers and I Need Them Now!
capacity testing, Stress Testing
Carbon Allocation Model (CarBAM), Power Is the Bottleneck to Growth
career stages, The Engineering Career at Microsoft
careers at Microsoft, The Engineering Career at Microsoft
recruiting (see )
in Test, The Engineering Career at Microsoft
Catlett, David, Thinking About Testability
CBO metric, Object-Oriented Metrics
CEIP (Customer Experience Improvement Plan), Quality Perception
CER (Corporate Error Reporting), Corporate Error Reporting
CFDs (control flow diagrams), Block Testing
chair of test leadership team, Anatomy of a Log File
changes in code, Other Tools
number of (code churn), Other Tools
tracking (see )
Check phase (PDCA cycle), Process Improvement
check-in systems, Stopping the Breaks
checklists for code reviews, Informal Reviews
churn, Other Tools
CIS (Cloud Infrastructure Services), Shifting to Internet Services as the Focus
CK metrics, Halstead Metrics
class-based complexity metrics, Halstead Metrics
classic Microsoft bugs, Bug Bars
ClassSetup attribute, Execution
ClassTeardown attribute, Execution
clean machines, How Do You Measure Performance?
cleanup phase (SEARCH test automation), What’s in a Test?, Reporting
closed (bug status), Anatomy of a Bug Report
cloud services, Shifting to Internet Services as the Focus
CLR (Common Language Runtime), A Language and Engine
CMMI (Capability Maturity Model Integrated), Formal Process Improvement Systems at Microsoft
code analysis (see static analysis)
code churn, Other Tools
code complexity, Common Automation Mistakes
in automated tests, Common Automation Mistakes
cyclomatic complexity, Basis Path Testing
measuring, Using Test Patterns, Measuring Cyclomatic Complexity, Object-Oriented Metrics
estimating (code smell), A Complex Problem
estimating test time, Starting with Testing
Halstead metrics, Measuring Cyclomatic Complexity
how to use metrics for, What to Do with Complexity Metrics
lines of code (LOC), A Complex Problem
object-oriented metrics, Halstead Metrics
quantifying, Risky Business
risk from, Analyzing Risk with Code Complexity
code coverage, Test Code Analysis
analysis tools for, Test Code Analysis
behavioral and exploratory testing, Structural Testing Techniques
combinatorial analysis and, Effectiveness of Combinatorial Analysis
with functional testing, The Need for Functional Testing
milestone criteria, Milestones
quality gates, Getting to Done
scripted tests, Structural Testing Techniques
statement vs. block coverage, Structural Testing Techniques
code reuse, Tools, Tools, Everywhere
code reviews, Snapshots
collateral data with, Time Is on My Side
measuring effectiveness of, Informal Reviews
code smell, A Complex Problem
code snapshots, Code Churn, Test Machine Configuration Savings, Export and Import
CodeBox portal, What’s the Problem?
Cole, David, Compatibility Testing
collaboration (see )
collecting data from customers (see )
combination tests, Combinatorial Testing Approaches
combinatorial analysis, Combinatorial Analysis in Practice
effectiveness of, Combinatorial Analysis in Practice
tools for, in practice, Combinatorial Testing Approaches
communication, Code Reviews and Inspections
cross-boundary collaboration (competency), I Need More Testers and I Need Them Now!
company values at Microsoft, The Microsoft Vision, Values and Why We “Love This Company!”
comparing documents, What Changed?
compatibility testing, Compatibility Testing
dogfooding (being users), Eating Our Dogfood, Testing Against Production
competencies, Testers at Microsoft Have Not Always Been SDETs
compilation errors, The Daily Build, Breaking the Build
complexity of code, Common Automation Mistakes
in automated tests, Common Automation Mistakes
cyclomatic complexity, Basis Path Testing
measuring, Using Test Patterns, Measuring Cyclomatic Complexity, Object-Oriented Metrics
estimating (code smell), A Complex Problem
estimating test time, Starting with Testing
Halstead metrics, Measuring Cyclomatic Complexity
how to use metrics for, What to Do with Complexity Metrics
lines of code (LOC), A Complex Problem
object-oriented metrics, Halstead Metrics
quantifying, Risky Business
risk from, Analyzing Risk with Code Complexity
complexity of test automation, To Automate or Not to Automate, That Is the Question
compound conditional clauses, testing, Summary of Decision Testing
compressibility (metric), Performance Test Metrics for Services, Page Load Time 1 and Page Load Time 2
computer-assisted testing, What’s in a Test?
computing innovations, waves of, Moving from Stand-Alone to Layered Services
condition testing, Summary of Decision Testing
conditional clauses, testing (see , )
conditions (test case attribute), The Value of a Test Case
confidence (competency), I Need More Testers and I Need Them Now!
with functional testing, The Need for Functional Testing
configurability of bug tracking system, Attributes of a Bug Tracking System
configuration data, collecting, Customers to the Rescue
configurations (test case attribute), The Value of a Test Case
conformance, in test time estimation, Starting with Testing
Connect site, Send a Smile Impact
container-based datacenters (container SKUs), Growing from Large Scale to Mega Scale
Content discipline, The Engineering Disciplines
context, bug, Anatomy of a Bug Report
continuous improvement (see )
contrast, display, Testing for Accessibility
control flow diagrams (CFDs), Block Testing
control flow graphs, Measuring Cyclomatic Complexity
control flow modeling, Structural Testing Techniques, Block Testing
control flow testing (see )
control testability, Thinking About Testability
Corporate Error Reporting (CER), Corporate Error Reporting
cost of quality, Good Logging Practices
cost of test automation, The Value of Automation
count of changes (churn metric), Other Tools
counters (performance), How Do You Measure Performance?
counting bugs, Common Mistakes in Bug Reports, How Not to Use the Data: Bugs as Performance Metrics
counting test cases, Test Case Mistakes, Tracking and Interpreting the Test Results
coupling, services, Platform Services vs. Top-Level Services, Integration Testing vs. Test Flags and emulation
coupling between object classes (metric), Object-Oriented Metrics
coverage method (WER), Emptying the Buckets
Creative discipline, The Engineering Disciplines
credit card processing, Loosely Coupled vs. Tightly Coupled Services
criteria for milestones, Milestones
quality gates as, Getting to Done
Critical attribute (BVTs), The Daily Build
cross-boundary collaboration (competency), I Need More Testers and I Need Them Now!
cross-referencing test cases with automation scripts, Putting It All Together
culture of quality, Overcoming Analysis Paralysis
Customer Experience Improvement Plan (CEIP), Quality Perception
customer feedback systems, Customer Feedback Systems
connecting with customers, Send a Smile Impact
emotional response, Test and WER
for services, Continuous Quality Improvement Program
testing vs. quality, Customer Feedback Systems
watching customers (see CEIP (Customer Experience Improvement Plan))
Windows Error Reporting, Games, Too!
customer impact of bugs, Anatomy of a Bug Report
customer-driven testing, Customers to the Rescue
customer-focused innovation, I Need More Testers and I Need Them Now!
Cutter, David, The Engineering Career at Microsoft
cyclomatic complexity, Basis Path Testing
measuring, Measuring Cyclomatic Complexity, Object-Oriented Metrics
practical interpretations, Using Test Patterns
Czerwonka, Jacek, Combinatorial Analysis in Practice

D

daily builds, A Home for Source Control
testing with virtualization, Test Machine Configuration Savings
data coverage (see , )
data sanitization, Production Data, so Tempting but Risky Too
DDE (defect detection effectiveness), Combinatorial Analysis in Practice
debug/checked builds, War, What Is It Good For?
Debuggable and Maintainable attribute (BVTs), The Daily Build
debuggers, exploratory testing with, Ask Questions
debugging after automated tests, Common Automation Mistakes
debugging scope, What Changed?
decision testing, Summary of Block Testing
(see also )
decisions in programs, counting (see , )
decomposing variable data (ECP), Equivalence Class Partitioning
dedicated teams for non-functional testing, Testing the “ilities”
defect detection effectiveness (DDE), Combinatorial Analysis in Practice
defect removal efficiency (DRE), Attributes of a Bug Tracking System
Deming, W. Edwards, Process Improvement
dependencies, services, Platform Services vs. Top-Level Services
dependency errors, Breaking the Build
deploying services with automation, Release Frequency and Naming
deployment test clusters (services), The Integrated Services Test Environment, The Deployment Test Cluster
depth of inheritance tree (metric), Halstead Metrics
descriptions for test patterns, Using Test Patterns
descriptions of bugs (in bug reports), Why Write a Bug Report?
design (test pattern attribute), Using Test Patterns
design, importance of, A Practical Approach to Test Case Design
design patterns, Practicing Good Software Design and Test Design, Test and WER
designing models, Modeling Basics
finite state models, Building a Finite State Model
designing test cases, A Practical Approach to Test Case Design
best practices, A Practical Approach to Test Case Design
estimating test time, Using Test Patterns
getting started, Starting with Testing
practical considerations, Testing the Good and the Bad
testability, Thinking About Testability
testing good and bad, Example Elements in a Test Design Specification
using test patterns, Practicing Good Software Design and Test Design
DeVaan, Jon, The Engineering Career at Microsoft
Development (SDE) discipline, The Engineering Disciplines
development models, Software Engineering at Microsoft
devices (see )
d�j� vu heuristic, Hidden Boundaries
Difficulty metric (Halstead), Measuring Cyclomatic Complexity
diff utilities, What Changed?
Director, Software Development Engineer in Test title, Becoming a Manager is Not a Promotion
Director of Test, Becoming a Manager is Not a Promotion
Director of Test Excellence, Informal Reviews
“disallow server cascading” failure, Common Bugs I’ve Seen Missed
disciplines, product engineering, The Engineering Disciplines, Software Test Engineers at Microsoft
display contrast (accessibility), Testing for Accessibility
distributed stress testing, Stress Testing
DIT metric, Halstead Metrics
diversity of Microsoft portfolio, The Microsoft Vision, Values and Why We “Love This Company!”
divisions at Microsoft, The Microsoft Vision, Values and Why We “Love This Company!”
DMAIC model, Process Improvement
Do phase (PDCA cycle), Process Improvement
document comparison tools, What Changed?
documentation of test cases (see entries at log; )
documenting code changes (see source control)
dogfooding, Eating Our Dogfood, Testing Against Production
“done,” defining, Getting to Done
down-level browser experience, Services vs. Packaged Product
DRE (defect removal efficiency), Attributes of a Bug Tracking System
Drotter, Stephen, The Engineering Career at Microsoft
duplicate (bug resolution value), Anatomy of a Bug Report
duplicate bugs, Common Mistakes in Bug Reports

E

E&D (Entertainment and Devices Division), The Microsoft Vision, Values and Why We “Love This Company!”
e-mail discussions in bug reports, Common Mistakes in Bug Reports
each choice (EC) approach, combinatorial testing, Combinatorial Testing Approaches
ease of use, bug tracking system, A Bug’s Life
“eating our dogfood”, Eating Our Dogfood, Testing Against Production
ECP (equivalence class partitioning), The Need for Functional Testing
analyzing parameter subsets, Analyzing Parameter Subsets
boundary condition tests, The ECP Tests
boundary value analysis with, The ECP Tests, Defining Boundary Tests
decomposing variable data, Equivalence Class Partitioning
example of, Equivalence Class Partitioning in Action
edges (control node graphs), Measuring Cyclomatic Complexity
education strategy, Ask Questions, Thinking About Testability
EE (Engineering Excellence) group, Learning How to Be a Microsoft SDET
effectiveness of code reviews, measuring, Informal Reviews
effort of test automation, determining, The Value of Automation
80:20 rule, Analyzing Risk with Code Complexity
Elop, Stephen, The Microsoft Vision, Values and Why We “Love This Company!”
emotional response from customers, Test and WER
employee orientation, Learning How to Be a Microsoft SDET
emulating services, Integration Testing vs. Test Flags and emulation
ending state (model), Model-Based Testing
engineering career at Microsoft, The Engineering Career at Microsoft
engineering disciplines, The Engineering Disciplines, Software Test Engineers at Microsoft
Engineering Excellence (EE) group, Learning How to Be a Microsoft SDET
Engineering Excellence (EE) team, Export and Import
Engineering Excellence Forum, Snapshots, Informal Reviews
engineering life cycles, Engineering Life Cycles
Agile methodologies, Spiral Model, Milestones
milestones, Agile Methodologies
process improvement, Process Improvement
formal systems for, Process Improvement
shipping software, Shipping Software from the War Room
software development models, Software Engineering at Microsoft
Engineering Management discipline, The Engineering Disciplines
engineering organizational models, Microsoft Is a Big Software Engineering Company
engineering workforce (Microsoft), size of, Testers at Microsoft Have Not Always Been SDETs
campus recruiting, I Need More Testers and I Need Them Now!
engineers, types of, Working Small in a Big Company
Entertainment and Devices Division (see )
environment, bug, Anatomy of a Bug Report
environmental sensitivity of automated tests, Test Code Analysis
equivalence class partitioning (ECP), The Need for Functional Testing
analyzing parameter subsets, Analyzing Parameter Subsets
boundary condition tests, The ECP Tests
boundary value analysis with, The ECP Tests, Defining Boundary Tests
decomposing variable data, Equivalence Class Partitioning
example of, Equivalence Class Partitioning in Action
estimation of test time, Using Test Patterns
ET (see )
Euler, Leonhard, Building a Finite State Model
examples for test patterns, Using Test Patterns
exception handling, block testing for, Block Testing
execution phase (SEARCH test automation), What’s in a Test?, Setup
exit criteria for milestones, Milestones
quality gates as, Getting to Done
expected practices, Mandatory Practices
experience quality, Testing Provides Information
expiration date set (metric), Performance Test Metrics for Services, Compressibility
exploratory testing, Structural Testing Techniques
exploratory testing (ET), Starting with Testing, Black Box, White Box, and Gray Box, Functional Testing Techniques
exporting virtual machines, Export and Import
external user access, bug tracking system, Attributes of a Bug Tracking System

F

facilitating testing, team for, Snapshots
Fagan inspections, Code Reviews and Inspections
Fail Perfectly attribute (BVTs), The Daily Build
Fail result (automated tests), Analysis
failure analysis, automatic, Solving Tomorrow’s Problems Today
failure count (test case metric), Tracking and Interpreting the Test Results
failure criteria in test cases, Anatomy of a Test Case
failure databases, The Match Game
failure matching, The Match Game
false alarms (metric), Continuous Quality Improvement Program
false negatives, with automated testing, To Automate or Not to Automate, That Is the Question
false positives, What to Do with Complexity Metrics, Common Automation Mistakes
with automated testing, To Automate or Not to Automate, That Is the Question
falsification tests, Example Elements in a Test Design Specification
fan-in and fan-out measurements, Object-Oriented Metrics
fast rollbacks with services, Fully Automated Deployments
feature area, bug, Anatomy of a Bug Report
feature crews (Agile methodologies), Milestones
features, Bug Triage
bugs vs., Bug Triage
in milestone criteria, Milestones
of services, Stateful Services
feedback systems (see )
Fiddler tool, Performance Test Metrics for Services, Compressibility
field replaceable units (FRUs), Growing from Large Scale to Mega Scale
film industry, product development as, The Shared Team Model
finite state machines (FSMs), Modeling Basics
finite state models, building, Building a Finite State Model
fix if time (bug priority), Bug Triage
fix number method (WER), Emptying the Buckets
fixed (bug resolution value), Anatomy of a Bug Report
fixed-constant values, Boundary Value Analysis
fixed-variable values, Boundary Value Analysis
font size (accessibility), Testing for Accessibility
forgotten steps in test cases, Anatomy of a Test Case
formal code reviews, Code Reviews and Inspections
forums, Microsoft, Send a Smile Impact
forward thinking in testing, Solving Tomorrow’s Problems Today
foundation (platform) services, Platform Services vs. Top-Level Services
frequency of release, services, Stateful Services
frequency of testing
performance testing, How Do You Measure Performance?
as test case attribute, The Value of a Test Case
frequency of testing, How Do You Measure Performance?
Friday the 13th bug, Analyzing Parameter Subsets
Frink, Lloyd, Software Test Engineers at Microsoft
frowny icon (Send a Smile), Smile and Microsoft Smiles with You
FRUs (field replaceable units), Growing from Large Scale to Mega Scale
FSMs (finite state machines), Modeling Basics
full automated service deployments, Release Frequency and Naming
fully automated tests (see )
functional testing, Functional Testing Techniques
boundary value analysis (BVA), Summary of Equivalence Class Partitioning
defining boundary values, Summary of Equivalence Class Partitioning
hidden boundaries, A New Formula for Boundary Value Analysis
using ECP tables with, The ECP Tests, Defining Boundary Tests
combinatorial analysis
effectiveness of, Combinatorial Analysis in Practice
tools for, in practice, Combinatorial Testing Approaches
equivalence class partitioning (ECP), The Need for Functional Testing
analyzing parameter subsets, Analyzing Parameter Subsets
boundary condition tests, The ECP Tests
boundary value analysis with, The ECP Tests, Defining Boundary Tests
decomposing variable data, Equivalence Class Partitioning
example of, Equivalence Class Partitioning in Action
need for, Functional Testing Techniques
vs. non-functional testing, Non-Functional Testing
structural testing vs., Structural Testing Techniques
functions, testing (see structural testing)
future of testing, Solving Tomorrow’s Problems Today
fuzz testing, Threat Modeling
fuzzy matching, To Automate or Not to Automate, That Is the Question
FxCop utility, Managed Code Analysis

G

game data, collecting, Customers to the Rescue
gatekeeper (check-in system), Stopping the Breaks
Gates, Bill, The Microsoft Vision, Values and Why We “Love This Company!”, The Shared Team Model, Software Test Engineers at Microsoft, The Microsoft Services Strategy
gauntlet (check-in system), Stopping the Breaks
General Manager of Test, Becoming a Manager is Not a Promotion
George, Grant, Testers at Microsoft Have Not Always Been SDETs
getting to done (Agile methodologies), Getting to Done
gimmicks, test techniques as, The Need for Functional Testing
glass box testing, Black Box, White Box, and Gray Box
global company, Microsoft as, The Engineering Disciplines
goals, Milestones
for milestones, Milestones, Getting to Done
for performance testing, How Do You Measure Performance?
for usability testing, Usability Labs
grammar models, Random Models
graph theory, Building a Finite State Model
gray box testing, Black Box, White Box, and Gray Box
Group Test Managers, Becoming a Manager is Not a Promotion
grouping bugs in bug reports, Common Mistakes in Bug Reports
groups of variables in ECP, Decomposing Variable Data

H

Halo 2 game, Customers to the Rescue
Halo 3 game, Microsoft Is a Big Software Engineering Company
Halstead metrics, Measuring Cyclomatic Complexity
happy path, testing, Example Elements in a Test Design Specification
hard-coded paths in tests, Common Automation Mistakes
hardware, Accessibility Testing
accessible technology tools, Accessibility Testing
device simulation framework, Setup
USB cart of death, Stress Testing
help phase (SEARCH test automation), What’s in a Test?, Cleanup
helping testers, team for, Snapshots
heuristics for equivalence class partitioning, Decomposing Variable Data
hidden boundary conditions, A New Formula for Boundary Value Analysis, Basis Path Testing
high-contrast mode, Testing for Accessibility
hiring testers at Microsoft, Testers at Microsoft Have Not Always Been SDETs
campus recruiting, I Need More Testers and I Need Them Now!
learning to be SDETs, Learning How to Be a Microsoft SDET
historical data, in test time estimation, Using Test Patterns
historical reference, test case as, The Value of a Test Case
hotfixes, Object-Oriented Metrics
how found (bug report field), Anatomy of a Bug Report
humorous bugs, Bug Bars
Hutcheson, Marnie, Structural Testing Techniques
Hyper-V, Daily Build Testing

I

IAccessible interface, User Interface Automation
IC Testers, The Test Architect
ICs (individual contributors), The Engineering Career at Microsoft
identification number validation, Basis Path Testing
ilities, list of, Beyond Functionality
(see also )
imaging technology, Setup
impact, The Microsoft Vision, Values and Why We “Love This Company!”
impact (competency), I Need More Testers and I Need Them Now!, Campus Recruiting
impact of bug on customer, Anatomy of a Bug Report
importing virtual machines, Export and Import
incubation, The Shared Team Model
industry recruiting, Campus Recruiting
influence (competency), I Need More Testers and I Need Them Now!, Campus Recruiting
informal code reviews, Code Reviews and Inspections
initial build process, about, The Daily Build
initiation phase (stress testing), Distributed Stress Testing
innovation, I Need More Testers and I Need Them Now!
customer focus, I Need More Testers and I Need Them Now!
(see also )
incubation, The Shared Team Model
in testing, Other Considerations
innovation in PC computing, Moving from Stand-Alone to Layered Services
inputs (test cases), The Value of a Test Case
fuzz testing, Threat Modeling
Inspect Objects tool, Testing for Accessibility
installation testing, Setup
INT environment, The Perf and Scale Cluster, The Deployment Test Cluster
integrated services test environment, The Perf and Scale Cluster, The Deployment Test Cluster
integration testing (services), Integration Testing vs. Test Flags and emulation
interactions within systems, Campus Recruiting
International Project Engineering (IPE), The Engineering Disciplines
internationalization, The Engineering Disciplines
Internet memo, The Microsoft Services Strategy
Internet services as Microsoft focus, The Microsoft Services Strategy
Internet Services Business Unit (ISBU), Working Small in a Big Company
interoperability of bug tracking system, Attributes of a Bug Tracking System
interpersonal awareness (competency), I Need More Testers and I Need Them Now!
interpreting test case results, Tracking and Interpreting the Test Results
interviewing for tester positions, I Need More Testers and I Need Them Now!
introduction (test strategy attribute), Ask Questions
invalid class data (ECP), Decomposing Variable Data
involvement with test automation, The Value of Automation
IPE (International Project Engineering), The Engineering Disciplines
ISBU (Internet Services Business Unit), Working Small in a Big Company
ISO 9000 program, Formal Process Improvement Systems at Microsoft
issue type (bug report field), Anatomy of a Bug Report
iterations, Agile methodologies, Getting to Done

J

jargon in test cases, Anatomy of a Test Case
JIT debuggers, The Stress Client
job titles for software test engineers, What’s in a Name?
moving from SDEs to SDETs, What’s in a Name?
SDET IC, The Test Architect
Test Architect, The Engineering Career at Microsoft, Test Lab Savings
in test management, The IC Tester
Test Manager, Becoming a Manager is Not a Promotion
Jorgensen’s formula, Defining Boundary Tests
Juran, Joseph, Overcoming Analysis Paralysis
just-in-time (JIT) debuggers, The Stress Client

K

key scenario (test strategy attributes), Ask Questions
keyboard accessibility (see )
keystrokes, simulating, User Interface Automation
K�nigsberg problem, Building a Finite State Model
knowledge testability, Thinking About Testability

L

large-scale test automation, Putting It All Together
Last Known Good (LKG) release, War, What Is It Good For?
layered services, S+S Example, Built on a Server
Lead Software Development Engineering in Test title, Becoming a Manager is Not a Promotion
leadership, Anatomy of a Log File
Leads (see )
Lean program, Formal Process Improvement Systems at Microsoft
learning how to be SDETs, Learning How to Be a Microsoft SDET
legacy client bugs, Continuous Quality Improvement Program
legal defense, bug reports as, Why Write a Bug Report?
Length metric (Halstead), Measuring Cyclomatic Complexity
length of program (lines of code), A Complex Problem
libraries, Test Code Is Product Code
libraries of applications for compatibility testing, Compatibility Testing, Tools, Tools, Everywhere
lifecycle, bugs, Bug Tracking
lifetime of automated tests, The Value of Automation
limitations of test patterns, Using Test Patterns
line metrics (churn metrics), Other Tools
linearly independent basic paths, Basis Path Testing
lines of code (LOC), A Complex Problem
Live Mail service, Microsoft Is a Big Software Engineering Company, Services vs. Packaged Product
Live Mesh, Shifting to Internet Services as the Focus
LKG (Last Known Good) release, War, What Is It Good For?
load tests, Testing the “ilities”, Stress Testing
(see also , )
Office Online, Test Flags
LOC (lines of code), A Complex Problem
Localization (IPE) discipline, The Engineering Disciplines
log file parsers, Reporting, Anatomy of a Log File
log files generated with test automation, Execution, Reporting
using for failure matching, The Match Game
long-haul tests, Testing the “ilities”
look-and-feel testing, Structural Testing Techniques, Structural Testing Techniques
loop structures, A New Formula for Boundary Value Analysis
boundary testing of, A New Formula for Boundary Value Analysis, Hidden Boundaries
structural testing of, Decision Testing
loosely coupled services, Platform Services vs. Top-Level Services, Integration Testing vs. Test Flags and emulation
love bug, Bug Bars
low-resource testing, Stress Testing
Luhn formula, Basis Path Testing

M

machine roles, The One Box
machine virtualization, Machine Virtualization
managing failures during tests, Network Topology Testing
test scenarios, Test Machine Configuration Savings, Snapshots
maintainability testing
testability, Thinking About Testability
maintainability of source code, complexity and, High Cyclomatic Complexity Doesn’t Necessarily Mean “Buggy”
maintainability testing, Beyond Functionality
managed code analysis, Managed Code Analysis
managed code test attributes, Execution
management career paths, The Engineering Career at Microsoft
test management, The IC Tester
managing bugs, Managing Bugs and Test Cases
attributes of tracking systems, A Bug’s Life
bug bars (limits on bugs assigned), How Not to Use the Data: Bugs as Performance Metrics
bug lifecycle, Bug Tracking
bug reports, Why Write a Bug Report?
common mistakes in, Bug Triage
using data effectively, Common Mistakes in Bug Reports
bug workflow, The Bug Workflow
classic Microsoft bugs, Bug Bars
false positives, What to Do with Complexity Metrics, Common Automation Mistakes
with automated testing, To Automate or Not to Automate, That Is the Question
triage, Anatomy of a Bug Report
managing test cases (see )
mandatory practices, Mandatory Practices
manual testing, Black Box, White Box, and Gray Box, Anatomy of a Test Case
(see also )
mashups, Moving from Stand-Alone to Layered Services, Emulation
matching failures, The Match Game
MBD (Microsoft Business Division), The Microsoft Vision, Values and Why We “Love This Company!”
MBT (see )
McCabe, Thomas, Basis Path Testing, Measuring Cyclomatic Complexity
mean time between failure (MTBF) testing, Stress Testing
measuring code complexity (see code complexity)
measuring performance, How Do You Measure Performance?
memory usage attribute (stress tests), The Stress Server
message loops, High Cyclomatic Complexity Doesn’t Necessarily Mean “Buggy”
metrics, Common Mistakes in Bug Reports
on bugs, Common Mistakes in Bug Reports
as performance metrics, How Not to Use the Data: Bugs as Performance Metrics
quota on finding, How Not to Use the Data: Bugs as Performance Metrics
for code churn, Other Tools
code complexity
Halstead metrics, Measuring Cyclomatic Complexity
how to use, What to Do with Complexity Metrics
object-oriented metrics, Halstead Metrics
on defect detection (see )
on emotional response, Smile and Microsoft Smiles with You
for performance, How Do You Measure Performance?
services, Production Data, so Tempting but Risky Too
on quality, Customers to the Rescue
for quality of services (QoS), Continuous Quality Improvement Program
smoke alarm metrics, Object-Oriented Metrics
on test cases, Tracking and Interpreting the Test Results
Microsoft Active Accessibility (MSAA), User Interface Automation
Microsoft Application Verifier tool, Application Libraries
Microsoft CIS (Cloud Infrastructure Services), Shifting to Internet Services as the Focus
Microsoft Connect, Send a Smile Impact
Microsoft Office, about, Microsoft Is a Big Software Engineering Company
Microsoft Office Online, Loosely Coupled vs. Tightly Coupled Services
Microsoft OneNote customer connections, Connecting with Customers
Microsoft Passport parental controlled, Loosely Coupled vs. Tightly Coupled Services
Microsoft Surface, Working Small in a Big Company
Microsoft Test Leadership Team (MSTLT), Anatomy of a Log File, Other Considerations
Microsoft Tester Center, Code Reviews and Inspections
Microsoft UI Automation framework, User Interface Automation
Microsoft Visual Studio 2008, Spec Explorer for, Modeling with Spec Explorer
Microsoft Visual Studio Team Foundation Server (TFS), Eating Our Dogfood
microsoft.public.* newsgroups, Send a Smile Impact
milestones, Agile Methodologies
in Agile methodologies, Getting to Done
quality milestone, Milestones
milk carton bug, Bug Bars
missing steps in test cases, Anatomy of a Test Case
mission statements, Microsoft, The Microsoft Vision, Values and Why We “Love This Company!”
mistakes in test cases, Anatomy of a Test Case
mixed mode service upgrades, Test Environments
mod 10 checksum algorithm, Basis Path Testing
model-based testing (MBT), Model-Based Testing
basics of modeling, Model-Based Testing
testing with model, Modeling Basics
automating models, Building a Finite State Model
designing models, Modeling Basics
finite state models, Building a Finite State Model
tips for modeling, A Language and Engine
tools for, Model-Based Testing Tools at Microsoft
modeling control flow, Structural Testing Techniques, Block Testing
modeling threats, Threat Modeling
modeling without testing, Grammar Models, A Language and Engine
models for engineering workforce, Microsoft Is a Big Software Engineering Company
models for software development, Software Engineering at Microsoft
monitoring code changes (see source control)
monitoring code changes (churn), Other Tools
monitoring code review effectiveness, Informal Reviews
monitoring performance, How Do You Measure Performance?
monkey testing, Random Models
mouse clicks, simulating, User Interface Automation
mouse target size (accessibility), Testing for Accessibility
movie industry, product development as, The Shared Team Model
moving quality upstream, Overcoming Analysis Paralysis
MQ (quality milestone), Milestones
services, Round Trip Analysis
MSAA (Microsoft Active Accessibility), User Interface Automation
MsaaVerify tool, Testing for Accessibility
MSN (Microsoft Network), Shifting to Internet Services as the Focus
MSTLT (Microsoft Test Leadership Team), Anatomy of a Log File, Other Considerations
MTBF testing, Stress Testing
Muir, Marrin, Testers at Microsoft Have Not Always Been SDETs
multiclient stress tests, The Stress Server
multiple bugs in single report, Common Mistakes in Bug Reports
must fix (bug priority), Bug Triage
Myers, Glenford, Decomposing Variable Data

N

n-wise testing, Combinatorial Testing Approaches
effectiveness of, Combinatorial Analysis in Practice
in practice, Combinatorial Testing Approaches
sufficiency of, Combinatorial Analysis in Practice
names for software test positions, What’s in a Name?
moving from SDEs to SDETs, What’s in a Name?
SDET IC, The Test Architect
Test Architect, The Engineering Career at Microsoft, Test Lab Savings
in test management, The IC Tester
Test Manager, Becoming a Manager is Not a Promotion
names for test patterns, Using Test Patterns
naming service releases, Stateful Services
native code analysis, Static Analysis
negative testing, Combinatorial Analysis in Practice
NEO (New Employee Orientation), Learning How to Be a Microsoft SDET
Net Promoter score, Continuous Quality Improvement Program
network topology testing, Daily Build Testing
new employee orientation (NEO), Learning How to Be a Microsoft SDET
newsgroups, Microsoft, Send a Smile Impact
no-known failure attribute (stress tests), The Stress Server
nodes (control flow graphs), Measuring Cyclomatic Complexity
non-functional (behavioral) testing, Structural Testing Techniques, Non-Functional Testing
automating, To Automate or Not to Automate, That Is the Question
non-functional testing, Eating Our Dogfood
accessibility testing, Eating Our Dogfood
compatibility testing, Compatibility Testing
dogfooding (being users), Eating Our Dogfood, Testing Against Production
dogfooding, Eating Our Dogfood, Testing Against Production
performance testing, What’s in a Test?, Testing the “ilities”
compatibility testing, Compatibility Testing
dogfooding (being users), Eating Our Dogfood, Testing Against Production
how to measure performance, How Do You Measure Performance?
Office Online, Test Flags
in other testing situations, How Do You Measure Performance?
services, metrics for, Production Data, so Tempting but Risky Too
stress testing, Stress Testing
security testing, Beyond Functionality, Usability Labs
stress testing, Stress Testing
architecture for, Distributed Stress Testing
Office Online newsgroups, Test Flags
team organization, Testing the “ilities”
usability testing, Beyond Functionality, Usability Testing
accessibility testing, Eating Our Dogfood
not repro (bug resolution value), Anatomy of a Bug Report
notification of bug assignment, Attributes of a Bug Tracking System
number of passes or failures (test case metric), Tracking and Interpreting the Test Results
number of tests, Defining Boundary Tests
boundary value analysis (BVA), Defining Boundary Tests
cyclomatic complexity and, Basis Path Testing
for performance testing, How Do You Measure Performance?
reducing with data partitioning (see )

O

object model, User Interface Automation
object-oriented metrics, Halstead Metrics
observable testability, Thinking About Testability
offering tester positions to candidates, I Need More Testers and I Need Them Now!
Office, about, Microsoft Is a Big Software Engineering Company
Office Live (see )
Office Online, Loosely Coupled vs. Tightly Coupled Services
Office Shared Services (OSS) team, The Shared Team Model
Office Online, Test Flags
OLSB (Online Live Small Business), Services vs. Packaged Product
one-box test platform (services), Test Environments, The Deployment Test Cluster
OneNote customer connections, Connecting with Customers
online services (see services)
open development, What’s the Problem?
open office space, Informal Reviews
operating systems (see )
Operations (Ops) discipline, The Engineering Disciplines
operators in programs, counting (see )
oracle (test pattern attribute), Using Test Patterns
oracles, Execution, Analysis
organization of engineering workforce, Microsoft Is a Big Software Engineering Company
orientation for new employees, Learning How to Be a Microsoft SDET
orthogonal arrays (OA) approach, combinatorial testing, Combinatorial Testing Approaches
OSS (Office Shared Services) team, The Shared Team Model
outdated test cases, The Value of a Test Case
output matrix randomization (PICT tool), Combinatorial Analysis in Practice
overgeneralization of variable data, Equivalence Class Partitioning
ownership of quality, The Match Game
Ozzie, Ray, The Microsoft Vision, Values and Why We “Love This Company!”, The Microsoft Services Strategy

P

packaged product vs. services, Services vs. Packaged Product
page load time metrics, Production Data, so Tempting but Risky Too, Performance Test Metrics for Services
page weight (metric), Performance Test Metrics for Services, Page Load Time 1 and Page Load Time 2
pair programming, Code Reviews and Inspections
pair testing, Exploratory Testing at Microsoft
pair-wise analysis, Combinatorial Testing Approaches
effectiveness of, Combinatorial Analysis in Practice
insufficiency of, Combinatorial Analysis in Practice
in practice, Combinatorial Testing Approaches
parameter interaction testing (see )
parental controlled with WLID, Loosely Coupled vs. Tightly Coupled Services
Pareto, Vilfredo, Analyzing Risk with Code Complexity
Pareto principle, Analyzing Risk with Code Complexity
parsing automatic test logs, Reporting, Anatomy of a Log File
partial production upgrades, services, Fully Automated Deployments
partitioning data into classes (see )
Partner SDETs, The Engineering Career at Microsoft
Partner Software Development Engineer in Test titles, The Test Architect
pass count (test case metric), Tracking and Interpreting the Test Results
pass rate (test case metric), Tracking and Interpreting the Test Results, Reporting
Pass result (automated tests), Analysis
pass/fail criteria in test cases, Anatomy of a Test Case
passion for quality (competency), I Need More Testers and I Need Them Now!
path testing (see )
patterns-based testing approach, Practicing Good Software Design and Test Design
PC computing innovations, waves of, Moving from Stand-Alone to Layered Services
PDCA cycle, Process Improvement
percentage of false alarms (metric), Continuous Quality Improvement Program
percentage of tickets resolved (metric), Continuous Quality Improvement Program
perception of quality, Testing Provides Information
perf and scale clusters, The Test Cluster, The Deployment Test Cluster
Perfmon.exe utility, How Do You Measure Performance?
performance, Common Mistakes in Bug Reports
(see also )
browser-based service performance metrics, Production Data, so Tempting but Risky Too
bug data as metrics of, How Not to Use the Data: Bugs as Performance Metrics
metrics for services, Production Data, so Tempting but Risky Too
in milestone criteria, Milestones
quality gates, Getting to Done
services and processing power, Growing from Large Scale to Mega Scale
performance counters, How Do You Measure Performance?
performance testing, What’s in a Test?, Testing the “ilities”
compatibility testing, Compatibility Testing
dogfooding (being users), Eating Our Dogfood, Testing Against Production
how to measure performance, How Do You Measure Performance?
Office Online, Test Flags
in other testing situations, How Do You Measure Performance?
services, metrics for, Production Data, so Tempting but Risky Too
stress testing, Stress Testing
performing arts organization, Microsoft as, The Shared Team Model
personas for accessibility testing, Accessibility Testing
pesticide paradox, The Need for Functional Testing
Petri nets, Petri Nets
PICT tool, Combinatorial Testing Approaches
pitfalls with test patterns, Using Test Patterns
Plan phase (PDCA cycle), Process Improvement
planning, Getting to Done
for performance bottlenecks, How Do You Measure Performance?
for services testing, Moving from Stand-Alone to Layered Services
for test automation, SEARCH at Microsoft
Platform Products and Services Division (see )
platform services, Built on a Server
platforms for test automation, To Automate or Not to Automate, That Is the Question
play production, shipping products as, The Shared Team Model
point of involvement with test automation, The Value of Automation
portability testing, Beyond Functionality
postbuild testing, Stopping the Breaks
postponed bugs, Bug Tracking, Anatomy of a Bug Report
power, growth and, Growing from Large Scale to Mega Scale
practical baseline path technique, Basis Path Testing
prebuild testing, Stopping the Breaks
predicted results (test case attribute), The Value of a Test Case
predicting quality perception, Testing Provides Information
PREfast tool, Static Analysis
Principle SDETs, The Engineering Career at Microsoft
Principle Software Development Engineer in Test titles, The Test Architect
Principle Test Managers, Becoming a Manager is Not a Promotion
Print Verifier, Application Verifier
prioritizing bugs, Bug Tracking, Anatomy of a Bug Report
bug severity, Anatomy of a Bug Report
Send a Smile program and, Smile and Microsoft Smiles with You
proactive approach to testing, The Match Game
problem (test pattern attribute), Using Test Patterns
Problem Reports and Solutions panel, Windows Error Reporting
problem solving competency, I Need More Testers and I Need Them Now!
process improvement, Process Improvement, A Home for Source Control
formal systems for, Process Improvement
services, Round Trip Analysis
processing power for services, Growing from Large Scale to Mega Scale
product code (see entries at code)
product engineering disciplines, The Engineering Disciplines, Software Test Engineers at Microsoft
product engineering divisions at Microsoft, The Microsoft Vision, Values and Why We “Love This Company!”
product releases (see )
Product Studio, Managing Bugs and Test Cases
product support, Testers at Microsoft Have Not Always Been SDETs
product teams, The Shared Team Model
Product Unit Manager (PUM) model, Microsoft Is a Big Software Engineering Company
production, testing against, Emulation
program decisions, counting (see cyclomatic complexity, measuring)
program length, measuring, A Complex Problem
Program Management (PM) discipline, The Engineering Disciplines, Software Test Engineers at Microsoft
programmatic accessibility, Eating Our Dogfood, Testing for Accessibility
progress tracking, The Value of a Test Case
Project Atlas, The Microsoft Vision, Values and Why We “Love This Company!”
project management, Bug Triage
bug prioritization, Bug Triage
competency in, I Need More Testers and I Need Them Now!
prototypes, Spiral Model
PSD (Platform Products and Services Division), The Microsoft Vision, Values and Why We “Love This Company!”
PUM (Product Unit Manager) model, Microsoft Is a Big Software Engineering Company
purpose (test case attribute), The Value of a Test Case

Q

QA (quality assurance), The Match Game
QoS (quality of service) programs, Round Trip Analysis
quality, cost of, Good Logging Practices
quality, passion for (competency), I Need More Testers and I Need Them Now!
quality, service, Stateful Services
quality assurance (QA), The Match Game
quality culture, Overcoming Analysis Paralysis
quality gates, Getting to Done
quality metrics, Customers to the Rescue
quality milestone, Milestones
services, Round Trip Analysis
quality of service (QoS) programs, Round Trip Analysis
quality perception, Testing Provides Information
quality tests (see non-functional testing)
Quests, Working Small in a Big Company
quotas for finding bugs, How Not to Use the Data: Bugs as Performance Metrics

R

rack units (rack SKUs), Growing from Large Scale to Mega Scale
Raikes, Jeff, The Microsoft Vision, Values and Why We “Love This Company!”
random model-based testing, Random Models
random selection, combinatorial testing, Combinatorial Analysis
random walk traversals, Modeling Software
random walk traversals (graph theory), Graph Theory and MBT
ranges of values in ECP, Decomposing Variable Data
RCA (root cause analysis), Continuous Quality Improvement Program
reactive approach to testing, The Match Game
reasons for code change, documenting, What Changed?
recruiting testers, I Need More Testers and I Need Them Now!
campus recruiting, I Need More Testers and I Need Them Now!
industry recruiting, Campus Recruiting
RedDog (see )
Redmond workforce, about, The Engineering Disciplines
regression tests, The Value of Automation
regular expressions, Random Models
Rehabilitation Act Section, Eating Our Dogfood
Rehabilitation Act Section 508, Eating Our Dogfood
related test patterns, identifying, Using Test Patterns
release/free builds, War, What Is It Good For?
releases, War, What Is It Good For?
Microsoft-speak for, War, What Is It Good For?
responsibility for managing, Mandatory Practices
of services, frequency and naming of, Stateful Services
reliability of bug tracking system, Attributes of a Bug Tracking System
reliability testing, Beyond Functionality, Testing the “ilities”
milestone criteria, Milestones
repeatability, test cases, The Value of a Test Case
repetition testing, Stress Testing
reporting bugs (see managing bugs)
reporting phase (SEARCH test automation), What’s in a Test?, Reporting
reporting user data (see customer feedback systems)
reproduction steps (repro steps), Anatomy of a Bug Report
Research discipline, The Engineering Disciplines
resolution (in bug reports), Anatomy of a Bug Report
resolved (bug status), Anatomy of a Bug Report
resource utilization, How Do You Measure Performance?
low-resource and capacity testing, Stress Testing
response from customers (see customer feedback systems)
responsiveness measurements, How Do You Measure Performance?
result types for automated testing, Analysis
reusing code, Tools, Tools, Everywhere
reviewing automated test results, Execution
risk analysis modeling, Grammar Models
risk estimation with churn metrics, Code Churn
risk management with services deployment, Fully Automated Deployments
risk with code complexity, Analyzing Risk with Code Complexity
risk-based testing, Analyzing Risk with Code Complexity
role of testing, Anatomy of a Log File
rolling builds, Breaking the Build
rolling upgrades, services, Fully Automated Deployments
round trip analysis (metric), Performance Test Metrics for Services, Round Trip Analysis
Royce, Winston, Waterfall Model
Rudder, Eric, The Engineering Career at Microsoft
run infinitely attribute (stress tests), The Stress Server

S

S+S (see )
SaaS (software as a service), Services vs. Packaged Product
(see also )
sanitizing data before testing, Production Data, so Tempting but Risky Too
scalability testing, Beyond Functionality, Testing the “ilities”
scale out (processing power), The Perf and Scale Cluster
scale up (system data), The Test Cluster
scenario voting, Connecting with Customers
scheduling, Getting to Done
code reviews, Taking Action
debugging scope and, What Changed?
test automation, To Automate or Not to Automate, That Is the Question
test case design, Testing the Good and the Bad
SCM (see source control)
scope of debugging, What Changed?
scope of testing, Testing the Good and the Bad
automated tests, SEARCH at Microsoft
scripted tests, code coverage of, Structural Testing Techniques
SDE (see , )
SDET Leads, Becoming a Manager is Not a Promotion
SDET Managers, Becoming a Manager is Not a Promotion
SDETs (Software Development Engineers in Test), What’s in a Name?
learning how to be, Learning How to Be a Microsoft SDET
recruiting (see recruiting testers)
using triangle simulations, The Need for Functional Testing
SEARCH acronym for test automation, What’s in a Test?, SEARCH at Microsoft
analysis phase, Execution
cleanup phase, Reporting
execution phase, Setup
help phase, Cleanup
reporting phase, Reporting
setup phase, SEARCH at Microsoft
Section 508 (Rehabilitation Act), Eating Our Dogfood
security, Production Data, so Tempting but Risky Too
data sanitization, Production Data, so Tempting but Risky Too
quality gates, Getting to Done
testing, Beyond Functionality, Usability Labs
self-host builds, War, What Is It Good For?
self-test build, War, What Is It Good For?
self-toast builds, War, What Is It Good For?
semiautomated tests, Anatomy of a Test Case
Send a Smile program, Smile and Microsoft Smiles with You
Senior SDET Leads, Becoming a Manager is Not a Promotion
Senior SDET Manager, Becoming a Manager is Not a Promotion
Senior SDETs, The Engineering Career at Microsoft
Senior Software Development Engineer in Test titles, The Test Architect
servers, building services on, Designing the Right S+S and Services Test Approach
service groups, Growing from Large Scale to Mega Scale
services, Testing Software Plus Services
dogfooding, Testing Against Production
loose vs. tight coupling, Platform Services vs. Top-Level Services, Integration Testing vs. Test Flags and emulation
Microsoft services strategy, Two Parts: About Services and Test Techniques
packaged product vs., Services vs. Packaged Product
performance test metrics, Production Data, so Tempting but Risky Too
platform vs. top-level, Built on a Server
processing power requirements, Growing from Large Scale to Mega Scale
S+S testing approaches, Moving from Stand-Alone to Layered Services
S+S testing techniques, Release Frequency and Naming
deployment automation, Release Frequency and Naming
performance test metrics, Production Data, so Tempting but Risky Too
test environment, Fully Automated Deployments
testing against production, Emulation
stand-alone and layered services, S+S Example
stateless vs. stateful, Waves of Innovation, Loosely Coupled vs. Tightly Coupled Services
testing S+S, Waves of Innovation
approaches for, Waves of Innovation
common bugs, Continuous Quality Improvement Program
continuous quality improvement, Round Trip Analysis
Services memo, The Microsoft Services Strategy
setup phase (SEARCH test automation), What’s in a Test?, SEARCH at Microsoft
Seven Bridges of K�nigsberg problem, Building a Finite State Model
severity, bug, Anatomy of a Bug Report
shared libraries, Test Code Is Product Code, Export and Import
Shared Team model, The Shared Team Model
shared teams, The Shared Team Model
shared test clusters, The Test Cluster, The Integrated Services Test Environment
sharing test tools, Test Code Is Product Code, Export and Import, What’s the Problem?
Shewhart cycle, Process Improvement
ship room, shipping software from, Shipping Software from the War Room
shipping software, Shipping Software from the War Room
shirts, ordering new, The Microsoft Vision, Values and Why We “Love This Company!”
shortest path traversal (graph theory), Graph Theory and MBT
should fix (bug priority), Bug Triage
shrink-wrap software, Services vs. Packaged Product
simple testability, Thinking About Testability
simplicity (see code complexity)
simplified baseline path technique, Basis Path Testing
simplified control flow diagrams (CFDs), Block Testing
single fault assumption, Summary of Equivalence Class Partitioning
Six Sigma program, Formal Process Improvement Systems at Microsoft
size issues (accessibility), Testing for Accessibility
size of Microsoft engineering workforce, Testers at Microsoft Have Not Always Been SDETs
campus recruiting, I Need More Testers and I Need Them Now!
Skip result (automated tests), Analysis, Reporting
SMEs as testers, What’s in a Name?
smiley icon (Send a Smile), Smile and Microsoft Smiles with You
Smith, Brad, The Microsoft Vision, Values and Why We “Love This Company!”
smoke alarm metrics, Object-Oriented Metrics
smoke tests, The Daily Build, Emulation
snapshots of code, Code Churn, Test Machine Configuration Savings, Export and Import
SOCK mnemonic for testability, Thinking About Testability
software as services (see )
software design, importance of, A Practical Approach to Test Case Design
Software Development Engineer in Test Manager title, Becoming a Manager is Not a Promotion
Software Development Engineer in Test titles, The Test Architect
software engineering at Microsoft, Engineering Life Cycles
Agile methodologies, Spiral Model, Milestones
milestones, Agile Methodologies
traditional models, Software Engineering at Microsoft
software features, Bug Triage
bugs vs., Bug Triage
in milestone criteria, Milestones
software libraries, Tools, Tools, Everywhere
Software Plus Services (S+S), Two Parts: About Services and Test Techniques, Moving from Stand-Alone to Layered Services
(see also )
common bugs with, Continuous Quality Improvement Program
continuous quality improvement, Round Trip Analysis
vs. SaaS (software as a service), Services vs. Packaged Product
testing approaches, Waves of Innovation
client support, Designing the Right S+S and Services Test Approach
loose vs. tight coupling, Platform Services vs. Top-Level Services, Integration Testing vs. Test Flags and emulation
platform vs. top-level, Built on a Server
release frequency and naming, Stateful Services
server builds, Designing the Right S+S and Services Test Approach
stateless vs. stateful, Loosely Coupled vs. Tightly Coupled Services
time-to-market considerations, Stateful Services
testing techniques, Release Frequency and Naming
deployment automation, Release Frequency and Naming
performance test metrics, Production Data, so Tempting but Risky Too
test environment, Fully Automated Deployments
testing against production, Emulation
software reliability (see )
software test engineers, Software Test Engineers at Microsoft
(see also )
career paths in Test, The Engineering Career at Microsoft
engineering careers, The Engineering Career at Microsoft
learning how to be, Learning How to Be a Microsoft SDET
recruiting (see recruiting testers)
sound features, accessibility of, Testing for Accessibility
source (bug report field), Anatomy of a Bug Report
source control, Code Churn
breaking builds, Breaking the Build
check-in systems, Stopping the Breaks
reasons for code changes, What Changed?
Spec Explorer tool, Model-Based Testing Tools at Microsoft
Windows 7 and, A Language and Engine
special values in ECP, Decomposing Variable Data
specifications for test design, Ask Questions, Thinking About Testability
spiral model, Waterfall Model
SQEs (software quality engineers), Testers at Microsoft Have Not Always Been SDETs
stand-alone applications, Setup
stand-alone services, S+S Example
stapler stress, Stress Testing
starting state (model), Model-Based Testing
starting the test process, Starting with Testing
state-based models (see )
stateless vs. stateful services, Loosely Coupled vs. Tightly Coupled Services
statement testing, Structural Testing Techniques
static analysis, Mandatory Practices, Static Analysis, Static Analysis
managed code analysis, Managed Code Analysis
native code analysis, Static Analysis
test code analysis, Test Code Analysis
status, bug, Anatomy of a Bug Report
Step attribute, Execution
steps in test cases, The Value of a Test Case, Anatomy of a Test Case
STEs (Software Test Engineers), What’s in a Name?
Stobie, Keith, I Need More Testers and I Need Them Now!, The IC Tester, What’s in a Test?
stopwatch testing, Testing the “ilities”
strategic insight (competence), I Need More Testers and I Need Them Now!
strategy, test, Ask Questions
stress testing, Stress Testing
architecture for, Distributed Stress Testing
Office Online newsgroups, Test Flags
structural testing, Structural Testing Techniques
basis path testing, Structural Testing Techniques
block testing, Structural Testing Techniques
condition testing, Summary of Decision Testing
decision testing, Summary of Block Testing
functional testing vs., Structural Testing Techniques
need for, Structural Testing Techniques
subject matter experts as testers, What’s in a Name?
support, product, Testers at Microsoft Have Not Always Been SDETs
support for test automation, To Automate or Not to Automate, That Is the Question
SupportFile attribute, Execution
Surface, Working Small in a Big Company
switch/case statement, testing, Block Testing
syntax elements in programs, counting. SeeHalstead metrics, The Daily Build
syntax errors, The Daily Build
system-wide accessibility settings, Accessibility Personas
systematic evaluation approaches, combinatorial testing, Combinatorial Analysis
systems, test (see )
systems interactions, Campus Recruiting

T

t-shirts, ordering new, The Microsoft Vision, Values and Why We “Love This Company!”
TAG (Test Architect Group), Test Lab Savings, Other Considerations
TAs (see )
TCMs (test case managers), Bug Bars, Test Case Mistakes, Tracking and Interpreting the Test Results
cross-referencing with automation scripts, Putting It All Together
TDSs (test design specifications), Ask Questions, Thinking About Testability
Team Foundation Server (TFS), Eating Our Dogfood
teams, Milestones
feature crews (Agile methodologies), Milestones
for non-functional testing, Testing the “ilities”
open office space, Informal Reviews
usability labs, Usability Testing
war team, Shipping Software from the War Room
Windows Stress team, The Stress Server
technical excellence (competency), I Need More Testers and I Need Them Now!
Technical Fellows, The Engineering Career at Microsoft
techniques as gimmicks, The Need for Functional Testing
templates for sharing test patterns, Using Test Patterns
Test (SDET) discipline, The Engineering Disciplines
(see also )
Test A Little attribute (BVTs), The Daily Build
test architects, Other Considerations
Test Architects (TAs), The Engineering Career at Microsoft, Test Lab Savings
test automation, Test Automation
automatic failure analysis, Solving Tomorrow’s Problems Today
developing, Campus Recruiting
elements of testing, User Interface Automation
exploratory testing vs., Black Box, White Box, and Gray Box
running the tests, Run, Automation, Run!
common mistakes with, Common Automation Mistakes
large-scale testing, Putting It All Together
SEARCH acronym for test automation, What’s in a Test?, SEARCH at Microsoft
analysis phase, Execution
cleanup phase, Reporting
execution phase, Setup
help phase, Cleanup
reporting phase, Reporting
setup phase, SEARCH at Microsoft
test code analysis, Test Code Analysis
testing, A Practical Approach to Test Case Design
tracking changes in, Code Churn
user interface automation, To Automate or Not to Automate, That Is the Question
value of, Test Automation
Test Broadly Not Deeply attribute (BVTs), The Daily Build
test case managers (TCMs), Bug Bars, Test Case Mistakes, Tracking and Interpreting the Test Results
cross-referencing with automation scripts, Putting It All Together
test cases, Bug Bars, Test Case Mistakes, Tracking and Interpreting the Test Results
analysis of, Test Code Analysis
anatomy of, The Value of a Test Case
common mistakes with, Anatomy of a Test Case
counting, Test Case Mistakes
cross-referencing with automation scripts, Putting It All Together
defined, Bug Bars, Cases and Points: Counting Test Cases
design of, A Practical Approach to Test Case Design
best practices, A Practical Approach to Test Case Design
estimating test time, Using Test Patterns
getting started, Starting with Testing
practical considerations, Testing the Good and the Bad
testability, Thinking About Testability
testing good and bad, Example Elements in a Test Design Specification
using test patterns, Practicing Good Software Design and Test Design
documenting with automated testing, Cleanup
executing automatically, Setup, Putting It All Together
(see also test automation)
milestone criteria, Milestones
tracking and interpreting results (metrics), Tracking and Interpreting the Test Results
tracking changes in, Code Churn
value of, The Value of a Test Case
workflow, The Bug Workflow
test cleanup, Reporting
test clusters, The One Box, The Deployment Test Cluster
deployment test clusters, The Integrated Services Test Environment, The Deployment Test Cluster
dogfood clusters, Testing Against Production
shared, The Test Cluster, The Integrated Services Test Environment
test code analysis, Test Code Analysis
test code snapshots, Code Churn, Test Machine Configuration Savings, Export and Import
test collateral, Putting It All Together
test controllers, Putting It All Together
test coverage (see )
test data, creating with grammar models, Random Models
test deliverables (test strategy attributes), Thinking About Testability
test design, importance of, A Practical Approach to Test Case Design
test design specifications (TDSs), Ask Questions, Thinking About Testability
test environment for services, Fully Automated Deployments
test excellence, Export and Import
Test Fast attribute (BVTs), The Daily Build
test flags, Integration Testing vs. Test Flags and emulation
test frequency, How Do You Measure Performance?
as test case attribute, The Value of a Test Case
performance testing, How Do You Measure Performance?
test harnesses, Execution, Cleanup
test innovation, Other Considerations
test leadership, Anatomy of a Log File
test logs (automated testing), Execution, Reporting
using for failure matching, The Match Game
test matrix for services testing, Moving from Stand-Alone to Layered Services
test oracles, Execution, Analysis
test pass, defined, Cases and Points: Counting Test Cases, Tracking and Interpreting the Test Results
test patterns, Practicing Good Software Design and Test Design
test points, Test Case Mistakes, Cases and Points: Counting Test Cases
test run, defined, Cases and Points: Counting Test Cases
test strategies, Ask Questions
test suite, defined, Cases and Points: Counting Test Cases
test therapists, Code Reviews and Inspections
test time, estimating, Using Test Patterns
test tools and systems, To Automate or Not to Automate, That Is the Question
automation (see )
bug management (see managing bugs)
build process, A Home for Source Control
breaking builds, The Daily Build
testing daily builds, Test Machine Configuration Savings
code churn, Other Tools
customer feedback systems, Customer Feedback Systems
connecting with customers, Send a Smile Impact
emotional response, Test and WER
for services, Continuous Quality Improvement Program
testing vs. quality, Customer Feedback Systems
watching customers (see CEIP (Customer Experience Improvement Plan))
Windows Error Reporting, Games, Too!
miscellaneous, Test Code Is Product Code
non-functional testing (see non-functional testing)
source control, Code Churn
breaking builds, Breaking the Build
check-in systems, Stopping the Breaks
reasons for code changes, What Changed?
static analysis, Mandatory Practices
managed code analysis, Managed Code Analysis
native code analysis, Static Analysis
test code analysis, Test Code Analysis
testability, Thinking About Testability
(see also )
TestCleanup attribute, Execution
Tester Center, Code Reviews and Inspections
tester DNA, What’s in a Name?, Testers at Microsoft Have Not Always Been SDETs
testing, future of, Solving Tomorrow’s Problems Today
testing against production, Emulation
Testing at Microsoft for SDETs class, Learning How to Be a Microsoft SDET
testing coverage, Test Code Analysis
analysis tools for, Test Code Analysis
behavioral and exploratory testing, Structural Testing Techniques
combinatorial analysis and, Effectiveness of Combinatorial Analysis
with functional testing, The Need for Functional Testing
milestone criteria, Milestones
quality gates, Getting to Done
scripted tests, Structural Testing Techniques
statement vs. block coverage, Structural Testing Techniques
testing techniques as gimmicks, The Need for Functional Testing
testing the tests, Static Analysis, Test Code Analysis
testing with models (see )
TestInitialize attribute, Execution
TestMethod attribute, Execution
text matrix for automated testing, SEARCH at Microsoft
TFS (Team Foundation Server), Eating Our Dogfood
ThinkWeek, Working Small in a Big Company
threat modeling, Threat Modeling
3(BC) formula, A New Formula for Boundary Value Analysis, Summary of Boundary Value Analysis
threshold method (WER), Emptying the Buckets
tickets resolved (metric), Continuous Quality Improvement Program
tidal wave bug, Common Bugs I’ve Seen Missed
tightly coupled services, Platform Services vs. Top-Level Services, Integration Testing vs. Test Flags and emulation
time for code reviews, monitoring, Taking Action
time investment (see )
time to detection (metric), Continuous Quality Improvement Program
time to document, The Value of a Test Case
time to market, services, Stateful Services
time to resolution (metric), Continuous Quality Improvement Program
titles for software test engineers, What’s in a Name?
moving from SDEs to SDETs, What’s in a Name?
SDET IC, The Test Architect
Test Architect, The Engineering Career at Microsoft, Test Lab Savings
in test management, The IC Tester
Test Manager, Becoming a Manager is Not a Promotion
titles of bugs (in bug reports), Why Write a Bug Report?
tool sharing, Test Code Is Product Code, Export and Import, What’s the Problem?
tools for accessibility technology, Accessibility Testing, Testing for Accessibility
top-level services, Built on a Server
topology testing, Daily Build Testing
tracking bugs (see managing bugs)
tracking code changes (see source control)
tracking code review data, Time Is on My Side
tracking test cases, Tracking and Interpreting the Test Results
tracking test progress, The Value of a Test Case
training as SDETs, Learning How to Be a Microsoft SDET
training strategy in test design, Ask Questions, Thinking About Testability
transitions (in models), Model-Based Testing
Petri nets, Petri Nets
trends in test failures, analyzing, Integrating AFA
triad (Test, Development, Program Management), The Engineering Disciplines
triage for managing bugs, Bug Tracking, Anatomy of a Bug Report
bug severity, Anatomy of a Bug Report
Send a Smile program and, Smile and Microsoft Smiles with You
triangle simulation (Weinberg’s triangle), The Need for Functional Testing
Trudau, Garry, The Microsoft Vision, Values and Why We “Love This Company!”
Trustworthy attribute (BVTs), The Daily Build
tsunami effect, Common Bugs I’ve Seen Missed
Turner, Kevin, The Microsoft Vision, Values and Why We “Love This Company!”

U

UI automation, To Automate or Not to Automate, That Is the Question
brute force approach, User Interface Automation
uncertainty, reducing with BGM, Grammar Models
unique tools for unique problems, Test Code Is Product Code
uniqueness of values in ECP, Decomposing Variable Data
unit tests, block testing for, Block Testing
universities, recruiting from, I Need More Testers and I Need Them Now!
upgrading services, Fully Automated Deployments
Usability and Design discipline, The Engineering Disciplines, Software Test Engineers at Microsoft
usability testing, Beyond Functionality, Usability Testing
accessibility testing, Eating Our Dogfood
usage data, collecting (see CEIP (Customer Experience Improvement Plan))
USB cart of death, Stress Testing
User Assistance and Education (see )
user interface, To Automate or Not to Automate, That Is the Question
automation of, To Automate or Not to Automate, That Is the Question
programmatic access to, Eating Our Dogfood, Testing for Accessibility
users, being (see )
UX (see )

V

valid class data (ECP), Decomposing Variable Data
values, Microsoft, The Microsoft Vision, Values and Why We “Love This Company!”
variable data, decomposing (ECP), Equivalence Class Partitioning
venture capital teams, internal, Working Small in a Big Company
verbose, test cases as, Anatomy of a Test Case
verification tests, Example Elements in a Test Design Specification
version number, bugs, Anatomy of a Bug Report
Vice President of Test, Becoming a Manager is Not a Promotion
VINCE (Verification of Initial Consumer Experience), Customers to the Rescue
Virtual Earth, Microsoft Is a Big Software Engineering Company
virtual teams for non-functional testing, Testing the “ilities”
virtualization, Machine Virtualization
managing failures during tests, Network Topology Testing
test scenarios, Test Machine Configuration Savings, Snapshots
visual freeze, defined, War, What Is It Good For?
Visual Round Trip Analyzer (VRTA), Performance Test Metrics for Services, Round Trip Analysis
Visual Studio 2008, Spec Explorer for, Modeling with Spec Explorer
Visual Studio Team Foundation Server (TFS), Eating Our Dogfood
VMs (see )
voice of customer, Continuous Quality Improvement Program
Voodoo Vince, Customers to the Rescue
VPNs (virtual private networks), Thinking About Testability

W

war room, shipping software from, Shipping Software from the War Room
Warn result (automated tests), Analysis
watching customers (see CEIP (Customer Experience Improvement Plan))
waterfall model, Software Engineering at Microsoft
waves of innovation in PC computing, Moving from Stand-Alone to Layered Services
Web services (see services)
weighted methods per class (metric), Halstead Metrics
weighted traversals (graph theory), Graph Theory and MBT
Weinberg’s triangle, The Need for Functional Testing
WER (Windows Error Reporting), Games, Too!
White, David, Testers at Microsoft Have Not Always Been SDETs
white box testing, Black Box, White Box, and Gray Box
assumption of bias in (false), Structural Testing Techniques
structure testing as, Structural Testing Techniques
Whittaker, James, The IC Tester
Whitten, Greg, Software Test Engineers at Microsoft
Windows 7, Spec Explorer and, A Language and Engine
Windows 95, supporting, Testers at Microsoft Have Not Always Been SDETs
Windows Error Reporting (WER), Games, Too!
Windows Live ID (WLID), Loosely Coupled vs. Tightly Coupled Services
Windows Live Mail service, Microsoft Is a Big Software Engineering Company, Services vs. Packaged Product
Windows Mobile, about, Microsoft Is a Big Software Engineering Company
Windows operating systems, Microsoft Is a Big Software Engineering Company
about, Microsoft Is a Big Software Engineering Company
accessibility settings, Eating Our Dogfood
Windows Powered Smart Display, User Interface Automation
Windows Sustained Engineering (SE) Team, Object-Oriented Metrics
WMC metric, Halstead Metrics
“won’t fix” bugs, Example Elements in a Test Design Specification, A Complex Problem, Bug Tracking
workflow for bugs, The Bug Workflow
workforce size at Microsoft, Testers at Microsoft Have Not Always Been SDETs
campus recruiting, I Need More Testers and I Need Them Now!

X

Xbox 360, about, Microsoft Is a Big Software Engineering Company

Z

zero bug bounce, Bug Triage
zero bugs concept, Bug Triage
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset