Home Page Icon
Home Page
Table of Contents for
Cover
Close
Cover
by Alan Page, Ken Johnston, and Bj Rollison
How We Test Software at Microsoft®
Special Upgrade Offer
Acknowledgments
Introduction
Who This Book Is For
What This Book Is About
Find Additional Content Online
Support for This Book
I. About Microsoft
1. Software Engineering at Microsoft
The Microsoft Vision, Values and Why We “Love This Company!”
Microsoft Is a Big Software Engineering Company
Developing Big and Efficient Businesses
The Shared Team Model
Working Small in a Big Company
Employing Many Types of Engineers
The Engineering Disciplines
Being a Global Software Development Company
Summary
2. Software Test Engineers at Microsoft
What’s in a Name?
Testers at Microsoft Have Not Always Been SDETs
I Need More Testers and I Need Them Now!
Campus Recruiting
Industry Recruiting
Learning How to Be a Microsoft SDET
The Engineering Career at Microsoft
Career Paths in the Test Discipline
The Test Architect
The IC Tester
Becoming a Manager is Not a Promotion
Test Managers
Summary
3. Engineering Life Cycles
Software Engineering at Microsoft
Traditional Software Engineering Models
Waterfall Model
Spiral Model
Agile Methodologies
Other Models
Milestones
Agile at Microsoft
Feature Crews
Getting to Done
Iterations and Milestones
Putting It All Together
Process Improvement
Formal Process Improvement Systems at Microsoft
Microsoft and ISO 9000
Shipping Software from the War Room
War, What Is It Good For?
Defining the Release—Microspeak
Mandatory Practices
Expected vs. Mandatory
One-Stop Shopping
Summary: Completing the Meal
II. About Testing
4. A Practical Approach to Test Case Design
Practicing Good Software Design and Test Design
Using Test Patterns
Estimating Test Time
Starting with Testing
Ask Questions
Have a Test Strategy
Thinking About Testability
Test Design Specifications
Example Elements in a Test Design Specification
Testing the Good and the Bad
Other Factors to Consider in Test Case Design
Black Box, White Box, and Gray Box
Exploratory Testing at Microsoft
Summary
5. Functional Testing Techniques
The Need for Functional Testing
Equivalence Class Partitioning
Decomposing Variable Data
Equivalence Class Partitioning in Action
Analyzing Parameter Subsets
The ECP Tests
Summary of Equivalence Class Partitioning
Boundary Value Analysis
Defining Boundary Tests
A New Formula for Boundary Value Analysis
Hidden Boundaries
Summary of Boundary Value Analysis
Combinatorial Analysis
Combinatorial Testing Approaches
Combinatorial Analysis in Practice
Effectiveness of Combinatorial Analysis
Summary of Combinatorial Analysis
Summary
6. Structural Testing Techniques
Block Testing
Summary of Block Testing
Decision Testing
Summary of Decision Testing
Condition Testing
Summary of Condition Testing
Basis Path Testing
Summary of Basis Path Testing
Summary
7. Analyzing Risk with Code Complexity
Risky Business
A Complex Problem
Counting Lines of Code
Measuring Cyclomatic Complexity
Halstead Metrics
Object-Oriented Metrics
High Cyclomatic Complexity Doesn’t Necessarily Mean “Buggy”
What to Do with Complexity Metrics
Summary
8. Model-Based Testing
Modeling Basics
Testing with Models
Designing a Model
Modeling Software
Building a Finite State Model
Automating Models
Graph Theory and MBT
Models for API Testing
Random Models
Grammar Models
Modeling Without Testing
Bayesian Graphical Modeling
Petri Nets
Model-Based Testing Tools at Microsoft
Spec Explorer
Modeling with Spec Explorer
Spec Explorer for Visual Studio
A Language and Engine
Modeling Tips
Summary
Recommended Reading and Tools
III. Test Tools and Systems
9. Managing Bugs and Test Cases
The Bug Workflow
Bug Tracking
A Bug’s Life
Attributes of a Bug Tracking System
Why Write a Bug Report?
Anatomy of a Bug Report
Bug Triage
Common Mistakes in Bug Reports
Using the Data
How Not to Use the Data: Bugs as Performance Metrics
Bug Bars
Test Case Management
What Is a Test Case?
The Value of a Test Case
Anatomy of a Test Case
Test Case Mistakes
Managing Test Cases
Cases and Points: Counting Test Cases
Tracking and Interpreting the Test Results
Summary
10. Test Automation
The Value of Automation
To Automate or Not to Automate, That Is the Question
User Interface Automation
What’s in a Test?
SEARCH at Microsoft
Setup
Execution
Analysis
Reporting
Cleanup
Help
Run, Automation, Run!
Putting It All Together
Large-Scale Test Automation
Common Automation Mistakes
Summary
11. Non-Functional Testing
Beyond Functionality
Testing the “ilities”
Performance Testing
How Do You Measure Performance?
Stress Testing
Distributed Stress Testing
Distributed Stress Architecture
The Stress Client
The Stress Server
Attributes of Multiclient Stress Tests
Compatibility Testing
Application Libraries
Application Verifier
Eating Our Dogfood
Accessibility Testing
Accessibility Personas
Testing for Accessibility
Testing Tools for Microsoft Active Accessibility
Usability Testing
Usability Labs
Security Testing
Threat Modeling
Fuzz Testing
Summary
12. Other Tools
Code Churn
Keeping It Under Control
Tracking Changes
What Changed?
Why Did It Change?
A Home for Source Control
Build It
The Daily Build
Breaking the Build
Stopping the Breaks
Static Analysis
Native Code Analysis
Managed Code Analysis
Just Another Tool
Test Code Analysis
Test Code Is Product Code
Even More Tools
Tools for Unique Problems
Tools for Everyone
Summary
13. Customer Feedback Systems
Testing and Quality
Testing Provides Information
Quality Perception
Customers to the Rescue
Games, Too!
Windows Error Reporting
The Way We WER
Corporate Error Reporting
Filling the Buckets
Emptying the Buckets
Test and WER
Smile and Microsoft Smiles with You
Send a Smile Impact
Connecting with Customers
Summary
14. Testing Software Plus Services
Two Parts: About Services and Test Techniques
Part 1: About Services
The Microsoft Services Strategy
Shifting to Internet Services as the Focus
Growing from Large Scale to Mega Scale
Power Is the Bottleneck to Growth
Services vs. Packaged Product
S+S Example
Moving from Stand-Alone to Layered Services
Part 2 Testing Software Plus Services
Waves of Innovation
Designing the Right S+S and Services Test Approach
Client Support
Built on a Server
Platform Services vs. Top-Level Services
Loosely Coupled vs. Tightly Coupled Services
Stateless to Stateful
Stateless Services
Stateful Services
Time to Market or Features and Quality
Release Frequency and Naming
Testing Techniques for S+S
Fully Automated Deployments
Test Environments
The One Box
The Test Cluster
The Perf and Scale Cluster
The Integrated Services Test Environment
The Deployment Test Cluster
Test Environments Summary
Integration Testing vs. Test Flags and emulation
Test Flags and Emulation Provide Agility
Test Flags
Emulation
Testing Against Production
Production Dogfood, Now with More Big Iron
Production Data, so Tempting but Risky Too
Performance Test Metrics for Services
Page Load Time 1 and Page Load Time 2
Page Weight
Compressibility
Expiration Date
Round Trip Analysis
Several Other Critical Thoughts on S+S
Continuous Quality Improvement Program
Common Bugs I’ve Seen Missed
Summary
IV. About the Future
15. Solving Tomorrow’s Problems Today
Automatic Failure Analysis
Overcoming Analysis Paralysis
The Match Game
Good Logging Practices
Anatomy of a Log File
Integrating AFA
Machine Virtualization
Virtualization Benefits
In the Office
Test Lab Savings
Test Machine Configuration Savings
Virtual Machine Test Scenarios
Daily Build Testing
Network Topology Testing
When a Failure Occurs During Testing
Export and Import
Snapshots
Test Scenarios Not Recommended
Code Reviews and Inspections
Types of Code Reviews
Formal Reviews
Informal Reviews
Checklists
Other Considerations
Activities
Taking Action
Time Is on My Side
More Review Collateral
Two Faces of Review
Tools, Tools, Everywhere
Reduce, Reuse, Recycle
What’s the Problem?
Open Development
Summary
16. Building the Future
The Need for Forward Thinking
Thinking Forward by Moving Backward
Striving for a Quality Culture
Testing and Quality Assurance
Who Owns Quality?
The Cost of Quality
A New Role for Test
Test Leadership
The Microsoft Test Leadership Team
Test Leadership Team Chair
Test Leadership in Action
The Test Architect Group
Test Excellence
Sharing
The Engineering Excellence Forum
Helping
Communicating
Keeping an Eye on the Future
Microsoft Director of Test Excellence
The Leadership Triad
Innovating for the Future
A. About the Authors
Index
About the Authors
Special Upgrade Offer
Copyright
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Next
Next Chapter
How We Test Software at Microsoft®
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset