Chapter 2. Software Test Engineers at Microsoft
In the beginning, there were no testers at Microsoft, no localization engineers, no program managers, and no usability engineers. In the beginning, there were just engineers; oh, and sales and marketing.
Chapter 1, introduced the 10 different engineering disciplines Microsoft uses today to organize and develop our engineering workforce. Even before we began to break out different engineering disciplines there were a few different titles, such as product support for shipped products. Positions like these were seen as a role for an engineer rather than a different career path. During these early days, all engineers had the same title and one common career path. Of course, in these early days Microsoft had less than 50 employees, software wasn’t even really an industry, and Microsoft was not yet a public company.
The move to supporting separate disciplines with distinct career paths has taken quite a while to evolve. Program Management and Usability are two of the older disciplines at Microsoft. Around 1990, the Usability discipline, engineers that work to make our software easy to use for real-world users, became a formal role and eventually a recognized discipline. Usability formed largely because of such features as the Microsoft Office Word mail merge (think snail mail and printing out form letters or stick-on labels) that just didn’t seem to work for the average person. Some customers might argue that we are still working to make mail merge easy to use, but that story is better suited for a book on how we design software at Microsoft.
Shortly after the creation of the software development position, the software tester position broke off as the second distinct discipline. As the story goes, the first-ever tester at Microsoft was a young high school intern by the name of Lloyd Frink, who started in June 1979. The Microsoft Archive team places the hiring of the first full-time tester to be 1983 followed by a wave of testers hired in 1985, as illustrated in Figure 2-1. Test as a separate standard title with a fully articulated career path did not show up until the late 1980s.
What’s in a Name?
A rose by any other name may still smell as sweet, but a title can have a significant impact on the prism through which a person, or in this case an engineer, looks out onto his world. Developers at Microsoft have the common title of Software Development Engineer, or SDE. They develop features by writing code. The formal title for a software tester at Microsoft is Software Development Engineer in Test, or SDET. The similarity in the names of the two disciplines is by design because testers at Microsoft are developers. Among other things, testers design tests, influence product design, conduct root cause analysis, participate in code reviews, and write automation. Occasionally testers check bug fixes or work on small features. Testing is a heavy workload, so writing features is not very common for testers.
The concept of hiring a software engineer with a passion for testing is powerful and is the biggest differentiator between the Microsoft approach to software testing and the typical industry approach. The most common conclusion drawn is that we hire these “coders” for test positions because we want to automate everything and eliminate manual testing. Although we do want testers who can write effective automation, that’s only a small part of the equation. Testers who understand programming concepts and computer architecture typically have the analysis skills that testing requires. They can also find bugs earlier and understand the root cause so that they can quickly find other similar bugs and implement early detection. This strong grounding in computer science—the same grounding a developer has—reinforces the tester skills and gives us a more dynamic and flexible workforce of testers.
A common question that comes up during industry events is why Microsoft doesn’t hire subject matter experts (SMEs) as testers. For example, international accounting rules are very complex and a tester with just a background in engineering wouldn’t know all the nuances. Another example is vertical products such as customer relationship management (CRM) solutions. The theory is that it would be better to hire an SME and Microsoft could focus on being really good at training our testers in computer science and engineering skills. The counterargument to this question is whether a company would hire a CPA, train that individual to be a world-class developer, and then have that individual write the company’s accounting solution. Of course, this approach just isn’t practical. Learning to be a great developer requires passion for technology and many years of formal training.
Just about every software company starts with a developer and teaches her the problem space and customer scenarios for the product she will be developing. This holds true whether the company develops operating systems or software to control the flow of power across electrical grids. With testing, we actually have two challenges. First, we must teach the engineer to be an SME for the product area she is working on, and second, we must teach her how to test.
The rule of thumb, therefore, is to hire someone who has solid engineering skills, who can code as well as an entry-level developer, and who has the other attributes we look for in a great tester. We call these attributes the tester DNA, and I discuss them later in this chapter.
As with any rule of thumb, there are exceptions. The vast majority of testers at Microsoft are developers in test, but in some areas, we do find that we need some of the team members to be SMEs. Global accounting rules specialists or researchers in speech recognition algorithms are two examples of areas in which we hire a lot of SMEs as testers. As we move into more consumer products, we have hired manufacturing process experts to test our designs against the needs of high-volume cost-controlled manufacturing. In these cases, the title of the SME test engineers is usually not SDET but rather something more appropriate such as Linguistics Test Engineer or Manufacturing Test Engineer.
Testers at Microsoft Have Not Always Been SDETs
Until 2005, Microsoft actually used two different titles for testers. Software Test Engineer (STE) and Software Development Engineer in Test (SDE/T). This dual-title process was very confusing. In some groups, the SDE/T title meant the employee worked on test tools, and in others it meant he had a computer science degree and wrote a lot of test automation. There wasn’t even a fully articulated career path for an SDE/T at this time. Table 2-1 lists the tasks SDE/Ts and STEs were responsible for.
Note
Here is an excerpt from the SDET Ladder Level Guide 2004, “An SDE/T should use either the test or development ladder level guide, whichever is most applicable.”
Common SDE/T tasks | Common STE tasks |
Develop test harness for test execution | Write test plans |
Develop specialty test tools for security or performance testing | Document test cases |
Automate API or protocol tests | Run manual tests |
Participate in bug bashes | Write automation for core tests |
Find, debug, file, and regress bugs | Find, file, and regress bugs |
Participate in design reviews | Participate in design reviews |
Participate in code reviews |
Even without a clear career path, the notion of having one foot in the tester world and one in the development world was attractive. Over time, the number of employees with the SDE/T title grew to the point that we decided to merge the two disciplines.
In 2002, there was a push to drop the SDE/T title because it was causing confusion in the workforce. In 2005, the title SDE/T was changed to SDET and there was a push to merge the STE and SDET career paths, as illustrated in Figure 2-2. Test Architect first showed up as a formally recognized title in 2003 and is included in the SDET calculations.
Going with a strong title reinforcing the similarity to the development discipline was critical to our strategy for ever increasing product quality and testing efficiency. The next three parts of this book delve into much more detail about the techniques we use to test software at Microsoft. The approach we take is possible only because of the skills we recruit for and develop in our test engineer workforce.
In the early stages, the use of the SDE/T title was simply to help with recruiting from colleges. Many candidates didn’t want to graduate with a computer science degree and not use their coding skills on the job. The SDE/T title and the original role of tools developer in test sold much better than the STE title.
Although all STEs were expected to be able to create automation when necessary, many of our products just didn’t warrant the high level of automation we expect today, so the amount of time STEs would spend writing code was a small percentage of their total time testing.
In 2001, a major change to Microsoft product support policies had a bigger impact on software testing than any other engineering discipline. The change was the length of support for our products. Most major products, including the Windows operating system, moved to a 10-year support commitment. The criticality of software in the enterprise and the long process to upgrade an enterprise from one operating system or productivity suite to another collided with our then policy of current minus one (N - 1) product support policies. As software shipping cycles became shorter, support windows needed to overlap. Instead of a policy based upon numbers of supported versions, we went with a model based on years, ranging from 3 years for consumer products that had an annual release cycle to 10 years for servers, operating systems, and critical productivity applications.
When we developed Windows 95, we never imagined supporting it until the year 2005. Although Windows 95 had a solid core set of test automation that validates most major functionality and key user scenarios, it also included vast numbers of documented (and undocumented) manual tests. Manual exploratory testing (covered in Chapter 4) was a common technique used on products shipped through the late 1990s. Armies of vendors and full-time employees were hired to do this button-pushing testing.
The change in support lengths meant test automation could be used for many more years and thus was more justifiable during the R&D phase of product development. That realization pushed us to emphasize the hiring of more SDE/Ts. As more computer science graduates flooded the ranks of the test organizations, we discovered an increased ability to affect design and drive improvement in testability.
Other factors such as increasing levels of integration, complexity, and challenging security testing such as threat modeling, fuzz testing, or fault injection have continued to drive our need for computer science and coding skills in test. Even the shift to services and the rapid rate of product shipping they go through is driving us to new models for automated testing. The impact of online services to our testing strategy is covered in Chapter 15.
I Need More Testers and I Need Them Now!
Microsoft continues to increase its engineering workforce every year. Test alone adds approximately 500 new positions a year. We have a nearly even split in hiring testers from other companies and computer science graduates straight from colleges both in the United States and in leading universities around the world.
A great SDET candidate possesses that tester DNA covered earlier in this chapter. We also look for some very specific skills or, in HR lingo, “competencies.” To prevent this section from reading too much like an HR manual, I just briefly cover what a competency is and how it relates to tester DNA.
Competencies describe behaviors that differentiate outstanding results from typical results. Competencies have different levels indicating relative strength. To begin with, most successful candidates have some but probably limited strength in most of our engineering competencies. Test shares the same set of competencies with all 10 engineering disciplines, but over time, some competencies such as analytical problem solving become more pronounced with testers than they might be with other disciplines.
Ten competencies are considered core to all engineers. There are additional competencies for individuals in, say, management roles, finance, or sales and marketing. Following are the 10 engineering competencies:
Analytical Problem Solving. Very critical for testers because problem decomposition and root cause analysis are key to driving quality upstream.
Customer-Focused Innovation. Does the candidate care about customers and see how software can help solve problems or wow them with fun experiences?
Technical Excellence. We look to see if the candidate understands networks, operating systems, and not just how to code but how to optimize code.
Project Management. For testers, this is more around personal time management as well as structuring a plan to get work that can have a lot of dependencies completed on time.
Passion for Quality. If you don’t have this, you need not apply for any engineering job let alone test.
Strategic Insight. This competency usually starts weak with new hires, but when we look to hire employees who can help us find that great breakthrough that vaults us ahead of the competition and adds to shareholder value, this competency must be present from the very beginning.
Confidence. Testers at Microsoft still get a lot of pushback on bugs. A tester must have confidence to push back when needed.
Impact and Influence. Influence comes from confidence and experience. Impact comes from knowing how to make a change happen. Starting off, most candidates exhibit this competency when they talk about how they drove a change in their current company or rallied their project team while in college.
Cross-Boundary Collaboration. Innovation often happens across organizational boundaries. An employee with a bunker mentality that just cares about her feature and her tests won’t be successful.
Interpersonal Awareness. This competency is about self-awareness. Many great candidates are self-critical and able to express how they are looking to improve their skills. Call this the continuous personal improvement plan.
Campus Recruiting
A big part of Microsoft’s hiring is done through campus recruiting. Campus recruiting is the term we use to describe when we hire someone within a year of his or her graduation from an undergraduate or graduate program. Hundreds of recruiters work year-round with university programs, developing relationships with schools and professors, and providing information about Microsoft and the different engineering careers we offer.
As candidates are identified, the Microsoft interviews are scheduled. These first interviews usually happen on campuses or at regional events, but sometimes candidates come to Microsoft for their interviews. Hiring managers such as SDET or SDE managers actually conduct the formal face-to-face interviews. To be a campus interviewer you must be an “A” interviewer. Yes, interviewing is so important to Microsoft we track and rate the interviewing skills of our employees.
It’s not unusual for the candidates to try to cram for their Microsoft interview by going out on the Internet and researching the sample questions former candidates have posted. The problem is we look at those sites too and continue to evolve our questions over time.
After several rounds of interviews by different “A” interviewers, a hire or no-hire decision is made. Whether an offer of full-time employment is made or not our goal is to ensure that the interview process is a good experience for all candidates. They wouldn’t have gotten this far in the process if they weren’t smart and motivated.
After the offer is made, we often have to go into sell mode. For tester jobs this is when we explain how test at Microsoft is very different from test at most other companies. For a number of years, we encouraged candidates to read such books as Testing Computer Software (2nd ed.) by Cem Kaner, Jack Falk, and Hung Quoc Nguyen (Wiley, 1999), but those books didn’t really cover the technical side of testing at Microsoft.
Fortunately, one of our Test Architects, Keith Stobie, was able to convince author Robert Binder to allow us to refer candidates to a portion of his book on testing: Testing Object-Oriented Systems: Models, Patterns, and Tools (Addison-Wesley, 1999), specifically, Chapter 3, “Testing: A Brief Introduction.” This chapter provides a concise description of testing very much in alignment with how Microsoft approaches testing. Although the book as a whole covers a great deal more than an introduction to testing, I still refer all testers to Chapter 3 of “the Binder book.”
Industry Recruiting
Industry recruiting is the term used to describe when we hire someone who has been working full time in software or a related industry for at least a year or more. You can consider industry to be more along the lines of hiring an experienced engineer as compared to a campus hire who is all about potential. We do hire entry level from industry, so it is not all about experience.
About half of our yearly hiring for SDETs comes through the campus recruiting pipeline, the other half through industry. Although some industry candidates do come from a test role at other companies, most, including the authors of this book, did not work full time in a testing position. When hiring from the industry, we often look for developers who have done some test work or who have been in a position where quality practices are prevalent. Usually, when we see résumés from individuals with a number of years in a test job, they might not have the computer science or programming skills we are looking for. The ideal industry candidate is usually someone who is working at a company where the role of product developer and software tester are combined.
The biggest shock to most industry candidates is the sheer size of the test discipline at Microsoft and the amount of clout we wield. The developer to tester ratio at Microsoft is about 1 to 1. Typical for industry is about 5 developers to 1 tester and sometimes 10 to 1 or higher. When you are outnumbered like that, a test organization is usually focused on survival rather than evolving their engineering skills and practices.
Learning How to Be a Microsoft SDET
After new employees starts at Microsoft, they go through the on-boarding process. This consists of going through New Employee Orientation (NEO) for the first few days. NEO combines all employees into a single class regardless of discipline. After NEO, new SDETs find their team administrator to find out what office they are in. Next, it’s off to find their immediate manager and find out who their mentor will be.
Among other things, the Engineering Excellence (EE) group at Microsoft is chartered to deliver technical training to employees. The first test course from Engineering Excellence a new SDET takes is Testing at Microsoft for SDETs. This class is usually taken within 12 months of the employee start date. Portions of this book cover much of what we teach in this 24-hour course. Although many of our technical classes and lecture series are online, milestone classes such as this one and the others in the SDET Training Roadmap, as shown in Figure 2-3, are taught in classrooms by experienced testers. This approach allows the students a rich opportunity for extensive discussions and to go deeper in many areas during exercises.
The Engineering Career at Microsoft
Any employee can change disciplines, and many do each and every year. In each engineering discipline, there are really only two options: to be an individual contributor (IC) or a manager. All junior engineers start as ICs. Each career path has major inflection points. For managers, it might be going from managing a team of individual contributors (a Lead position) to managing other Leads. For an IC, the scope expands from affecting a product to affecting a product line. These inflections are called career stages. Our use of career stages is based largely upon the work Stephen Drotter did while at General Electric.[1] Each career stage comes with a full career stage profile that helps each employee see the expected results for the next career step.
It is fairly common for an employee to “test the water” of the management career path, and then to opt back to being an IC. As senior ICs continue to develop, they require many of the same leadership skills and business acumen skills of senior engineering managers, as shown in Figure 2-4. In some cases, even very senior engineers can jump back and forth between IC and management roles.
For example, David Cutler joined Microsoft in 1988 to drive the creation of Microsoft Windows NT, which is still at the core of every release of the Windows operating system. Although David has managed teams during his career at Microsoft, he is generally considered an Architect and has risen to be the most senior individual contributor at Microsoft. He is recognized for his technical prowess, knowledge of systems architecture, and industry influence rather than his skills managing and developing employees.
Two other examples include Eric Rudder (Senior Vice President, Technical Strategy) and Jon DeVaan (Senior Vice President, Windows Core Operating System Division). Both men have spent multiple years in small team strategic leadership roles, and both have managed organizations made up of thousands of engineers.
The highest level a person can go on the individual contributor career path is Technical Fellow. This title is equivalent to being a Senior Vice President on the management career path. Technical Fellows and Distinguished Engineers (equivalent to a Corporate Vice President) often participate in industry efforts such as standards bodies and are viewed in the software industry as the best in their area of expertise. These senior engineers work on real shipping products but also do other company-wide influencing work such as participating in the Bill Gates ThinkWeek process by submitting their own whitepapers and commenting on others.
Career Paths in the Test Discipline
In some companies, test is seen as a junior job with development being a position that a tester could potentially grow into. At Microsoft, the test position is completely parallel to development with all the same career possibilities.
The Test Architect
In 1999, the Test Architect title was created for senior ICs with product-wide impact. The Test Architect title reflects the broad product impact role the SDET is working in, whereas the more common Senior, Principle, or Partner SDET titles are used when an individual focuses on features of a product. An important point to keep in mind in this discussion is that a Test Architect is a role and not a position. Although there is definitely a path for a senior tester to become a Test Architect, not all senior testers become Test Architects. An organization will generally choose to create a role for a Test Architect when there is sufficient business need or strategic requirement to do so. You will also see senior testers who function very much like a Test Architect, but who do not have the Test Architect title. This is a discussion of the role of a Test Architect, and not a discussion of the title.
There is no “typical” Test Architect role. Test Architects focus on a diverse set of goals and perform a wide variety of tasks. Some spend time developing testing infrastructure, testing authoring frameworks, or evaluating features to create complex tests. Some are in charge of a particular technology for their group. Others spend time consulting on how to improve test effectiveness. The common thread across all Test Architect roles and the primary responsibility of a Test Architect is to provide technical leadership and strategic direction for their testing organization. The level of the Test Architect generally dictates whether their scope is primarily focused on a family of features, a product line, or across an entire division. It is also expected that in addition to accountability to the current product senior Test Architects consistently look beyond the current release and possibly have several deliverables not tied to a particular product release.
A Test Architect is expected to be able to effect change not only across the testing community, but between development and program management as well. Test Architects must drive quality across all disciplines, providing guidance, feedback, and suggestions to improve quality practices across an entire engineering team.
It is also important to note what the Test Architect title is not. Test Architect is not a title that is awarded simply by level or experience. Creation of a Test Architect is an investment that combines an area in need of assistance with a strong individual capable of delivering invigorating change. It is also worth reiterating that Test Architect is not a career track. The skills of a Test Architect align with the career stage profile criteria for similar levels with an emphasis on cross-organizational communication and ability to drive change across an organization.
The IC Tester
The IC career path in test starts at SDET 1 (also known as IC1) and progresses through Partner SDET (IC6), as shown in Table 2-2. The major differentiators of the different levels are technical depth, technical breadth, and scope of influence. An SDET 1 is usually learning test, learning about Microsoft and how we develop software, and finding bugs in a very well defined feature. A more senior SDET might specialize in an area such as performance testing or security testing. By the time an SDET reaches Partner level, she should have spent time in many of these roles and probably a few years as a Test Architect.
Career stage name | Software Development Engineer in Test | Software Development Engineer in Test 2 | Senior Software Development Engineer in Test | Principal Senior Software Development Engineer in Test | Partner Software Development Engineer in Test |
Customer impact | Seeks out customer feedback by means of PSS and other channels to clarify features and write test cases | Interacts directly with customers to provide critical feedback on feature areas and to develop test cases | Addresses customer expectations concerning product integration and creating specific scenarios or personas | Implements customer connection techniques that improve the interaction between customers and the organization | Leads deep customer understanding across the product line to improve designs |
Test impact | Clarifies how features should work to eliminate ambiguous requirements | Provides critical feedback that improves specifications and technical designs | Identifies design patterns that are at high risk for generating future bugs | Leads innovation in test methods and technologies across the major product | Leads innovation in test methods and technologies across the product |
Scope of influence expands from a narrowly defined feature to a set of features, a full product such as Microsoft Office Word or Windows Media Player, and finally to a product line such as Office or Windows. The influence can be broad based on all aspects of testing such as the Test Architect (TA) role, or it might center around a deep technical area such as protocol security.
The IC career path for engineers does not stop at Partner SDET, but the test career path does. Partner SDET is one level below Distinguished Engineer (Corporate VP level IC). This is not because Microsoft does not see a need for a Distinguished Engineer (DE) in the test discipline. It is our belief that as engineers progress further and further along their career paths they start to look and behave more similarly to each other and the discipline distinction becomes less valuable. In a sense, every IC from every one of the 10 engineering disciplines can progress to the point that they are back to being just another engineer.
Many well-known industry luminaries exist in the Microsoft tester ranks with successful blogs, frequent conference appearances, and books. James Whittaker, author of the How to Break Software series, joined Microsoft in 2005 as a member of the TrustWorthy Computing team focused on software reliability. Keith Stobie writes the TestMuse blog on Spaces.live.com and is active in planning and supporting the Pacific Northwest Software Quality Conference (PNSQC) held annually in Portland, Oregon. The Test Verification and Metrics team headed by Tom Ball is a small team of researchers producing new and unique results around the use of data to drive product quality improvements.[2] Every member of the team is recognized widely for their research on test metrics.
These are just a few examples of how we invest in developing and supporting world-class test experts at Microsoft.
Becoming a Manager is Not a Promotion
Test management is the other major career path for an SDET. A test manager can move up the ranks, managing larger teams and areas. Of course, the management career path is one of an ever narrowing pyramid, so many test managers plateau long before reaching VP. There is one more major inflection point for the test management career path, as shown in Table 2-3, and that is to become a general manager. As described in Chapter 1, general engineering managers run organizations made up usually of Dev, Test, and Program Management (PM).
Lead Software Development Engineer in Test | Software Development Engineer in Test Manager | Director, Software Development Engineer in Test | |
Career stage | Manager | Manager of managers | Functional leader |
Product scope | Feature Areas | Product | Product Line |
An SDET Lead works on a collection of feature areas, a highly complex feature area or component forming a small subsystem, or a simple product. Examples of feature areas include the speech recognition server, the C# compiler, the graphics engine for Microsoft Office PowerPoint, and the IP stack. | An SDET Manager works on a major product or highly complex feature areas forming a product, a large subsystem, or a simple product line. SDET Managers are major contributors to the product line. Examples of a product include Word, Microsoft Money, and the Windows shell. | An SDET Director works on a product line generally representing a Profit and Loss center (P&L) or a highly complex system or architecture underlying a product line. Examples of a product line include Windows, Office, MSN, and Microsoft Exchange. | |
Recruiting | Leads recruiting processes for the team | Proactively optimizes the group’s recruiting processes and practices | Leads a comprehensive and effective recruiting plan for the product line |
One question common at Microsoft, as well as in the whole software industry, is what it takes to get “promoted” into test management. That is a fairly loaded question because it implies that managers get paid more or perhaps get a better office than individual contributors do. The reality is that when an engineer moves into a management position, it is a lateral move—that is, there is no promotion involved. Future promotions are based upon the engineer’s skills and individual results in leading his team. This is a different model from that used in most companies and public sector jobs where the job defines the pay scale and management jobs are almost always a higher pay scale than nonmanagement jobs are.
SDET Lead is the first rung of management in Test. An SDET Lead manages a team ranging in size from 2 to 10 employees. The team size is most often based upon the work needed to ship a specific feature or component such as printing, graphics, or a shared feature. The team might also be organized to drive a specialized test area such as performance, scale, or security. All of these employees report directly to the SDET Lead. There are no other layers of management between them and their employees.
It is important to note that technical complexity and the skills of the SDET Lead are more important in determining career advancement and rewards than is the size of the test team the SDET Lead manages. Security is an example where a small, highly skilled team can have a major impact on product quality and where you might find a more senior-level SDET Lead.
With smaller teams, the SDET Lead is expected to do a lot of his own testing, coding, analysis, and filing of bugs. All members of the product team from the highest executive down to the most junior new hire are expected to file product bugs when they find them. As a team becomes larger, more management tasks fall on the Lead and the less time he has for hands-on engineering work. Regardless of team size, SDET Leads are expected to have strong technical skills and function as technical leaders for their teams. Often, the SDET Lead is the most knowledgeable engineer on the team for a feature area and is typically one of the best testers and developers on the team.
The expectation for Leads to be very hands-on and technical is consistent with this expectation for all engineering disciplines. Development Leads often contribute to product development as much as any member on their team. Program Management Leads often design the most complex features or handle the most complex coordination issues. The expectation that all engineers, whether in a management role or not, are technical and hands-on is core to Microsoft DNA. It is easy to trace this culture back to the very early days of Microsoft, when Bill Gates would read through code at night and even rewrite portions.
Test Managers
Test Managers have many different titles and a wider variation in roles than SDET Leads do. As discussed in Chapter 1, Microsoft is a big company, and for every rule there are exceptions. Titles for management positions also have rules and variations. Table 2-4 lays out the most common titles and how they typically reflect team size and roles.
Title | Team size | Org depth |
SDET Lead or Senior SDET Lead | 2-10 | 1 |
Test Manager, Senior SDET Manager, or SDET Manager | 15-50 | 2 |
Group Test Manager, Principal Test Manager, or Director of Test | 30-100 | 3-4 |
General Manager or VP of Test | 200+ | 4-5 |
Test Managers are less hands-on with respect to writing and running tests, but every tester, regardless of level, files bugs in every product cycle. Test Managers are still expected to be technical but do tend to focus more on the processes and tools used in testing than how best to automate a particular class of tests.
A Test Manager spends a lot of time focused on how to develop and improve the skills of the test team and working with Product Management on assessing product quality and readiness to ship.
Summary
Microsoft takes a unique approach to software testing compared to industry norms. We have more test engineers than developers, and we emphasize software engineering skills with all testers. This unique approach goes all the way down to the title we give testers: Software Development Engineer in Test (SDET). By design, this title is virtually the same as the Software Development Engineer title used for developers.
With Microsoft adding more than 500 new tester positions every year, we have a very active recruiting program to hire experienced engineers from other companies as well as graduates from computer science and related programs upon graduation. Most new testers do not know much about software testing, so we have a very strong training program to teach software testing techniques.
Our emphases in very technical approaches to software testing allow us to have a fully articulated career path for testers both in management and as individual contributors (ICs). ICs can have the ability to grow to the most senior levels of engineering just as they do in the development discipline.
The 9,000 test engineers at Microsoft play a major role in developing our products and ensuring that the products ship with high quality. This vibrant community of engineers uses a wide range of engineering techniques to drive continuous improvement in our engineering practices and products.
[1] Ram Charan, Stephen Drotter, and James Noel, The Leadership Pipeline: How to Build the Leadership Powered Company (San Francisco: Jossey-Bass, 2000).
[2] Microsoft Research, “Software Reliability Research,” http://research.microsoft.com/research/srr.