Software Testing with Visual Studio 2010

Use Visual Studio 2010's breakthrough testing tools to improve quality throughout the entire software lifecycle

Together, Visual Studio 2010 Ultimate, Visual Studio Test Professional 2010, Lab Management 2010, and Team Foundation Server offer Microsoft developers the most sophisticated, well-integrated testing solution they've ever had. Now, Microsoft MVP and VS testing guru Jeff Levinson shows exactly how to use Microsoft's new tools to save time, reduce costs, and improve quality throughout the entire development lifecycle.

Levinson shows how Microsoft's new tools can help you finally overcome long-standing communication, coordination, and management problems. You'll discover how to perform first-rate functional testing; quickly create and execute tests and record the results with log files and video; and create bugs directly from tests, ensuring reproducibility and eliminating wasted time. Levinson offers in-depth coverage of Microsoft's powerful new testing metrics, helping you ensure traceability all the way from requirements through finished software.

Coverage includes:

· Planning your tests using Microsoft Test Manager (MTM)

· Creating test settings, structuring test cases, and managing the testing process

· Executing manual tests with Microsoft Test Manager and Test Runner

· Filing and resolving bugs, and customizing your bug reporting process

· Automating test cases and linking automated tests with requirements

· Executing automated test cases through both Visual Studio and Microsoft Test Manager

· Integrating automated testing into the build process

· Using Microsoft's Lab Management virtualization platform to test applications, snapshot environments, and reproduce bugs

· Implementing detailed metrics for evaluating quality and identifying improvements

Whether you're a developer, tester, manager, or analyst, this book can help you significantly improve the way you work and the results you deliver - both as an individual right now , and as a team member throughout your entire project .

List of contents

Foreword    xvii
Preface    xxi
Acknowledgments    xxix
About the Author    xxxi
1  State of Testing    1
Software Testing Challenges    1
The Need for Testers    3
A Different Approach    5
Fixing Communication    5
Increasing Project Visibility    6
What Are the Tools Designed to Do?    7
Metrics    10
Citations    12
2  Software Quality and Testing Overview    13
Software Quality    13
Requirements    14
Business Value    14
Expectations    15
Nonfunctional Requirements    15
Where Do You Build Quality?    17
Process and Quality    19
Software Testing    19
The Testing Mindset    20
Software Testing Strategies    21
Types of Software Testing    22
Test Management    27
After the Product Is Shipped or Deployed    27
3  Planning Your Testing    29
Microsoft Test Manager    30
Test Plans    36
Properties    38
Contents    43
Adding Suites and Test Cases to Your Plan    46
Testing Configurations    48
Managing Test Configurations    49
Assigning Test Configurations    51
Assigning Testers    53
Test Case Planning Workflow    55
Analysis and Initial Design    56
Construction    61
User Acceptance Testing    62
Common Scenarios    64
Scheduling and Tracking Test Case Creation and Execution    64
Feature Driven Development    65
Moving from One Iteration to Another    67
Handling Different Test Configurations    68
4  Executing Manual Tests    71
Using the Test Case Work Item Type    72
Shared Steps    75
Data Driven Test Cases (Test Parameters)    77
Running Your First Tests    79
Test Runner    80
Examining Test Results    92
Test Run Results    93
Detailed Test Results    95
Exploratory Testing with MTM    101
5  Resolving Bugs    107
A Bug's Life    107
Customer Reported Bug    110
Test Team Reported Bug    110
Triaging the Bug    110
Reactivations    111
Bug Differences and Modifications    111
The Generated Bug    116
How a Developer Uses IntelliTrace    120
Fixing the Bug    122
Associated Changesets    124
Associated Work Items    124
Impacted Tests    125
Setting the Build Quality    125
Assigning a New Build    127
Verifying That the Bug Is Fixed    129
Dealing with Impacted Tests    131
6  Automating Test Cases    135
To Automate or Not to Automate    136
The Automated Testing Framework    139
Creating an Automated Test from a Manual Test    141
Examining a Generated Web Application Coded UI Test    142
Adding Validations    157
Adding Additional Recorded Steps    164
Parameterized Coded UI Tests    166
Handling Issues Due to Inconsistency    168
Resolving the Data Inconsistency    169
Handling Dynamic Values    172
Other Tips    177
Combining Multiple Tests    178
Associating Coded UI Tests and Test Cases    178
7  Executing Automated Test Cases    183
Executing Automated Tests Through Visual Studio    183
Local Execution    184
Local Execution with Remote Collection    184
Remote Execution    185
Executing Automated Tests from the Command Line    190
Executing Automated Tests in MTM    191
Creating an Automated Build    191
Setting Up the Physical Environment    193
Running a Coded UI Test Through MTM    196
Executing Automated Tests with Team Build    202
Automated Testing Gotchas    205
Custom Dialogs    205
Cleaning Up Your Tests    207
8  Lab Management    209
Managing Virtual Environments Through MTM    210
Finishing Virtual Environment Configuration    217
Automated Test Settings    221
Lab Management Workflow    222
Executing a Lab Build    231
Running Automated Tests Through MTM    233
Manual Tests in a Virtual Environment    234
9  Reporting and Metrics    239
Understanding the Reporting Structure    240
Built-In Reports    242
Bug Status    244
Bug Trends    245
Reactivations    246
Build Quality Indicators    246
Build Success over Time    248
Build Summary    249
Stories Overview    250
Test Case Readiness    251
Test Plan Progress    252
Excel Services Reports (Dashboards)    253
Reporting with Microsoft Excel    254
Creating a Generated Report    255
The Testing Measures    256
Metrics    268
What to Measure    271
First-Time Defect Rate    273
Bug Reactivations    276
General Bug Counts    277
Index       283
 

About the author

Jeff Levinson has 16 years of experience in software development in many different roles-developer, tester, architect, project manager, and scrum master at several large companies. Jeff is currently a Senior Application Lifecycle Management (ALM) Consultant for Northwest Cadence, which is a company that specializes in Team Foundation Server, Visual Studio, methodologies, and process improvement. In his day-to-day work, Jeff helps teams, organizations, and companies adopt more efficient processes, improve quality, and reduce costs associated with software development. Jeff is a frequent speaker at industry events and writes a twice-monthly column for Visual Studio Magazine Online. This is his fourth book on software development. His other books are Building Client/Server Applications with VB.NET (Apress 2003), Pro Visual Studio Team System 2005 (Apress 2006), and Pro Visual Studio Team System with Team Edition for Database Professionals (Apress 2007). Jeff has a master's degree in software engineering from Carnegie Mellon University and is an MCP, MCAD, MCSD, MCDBA, and MCT.
 
Jeff currently lives in Washington State with his wife and two children.
 

Summary

Use Visual Studio 2010's Breakthrough Testing Tools to Improve Quality Throughout the Entire Software Lifecycle
 
Together, Visual Studio 2010 Ultimate, Visual Studio Test Professional 2010, Lab Management 2010, and Team Foundation Server offer Microsoft developers the most sophisticated, well-integrated testing solution they've ever had. Now, Microsoft MVP and VS testing guru Jeff Levinson shows exactly how to use Microsoft's new tools to save time, reduce costs, and improve quality throughout the entire development lifecycle.
 
Jeff demonstrates how Microsoft's new tools can help you finally overcome long-standing communication, coordination, and management challenges. You'll discover how to perform first-rate functional testing; quickly create and execute tests and record the results with log files and video; and create bugs directly from tests, ensuring reproducibility and eliminating wasted time. Levinson offers in-depth coverage of Microsoft's powerful new testing metrics, helping you ensure traceability all the way from requirements through finished software.
 
Coverage includes
 
•    Planning your tests using Microsoft Test Manager (MTM)
•    Creating test settings, structuring test cases, and managing the testing process
•    Executing manual tests with Microsoft Test Manager and Test Runner
•    Filing and resolving bugs, and customizing your bug reporting process
•    Automating test cases and linking automated tests with requirements
•    Executing automated test cases through both Visual Studio and Microsoft Test Manager
•    Integrating automated testing into the build process
•    Using Microsoft's Lab Management virtualization platform to test applications, snapshot environments, and reproduce bugs
•    Implementing detailed metrics for evaluating quality and identifying improvements
 
Whether you're a developer, tester, manager, or analyst, this book can help you significantly improve the way you work and the results you deliver-both as an individual right now, and as a team member throughout your entire project.
 

Product details

• Authors: Steven Borg, Jeff Levinson
• Publisher: Addison-Wesley Longman, Amsterdam
• Languages: English
• Product format: Paperback / Softback
• Released: 16.03.2011
• EAN: 9780321734488
• ISBN: 978-0-321-73448-8
• No. of pages: 336
• Dimensions: 197 mm x 225 mm x 1 mm
• Weight: 565 g
• Illustrations: w. ill.
• Series: Microsoft .net Development; The Agile Software Development Series; Addison-Wesley; Addison-Wesley; The Agile Software Development Series
• Subject: Natural sciences, medicine, IT, technology > IT, data processing > IT