Advanced Windows Debugging ebook free download

  

  

“Who says you can’t bottle experience? Between the covers is a wealth of information that

clearly demonstrates how to take a logical approach to finding and eliminating bugs. This

is an absolute must-have book for anyone who develops, tests, or supports software for

Microsoft Windows.”

  —Bob Wilton, Escalation Engineer, Critical Problem Resolution Team, Microsoft Corporation

“I have been fortunate enough to personally work with the authors on extremely demand-

ing systems projects for more than eight years. This volume contains the kind of stuff we

all wish we had known back at the beginning of those projects—the kind of stuff that the

debugging guru tells you over a coffee-spilled keyboard on February 29 only because an

extra day showed up and he has the afternoon free; the kind of stuff that only comes

from actually building and then debugging complex systems projects instead of just read-

ing about somebody else doing it.

  

“ Most books leave the advanced cases as ‘exercises to the reader’ or to ‘other, more

advanced books,’ and those never seem to materialize. This book is one of those very rare

‘other’ books. Get two copies. You will always be lending the other one out.”

  —Raymond McCollum, Architect, Microsoft Forefront Security Products

“This book by Microsoft authors Mario and Daniel is an excellent reference for both

intermediate and advanced debuggers. In-depth examples showing how to debug intri-

cate problems, such as stack and heap corruptions, make this book stand out among cur-

rent available literature on debugging Win32 software on Windows. The book is highly

practical and is filled with numerous debugging tricks and strategies.”

  —Kinshuman, Development Lead, Windows Core OS Division

“I am pleased to see this guided tour through a comprehensive set of clever debugging

techniques. It does not only tell how to deal with tough diagnosis problems, but it also

explains the mechanisms behind the techniques used. The pragmatic approach taken in

Advanced Windows Debugging makes it a good resource to understand several key

Windows areas.”

  —Adrian Marinescu, Software Architect, Microsoft Corporation

  

“Advanced Windows Debugging fills the need for good documentation about debugging

and fixing software defects. The book is based on the authors’ valuable experience of

tracking down the cause of various classes of software bugs. It includes representative

examples of typical defects, the tools used to investigate these defects, and step-by-step

instructions for using these tools. Software developers and testers will greatly benefit

from becoming familiar with these examples.”

  —Daniel Mihai, Software Design Engineer, Developer Productivity Tools, Microsoft

“I wrote the WinDbg symbol handler, Symbol Server, and Source Server. Even so, I can’t

get my own wife to use WinDbg. She thinks it is hard to use, and, consequently, she

hasn’t learned of the potential of this toolset. I am buying a copy of this book, so she can

learn it. The chapters on postmortem debugging and memory corruption are essential

reading that provide real insight into the internals of the runtime and OS in the context

of a program fault. Mario and Daniel’s understanding of debugging comes from being

asked to resolve completely unexplained bugs in unfamiliar target programs. This is what

industrial strength debugging is all about.”

  —Pat Styles, Microsoft

  A DVANCED W

  

INDOWS

D EBUGGING

  This page intentionally left blank

  A DVANCED W

  

INDOWS

D EBUGGING Mario Hewardt Daniel Pravat

  Upper Saddle River, NJ • Boston • Indianapolis • San Francisco New York • Toronto • Montreal • London • Munich • Paris • Madrid Cape Town • Sydney • Tokyo • Singapore • Mexico City Many of the designations used by manufacturers and sellers to distinguish their products Editor-in-Chief are claimed as trademarks. Where those designations appear in this book, and the pub- Karen Gettman lisher was aware of a trademark claim, the designations have been printed with initial

  Acquisitions Editor capital letters or in all capitals.

  Joan Murray The authors and publisher have taken care in the preparation of this book, but make no Senior Development Editor expressed or implied warranty of any kind and assume no responsibility for errors or omis- Chris Zahn sions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.

  Managing Editor Gina Kanouse The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers

  Copy Editor and content particular to your business, training goals, marketing focus, and branding Rhonda Tinch-Mize interests. For more information, please contact:

  Indexer U.S. Corporate and Government Sales Brad Herriman (800) 382-3419

  Proofreader corpsales@pearsontechgroup.com Karen A. Gill

  For sales outside the United States please contact: Editorial Assistant

  International Sales Kim Boedigheimer international@pearsoned.com Cover Designer Chuti Prasertsith Composition TnT Design

  Visit us on the W Library of Congress Cataloging-in-Publication Data: Hewardt, Mario.

  Advanced windows debugging / Mario Hewardt, Daniel Pravat. p. cm. Includes index.

  

ISBN 0-321-37446-0 (pbk. : alk. paper) 1. Microsoft Windows (Computer file) 2. Operating systems (Computers)—

Management. 3. Debugging in computer science. I. Pravat, Daniel. II. Title. QA76.76.O63H497 2007 005.4’46—dc22

  2007030163 Copyright © 2008 Pearson Education, Inc. All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission

must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in

any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permis-

sions, write to:

  Pearson Education, Inc. Rights and Contracts Department 501 Boylston Street, Suite 900 Boston, MA 02116 Fax (617) 671 3447

  

This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, v1.0

or later

ISBN-13: 978-0-321-37446-2

  ISBN-10: 0-321-37446-0 Text printed in the United States on recycled paper at Edwards Brothers in Ann Arbor, Michigan. First printing October 2007.

  

To my wife Pia, whose support, patience, and encouragement helped

make this book a reality. To the familia who taught and encouraged

me to follow my dreams and passions.

  

Mario Hewardt

To Claudia, Alexis, and Edward

Daniel Pravat

  This page intentionally left blank

  ONTENTS C

  

  

  

ix

  Contents

  

  Contents

  

Chapter 14: Power Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .691 Debug Diagnostic Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .691

  

  

  

  Software has one goal: simplify. If there’s a workflow that can be optimized or auto- mated, data that can be stored or processed more efficiently, software steps in to fill the job. While simplifying, software must not introduce undo complexity, and there- fore should install with minimal user interaction, seamlessly integrate services and data from other applications and multiple sources, and be resilient to changes in its soft- ware and hardware environment. For the most part, software magically just works.

  However, while software strives to simplify the experiences of end users and administrators, it has become more and more complex. Whether it’s the amount of the data they work with, the number of applications with which they communicate, their degree of internal parallelism, or the APIs they import directly and indirectly from the software stack upon which they run, most of software’s apparent simplicity hides a world of subtle timings, dependencies, and assumptions that run between lay- ers of software, often across different applications and even computers. Just deter- mining which component is at fault—much less why, for a problem that surfaces as a crash in a library, a meaningless error message, or a hang—is often daunting.

  The reason you’re reading this book is that you develop, test, or support software, and therefore face breakdowns in software’s myriad moving parts that you are charged with investigating through to a root cause and maybe fixing. Success in this endeavor means identifying the source of a problem as quickly and efficiently as pos- sible, which requires knowing what to look with, where to look, and how to look. In other words, succeeding means knowing what tools are at your disposal, which ones are the most effective for a class of failures, and how to apply the tool’s features and functionality to quickly narrow in on the source of a problem.

  Learning how to troubleshoot and debug Windows applications on the job has, for the most part, been the only option, but when you debug an application failure, knowing about that one obscure tool or scenario-specific debugger command can mean the difference between instantly understanding a problem and spending hours or even days hunting it without success. That’s why a book like this pays for itself many times over.

  Advanced Windows Debugging takes the combined knowledge and years of

  hands-on experience of not just Mario and Daniel, but also the Microsoft Customer

  Support Services and the Windows product and tools development teams and puts it at your fingertips. There’s no more authoritative place to learn about how the Windows heap manager influences the behavior of buffer overflows or what debugger extension command you should use to troubleshoot DCOM hangs, for example. I’ve been debugging my own Windows applications and device drivers for over 10 years, but when I reviewed the manuscript, I learned about new techniques, tools, and debugger commands that I’d never come across and that I’ve already found use for.

  We all earn our pay and reputations not by how we debug, but by how quickly and accurately we do it. Whether you’ve been debugging Windows applications for years or are just getting started, Mario and Daniel equip you well for your bug hunting expeditions. Happy hunting! Mark Russinovich Technical Fellow, Platform and Services Division Microsoft Corporation

   Not long ago, we were reminiscing about a really tough problem we faced at work.

  The Quality Assurance team was running stress tests on our product, and every four or five days, a crash would rear its ugly head. Sure, we had debugged the crash as far as we thought possible, and we had done extensive code reviews to try to figure it out, but alas, not enough information could be gained to get to the bottom of it. After sev- eral weeks of unfruitful attempts, we started looking for alternative approaches. During a random hallway conversation, someone happened to casually mention a tool called gflags. Having never heard of this tool before, we set out to do some research to find out how it could help us get to the bottom of our crash. Unfortunately, the learning process proved to be somewhat difficult. First, finding information about the tool proved to be a real challenge. There was a ton of great information in the refer- ence documentation that came with the tools, but it was hard to figure out how to actually get started. We quickly realized that without some basic guidance, there was little hope for us to be able to utilize the tool. Naturally, we decided to ask the per- son who had happened to mention the tool if he knew of any documentation or point- ers. He gave us some brief descriptions of the tool and, perhaps more importantly, the names of other people who had worked with the tool extensively. What followed was a series of long and instructive conversations, and bit by bit the basic idea behind the tool started falling into place.

  Did we ever get to the bottom of the crash? Yes—we did. As a matter of fact, enabling the correct tool while running our stress tests pinpointed the problem to such accuracy that it only took an hour of code reviewing to locate and fix the misbe- having code. Had we known about this tool and how to use it from the start, we would have saved several weeks of work. From that point on, we dedicated quite a lot of time to furthering our understanding of the tools and how they can help while trying to troubleshoot misbehaving code.

  Over the years, the Windows debuggers and tools have matured and grown and become increasingly powerful. The amount of timesaving features now available is truly mind-boggling. What is equally mind-boggling is that after several years, the native debuggers and tools are still relatively unknown to developers. The few devel- opers who do find out that these tools exist have to go through a similarly painful learn- ing process as we did years ago. We were fortunate to have the luxury of working with

  xv Preface

  engineers at Microsoft (some of whom wrote the tools), but without this luxury, many hopeful developers end up at a dead end and are never able to reap the benefits of the tools. This unfortunate problem of a lack of learning material also turned out to be a great opportunity for a solution, and thus the idea for this book was born. The key to enable developers to gain the knowledge required is to provide a central repository of concise information that fully explains the ins and outs of the debugging tools and processes. The book you are holding serves as that key and is the net result of three years of writing and over 15 years of collective debugging experience.

  We hope that you will enjoy reading this book as much as we enjoyed authoring it and that it will open up the door to a truly amazing world of highly efficient soft- ware troubleshooting and debugging. Knowing how to use the tools and techniques described in this book is a critical part of a computer scientist’s work and can teach you how to very efficiently troubleshoot some of the toughest problems in software.

  Who Is This Book For?

  The short answer to this question is anyone who is involved in any facet of software development and has a strong desire to learn what is actually happening deep inside Windows. Although the technical nature of the book might make you believe that its content is only intended for advanced system engineers, this is absolutely not true. One of the key points of this book is the removing of the magic. For various reasons, a lot of software engineers believe that there is a magical relationship between the software they are working on and the operating system. When a problem surfaces that requires the analysis of operating system components (such as RPC/COM or the Windows heap manager), this preconceived notion of magic prevents them from venturing inside Windows to gain more information that can potentially help them solve the problem. To make effective use of this book, you will have to learn how to remove this precon- ceived notion and truly be of the mind-set that there is no magic behind-the-scenes. The core Windows components should be viewed as an extension of your product and not as a separate and magical layer. After all, it’s all just code—some of which just hap- pened to be written by other people. If you can adjust your mind-set to accept this, you will have taken your first steps to mastering the art of Windows debugging.

  Software Developers

  Anyone from a low-level system developer to a high-level RAD developer will benefit from reading this book. Whether your preference is writing Windows-based software in assembly language or by using the .NET framework, there is a ton of useful infor- mation to be learned about the tools and techniques behind Windows debugging.

  Preface

  Over the years, we’ve had several discussions with higher-level RAD developers who claim that they really don’t see the need to learn about these low-level topics. After all, the beauty of writing code at a higher level is that all of the low-level intricacies are abstracted and hidden away from the developer. We couldn’t agree more. However, our claim is that although abstractive programming allows the developer not to have to focus on low-level details, it does not negate the need to know how the abstraction really works. The substance behind this claim is simple. What you are working with is really just that—an abstraction. Usage of this abstraction in a design that it was not suited for can cause serious problems in your software; and, in such a case, without a solid understanding of how the abstraction works, it can mean the difference between shipping your product on time and slipping the release date by several months.

  Another key factor when considering mastering the Windows debuggers and tools is related to the debugging of live production servers. While every attempt should be made to fix bugs before shipping a product, we all know that some bugs might slip through the cracks. When these bugs do surface post release, it can be a real headache tracking them down. Customers who encounter the bugs on live produc- tion servers are typically very sensitive to downtime and configuration changes, mak- ing it impossible to install a complex debugger package. The Debugging Tools for Windows, on the other hand, enables live debugging with no server configuration change and no installation requirements. In short, it enables customers to keep a pris- tine server during the troubleshooting process.

  Quality Assurance Engineers

  Just as software developers will find the information in this book useful in their day-to- day tasks, so will quality assurance engineers. Quality assurance typically runs a battery of tests on any given component being tested. During this time, any number of bugs can surface. Whether they are memory corruptions, resource leaks, or hangs, knowing what extended instrumentation to enable during the test run can dramatically reduce the time it takes for root cause analysis. For instance, imagine that quality assurance is tasked with stress testing a credit card authorization service. One of the goals is that the service must be capable of surviving one week of continuous and simultaneous hammering by client requests. On day six, the service starts reporting errors for all client requests. At this point, the developers responsible for the service are called in to analyze the problem. It doesn’t take long for them to figure out that the server has run out of memory, presumably due to a small memory leak that accumulates over time. After six days of accumulated leaks, figuring out the source of the leak, however, is a much bigger challenge that can take days of debugging and code reviewing. Had the correct extended instrumentation been enabled while running these tests, the time it would have taken to analyze the leak could have been greatly reduced.

  Preface Product Support Engineers

  In much the same way that quality assurance uses the Windows debuggers and tools to make root cause analysis more efficient, so can the product support engineers. Product support faces many of the same problems that quality assurance and software developers face on a day by day basis. The key difference, however, is the environ- mental constraints that they work under. The constraints can include not having full access to the server exhibiting the problems, having a limited amount of time avail- able for troubleshooting the server, having limited access to customer source code, and other issues.

  The information presented in this book will give product support engineers a great deal of ammunition when tackling these tough problems. Knowing how to debug customer problems with minimal downtime and minimal system configuration changes enables product support engineers to much more efficiently and nonintru- sively gather the required data to get to the bottom of the problem.

  Where There Is a Will, There Is a Way

  It should come as no surprise that the material presented in this book is highly tech- nical in nature. We are not going to try and convince you that you don’t need to know anything about Windows internals to benefit from the book because the simple truth is that you do. As with any technically oriented book, a certain amount of knowledge is assumed.

  Curiosity and a Will to Learn

  While writing this book, we came to the realization that some of the areas of Windows we were writing about had been taken for granted. Sure, most of the time we knew that those areas worked a certain way, but we did not know exactly what made them

  

work that way. We could have simply accepted the fact that they just work, but curios-

  ity got the best of us (as it usually does). We spent quite a lot of time researching the topics and trying to connect the dots. The net result was a more in-depth under- standing of Windows, which, in turn, allowed us to more efficiently debug problems.

  The basic principle behind learning anything is that there must be a will to learn. Depending on your background, some of the high-level material in the book might feel intimidating. Embrace this intimidation, and you will be in a stronger position to fully grasp and understand the contents of this book.

  If you possess the will to learn and have a great deal of curiosity, you will be well on your way to becoming an expert in Windows debugging.

  Preface

C/C++

  All the sample code throughout the book is written in C/C++, and as such a good understanding of the language as well as its object layout is required. If some of the language concepts in the book are unfamiliar to you and you want to brush up on your C/C++ skills, we recommend the following books:

  The C++ Programming Language (3rd Edition), by Bjarne Stroustrup, Boston: Addison-Wesley, 2000. Inside the C/C++ Object Model, by Stanley B. Lippman, Reading, MA: Addison-Wesley, 1996.

  Windows Internals

  This book is about advanced Windows debugging, and as such parts of the book are dedicated to describing the internals of several integral Windows components (for example, heap manager, RPC, security subsystem). Our intentions are not to fully explain all aspects of these components but rather to give a brief but in-depth sum- mary of how the component functions in relationship to the debugging scenarios being illustrated. If you want to take your knowledge of the internals of Windows even further, we strongly recommend reading

  Microsoft Windows Internals, Fourth Edition: Microsoft Windows Server

2003, Windows XP, and Windows 2000, by Mark E. Russinovich and David

  A. Solomon. Redmond, WA: Microsoft Press, 2004.

  Organization

  The book consists of three major parts. In this section, we provide a short description of the contents of each chapter.

  Part I: Overview Part I lays the groundwork. It provides an overview of the tools and debuggers and

  lets you familiarize yourself with the fundamentals of the debuggers. Even if you are already familiar with the Windows debuggers, we strongly encourage you to, at the very least, skim through these chapters, as they contain a ton of valuable information.

  Preface

  Chapter 1, “Introduction to the Tools,” provides a high-level introduction to the tools used throughout the book. Topics such as download locations, installation instructions, and usage scenarios are detailed.

  Chapter 2, “Introduction to the Debuggers,” introduces the reader to the funda- mentals of the Windows debuggers. Basic concepts such as what debuggers are avail- able, how to use them, and how to configure them are covered.

  Chapter 3, “Debuggers Uncovered,” provides a more in-depth examination of user mode debuggers. A minimalist implementation of a debugger is provided, as well as looking at more advanced topics such as how the exception dispatch mechanism works.

  Chapter 4, “Managing Symbol and Source Files,” discusses how to maintain two of the most critical pieces of information during debugging: symbol files and source files. It gives a brief description of what symbol and source servers are, how to use them in association with the debuggers, and how to effectively manage them by set- ting up symbol servers and maintaining source servers for your organization.

  Part II: Applied Debugging The focus of Part II is to provide the reader with the opportunity to analyze common

  programming mistakes using the Windows debuggers. Each of the chapters in this section is focused on a particular category of problems, such as memory corruption, memory leaks, and RPC/COM. Each chapter begins with an overview of the Windows component(s) involved followed by one or more scenarios that illustrate common programming mistakes in that area.

  With the exception of Chapters 5 and 6, the chapters in Part II are standalone and can be read in any order.

  Chapter 5, “Memory Corruption Part I—Stacks,” and 6,” Memory Corruption Part II—Heaps,” take a close look at a very common problem that plagues develop- ers on a daily basis: memory corruptions. Chapter 5 focuses on stack corruptions, and Chapter 6 on heap corruptions. Each chapter begins by explaining the overall con- cept behind the type of memory being examined (stack and heap) and is followed by a number of common scenarios under which the corruption can occur. Each scenario has associated sample code and a walk-through of the process that is used during debugging and root cause analysis.

  Chapter 7, “Security,” discusses common security-related problems that often surface during development. Quite often, developers face situations in which an API returns an access denied error code without any more in-depth information, making it hard to understand or track down where the error is coming from. This chapter will show several security-related examples of code and how to use the debuggers and appropriate tools to get to the bottom of the issue.

  Preface

  Chapter 8, “Interprocess Communication,” focuses solely on interprocess com- munication debugging. Arguably perhaps the most used interprocess communication protocol in Windows but also the most magical is RPC/LPC. Knowing how to trou- bleshoot this important component is paramount when working with most applica- tions. Using the debuggers, this chapter will show how you can track identity, analyze RPC failures, and much more.

  Chapter 9, “Resource Leaks,” details a very common problem with software today: resource leaks. The most common form of resource leaks is related to memo- ry but not limited to it. Other examples includes registry keys, file handles, and so on. This chapter takes a look at the resource leak problem by showing a number of sce- narios and associated sample code, as well as how to use the debuggers and tools to efficiently track them down.

  Chapter 10, “Synchronization,” discusses the topic of application hangs and how to most efficiently make use of the debuggers to track down synchronization prob- lems such as deadlocks and lock contentions. A number of different synchronization scenarios are examined with associated debug sessions that give an in-depth view of the analysis process.

  Part III: Advanced Topics Part III is an advanced section that consists of chapters that discuss topics such as

  postmortem debugging 64-bit debugging, Windows Vista fundamentals, and much more. The goal of these chapters is not to provide an exhaustive examination of each area, but rather provide just enough fundamentals for the reader to get started in the topic explained.

  Chapter 11, “Writing Custom Debugger Extensions,” talks about custom debug- ger extensions. Even though the Windows debuggers pack an extremely powerful set of commands and tools, there are times when you want to automate certain aspects of your own application debugging sessions. This chapter details how the extensibili- ty model of the debuggers works and describes an example of a sample custom debugger extension.

  Chapter 12, “64-Bit Debugging,” introduces the basic concepts of debugging 64- bit architectures. Basic concepts such as stack traces, function calls, and parameter passing are discussed to enable the reader to get started on debugging these power- ful architectures.

  Chapter 13, “Postmortem Debugging,” discusses postmortem debugging, which is an incredibly useful way of troubleshooting problems when there is no means of debugging a problem at the point of occurrence. This is a very common form of debugging once the product has shipped and problems surface on the customer site.

  Preface

  Chapter 14, “Power Tools,” discusses two powerful tools that can be used to auto- mate the debugging process. The first tool is called DebugDiag, and it provides an excellent way of automating resource leak debugging. The other tool is a command called analyze, which automates the initial fault analysis process.

  Chapter 15, “Windows Vista Fundamentals,” details some of the fundamentals behind Windows Vista. With the introduction of the new generation Windows plat- form, certain aspects of the operating system have changed dramatically, and some of the key changes are outlined in this chapter.

  Required Tools

  All the tools required to make full use of this book are available as downloads free of charge. The new Windows Drivers Kit contains a complete command-line C/C++ development environment and a great set of associated development tools.

  Sample Code

  As software engineers, we spend a great deal of our time hunting for the ultimate treasure of writing perfect code. While writing this book, we were faced with quite the opposite chore—the need to write not-so-perfect code to illustrate common pro- gramming mistakes.

  The sample code is structured to achieve one goal: present examples of common programming mistakes in the shortest and most concise fashion as to not pollute the basic principle of the programming mistake being examined. To satisfy the goal of short and concise examples, we had to, at times, concoct examples rather than use real-life examples. Even though the sample code is “made up,” it serves to simulate real-life examples, and every effort was made to ensure that the example stays true to the problem being examined.

  All sample code is written in C/C++. We chose this language for two simple reasons:

  ■ C/C++ is predominantly used in Windows development. ■

  In order not to obscure the debugging concepts discussed with higher-level abstractions, we chose the language that is most commonly used and also clos- est to the core.

  Preface

  All sample code is compiled and tested using the Windows Drivers Kit. The WDK was chosen so that readers would be able to enjoy learning the art of Windows debug- ging without being required to purchase a complete developer suite.

  The source code assumes a Unicode environment, and as such Win32 API calls, as seen in the debugger, will be illustrated using the Unicode version of the API. For example, the sample code might show a call to the CreateProcess API, but when

  CreateProcessW

  working in the debugger, the API will be utilized. The API shown in the debugger is prefixed by the module name implementing the API. One exam-

  CreateProcessW

  ple is the API, which is implemented in kernel32.dll. It is often required to specify both the module name and the API name separated by the (!)

  kernel32!CreateProcessW character ( ).

  All sample code and binaries are available on the book’s Web site ries being available, the site acts as a symbol and source code server for the book’s binaries. When you try out the debugging sessions illustrated in the book, there is no need to download all the symbols for the binaries; rather, point your debuggers sym- bol path directly to the book’s symbol server, and you can debug with remote symbols. The sources are also retrieved by the source servers from the book’s Web site.

  To provide a consistent learning experience, the binaries on the book’s Web site have been built as nonoptimized and checked releases for the x86 architecture using the Windows XP platform. We chose to use Windows XP as the common denomina- tor due to its widespread usage. If you choose to build the samples on your own using a different target platform, there might be minor variations in the debug output.

  To build the samples on your own, simply open a WDK build window and type

  build /ZCc

  from the directory containing the makefile. If the source code being compiled requires additional steps, those steps will be spelled out in the chapter dis- cussing the sample code.

  Throughout the book, it is assumed that all binaries have been downloaded from the Web site and copied to the local hard drive (keeping the folder structure intact)

C:\AWDBIN

  to the following location: , and the sources have been downloaded to the

  C:\AWD folder.

  Conventions mono-

  Code, command-line activity, and syntax descriptions appear in the book in a

  spaced

  font. Many of the examples and walk-throughs in this book show a great deal of what is known as debug spew. Debug spew simply refers to the output that the

  Preface

  debugger displays as a result of some action that the user takes. Typically, this debug spew consists of information shown in a very compact and concise form. In order to effectively reference bits and pieces of this data and make it easy for you to follow, the boldface and italic types are used. Additionally, anything with the boldface type in the debug spew indicates commands that you will be entering. The following exam- ple illustrates the mechanism.

  0:000> ~*kb . 0 Id: 924.a18 Suspend: 1 Teb: 7ffdf000 Unfrozen ChildEBP RetAddr Args to Child 0007fb1c 7c93edc0 7ffdf000 7ffd4000 00000000 ntdll!DbgBreakPoint 0007fc94 7c921639 0007fd30 7c900000 0007fce0 ntdll!LdrpInitializeProcess+0xffa 0007fd1c 7c90eac7 0007fd30 7c900000 00000000 ntdll!_LdrpInitialize+0x183 00000000 00000000 00000000 00000000 00000000 ntdll!KiUserApcDispatcher+0x7 0:000> dd 0007fd30 0007fd30 00010017 00000000 00000000 00000000 0007fd40 00000000 00000000 00000000 ffffffff 0007fd50 ffffffff f735533e f7368528 ffffffff 0007fd60 f73754c8 804eddf9 8674f020 85252550 0007fd70 86770f38 f73f4459 b2f3fad0 804eddf9 0007fd80 b30dccd1 852526bc b30e81c1 855be944 0007fd90 85252560 85668400 85116538 852526bc 0007fda0 852526bc 00000000 00000000 00000000

  ~*kb

  In this example, you are expected to type in the debug session. The result of entering that command shows several lines, with the most critical piece of informa-

  0007fd30. dd 0007fd30

  tion being Next, you should enter the command illustrat-

  0007fd30 ed to glean more information about the previously highlighted number .

  All tools used in this book are assumed to be launched from their installation fold-

  C:\Program Files\

  er. For example, if the Windows debuggers are installed in the

  Debugging Tools for Windows

  folder, the command line for launching windbg.exe will be shown as

  C:\>windbg

  Preface Supported Windows Versions

  Windows XP or higher is required to fully make use of this book. All sample code and debugging scenarios have been run on Windows XP SP2 or Windows Server 2003 SP1, depending on the requirements of the specific scenario. Please note that service packs or even specific patches can change the result of various commands, although these changes will not affect the overall outcome of what is being illustrated with the debug session.

  Chapter 15, “Windows Vista Fundamentals,” covers the most important changes made in Windows Vista and includes debug sessions that must be run on a machine running Windows Vista.

  Furthermore, all samples and debug sessions were run using the 32-bit version of Windows. Samples used in Chapter 12, “64-Bit Debugging,” were run using the 64- bit version of Windows XP.

  Support

  While every attempt has been made to make this book 100% accurate, without a doubt errors will be found. If you encounter an error in this book, feel free to contact us using any of the following resources:

  Email: marioh@advancedwindowsdebugging.com or daniel@advancedwindowsdebugging.com . Alternatively is monitored and can be used to report erroneous information. As corrections are made, they will be posted to the errata section of the Web site.

  This page intentionally left blank

  

  Writing a technical book is a large-scale effort, far more substantial than we had orig- inally anticipated. As authors, we provided the raw material and the first draft of the book, but throughout the project, a number of people shared their insights and expertise to make this book worth the time spent reading it.

  Thanks to all the team members at Addison Wesley, especially Elizabeth Peterson, Jana Jones, Curt Johnson, Joan Murray, and Gina Kanouse. Chris Zahn also played an instrumental role in editing the book and in correcting our self-styled syntax.

  As with any technical publication, technical accuracy is of utmost importance. We were fortunate to have great engineers (many of them own the specific technology areas discussed in the book) look at the material and provide feedback. Thanks go to Mark Russinovich, Ivan Brugiolo, Pat Styles, Pavel Lebedynskiy, Daniel Mihai, Doug Ellis, Cristi Vlasceanu, Adrian Marinescu, Saji Abraham, Kamen Moutafov, Kinshuman Kinshumann, Bob Wilton, Raymond McCollum, Viorel Mititean, Andy Cheung, Saar Picker, Drew Bliss, Jason Cunningham, Adam Edwards, Jen-Lung Chiu, Alain Lissoir, and Brandon Jiang.

  Special thanks go to Mark Russinovich for not only reviewing the book but also writing the foreword. Mark’s remarkable body of work is well known among software developers and has been a great influence on us and countless other engineers.

  Ivan Brugiolo was also instrumental in reviewing and providing in-depth feed- back. Ivan was incredibly generous with his spare time, sharing knowledge that has added considerable value to this book.

  We also want to extend our gratitude to Alexandra Hewardt for designing and implementing the book’s Web page.

  xxvii

   Mario Hewardt is a senior design engineer with Microsoft

  Corporation and has worked extensively in the Windows system level development arena for the past nine years. Throughout five releases of Windows (starting with Windows 98), he has worked primarily in the server and desktop management arena, focusing the majority of his time on ensuring the reliability, robustness, and _© security of the product.

   Daniel Pravat is a senior design engineer with Microsoft

  Corporation and was actively involved in releasing several windows components in multiple Windows releases. Prior to joining Microsoft, he developed telecommunication software for computer- based telephony servers. He expects all software applications to be reliable, predictable, and efficient.

  Photo by Eduard Koller

  

  

  Chapter 1 Introduction to the Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

Chapter 2 Introduction to the Debuggers . . . . . . . . . . . . . . . . . . . . . . . .29

Chapter 3 Debuggers Uncovered . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123

Chapter 4 Managing Symbol and Source Files . . . . . . . . . . . . . . . . . . .179

  This page intentionally left blank

  

  

  Many books and articles have been written about the importance of proper software design and engineering principles. Some of the publications take a very balanced approach between methodology and practice, whereas others focus mostly on methodology. Books written about the importance of object-oriented design and pro- gramming, design patterns, or modular programming are all great examples of methodologies that help us write better software. Without a doubt, proper software methodologies are the precursors to all successful software projects. However, they are not the sole contributors to the success of the software. Regardless of how well we think that we can design software and regardless of how accurate we believe our scheduling to be, mysterious problems always plague us during the development process. Hectic schedules, complex component interactions, and legacy code are just some of the reasons why we cannot practically anticipate and solve all the problems by simply employing good development methodologies. In addition to the method- ologies, we have to know how to troubleshoot complex problems in a cost- and time- efficient manner.

  This chapter introduces you to invaluable tools that will be of great aid in the trou- bleshooting process, as well as help reduce the time and money spent on handling a wide range of common problems. A lot of the problems that we discuss in this book leave developers feeling frustrated because of their complex nature. Even if a devel- oper has an idea of how to manually approach a particular problem, the effort of track- ing it down is typically very costly. Unbeknownst to many developers, help is out there; the help comes in the form of incredible tool sets that aid developers in tracking down and solving a lot of these types of problems. Not only does it help with the problem solving, but it does so in a very efficient manner.

  This chapter provides an introduction to the tools used throughout the book. Each tool is discussed in detail, and the coverage includes important information, such as common usage scenarios, install points, and background information on how the tools do their work The tool descriptions are not exhaustive sources for all the various usage scenarios; rather, they serve as high-level overviews of the tools. Each of the tools list- ed is used in other parts of the book to illustrate the usage of the tool to solve a real

  3

Chapter 1 Introduction to the Tools

  problem. This chapter can be viewed as an introduction to the tool set that comple- ments its practical usage scenario in subsequent chapters in the book.

  Note that the tools this chapter describes are the latest versions of each tool avail- able at the time of writing. Newer versions might have been published by the time you read this chapter. This does not constitute a problem, as the general tool behav- ior generally stays the same.

  

  Usage Scenarios Memory leak detection Current Version

  1.25 Download Point ftp://ftp.microsoft.com/PSS/Tools/Developer Support Tools/LeakDiag

  Analysis Mechanism Log Files The Leak Diagnosis tool (LeakDiag) is a tool used during the memory leak detection process. It goes well beyond the basic capabilities of showing how much memory a process has leaked to detailed information, such as the exact stack trace that resulted in the allocation and allocation statistics.

  The installation process for LeakDiag is trivial. Download leakdiag125.msi from the download point and use the default settings during the install process. The appli-

  C:\LEAKDIAG cation is, by default, installed into and can run in two modes.

  Specifically, it has a command-line version and a graphical user interface (GUI) ver- sion. The command-line version is called ldcmd.exe, and the GUI version is called leakdiag.exe. Both can be executed from the command line or by going to the Start button and selecting All Programs, LeakDiag.