Prentice Hall Running Xen A Hands On Guide To The Art Of Virtualization Apr 2008 ISBN 0132349663
Running Xen: A Hands-On Guide to the Art of Virtualization
by Jeanna N. Matthews; Eli M. Dow; Todd Deshane;
Wenjin Hu; Jeremy Bongio; Patrick F. Wilbur;
Brendan JohnsonPublisher: Prentice Hall Pub Date: April 10, 2008
Print ISBN-10: 0-13-234966-3 Print ISBN-13: 978-0-13-234966-6 eText ISBN-10: 0-13-207467-2 eText ISBN-13: 978-0-13-207467-4
Pages: 624
Overview
"This accessible and immediately useful book expertly provides
the Xen community with everything it needs to know to download, build, deploy and manage Xen implementations."- –Ian Pratt, Xen Project Leader VP Advanced Technology, Citrix Systems
The Real—World, 100% Practical Guide to Xen Virtualization in Production Environments
Using free, open source Xen virtualization software, you can save money, gain new flexibility, improve utilization, and simplify everything from disaster recovery to software testing. Running Xen brings together all the knowledge you need to create and manage high—performance Xen virtual machines in any environment. Drawing on the unparalleled experience of a world—class Xen team, it covers everything from installation to administration–sharing field-tested insights, best practices, and case studies you can find nowhere else.
The authors begin with a primer on virtualization: its concepts, uses, and advantages. Next, they tour Xen's capabilities, explore the Xen LiveCD, introduce the Xen hypervisor, and walk you through configuring your own hard—disk—based Xen installation. After you're running, they guide you through each leading method for creating "guests" and migrating existing systems to run as Xen guests. Then they offer comprehensive coverage of managing and securing Xen guests, devices, networks, and distributed resources. Whether you're an administrator, data center manager, developer, system integrator, or ISP, Running Xen will help you achieve your goals with Xen—reliably, efficiently, with outstanding performance, and at a surprisingly low cost.
- Understanding the Xen hypervisor: what it does, and how it works
- Using pre-built system images, including compressed file systems
- Managing domains with the xm console
- Populating and storing guest images
- Planning, designing, and configuring networks in Xen •Utilizing Xen security: special purpose VMs, virtual network segments, remote access, firewalls, network monitors, sHype access control, Xen Security Modules (XSM), and more
- Managing guest resources: memory, CPU, and I/O
- Employing Xen in the enterprise: tools, products, and techniques
Running Xen: A Hands-On Guide to the Art of Virtualization
by Jeanna N. Matthews; Eli M. Dow; Todd Deshane;
Wenjin Hu; Jeremy Bongio; Patrick F. Wilbur;
Brendan JohnsonPublisher: Prentice Hall Pub Date: April 10, 2008
Print ISBN-10: 0-13-234966-3 Print ISBN-13: 978-0-13-234966-6 eText ISBN-10: 0-13-207467-2 eText ISBN-13: 978-0-13-207467-4
Pages: 624
Copyright
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and the publisher was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein.
The publisher offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests. For more information, please contact: U.S. Corporate and Government Sales (800) 382-3419
For sales outside the United States please contact: International Sales
Visit us on the Web:
Library of Congress Cataloging-in-Publication Data: Matthews, Jeanna N.
Running Xen : a hands- on guide to the art of virtualization / Jeanna N. Matthews, Eli M. Dow, Todd Deshane, Wenjin Hu, Jeremy Bongio, Patrick F. Wilbur, Brendan Johnson. p. cm.
ISBN 0-13-234966- 3 (pbk. : alk. paper) 1. Xen (Electronic resource) 2. Virtual computer systems.
3. Computer organization. 4. Parallel processing (Electronic computers) I. Title. QA76.9.V5M38 2008 005.4'3--dc22 2007052439 Copyright © 2008 Pearson Education, Inc.
All rights reserved. Printed in the United States of America. This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise. For information regarding permissions, write to: Pearson Education, Inc Rights and Contracts Department 501 Boylston Street, Suite 900 Boston, MA 02116 Fax (617) 671 3447
ISBN-13: 978-0-132-34966-6 Text printed in the United States on recycled paper at Courier Stoughton in Stoughton, Massachusetts.
First printing April 2008
Editor-in-Chief
Mark Taub
Acquisitions Editor
Debra Williams Cauley
Development Editor
Managing Editor
Gina Kanouse
Project Editor
Chelsey Marti
Copy Editor
Geneil Breeze
Indexer
Erika Millen
Proofreader
Meg Shaw
Technical Reviewers
Jessie Yu Ken Hess Jose Renato Santos Andrew Warfield Tom "Spot" Callaway Simon Crosby Tom Doeppner Dan Kuebrich Jim Owens Zach Shepherd Keegan M. Lowenstein Ryan Kornheisl Igor Hernandez Alexander M. Polimeni Erika Gorczyca Justin Bennett Joseph Skufca Mathew S. McCarrell Krista Gould Ron Arenas
Publishing Coordinator
Kim Boedigheimer
Cover Designer
Alan Clements
Senior Compositor
Gloria Schurick
Dedication
This book is dedicated to the Applied Computing
Laboratories at Clarkson University and especially to the
Clarkson Open Source Institute. May the labs always be a place for contribution, for achievement, for experimentation, and for fun.Foreword
The Xen open source hypervisor is changing the world of virtualization. It encourages the broad distribution of a common industry standard hypervisor that runs on a wide range of architectures from super computers to servers to clients to PDAs. By focusing on the hypervisor, the "engine" of virtualization, rather than a specific product embodiment, the Xen open source project enables multiple vendors and the community to combine the common cross platform virtualization features of Xen into exciting new products and service offerings.
To date, the community around the Xen hypervisor has been squarely in the camp of developers and expert users. While the Xen-users mailing list offers a friendly and useful source of advice for those wanting to deploy and manage Xen-based environments, the new user might find herself in need of advice about best practice and step-by-step instructions for the deployment of Xen. Running Xen: A Hands-on Guide to the Art
of Virtualization speaks directly to this critical need. It provides
users with everything they need to know to download, build, deploy, and manage Xen implementations. To the authors, a set of Xen contributors, practitioners, and researchers, I would like to say thank you on behalf of the broader Xen community for an accessible and immediately useful book. Code might rule, but "know-how" builds the community itself. Clear information, advice, and documentation like this book will allow the Xen project to grow and solidify its user base, to renew its creativity and innovation, to focus itself on a larger set of new virtualization initiatives. To the readers, I would like to say welcome to the community of Xen users. We look forward to your involvement and contributions! We believe this book will provide you with an excellent introduction to running Xen.
Ian Pratt, Xen Project Leader
VP Advanced Technology, Citrix Systems
Preface
We began using Xen in the fall of 2003 soon after reading the paper "Xen and the Art of Virtualization" published in the Symposium on Operating Systems Principles (SOSP). After attending SOSP and talking to some of the authors, Jeanna Matthews returned excited about Xen. She and her graduate operating systems course at Clarkson University decided to repeat and extend the results reported in that paper. That class included two of the coauthors for this book, Eli Dow (currently at IBM) and Todd Deshane (currently completing his Ph.D.), who were both studying for their Master's degrees at the time. In the process of repeating the results from the 2003 Xen paper, we learned a lot about running Xen—much of it the hard way! Our goal for this book was to write exactly the material we wished was available when we first started using Xen.
In July 2004, we published the paper "Xen and the Art of Repeated Research," describing our experience with Xen and presenting the results we obtained repeating and extending the results. All the authors, in addition to being a part of the Fall 2003 graduate operating systems course, were also members of the Applied Computing Laboratories at Clarkson University, specifically the Clarkson Open Source Institute (COSI) and the Clarkson Internet Teaching Laboratory (ITL). These labs were founded to provide students with hands-on experience with cutting-edge computing technologies and to form a community in which everyone both learns and teaches. Other students in the labs—both graduate and undergraduate—began to use Xen as the basis for both production systems and for research projects. Through the years, we have used Xen as the basis for a number of academic papers as well as the basis of award- winning team projects. In the process, we have learned a lot about running Xen. It is our goal in this book to share this knowledge with you and to make your experience running Xen as smooth and simple as possible. deploying Xen systems. It walks the reader through the basics, from installing Xen to using prebuilt guest images. It even tells readers how to experiment with Xen using only a Xen LiveCD. It covers the basics of virtualizations and important elements of all Xen systems like the hypervisor and Domain0. It explains the details of the commands for managing guest domains. It
xm
helps users deploy custom guest images based on operating systems from Linux to Windows. It covers more advanced topics such as device virtualization, network configuration, security, and live migration. We hope you will find it a good mix of introductory and advanced topics that will prove useful from your first Xen deployment experiment to running production Xen systems.
quick introduction to virtualization in general and to Xen in particular.
provides an overview of Xen's functionalities by exploring the Xen LiveCD.
hypervisor that is the core of any Xen system and some other trusted components such as Domain0 and . We build on
xend
that common understanding of the Xen hypervisor by concretely showing you how to install and configure your own hard-disk- based Xen installation in
and Installation of Xen Domain0." After you have your own hypervisor installation up and running, this book eases you into using guest images by first showing you how to download and use images available from the Internet in
, "Managing Unprivileged Domains," covers the basics of administering the running DomUs or unprivileged guest domains. You are then guided
through the various methods of creating your own custom guest images in
, "Populating Guest Images." Now that you
have all these guests,
"Storing Guest Images,"
covers a variety of choices for storing guest images for online use as well as backup and sharing. management topics including device management
"Guest Save, Restore and Live Migration"). We conclude with a survey of
some of the popular administrative tools available for your Xen systems in
commands and their output. We use the command prompt to indicate where the command should be run. For example, the following would indicate a command to be run as root on the privileged domain, Domain0: [root@dom0 ]# The following would indicate a command to be run as any user in a regular guest domain: [user@domU]$ Watching these command prompts will help you identify which of the many guests in your Xen system should be used for running any given command. It is our intention to maintain a website with additional information and materials relevant to the book. We have registered the domain, runningxen.com, for this purpose and are working on assembling materials. We invite you to check on our progress and to send questions or suggestions.
Acknowledgments
We are indebted to the many people who provided feedback and suggestions on the book's content. Simon Crosby provided key feedback on the overall content of the book. Keir Fraser answered a number of technical questions with amazing speed
work to the open source community.
Several of the authors attended the Xen Summit at IBM T.J. Watson Research in April 2007 and we would like to express our gratitude to all the organizers and attendees. Many people provided invaluable feedback and advice in various conversations—short and long. We would especially like to thank Sean Dague, who provided excellent overall Xen advice and feedback throughout this process, and Jose Renato Santos, who provided detailed feedback on the networking material in the book. In general, online materials from all the Xen summits were an invaluable resource for us, as was the Xen Wiki, Xen mailing lists, and other similar resources. We appreciate the efforts of all the individuals who contributed to those materials.
We would like to thank everyone who read early drafts of the book. Jessie Yu in particular went above and beyond the call of duty in reviewing and helping to revise many chapters. Jim Owens provided valuable early feedback for
"Spot" Callaway from Red Hat gave us some excellent suggestions about
did some early writing on the security chapter and provided valuable comments in the initial phases of organizing the text. Lindsay Hoffman and Barbara Brady provided detailed comments on the writing in
number of chapters and helped test many of the instructions in the book. Anthony Peltz also helped with testing. We would also like to thank everyone who helped with final revisions. In the last days before the manuscript was submitted, a small army of people volunteered to do a fresh read of many chapters, finding everything from typos to substantial problems. We would like to thank Zach Shepherd, Keegan M. Lowenstein, Igor Hernandez, Alexander M. Polimeni, Erika Gorczyca, Justin Bennett, Joseph Skufca, Mathew S. McCarrell, Krista Gould, and Ron Arenas. We couldn't have done it without you! Tom Doeppner and Dan Kuebrich from Brown University also provided some very helpful feedback on
especially like to thank Michael Thurston and Ken Hess for their excellent suggestions. Beside the authors, we believe they are the only ones who have read the entire book! We would like to thank many members of the Clarkson Open Source Institute and Clarkson Internet Teaching Laboratory who over time added to our understanding and hands-on experience with Xen. Bryan Clark (now at Red Hat), Steven Evanchik (now at VMware), Matt Finlayson, and Jason Herne (both now at IBM) were all coauthors on the 2004 "Xen and the Art of Repeated Research" paper. Jason Herne, Patricia Jablonski, Leslie Cherian, and Michael McCabe were all coauthors on the 2005 "Data Protection and Rapid Recovery From Attack With A Virtual Private File Server and Virtual Machine Appliances" paper, which used Xen for some of the prototypes being tested. Madhu Hapauarachchi, Demetrios Dimatos, Gary Hamilton, Michael McCabe, and Jim Owens were coauthors on the 2007 paper "Quantifying the Performance Isolation Properties of Virtualization Systems." Justin Basinger, Michael McCabe, and Ed Despard were part of the Xenophilia project that won second place in the 2005 Unisys Tuxmaster competition. Cyrus Katrak and Zach Shepherd have been key to the deployment of Xen in our production environment in the Applied CS labs. They have both been a crucial source of advice and feedback. We would like to thank the OpenSolaris Xen Community Leaders, especially Todd Clayton, Mark Johnson, John Levon, and Christopher Beal, for their quick and helpful responses over e-mail and IRC during our testing of Xen on OpenSolaris. We would like to have included more coverage of Solaris in this book. Additional support in Solaris for Xen beyond what is covered in this book is expected soon. We would like to thank our editor, Debra Williams Cauley, for her help and encouragement through this entire process. Thanks also to Catherine Nolan who initially contacted us about this project.
Richard A. Wilbur provided access to early HVM-enabled equipment used in testing. Jeanna Matthews would like to thank her husband Leonard Matthews and children Robert and Abigail Matthews for their patience and love throughout this whole process. She would also like to thank her current and former students—including the six other authors on this book—for all she continues to learn from them. Eli M. Dow would like to thank his parents, Terry and Mona, as well as his siblings, Ian and Ashley, for everything. He would also like to thank IBM and the Test and Integration Center for Linux for their support during the writing process. Specifically he wishes to acknowledge Frank Lefevre, Duane Beyer, Robert Jay Brenneman, Phil Chan, Scott Loveland, and Kyle Smith for their insightful conversations regarding virtualization and this book in particular. Eli would also like to thank the wonderful faculty and staff at Clarkson University who made his academic career such a wonderful experience. Lastly he would like to thank his significant other, Jessie, for her enduring patience during the writing process. Todd Deshane would like to thank his significant other, Patty, for her support during the writing of this book. Wenjin Hu would like to thank his mom, Yajuan Song, and his dad, Hengduo Hu, for their constant support of his study at Clarkson University, and his friend, Liang Zheng, for emotional support.
Patrick F. Wilbur would like to thank his mother, Claudia, his father, Richard, and his significant other, Krista, for their support and patience throughout the development of this book.
About the Authors
Jeanna Matthews is an associate professor of Computer
Science at Clarkson University (Potsdam, New York) where she leads several hands-on computing laboratories including the Clarkson Open Source Institute and Clarkson Internet Teaching Laboratory. Students in these labs and in her classes have been winners in a number of prestigious computing contests including the 2001, 2002, and 2004 IBM Linux Challenge, the 2005 IBM North American Grid Scholar's Challenge, the 2005 Unisys Tuxmaster competition, and the 2006 VMware Ultimate Virtual Appliance Challenge. Her research interests include virtualization, operating systems, computer networks, and computer security. She is actively involved in the Association for Computing Machinery as treasurer of the Special Interest Group on Operating Systems, editor of Operating Systems Review, and is a member of the Executive Committee ACM's U.S. Public Policy Committee, US-ACM. She is also the author of a computer networking textbook, Computer Networking: Internet
Protocols in Action, that has been translated into several
languages. Jeanna received her Ph.D. in Computer Science from the University of California at Berkeley in 1999.
Eli M. Dow is a software engineer in IBM's Test and Integration
Center for Linux in Poughkeepsie, NY. He holds a B.S. degree in Computer Science and Psychology as well as an M.S. in Computer Science from Clarkson University. He is passionate about open source software and is an alumnus and founding member of the Clarkson Open Source Institute. His interests include virtualization, Linux systems programming, the GNOME desktop, and human-computer interaction. He is the author of numerous IBM developerWorks articles focused on Linux and open source software. Additionally, he has coauthored two books on the mainframe hypervisor z/VM, entitled Introduction
to the New Mainframe: z/VM Basics and Linux for IBM System
z9 and IBM zSeries. His first published experience with Xen was
coauthoring an early academic paper entitled "Xen and the Art of Repeated Research." Recently he has focused on developing highly available, enterprise customer solutions deployed on virtualized Linux using the z/VM hypervisor.
Todd Deshane expects to obtain a Ph.D. in Engineering
Science from Clarkson University in 2008. He also has a Master of Science in Computer Science and a Bachelor of Science in Software Engineering from Clarkson. While at Clarkson University, he has had a variety of research publications—many involving Xen. In 2005, a project that was based on Todd's Master's thesis—an open source collaborative, large database explorer—won first place in the Unisys TuxMaster competition. Todd's primary academic and research interests are in the area of operating system technologies, such as virtual machine monitors, high availability, and file systems. His doctoral dissertation focuses on using these technologies to provide desktop users with an attack-resistant experience, with automatic and autonomic recovery from viruses, worms, and adverse system modifications. During his Ph.D. years, Todd has been a teaching assistant and an IBM Ph.D. Fellowship recipient. At IBM, Todd has worked on internship projects involving Xen and IBM technologies. Todd enjoys teaching, tutoring, and helping people.
Wenjin Hu graduated from Clarkson University in 2007 with a
Master's degree of Computer Science and is currently working on his Ph.D. His Masters thesis was "A Study of the Performance Isolation Properties of Virtualization Systems." His research field is applying virtualization techniques to operating systems and security.
Jeremy Bongio is currently a Master's student at Clarkson
University. He won second place in the Unisys Tuxmaster competition in 2005 with a project called Xenophilia, an early effort to make Xen more user friendly. He is a current member and former student director of the Clarkson Open Source Institute, where he actively learns and experiments with different kinds of virtualization.
Patrick F. Wilbur is currently pursuing graduate studies in
Computer Science at Clarkson University. His interests include operating systems, systems and application security, natural language processing, and home automation. In his spare time, Patrick enjoys composing music, experimenting with amateur radio, storm chasing, and working on various electronics, software, and carpentry projects around the house. He is currently a member of the Clarkson Open Source Institute, a volunteer at the Applied Computer Science Laboratories at Clarkson University, an emergency communications volunteer, and a member of the Association for Computing Machinery.
Brendan Johnson graduated from Clarkson University in 2002
with a Bachelor's degree in Computer Science and a minor in Mathematics. Brendan continued his education at Clarkson University and obtained a Master's of Science in Computer Science with a thesis in quantum computing. Brendan is currently a senior software architect at Mobile Armor, a world leading "Data At Rest" encryption software company.
Chapter 1. Xen—Background and Virtualization Basics Xen is a virtual machine monitor (hypervisor) that allows you to
use one physical computer to run many virtual computers—for example, running a production Web server and a test server on the same physical machine or running Linux and Windows simultaneously. Although not the only virtualization system available, Xen has a combination of features that make it uniquely well suited for many important applications. Xen runs on commodity hardware platforms and is open source. Xen is fast, scalable, and provides server-class features such as live migration. This chapter discusses common uses and types of virtualization, describes the history of virtualization and origins of Xen, provides a brief overview of the Xen architecture, and compares Xen with other virtualization systems.
Common Uses and Benefits of Virtualization
Virtual machine monitors provide a convenient way to use the same physical computer hardware for many different tasks. Operating systems have been doing this for years simply by enabling users to run many different applications at once, such as Web browsers, database servers, and games. However, without virtualization, the act of choosing an operating system and system configuration to run on your physical computer has the unfortunate side effect of closing off many other options. For example, if you run Linux to develop and test programs in that environment, you may not be able to run programs written exclusively for Windows. Also, if you run the newest and fully patched version of Windows, reproducing problems experienced by customers on earlier versions may be difficult. Additionally, if your Web server and database server require different versions of a system library, they may not be able to run on the same system. Without virtualization in each of these examples, you would need to maintain many physical machines, each with a special software configuration, even if the computing resources in one machine are sufficient to run all of your applications at once.
Virtual machine monitors (hypervisors) are becoming increasingly important in modern computing because they allow many different operating systems and software configurations to exist on the same physical machine. The hypervisor controls the underlying hardware, allowing it to be used by many guest systems at once, and gives each guest system the illusion that it is running on its own private hardware. The hypervisor abstracts the physical resources of the host computer into discrete virtual counterparts that can be allocated for use by individual guests. Virtual guests treat their virtual hardware as if it were real, and the hypervisor ensures that this illusion is seamless. Additionally, hypervisors must ensure some level of isolation between guests. In a way, hypervisors act as both magician and traffic cop.illustrates the relationship between the physical hardware, the hypervisor, and the guest virtual machines.
Figure 1.1. The hypervisor sits between the guest
domains and the physical hardware.
[View full size image]
Virtual machine monitors also provide a uniform interface to the hardware. This uniform interface shields guest systems from some lower level details of the physical computing resources and provides portability, which is another key benefit of virtualization. In fact, many modern hypervisors allow guest systems to move from one physical machine to another without interruption. Guest system configurations can easily be developed on one machine and then deployed on many systems. This eases the job of managing or deploying software on a collection of machines with different hardware characteristics. Guest systems can even migrate from one physical computer to another while running. Xen calls this live
migration. Some benefits of virtualization are as follows.
Debugging operating systems is time consuming and requires exceptionally skilled programming. Virtualization can ease the burden by allowing a developer to test new operating systems as a guest on a more stable host. This technique has been used for many years and has proven effective. Similarly, security researchers can create guest operating systems that are isolated from one another as well as the host. Such guests allow researchers to study the effects of worms, Trojans, and viruses, without affecting the host system. These isolated guests are colloquially referred to as being "sandboxed." Sandboxed guests can also be used to test updates or cutting-edge software before applying them to production systems. Another benefit of virtualization is the ability to recover quickly from software problems caused by deliberate malicious attacks or accidental malfunctions. By maintaining a copy of a stable guest image, recovering from an attack can be as simple as rolling back to this trusted saved point. Virtualization can provide higher availability by relocating guests when a server machine is inoperable. Server environments can have many physical machines, each running a number of guest systems. Guest systems can be moved seamlessly between physical machines to balance the load dynamically, thus using the aggregate resources most efficiently. Many enterprise customers have enjoyed these benefits on exotic hardware platforms for many years. Xen now provides these advantages to a wider audience. Other benefits of virtualization become especially clear in a server environment. One example is the ability to consolidate many services on one physical machine while still allowing each service to be administered independently. In a multihosting environment, a service provider may run guest systems belonging to many different individuals or businesses on the same physical machine. Each entity could have its own root or administrative access, make its own choices as to what software to run, and administer its own virtual guest autonomously without any need to consult or coordinate with the owners of the other guest systems. Most of the advantages of virtualization, especially on commodity platforms such as x86, derive from the abundance of computing power available on a single machine. As the power of the average system has increased, the amount of computing power that goes underutilized is also increasing—especially with
provides a way to take advantage of this latent computing power by consolidation onto today's increasingly powerful physical machines. Hypervisors can be especially useful for developers, because the developers no longer need to restart physical machines to switch between various operating systems. Multiboot configurations are just not sufficient for these developers any longer. Developers requiring this functionality are becoming more common as more applications are made multiplatform.From a business perspective, virtualization can provide a reduced total cost of ownership (TCO). Hardware is utilized more fully when multiple operating systems coexist on a single physical machine. Imagine running just two virtual machines on each server a company owns. This would mean 50 percent of the hardware would be needed for the same computing infrastructure. Now, we do not mean to imply every computer should be running simultaneous virtualized guest operating systems, but often many machines sit practically idle, and these computers are prime candidates for consolidation via virtualization. Training costs for employees can be decreased when using virtualization because it allows several different training configurations (operating systems and applications) to coexist on a single platform, thus fewer computers are needed for training, and reconfiguration is minimized between different training sessions.
In many business environments, users are afforded the advantage of being able to virtualize legacy operating systems and applications on modern hardware platforms. Typically, migration of these applications to current architectures is too costly. Even if migration was successful, those applications would need debugging for many years to be as robust as the original applications. With a virtual machine, users are free to execute legacy products in a protected virtual environment without the fear of some rogue legacy application bringing the system to a halt. The final benefit of virtualization that bears mentioning is decreased power consumption and cooling infrastructure. Servers running virtualized at higher utilization make more efficient use of power than many systems functioning at low capacity. Because smaller space is occupied by the computing infrastructure, there is more room to adequately cool today's very dense and very warm data centers. In some cases a substantial cost savings for air conditioning can be realized.
Chapter 1. Xen—Background and Virtualization Basics Xen is a virtual machine monitor (hypervisor) that allows you to
use one physical computer to run many virtual computers—for example, running a production Web server and a test server on the same physical machine or running Linux and Windows simultaneously. Although not the only virtualization system available, Xen has a combination of features that make it uniquely well suited for many important applications. Xen runs on commodity hardware platforms and is open source. Xen is fast, scalable, and provides server-class features such as live migration. This chapter discusses common uses and types of virtualization, describes the history of virtualization and origins of Xen, provides a brief overview of the Xen architecture, and compares Xen with other virtualization systems.
Common Uses and Benefits of Virtualization
Virtual machine monitors provide a convenient way to use the same physical computer hardware for many different tasks. Operating systems have been doing this for years simply by enabling users to run many different applications at once, such as Web browsers, database servers, and games. However, without virtualization, the act of choosing an operating system and system configuration to run on your physical computer has the unfortunate side effect of closing off many other options. For example, if you run Linux to develop and test programs in that environment, you may not be able to run programs written exclusively for Windows. Also, if you run the newest and fully patched version of Windows, reproducing problems experienced by customers on earlier versions may be difficult. Additionally, if your Web server and database server require different versions of a system library, they may not be able to run on the same system. Without virtualization in each of these examples, you would need to maintain many physical machines, each with a special software configuration, even if the computing resources in one machine are sufficient to run all of your applications at once.
Virtual machine monitors (hypervisors) are becoming increasingly important in modern computing because they allow many different operating systems and software configurations to exist on the same physical machine. The hypervisor controls the underlying hardware, allowing it to be used by many guest systems at once, and gives each guest system the illusion that it is running on its own private hardware. The hypervisor abstracts the physical resources of the host computer into discrete virtual counterparts that can be allocated for use by individual guests. Virtual guests treat their virtual hardware as if it were real, and the hypervisor ensures that this illusion is seamless. Additionally, hypervisors must ensure some level of isolation between guests. In a way, hypervisors act as both magician and traffic cop.illustrates the relationship between the physical hardware, the hypervisor, and the guest virtual machines.
Figure 1.1. The hypervisor sits between the guest
domains and the physical hardware.
[View full size image]
Virtual machine monitors also provide a uniform interface to the hardware. This uniform interface shields guest systems from some lower level details of the physical computing resources and provides portability, which is another key benefit of virtualization. In fact, many modern hypervisors allow guest systems to move from one physical machine to another without interruption. Guest system configurations can easily be developed on one machine and then deployed on many systems. This eases the job of managing or deploying software on a collection of machines with different hardware characteristics. Guest systems can even migrate from one physical computer to another while running. Xen calls this live
migration. Some benefits of virtualization are as follows.
Debugging operating systems is time consuming and requires exceptionally skilled programming. Virtualization can ease the burden by allowing a developer to test new operating systems as a guest on a more stable host. This technique has been used for many years and has proven effective. Similarly, security researchers can create guest operating systems that are isolated from one another as well as the host. Such guests allow researchers to study the effects of worms, Trojans, and viruses, without affecting the host system. These isolated guests are colloquially referred to as being "sandboxed." Sandboxed guests can also be used to test updates or cutting-edge software before applying them to production systems. Another benefit of virtualization is the ability to recover quickly from software problems caused by deliberate malicious attacks or accidental malfunctions. By maintaining a copy of a stable guest image, recovering from an attack can be as simple as rolling back to this trusted saved point. Virtualization can provide higher availability by relocating guests when a server machine is inoperable. Server environments can have many physical machines, each running a number of guest systems. Guest systems can be moved seamlessly between physical machines to balance the load dynamically, thus using the aggregate resources most efficiently. Many enterprise customers have enjoyed these benefits on exotic hardware platforms for many years. Xen now provides these advantages to a wider audience. Other benefits of virtualization become especially clear in a server environment. One example is the ability to consolidate many services on one physical machine while still allowing each service to be administered independently. In a multihosting environment, a service provider may run guest systems belonging to many different individuals or businesses on the same physical machine. Each entity could have its own root or administrative access, make its own choices as to what software to run, and administer its own virtual guest autonomously without any need to consult or coordinate with the owners of the other guest systems. Most of the advantages of virtualization, especially on commodity platforms such as x86, derive from the abundance of computing power available on a single machine. As the power of the average system has increased, the amount of computing power that goes underutilized is also increasing—especially with
provides a way to take advantage of this latent computing power by consolidation onto today's increasingly powerful physical machines. Hypervisors can be especially useful for developers, because the developers no longer need to restart physical machines to switch between various operating systems. Multiboot configurations are just not sufficient for these developers any longer. Developers requiring this functionality are becoming more common as more applications are made multiplatform.From a business perspective, virtualization can provide a reduced total cost of ownership (TCO). Hardware is utilized more fully when multiple operating systems coexist on a single physical machine. Imagine running just two virtual machines on each server a company owns. This would mean 50 percent of the hardware would be needed for the same computing infrastructure. Now, we do not mean to imply every computer should be running simultaneous virtualized guest operating systems, but often many machines sit practically idle, and these computers are prime candidates for consolidation via virtualization. Training costs for employees can be decreased when using virtualization because it allows several different training configurations (operating systems and applications) to coexist on a single platform, thus fewer computers are needed for training, and reconfiguration is minimized between different training sessions.
In many business environments, users are afforded the advantage of being able to virtualize legacy operating systems and applications on modern hardware platforms. Typically, migration of these applications to current architectures is too costly. Even if migration was successful, those applications would need debugging for many years to be as robust as the original applications. With a virtual machine, users are free to execute legacy products in a protected virtual environment without the fear of some rogue legacy application bringing the system to a halt. The final benefit of virtualization that bears mentioning is decreased power consumption and cooling infrastructure. Servers running virtualized at higher utilization make more efficient use of power than many systems functioning at low capacity. Because smaller space is occupied by the computing infrastructure, there is more room to adequately cool today's very dense and very warm data centers. In some cases a substantial cost savings for air conditioning can be realized.
Types of Virtualization
Many technical details of virtualization are similar, yet various approaches exist to solve problems associated with the different implementations. Four major virtualization architectures in modern computing provide the illusion of complete stand-alone systems: emulation, full virtualization, paravirtualization, and operating system level virtualization. For completeness, we also briefly discuss two other types of virtualization—library and application level virtualization—even though they are not capable of running complete stand-alone systems with full operating systems.
Modern operating systems on personal computers usually provide a weak form of isolation through the use of individual processes, with generous facilities for sharing data between the processes. Most PCs are designed for a single user, so sharing is often given precedence over isolation. A modern PC might have any number of programs running as separate processes. Each has its own unique process identifiers obtained from a global pool but shares a common underlying file system. In contrast, hypervisors have been designed to obtain much stronger isolation between virtual machines. Most hypervisors provide no more sharing support between guest instances than disjointed physical computers on the same network.
Each virtualization technique trades some level of isolation for increased sharing of resources among its guests. Typically, stronger isolation comes at the cost of some performance. This is due to the overhead required to implement strong isolation mechanisms. Conversely, weaker isolation can relax the implementation requirements in ways that can increase performance.
Emulation
In emulation, the virtual machine simulates the entire hardware set needed to run unmodified guests for completely different hardware architecture. This is illustrated in . Typically, emulation is used to create new operating systems or microcode for new hardware designs before that hardware is physically available. Examples include PearPC, Bochs, and the nonaccelerated form of QEMU.
Figure 1.2. Emulator virtual machines provide a virtual
computing architecture that is not the same as the actual
physical architecture of the host machine. Operating
systems intended for the emulated hardware are
executed unmodified.
Full Virtualization
Full virtualization (also called native virtualization) is similar to
emulation. As in emulation, unmodified operating systems and applications run inside a virtual machine. Full virtualization differs from emulation in that operating systems and applications are designed to run on the same architecture as the underlying physical machine. This allows a full virtualization system to run many instructions directly on the raw hardware. The hypervisor in this case polices access to the underlying hardware and gives each guest operating system the illusion of having its own copy. It no longer must use software to simulate a different basic architecture.illustrates full virtualization.
Figure 1.3. The full virtualization hypervisor presents the
actual physical hardware "P" to each guest so that
operating systems intended for the underlying
architecture may run unmodified and unaware that they
are being run virtualized.
For x86, virtualization systems are often classified as full virtualization if they can run unmodified guest operating system binaries. However, some of these still make some simplifying changes to x86 for easier virtualization and still achieve high performance. The x86 architecture is notoriously difficult to virtualize. Because of this, virtualization specifics (Intel's VT and AMD's AMD-V, discussed in the "
running an operating system within a Xen virtual machine simpler. They support these changes with clever methods, such as on-the-fly binary translation of instructions that are not desired in their simplified x86 architecture.
Major vendors of full virtualization include VMware Workstation,