Apress Cost Based Oracle Fundamentals Nov 2005 ISBN 1590596366 pdf

  

Cost-Based Oracle

Fundamentals ■ ■ ■ Jonathan Lewis

  Cost-Based Oracle Fundamentals Copyright © 2006 by Jonathan Lewis

All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means,

electronic or mechanical, including photocopying, recording, or by any information storage or retrieval

system, without the prior written permission of the copyright owner and the publisher.

  ISBN (pbk): 1-59059-636-6 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1

Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence

of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark

owner, with no intention of infringement of the trademark. Lead Editor: Tony Davis Technical Reviewers: Christian Antognini, Wolfgang Breitling

Editorial Board: Steve Anglin, Dan Appleman, Ewan Buckingham, Gary Cornell, Tony Davis, Jason Gilmore,

Jonathan Hassell, Chris Mills, Dominic Shakeshaft, Jim Sumser Production Director and Project Manager: Grace Wong Copy Edit Manager: Nicole LeClerc Senior Copy Editor: Ami Knox Assistant Production Director: Kari Brooks-Copony Production Editor: Katie Stence Compositor: Susan Glinert Proofreaders: Lori Bring, Kim Burton, Nancy Sixsmith Indexer: Valerie Perry Artist: April Milne Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski

Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor,

orders-ny@springer-sbm.com, or

  New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail http://www.springeronline.com. visit

For information on translations, please contact Apress directly at 2560 Ninth Street, Suite 219, Berkeley, CA

info@apress.com, or visit http://www.apress.com.

  94710. Phone 510-549-5930, fax 510-549-5939, e-mail

The information in this book is distributed on an “as is” basis, without warranty. Although every precaution

has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to

any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly

by the information contained in this work. http://www.apress.com in the Source Code section.

  The source code for this book is available to readers at

  Contents at a Glance Foreword

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

  About the Author

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

  About the Technical Reviewers

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

  Acknowledgments

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

  Introduction

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

  CHAPTER 1 What Do You Mean by Cost? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 ■

  CHAPTER 2 Tablescans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 ■

  CHAPTER 3 Single Table Selectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 ■

  CHAPTER 4 Simple B-tree Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 ■

  CHAPTER 5 The Clustering Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 ■

  CHAPTER 6 Selectivity Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 ■

  CHAPTER 7 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 ■

  CHAPTER 8 Bitmap Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 ■

  CHAPTER 9 Query Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207 ■

  CHAPTER 10 Join Cardinality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 ■

  CHAPTER 11 Nested Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 ■

  CHAPTER 12 Hash Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 ■

  CHAPTER 13 Sorting and Merge Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 ■

  CHAPTER 14 The 10053 Trace File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403 ■

  APPENDIX A Upgrade Headaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 ■

  APPENDIX B Optimizer Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 ■

  INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 ■

  Contents Contents Foreword

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

  About the Author

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

  About the Technical Reviewers

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

  Acknowledgments

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

  Introduction

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

CHAPTER 1 What Do You Mean by Cost? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

  ■ Optimizer Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 So What Is the Cost? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Transformation and Costing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 WYSIWYG? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

  CHAPTER 2 Tablescans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 ■

  Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Onwards and Upwards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Effects of Block Sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 CPU Costing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 The Power of CPU Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

  The BCHR Is Dead! Long Live the BCHR! . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Parallel Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Index Fast Full Scan

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

  Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

  CHAPTER 3 Single Table Selectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 ■

  Getting Started

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

  Null Values

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

  vi C O N T E N T S ■

  Using Lists

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

  10g Update

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

  Range Predicates

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

  10g Update

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

  Two Predicates

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

  Problems with Multiple Predicates

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

  Summary

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

  Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

  

CHAPTER 4 Simple B-tree Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

■

  Basics of Index Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

  Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Effective Index Selectivity

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

  Effective Table Selectivity

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

  clustering_factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Putting It Together

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

  Extending the Algorithm

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

  The Three Selectivities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 CPU Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

  Loose Ends

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

  

CHAPTER 5 The Clustering Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

■

  Baseline Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Reducing Table Contention (Multiple Freelists) . . . . . . . . . . . . . . . . . 90 Reducing Leaf Block Contention (Reverse Key Indexes) . . . . . . . . . 94 Reducing Table Contention (ASSM) . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Reducing Contention in RAC (Freelist Groups) . . . . . . . . . . . . . . . . . . 99 Column Order . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

  Extra Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Correcting the Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 The sys_op_countchg() Technique . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Informal Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

  Loose Ends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

  C O N T E N T S vii ■

  

CHAPTER 6 Selectivity Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

■

  Different Data Types

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

  Date Values

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

  Character Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Daft Data Types

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

  Leading Zeros

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

  Deadly Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 Discrete Dangers

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

  10g Update

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

  Surprising sysdate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 Function Figures

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

  Correlated Columns

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

  Dynamic Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 Optimizer Profiles

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

  Transitive Closure

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

  Constraint-Generated Predicates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Summary

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

  Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

  

CHAPTER 7 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

■

  Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Generic Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Histograms and Bind Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 When Oracle Ignores Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

  Frequency Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 Faking Frequency Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Warning to Fakers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 “Height Balanced” Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 The Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 Data Problems Revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Daft Datatypes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Dangerous Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

  viii C O N T E N T S ■

  

CHAPTER 8 Bitmap Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

■

  Getting Started

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181

  The Index Component

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

  The Table Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188 Bitmap Combinations

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

  Low Cardinality

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

  Null Columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 CPU Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

  Interesting Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

  Multicolumn Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 Bitmap Join Indexes

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

  Bitmap Transformations

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

  

CHAPTER 9 Query Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

■

  Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208 Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Filter Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215

  Scalar Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Subquery Factoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 Complex View Merging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230 Pushing Predicates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

  General Subqueries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234 Subquery Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Categorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Semi-Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 Anti-Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246 Anti-join Anomaly . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248 Nulls and Not In . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 The ordered Hint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250

  Star Transformation Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 Star Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258 The Future . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

  C O N T E N T S ix ■

  CHAPTER 10 Join Cardinality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 ■

  Basic Join Cardinality

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

  Biased Joins

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

  Join Cardinality for Real SQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Extensions and Anomalies

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

  Joins by Range

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

  Not Equal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 Overlaps

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

  Histograms

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280

  Transitive Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Three Tables

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

  Nulls

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

  Implementation Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Difficult Bits!

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

  Features

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301

  An Alternative Viewpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Summary

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

  Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

  CHAPTER 11 Nested Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 ■

  Basic Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Worked Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Sanity Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

  CHAPTER 12 Hash Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 ■

  Getting Started

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319

  The Optimal Hash Join

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

  The Onepass Hash Join

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325

  The Multipass Hash Join

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331

  Trace Files

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

  Event 10104

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336

  Event 10053

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

  Headaches

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

  Traditional Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

  Modern Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

  x ■

  C O N T E N T S Comparisons

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

  Multitable Joins

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347

  Summary

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

  Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350

CHAPTER 13 Sorting and Merge Joins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353 Getting Started

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

  A Merge Join Without the First Sort . . . . . . . . . . . . . . . . . . . . . . . . . 384 The Cartesian Merge Join

  

CHAPTER 14 The 10053 Trace File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

The Query . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404 The Execution Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405 The Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 The Trace File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Parameter Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 Query Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 Stored Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 Single Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414 Sanity Checks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416 General Plans . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416

  ■

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400

  Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398

  Final Warning

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

  Indexes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 Set Operations

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387

  Aggregates and Others

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

  Memory Usage

  The Merge Mechanism

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

  Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375 Merge Joins

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

  ■

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

  Real I/O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 Cost of Sorts

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365

  pga_aggregate_target

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364

  CPU Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360 sort_area_retained_size

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359

  10053 trace

  C O N T E N T S xi ■

  Join order[3]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

  Join order[4]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

  Join order[5]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425

  Join order[6]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429

  Join order[7]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429

  Join order[8]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433

  Join order[9]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435

  Join order[10]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435

  Join order[11]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

  Join order[12]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

  Join order[13]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

  Join order[14]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

  Join order[15]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

  Join order[16]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

  Join order[17]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

  Join order[18]

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

  Join Evaluation Summary

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

  Test Cases

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

  APPENDIX A Upgrade Headaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 ■ dbms_stats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 Frequency Histograms

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

  CPU Costing

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

  Rounding Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Bind Variable Peeking

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

  Nulls Across Joins

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456

  B-tree to Bitmap Conversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456 Index Skip-Scans

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456

  AND-Equal

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456

  Index Hash Join . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 In-List Fixed

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457

  Transitive Closure

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458

  sysdate Arithmetic Fixed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 458 Indexing Nulls

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

  pga_aggregate_target

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459

  Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460 Grouping

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

  Sanity Checks

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460

  xii C O N T E N T S ■ optimizer_mode

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

  Descending Indexes

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

  Complex View Merging

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

  Unnest Subquery

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

  Scalar and Filter Subqueries

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461

  Parallel Query Changes x 2

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

  Dynamic Sampling

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

  Temporary Tables

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

  Dictionary Stats

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

  

APPENDIX B Optimizer Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465

■ optimizer_features_enable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 The 10053 Trace File

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467

  v$sql_optimizer_env

  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473

  

INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 475 ■ Foreword

  rthur C. Clarke once wrote that “any sufficiently advanced technology is indistinguishable

  A

  from magic.” I believe that observation to be entirely accurate. Someone else later observed that “any technologist with sufficient knowledge is indistinguishable from a magician.” With that in mind, what you have in your hands right now is a book on magic.

  However, for as long as I’ve known the author of this book, Jonathan Lewis (some 11 years according to my research on Google going back over the newsgroup archives), he has never been content to accept “magic.” He wants to know why something happens the way it does. So, fundamentally, his book is all about understanding: understand v. understood, (-std) understanding, understands

   1. To perceive and comprehend the nature and significance of

   2. To know thoroughly by close contact or long experience with More precisely, it is all about understanding the Oracle cost based optimizer (CBO), how it works, and why it does what it does. Jonathan conveys to us his understanding of the Oracle

  CBO through practice and example, and with this understanding, with this knowledge, new options and solutions become available.

  Put simply, the Oracle CBO is a mathematical model; you feed it inputs (queries, statistics), and it produces outputs (query plans). In order to use the CBO successfully, it is critical that you understand what these inputs are and how the CBO uses them. Consider the following question, one that I’ve tried to answer many times: What is the best way to collect statistics, what statistics should I gather? Seems simple enough, very straightforward—there should be an answer and there is, but it is no “one size fits all” answer. It depends on your circumstances, your data distribution, your queries, the type of system you have (transactional, data warehouse)—a whole host of factors—and it is only through an understanding of how the CBO works and of how these factors affect the CBO that you’ll be able to answer the question for yourself.

  My favorite chapter in this book is Chapter 7, which provides an excellent discussion on what histograms do, how the CBO uses them, and some of the myths that surround them (they are a frequently misunderstood input to the CBO). One of the reasons it is my favorite chapter is because I still remember hearing this chapter for the first time (not reading it, hearing it). It was about three years ago, at the NoCOUG (Northern California Oracle User Group) meeting. I attended Jonathan’s Histograms session and, for the first time, felt that I truly understood how histograms worked in Oracle. Jonathan provided practical information, of immediate day-to- day use, and as a result I was in a much better position to answer the question just posed. In this book, you will find many such topics described in that same practical, immediately usable way.

  The insights that Jonathan provides into the workings of the cost based optimizer will make a DBA a better designer and a developer a better SQL coder. Both groups will become

  xiv F O R E W O R D ■

  Time and time again, it has been proven that you can only use a tool safely and effectively once you understand how it works. This is true of software, hardware, and pretty much every- thing else in life. In which case, what you have in your hands is a book that will allow you to make safe and effective use of the tool that is the Oracle CBO.

  Thomas Kyte

  VP (Public Sector), Oracle Corporation About the Author JONATHAN LEWIS is a qualified teacher with a mathematics degree from

  ■

  Oxford University. Although his interest in computers came to light at the tender age of about 12—in the days when high-technology meant you used a keyboard, rather than a knitting needle to punch holes in cards—it wasn’t until he was four years out of university that he moved into computing professionally. Apart from an initial year as an incompetent salesman, he has been self-employed his entire career in the computer industry. His initiation into Oracle was on version 5.1 running on a PC, which he used to design and build a risk-management system for the crude-trading floor of one of the major oil companies. He had written the first version of the system using a PC program called dBase III—which did use tables and indexes and made some claims to being a relational database management system. Suddenly he found out what a proper Relational Database Management System ought to be able to do.

  Since that day of revelation, Jonathan has focused exclusively on the use and abuse of the Oracle RDBMS. After three or four years of contract work helping to build large systems, he decided to move from the “contractor” market to the “consultant” market, and now shares his time evenly between short-term consultancy work, running seminars, and “research.” www.ukoug.com),

  At the time of writing, he is one of the directors of the UK Oracle User Group ( and a regular contributor to their quarterly magazine and presenter at their Special Interest Groups (particularly the RDBMS and UNIX groups). Whenever the opportunity arises, he tries to find time for speaking to user groups outside the UK, sometimes as short evening events at www.jlcomp.demon.co.uk) of the end of a day’s consultancy work. He also maintains a web site ( articles and notes on things that you can do with the Oracle RDBMS.

  Jonathan has been married to Diana (currently a primary school teacher after many years of being an accountant) for nearly 20 years, and has two children: Anna (actress and trumpet player) and Simon (rugby and saxophone player). Given the choice between working on a Saturday and supporting a rugby match or school concert—it’s the school event that comes first.

  About the Technical Reviewers CHRISTIAN ANTOGNINI has focused on understanding how the Oracle database engine works

  ■

  since 1995. His main interests range from logical and physical database design, the integration of databases with Java applications, to the cost based optimizer, and basically everything else related to performance management and tuning. He is currently working as a senior consultant www.trivadis.com) in Zürich, Switzerland. If he is not helping one of and trainer at Trivadis AG ( his customers to get the most out of Oracle, he is somewhere lecturing on optimization or new Oracle database features for developers.

  Christian lives in Ticino, Switzerland, with his wife, Michelle, and their two children, Sofia and Elia. He spends a great deal of his spare time with his wonderful family and, whenever possible, reading books, enjoying a good movie, or riding one of his BMX bikes.

  WOLFGANG BREITLING was born in Stuttgart, Germany, and studied mathematics, physics, and ■

  computer sciences at the University of Stuttgart. After graduating, he joined the QA department of IBM Germany’s development laboratory in Böblingen. There he became one of a team of two to develop an operating system to test the hardware of the /370 model machines developed in the Böblingen lab.

  Wolfgang’s first direct foray into the field of performance evaluation/tuning was the devel- opment of a program to test the speed of individual operating codes of the /370 architecture. After IBM Germany, he worked as a systems programmer on IBM’s hierarchical databases DL/1 and IMS in Switzerland before emigrating to his current home in Calgary, Canada. Following several years as systems programmer for IMS and later DB2 databases on IBM mainframes, he got on the project to implement Peoplesoft on Oracle.

  In 1996, he became an independent consultant specializing in administering and tuning Peoplesoft on Oracle. In that capacity, Wolfgang has been engaged in several Peoplesoft instal- lation and upgrade projects. The particular challenges in tuning Peoplesoft, with often no access to the SQL, motivated him to explore Oracle’s cost based optimizer in an effort to better understand how it works and use that knowledge in tuning. He has shared the findings from this research in papers and presentations at IOUG, UKOUG, local Oracle user groups, and other conferences and newsgroups dedicated to Oracle performance topics.

  Wolfgang has been married to his wife Beatrice for over 30 years and has two children, Magnus and Leonie. When he is not trying to decipher the CBO, Wolfgang enjoys canoeing and hiking in the Canadian prairies and Rocky Mountains.

  Acknowledgments

  irst and foremost, I have to thank my wife and children for putting up with my distant expres-

  F

  sions, long silences, and my habit of being physically present while mentally absent. They have been very patient with my mantra to “give me just five more minutes.” I would like to thank those Oracle experts and adventurers around the world who (know- ingly or unknowingly) have extended my knowledge of the database. In particular, though not exclusively, I owe thanks to (in alphabetical order) Steve Adams, Wolfgang Breitling, Julian Dyke, K. Gopalakrishnan, Stephan Haisley, Anjo Kolk, Tom Kyte, James Morle, and Richmond Shee. Each of them has, at various times, supplied me with insights that have allowed me to open up whole new areas of investigation and understanding. Insights are so much more important than answers.

  Christian Antognini and Wolfgang Breitling merit special thanks for reading the draft chap- ters, and for making a number of suggestions to improve the quality of the book. Any outstanding errors are, of course, all my own work.