COP 5725 Fall 2012 Database Management Systems

University of Florida, CISE Department Prof. Daisy Zhe Wang Overview of Storage and Indexing File Organizations

Indexes

Cost Model Performance Tuning Data on External Storage

Basic Abstraction of Data in DBMS: A File of records
• File organization: Method of arranging a file of records
on external storage.

is sufficient to physically locate record

– Record id (rid)
– Indexes

External Storage:

– Disks: random access at fixed cost, cheaper for block I/O
– Tapes: only read pages in sequence, cheaper, archival

Architecture:

make calls to the buffer manager

– File and index layers
– Buffer manager stages pages from external storage to main

memory buffer pool

– Disk manager keeps track of pages used by files, and empty

slots on pages Alternative File Organizations Many alternatives exist, each ideal for some situations, and not so good in others:

– access is a file scan retrieving all records.

Heap (random order) files: Suitable when typical

– Sorted Files: Best if records must be retrieved in some order, or only a `range’ of records is needed. Indexes: Data structures to organize records via
– trees or hashing.

Like sorted files, they speed up searches for a subset of • records, based on values in certain (“search key”) fields Updates are much faster than in sorted files. • Indexes

• An on a file speeds up selections on the

index for the index. search key fields

– Any subset of the fields of a relation can be the search key for an index on the relation.
– fields that uniquely identify a record in a relation).

is not the same as (minimal set of Search key key

An index contains a collection of data entries and supports efficient retrieval of all data

k* k entries with a given key value .

– Given data entry k*, we can find record with key k in at most one disk I/O. (Details soon …)

B+ Tree Indexes 

Leaf pages contain data entries , and are chained (prev & next)



Non-leaf pages have index entries; only used to direct searches: P 0 K 1 P 1 K 2 P 2

K m P m index entry Non-leaf Pages Pages (Sorted by search key) Leaf ^{Index entries} ^{direct search for} ^{data entries} Example B+ Tree

Root

30 14* 16* ^{33* 34* 38* 39*} ¹³ ⁵ ^{7* 5* 8* 22* 24*} ²⁷ ^{27* 29*} Entries <= 17 Entries > 17 Note how data entries in leaf level are sorted

Find 28*? 29*? All > 15* and < 30*
Insert/delete: Find data entry in leaf, then change it. Need to adjust parent sometimes.

– And change sometimes bubbles up the tree ^{2* 3*}

Hash-Based Indexes

Good for equality selections.
Index is a collection of buckets .

– Bucket = primary page plus zero or more

overflow pages.

– Buckets contain data entries.

h : h

Hashing function

( r ) = bucket in which (data entry for) record r belongs. h looks at the search key fields of r. Example Hash-Based Index

k* Alternatives for Data Entry in Index k* we can store:

In a data entry

– Data record with key value k, or
– <k, rid of data record with search key value k>, or
– <k, list of rids of data records with search key k>

Choice of alternative for data entries is
orthogonal to the indexing technique used to

locate data entries with a given key value k.

– Examples of indexing techniques: B+ trees, hash- based structures

Alternative 1:

If this is used, index structure is a file organization – for data records (instead of a Heap file or sorted file).

– records can use Alternative 1.

At most one index on a given collection of data

– file

Pros : save the I/O to fetch the record in a different

–
containing data entries is high. Implies size of
auxiliary information in the index is also large, typically.

Cons : If data records are very large, # of pages Alternatives for Data Entries (Contd.)

Alternatives 2 and 3:

– Data entries typically much smaller than data records. So, better than Alternative 1 with large data records, especially if search keys are small.
– Alternative 3 more compact than Alternative 2, but leads to variable sized data entries even if search keys are of fixed length.

Index Classification (I)

Primary vs.

secondary : If search key contains primary key, then called primary index.

– Duplicates
– Unique index: Search key contains a candidate key.

Index Classification (II)

index: the order of data records is

Clustered the same as, or `close to’, order of data entries in the index

– Alternative 1 implies clustered In practice, clustered also implies Alternative 1
– (since sorted files are rare).
–
A file can be clustered on at most one search key.

Vs. Unclustered index

Cost of retrieving data records through index

varies greatly based on whether index is clustered or not! Clustered vs. Unclustered Index

Suppose that Alternative (2) is used for data entries, and that the data records are stored in a Heap file.
To build clustered index

– sort the heap file on the search key of the tree index
– overflow pages may be needed for inserts

Data entries ^CLUSTERED _{(Index File)} _{(Data file)} ^{Data entries} ^UNCLUSTERED Comparing File Organizations

Heap files (random order; insert at eof)
Sorted files, sorted on <age, sal>
Clustered B+ tree file, Alternative (1), search key

<age, sal>

• Heap file with unclustered B + tree index on
search key

<age, sal>

Heap file with unclustered hash index on search key

<age, sal> Operations to Compare

• Scan: Fetch all records from disk
Equality search
Range selection
Insert a record
Delete a record

Cost Model for Our Analysis

We ignore CPU costs, focus on I/O cost, assume:

– B: The number of data pages R: Number of records per page
– –

D: (Average) time to read or write a disk page

F: Fanout in B+ Tree –

–
pre-fetching a sequence of pages (i.e., blocked

I/O); thus, even I/O cost is only approximated.
Average-case analysis with simplistic assumptions.

Measuring number of page I/O’s ignores gains of

–  Good enough to show the overall trends!

Assumptions in Our Analysis

Heap Files:

– Equality search on a candidate key: results in exactly one match

Sorted Files:

– Files compacted after deletions.

Indexes:

– Alt (2), (3): data entry size = 10% size of record
– Hash: No overflow buckets.

80% page occupancy => File size = 1.25 data size

– Tree: 67% occupancy (this is typical).

Implies file size = 1.5 data size

Cost of Operations

(a) Scan (b) (c ) Range (d) Insert (e) Delete Equality

(1) Heap (2) Sorted (3) Clustered (4) Unclustered Tree index (5) Unclustered Hash index

 Several assumptions underlie these (rough) estimates!

2 B D(log

(3) Clustered

Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete (1) Heap BD

0.5BD BD

2D Search

(2) Sorted BD

Dlog

2 B +

# pgs with match recs) Search

1.5BD

Dlog

D Search +D (4) Unclust.

BD(R+0.125) 2D BD Search

Hash index

0.15B + # pgs w. match recs)

0.15B) D(log

D(1 + log

Tree index BD(R+0.15)

1.5B + # pgs w. match recs)

1.5B D(log

2D
2D (5) Unclust.
2D
2D
D

2 B D(log

(3) Clustered

Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete (1) Heap BD

0.5BD BD

2D Search

(2) Sorted BD

Dlog

2 B +

# pgs with match recs) Search

1.5BD

Dlog

D Search +D (4) Unclust.

BD(R+0.125) 2D BD Search

Hash index

0.15B + # pgs w. match recs)

0.15B) D(log

D(1 + log

Tree index BD(R+0.15)

1.5B + # pgs w. match recs)

1.5B D(log

2D
2D (5) Unclust.
2D
2D

Assumptions in Our Analysis

Heap Files:

– Equality search on a candidate key: results in exactly one match

Sorted Files:

– Files compacted after deletions.

Indexes:

– Alt (2), (3): data entry size = 10% size of record
– Hash: No overflow buckets.

80% page occupancy => File size = 1.25 data size

– Tree: 67% occupancy (this is typical).

Implies file size = 1.5 data size
D

2 B D(log

(3) Clustered

Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete (1) Heap BD

0.5BD BD

2D Search

(2) Sorted BD

Dlog

2 B +

# pgs with match recs) Search

1.5BD

Dlog

D Search +D (4) Unclust.

BD(R+0.125) 2D BD Search

Hash index

0.15B + # pgs w. match recs)

0.15B) D(log

D(1 + log

Tree index BD(R+0.15)

1.5B + # pgs w. match recs)

1.5B D(log

2D
2D (5) Unclust.
2D
2D

Assumptions (contd.)

Scan via indexes:

– Scan data-entries in index plus fetching actual records from data files

Range searches:

– We use tree indexes to restrict the set of data records fetched, but ignore hash indexes.

Assumptions in Our Analysis

Heap Files:

– Equality search on a candidate key: results in exactly one match

Sorted Files:

– Files compacted after deletions.

Indexes:

– Alt (2), (3): data entry size = 10% size of record
– Hash: No overflow buckets.

80% page occupancy => File size = 1.25 data size

– Tree: 67% occupancy (this is typical).

Implies file size = 1.5 data size
D

2 B D(log

(3) Clustered

Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete (1) Heap BD

0.5BD BD

2D Search

(2) Sorted BD

Dlog

2 B +

# pgs with match recs) Search

1.5BD

Dlog

D Search +D (4) Unclust.

BD(R+0.125) 2D BD Search

Hash index

0.15B + # pgs w. match recs)

0.15B) D(log

D(1 + log

Tree index BD(R+0.15)

1.5B + # pgs w. match recs)

1.5B D(log

2D
2D (5) Unclust.
2D
2D

Assumptions in Our Analysis

Heap Files:

– Equality search on a candidate key: results in exactly one match

Sorted Files:

– Files compacted after deletions.

Indexes:

– Alt (2), (3): data entry size = 10% size of record
– Hash: No overflow buckets.

80% page occupancy => File size = 1.25 data size

– Tree: 67% occupancy (this is typical).

Implies file size = 1.5 data size
D

2 B D(log

(3) Clustered

Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete (1) Heap BD

0.5BD BD

2D Search

(2) Sorted BD

Dlog

2 B +

# pgs with match recs) Search

1.5BD

Dlog

D Search +D (4) Unclust.

BD(R+0.125) 2D BD Search

Hash index

0.15B + # pgs w. match recs)

0.15B) D(log

D(1 + log

Tree index BD(R+0.15)

1.5B + # pgs w. match recs)

1.5B D(log

2D
2D (5) Unclust.
2D
2D
D

2 B D(log

(3) Clustered

Cost of Operations (a) Scan (b) Equality (c ) Range (d) Insert (e) Delete (1) Heap BD

0.5BD BD

2D Search

(2) Sorted BD

Dlog

2 B +

# pgs with match recs) Search

1.5BD

Dlog

D Search +D (4) Unclust.

BD(R+0.125) 2D BD Search

Hash index

0.15B + # pgs w. match recs)

0.15B) D(log

D(1 + log

Tree index BD(R+0.15)

1.5B + # pgs w. match recs)

1.5B D(log

2D
2D (5) Unclust.
2D
2D

Understanding the Workload in the workload:

For each query

– Which relations does it access? Which attributes are retrieved?
– –

Which attributes are involved in selection/join conditions ? How selective are these conditions likely to be?

For each in the workload:

update Which attributes are involved in selection/join conditions?

– How selective are these conditions likely to be?
– INSERT/DELETE/UPDATE

The type of update ( ), and the attributes that are affected.

Choice of Indexes

What indexes should we create?

– Which relations should have indexes?
– What field(s) should be the search key ?
– Should we build multiple indexes?

• For each index, what kind of an index
should it be?

– Clustered?
– Hash/tree?

Choice of Indexes (Contd.)

One approach: – Consider the most important queries in turn.

– Consider the best plan using the current indexes, and see if a better plan is possible with an additional index* – If so, create it.

Before creating an index, consider the impact on updates in the workload!

– slower. Require disk space, too.

Trade-off: Indexes can make queries go faster, updates

WHERE clause are candidates for index keys.

Index Selection Guidelines (I)

Attributes in

– Exact match condition suggests hash index.
– Range query suggests tree index.
– Clustering especially useful for:

range queries; • equality queries if there are many duplicates.

Index Selection Guidelines (II)

Multi-attribute search keys should be considered

WHERE

when a clause contains several conditions.

– Only when all conditions a equality conditions, then use hash index, otherwise, use tree index.
Order of attributes in the search key is important for
– range queries.
– Such indexes can sometimes enable index-only strategies for important queries.

Index Selection Guidelines (III)

• Try to choose indexes that benefit as many queries
as possible.
• Choose clustered index based on importance of the
queries and the amount of speed up

– For queries with index-only strategies, clustering does
not matter

B+ tree index on E.age can be used to get qualifying tuples.

– How selective is the condition?
– Is the index clustered?

Consider the

– Selectivity?

E.dno,

FROM

E.dno

SELECT

E.dno

GROUP BY

E.age>10

WHERE

Emp E

FROM

(*)

COUNT

SELECT

E.age>40

WHERE

Emp E

FROM

E.dno

SELECT

– Index on
– Clustered?

Equality queries and duplicates:

– Clustering on

E.hobby helps! Indexes with Composite Search Keys

E.dno ?

E.age or

GROUP BY query.

Examples of Clustered Indexes

Emp E

Composite Search Keys : for

Examples of composite key multiple conditions in WHERE indexes using lexicographic order.

– Equality query : Every field value is

11,80

11 equal to a constant value. E.g. wrt <sal,age> index:

12 12,10 name age sal

age=20 and sal =75

12,20

– 13,75 bob

13 Range query: Some field value is not a constant. E.g.:

<age, sal> cal

11 80 <age>

age =20; or age=20 and sal > 10

joe 12

20 10,12 sue 13

Data entries in B+ tree index

20 20,12

sorted by search key to support

Data records 75,13

75 sorted by name range, group by queries.

80,11 80 <sal, age>

<sal> Data entries in index Data entries sorted by <sal,age> sorted by <sal> Composite Search Keys

AND

=30 =4000

To retrieve Emp records with age sal

– Hash/Tree? Clustered? On <age,sal>, <age>, <sal>,<sal,age>?

AND

• If condition is: 20< <30 3000< <5000:

age sal

– > or < >?

Clustered? Tree/Hash? On <

age,sal sal,age

AND

If condition is: =30 3000< <5000:

age sal

– Tree/Hash?

Index key < > or < >? Clustered? age,sal sal,age

Composite indexes are larger, updated more often
Better chance to have index-only evaluation plan

Index-Only Plans

SELECT AVG

<E. age,E.sal> or <E.sal, E.age>

<E.dno,E.sal> Tree index!

5000 <E.dno> Tree Index!

AND

3000

BETWEEN

E.sal

AND

E.age=25

WHERE

Emp E

FROM

(E.sal)

E.dno

SELECT

GROUP BY

Emp E

FROM

(E.sal)

MIN

E.dno,

SELECT

E.dno

GROUP BY

Emp E

FROM

(*)

COUNT

E.dno,

Tree index!

Index-Only Plans (Contd.)

SELECT COUNT

E.dno, (*)

Tree index

FROM

Emp E

Unclustered

WHERE

E.age=30

<dno,age> or

GROUP BY

E.dno <age,dno>

– Which is better?

SELECT COUNT

E.dno, (*)

FROM

Emp E

WHERE

E.age>30

GROUP BY

E.dno Summary

• Many alternative file organizations exist, each
appropriate in some situation.
If selection queries are frequent, sorting the file or building an index is important.

– Hash-based indexes only good for equality search.
– Sorted files and tree-based indexes best for range search; also good for equality search. (Files rarely kept sorted in practice; B+ tree index is better.)

Index is a collection of data entries plus a way to quickly find entries with given key values.

Summary (Contd.)

Data entries can be actual data records, <key, rid> pairs, or <key, rid-list> pairs.
Can have several indexes on a given file of data records, each with a different search key.
Indexes can be classified as clustered vs.

unclustered, primary vs. secondary. Differences have important consequences for utility/performance.

Summary (Contd.)

Understanding the nature of the

workload for the application, and the performance goals, is essential to developing a good physical design.

Indexes must be chosen to speed up important queries (and perhaps some updates!).

– Index maintenance overhead on updates to key fields.
– Choose indexes that can help many queries, if possible.
– Build indexes to support index-only strategies.
– Clustering is an important decision; only one index on a given relation can be clustered!
– Order of fields in composite index key can be important.

Exercise

Index-only plan?

SELECT COUNT

E.dno, (*)

FROM

Emp E

WHERE

E.age>10

GROUP BY

E.dno

COP 5725 Fall 2012 Database Management Systems

F: Fanout in B+ Tree –

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

Dokumen yang terkait

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

Dukungan

Links

COP 5725 Fall 2012 Database Management Systems

F: Fanout in B+ Tree –

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

2 B D(log

1.5BD

Dokumen yang terkait

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

COP 5725 Fall 2012 Database Management Systems

Dokumen yang Anda mencari sudah siap untuk unduhkan