Reserved Free Space Filesystem Performance Issues

Figure 6.5: Berkeley−style filesystem: Blocks and fragments. In this example, a hypothetical 25KB large file was located in the three 8 KB blocks and one 1 KB block fragment. Upon a change in the file size to 51 KB, the file will consume six 8 KB blocks and three block fragments. In both cases, the remaining block fragments will be used for other files, so the wasted space is minimized.

6.2.5.2 Reserved Free Space

File transfer efficiency can be improved by the introduction of the 10 filesystem free space. We briefly mentioned the 10 free space in Chapter 5 regarding the command df. We elaborate on this issue in more detail here. The disk space always tends to be fragmented. The filesystem content is changing dynamically, old files are deleted and new files created. Upon the filesystem creation, the empty disk space will be quickly filled with data. Normally the filesystem tries to keep all file blocks together, so the access to the file could be faster. But the files are also deleted, and many gaps in the disk space remain after the file blocks are removed. This is known as disk fragmentation. These gaps are reused, and reused, but the fragmentation of the overall disk space through time is unavoidable. Fragmented space requires more time to store and access files. Simply, the time spent in seeking and transferring chaotically allocated small chunks of the file blocks is much larger than if the blocks are allocated in larger chunks. Statistically, if 10 of the available storage space is sacrificed and not used, the performance benefits can be significant. This space is already badly fragmented and too expensive to be used. The remaining space offers more contiguous space for faster file allocation. Remember that this free space is dynamically allocated and is changing through time. It always contains the most fragmented storage space in that point of time. In addition, this 10 of space remains a forbidden zone only for users. Superuser and high−priority processes are still allowed to use that space. The basic assumption is that these processes are beyond introduced restrictions. Those are system−related processes and should not be interrupted despite expected low−performance behavior of the system. There is an odd consequence of this implementation. Occasionally the df−k command can report filesystem consumption larger than 100. Although it could be quite confusing, it is still normal system behavior. Your system will not crash soon. It also does not mean that your data will spill over the edges of your disk. It simply means that 10 of reserved free space of this filesystem was 153 The 10 free filesystem space was introduced in the Berkeley UNIX. It has to be specified when the filesystem is created. It can be disabled at any time during the life of the filesystem. The reserved space can be returned for regular use at any time. The opposite is not possible: there is no way to introduce the 10 free space in an existing filesystem. If needed, the filesystem must be recreated, whatever it takes.

6.3 Logical Filesystem Layout

The physical approach to managing disk space is easier to understand, but it carried a number of restrictions caused by the disk hardware itself: How to overcome the finite size of a disk unit? What to do when the maximum size of the filesystem is below that needed? How to provide redundancy? And many other issues needed to improve overall system performances. The problem was especially acute in the management of large databases. A solution was found in a different, logical approach in managing disk space. Existing physical storage entities partitions and disks could be combined and presented as arbitrary large logical storage entities. They then appear simply as storage entities to the operating system. The obvious benefits of such an approach are its inherent flexibility and increased capabilities. For a better understanding of the terminology, here are a few introduction notes. Generally, the term physical refers to a real situation — what something physically looks like. The term logical refers to the way something is presented to the users. The relationship between physical and logical entities must be strictly defined and established. Once this bidirectional relationship is done, further management can be completely shifted to the logical layer. The required division of the storage space continues over the logical entities in the almost identical way we have already discussed. Of course, in real life everything is mapped back to physical entities, because they are the real providers of the needed storage space. The basic logical entity was named the logical volume although this name is not used explicitly on all UNIX platforms. The most common name for the whole suite is the logical volume manager LVM. UNIX vendors do not have a uniform approach regarding the LVM. There are several mutually incompatible versions designed by different manufacturers. We cannot even discuss BSD−like and System V−like versions; simply, the LVM appeared much later. LVM is a new, vendor−flavored product. This section briefly covers three LVM versions: AIX, HP−UX, and Solaris. It should be sufficient to help us become familiar with this important topic. However, it is fair to mention that the third−party vendor VERITAS is probably the leading designer in this field. As a matter of fact, VERITAS also contributed a great deal to all three of the versions examined. The terminology used by the different vendors is also very vendor−specific. The same entities are named in different ways, making a complete description quite confusing. Unfortunately, issues that are already complex enough sometimes sound even more complicated due to the naming ambiguities.

6.3.1 Logical Volume Manager — AIX Flavor

AIX started early the trend toward a logical approach to disk treatment. Since AIX 3.1, physical volumes correspond to the physical partition were divided into a large number of relatively small disk chunks by default their size was 4 MB. They were called physical partitions, but we will use 154