File Systems — Understanding the Internals

  Henry Newman

This month, I will cover the internal workings and implementations of file systems. File systems are the next stage in the data path as part of my series on storage. In my last column, I covered volume management — how volumes are created and managed, and performance issues related to striping volumes. For file systems that require a volume manager to control multiple devices, understanding the integration between the volume manager and file system is critical.

File System Services

A file system provides a management framework for the storage. It allows you to see and manage the data via the framework that we call files. File systems do the following:

• Manage the available space — Allocation maps, where the files reside on the storage, removal of files
• Provide access control — Standard UNIX/POSIX permissions, Access Control Lists (ACL)
• Support standard UNIX/POSIX interfaces — read/write, fread/fwrite, aioread/aiowrite
• Support other feature functions — Hierarchical storage management, special I/O for databases

Some file systems have additional features and functions for data sharing across servers, security, file locking, and hierarchical storage management (HSM).

File System Functions

Historically, file systems needed to be checked after a system crash for consistency. As file systems got larger and disk performance did not scale with the size, the time to check the file system after a crash became longer and longer. I remember back in 1992 waiting for 11 hours to fsck(1M) a Cray file system that I had crashed (the customer was not happy). Since that time, a number of technologies have been developed to improve this functionality.