CAPFS - A Content Addressable Parallel File System

The ever-increasing gap in performance between CPU/memory technologies and the I/O subsystem (disks, I/O buses) in modern workstations has exacerbated the I/O bottlenecks inherent in applications that access large disk resident data sets. A common technique to alleviate the I/O bottlenecks on clusters of workstations, is the use of parallel file systems due to the increased I/O bandwidth made possible by accessing large number of disks in parallel. Designing a fault-tolerant parallel file system that provides consistency in the face of multiple writers and readers is increasingly becoming more important as clusters scale. Traditional techniques to provide file system consistency for both meta-data and data use variants of locking techniques that fall short if fault tolerance is a design requirement. We propose a simpler and a more robust lockless technique atop a content-addressable storage layer, that can be used for performing atomic updates and can hence provide sequential consistency semantics in a parallel file system, when the underlying network is fully connected.

Completed Features

Partially completed Features/Works-in-progress