HPFS is a file metadata server based on Ceph RADOS, designed to address the bottleneck issues commonly encountered in Ceph file systems. Unlike traditional setups that rely on third-party storage for metadata, HPFS directly stores metadata on RADOS. This approach ensures metadata stability akin to that of Ceph, which has undergone over a decade of iterative development and is known for its robust data stability. By avoiding third-party plugins for metadata storage, we eliminate potential security risks associated with external data handling. One of the known shortcomings of Ceph is the single-point issue with its Metadata Server (MDS), which tends to underperform in high IOPS scenarios. HPFS has redesigned the metadata architecture to create a distributed framework, allowing metadata access to scale with the concurrent access capabilities of RADOS objects. Additionally, HPFS has been specifically optimized for scenarios involving intensive small file operations, such as AI visual training and graphic rendering, significantly enhancing IOPS performance under multi-client and concurrent conditions—often outperforming CephFS by orders of magnitude. If you manage environments with hundreds of millions of files requiring frequent metadata access, consider evaluating the performance of HPFS. We have now made some of HPFS’s features available for free to facilitate community interaction and collaborative discussion.

github url:
https://github.com/ptozys2/hpfs