SeaweedFS
Distributed storage system with S3-compatible object storage, POSIX file system, and Iceberg table support for billions of files.
SeaweedFS is a distributed storage system designed to handle billions of files with O(1) disk access and effortless horizontal scaling. It provides S3-compatible object storage, a POSIX-compliant file system, and Iceberg table support.
Key Features
- S3 Compatible API - Full S3 API support for object storage workloads.
- Distributed File System - POSIX-compliant filer with FUSE mount support for seamless file access.
- Massive Scale - Designed for billions of files with O(1) disk access time.
- Erasure Coding - Built-in data redundancy without the overhead of full replication.
- Tiered Storage - Hot and cold storage tiers with automatic data migration.\n- FUSE Mount - Mount SeaweedFS as a local file system on any machine.
- Hadoop HDFS Compatible - Drop-in replacement for HDFS in big data workloads.
- Iceberg Tables - Native support for Apache Iceberg table format.
- Replication - Cross-datacenter replication for disaster recovery.
- Kubernetes Native - Helm charts and operators for easy K8s deployment.
- Web UI - Built-in web interface for file browsing and management.
Architecture
SeaweedFS uses a master-volume-filer architecture. The master server manages volume assignments, volumes store actual data with O(1) lookup, and the filer provides a directory structure on top. This separation allows independent scaling of metadata and data.
Self-Hosting
SeaweedFS can be deployed as a single binary or via Docker. It supports standalone, replication, and erasure coding modes for different reliability requirements.