SeaweedFS

SeaweedFS is a distributed storage system designed to handle billions of files with O(1) disk access and effortless horizontal scaling. It provides S3-compatible object storage, a POSIX-compliant file system, and Iceberg table support.

Key Features

• S3 Compatible API - Full S3 API support for object storage workloads.
• Distributed File System - POSIX-compliant filer with FUSE mount support for seamless file access.
• Massive Scale - Designed for billions of files with O(1) disk access time.
• Erasure Coding - Built-in data redundancy without the overhead of full replication.
• Tiered Storage - Hot and cold storage tiers with automatic data migration.\n- FUSE Mount - Mount SeaweedFS as a local file system on any machine.
• Hadoop HDFS Compatible - Drop-in replacement for HDFS in big data workloads.
• Iceberg Tables - Native support for Apache Iceberg table format.
• Replication - Cross-datacenter replication for disaster recovery.
• Kubernetes Native - Helm charts and operators for easy K8s deployment.
• Web UI - Built-in web interface for file browsing and management.

Architecture

SeaweedFS uses a master-volume-filer architecture. The master server manages volume assignments, volumes store actual data with O(1) lookup, and the filer provides a directory structure on top. This separation allows independent scaling of metadata and data.

Self-Hosting

SeaweedFS can be deployed as a single binary or via Docker. It supports standalone, replication, and erasure coding modes for different reliability requirements.