S4Core

High-performance Rust-based S3-compatible object storage with deduplication, SQL queries, and federation.

Open source alternative to:

MinIO

RustFS

S4Core is an open-source, S3-compatible object storage server written in Rust. It claims to be 3x faster than MinIO and RustFS while solving the inode exhaustion problem through append-only log storage.

Key Features

Full S3 API Compatibility - Works with AWS CLI, boto3, and any S3-compatible client.
Inode Problem Solved - Append-only volume files eliminate inode exhaustion (1 billion objects = ~1000 files).
Content Deduplication - Automatic deduplication saves 30-50% storage space.
S3 Select SQL - Query CSV/JSON/Parquet objects with full SQL powered by Apache DataFusion, including multi-object glob patterns.
Object Versioning - S3-compatible versioning with delete markers and lifecycle policies.
Atomic Operations - Rename directories with millions of files in milliseconds.
IAM & Admin API - Role-based access control (Reader, Writer, SuperUser) with JWT authentication.
Federation - Leaderless quorum replication for high availability (N=3, W=2, R=2).
Strict Consistency - Data is guaranteed to be written before returning success.

Architecture

S4Core uses a Bitcask-style storage approach with append-only volume files (~1GB each) and metadata stored in fjall (LSM-tree, MVCC, LZ4 compression). This ensures minimal inode usage, maximum write performance, atomic metadata operations, and fast crash recovery.

Self-Hosting

Available as Docker images (Community and Enterprise editions) or can be built from source with Rust. The CE edition supports single node or up to 3-node clusters with quorum replication.