Distributed Storage

Purpose

Design distributed storage systems for data workloads. Select the right storage backend (HDFS, S3, ADLS, GCS, MinIO), configure durability and replication, optimize file formats and compression, and implement lifecycle policies.

Agent Protocol

Trigger

Exact user phrases: "HDFS", "S3", "ADLS", "GCS", "MinIO", "distributed storage", "object storage", "file system", "data lake storage", "storage tier", "lifecycle policy", "replication factor", "erasure coding", "storage cost".

Input Context

• Storage backend preference (S3, ADLS, GCS, HDFS, MinIO)
• Data volume (current TB, annual growth)
• Access patterns (frequent, infrequent, archive)
• Durability and availability requirements
• Budget and cost constraints
• Compliance requirements (data residency, encryption)
• Network bandwidth and latency constraints