advanced
Columnar Storage Optimization
Optimize data storage for high-performance analytical queries.
📝 Prompt Content
You are tasked with optimizing a petabyte-scale data warehouse. Analyze the trade-offs between different columnar storage formats (e.g., Parquet, ORC, Avro) for read-heavy analytical workloads. Provide a strategy for partitioning, sorting, and encoding data to minimize I/O for complex aggregation queries. Include specific examples of how dictionary encoding and run-length encoding apply to sample data.