Query Pattern Mining for Storage Optimization in Analytical Data Platforms

Abstract

Analytical data platforms often store large tables in layouts that do not match how users actually query them. This article presents a query pattern mining approach for improving storage design by extracting repeated predicates, projection groups, join paths, and recent-window access behavior from analytical query logs. The proposed method normalizes query histories, applies frequency and recency weighting, builds a column access graph, and converts stable workload patterns into storage actions such as partition redesign, predicate-aware clustering, join-key co-location, compact file grouping, and cold-tier movement. The analysis shows that query patterns become more stable as workload windows accumulate, allowing the platform to distinguish persistent analytical behavior from occasional exploratory queries. The results also show that adaptive layouts built from mined query patterns improve scan reduction, partition pruning, and read amplification control more effectively than static storage strategies. These findings indicate that analytical storage optimization should be guided by observed query behavior rather than fixed table design alone.