Approximate Query Processing over Data Lakes Using Semantic Sampling and Vector-Similarity Guided Stratification for E-Commerce Marketplace BI

Abstract

E-commerce marketplaces increasingly rely on data lakes that consolidate event streams, transactional records, product catalogs, seller operations, logistics telemetry, and customer interactions. Business intelligence workloads over such lakes are dominated by exploratory aggregates, repeated slice-and-dice filtering, and wide group-by summaries whose exact evaluation can be expensive under high data volume, heterogeneous formats, and evolving schemas. Approximate query processing offers a practical path to interactive latency by trading small, controlled error for substantial reductions in scanned data, but classical sampling and stratification strategies often underperform in marketplaces because relevant subpopulations are defined by high-cardinality, semantically rich attributes such as textual product descriptions, user intent signals, and behavioral embeddings rather than by stable, low-dimensional keys. This paper develops an approximate query processing framework for data-lake-based marketplace BI that combines semantic sampling with vector-similarity guided stratification. The central idea is to represent tuples, entities, and query predicates in a shared embedding space, maintain semantic samples that preserve rare but decision-critical regions, and at query time allocate sampling effort toward strata whose semantic proximity to the query indicates higher contribution and variance. The approach yields unbiased or asymptotically unbiased estimators for common aggregates, supports group-by over dynamic taxonomies, and provides error estimation that remains informative under distribution shift. The proposed design integrates with lakehouse storage, catalog metadata, and feature stores while supporting incremental maintenance. Analytical modeling and empirical considerations highlight conditions under which semantic stratification reduces variance relative to key-based baselines, particularly for long-tail products, sparse conversion events, and cross-domain joins.