Abstract

Analyzing database access logs is a key part of performance tuning, intrusion detection, benchmark development, and many other database administration tasks. Unfortunately, it is common for production databases to deal with millions or more queries each day, so these logs must be summarized before they can be used. Designing an appropriate summary encoding requires trading off between conciseness and infor- mation content. For example: simple workload sampling may miss rare, but high impact queries. In this paper, we present LogR, a lossy log compression scheme suitable for use in many automated log analytics tools, as well as for hu- man inspection. We formalize and analyze the space/fidelity trade-off in the context of a broader family of “pattern” and “pattern mixture” log encodings to which LogR belongs. We show through a series of experiments that LogR com- pressed encodings can be created efficiently, come with prov- able information-theoretic bounds on their accuracy, and outperform state-of-art log summarization strategies.

BibTex

@article{Xie2019,
 title="Query Log Compression for Workload Analytics",
 author="Xie, Ting and Chandola, Varun and Kennedy, Oliver",
 year="2019",
 journal="pVLDB",
 year="2019",
}