ClickHouse : How to optimize disk space usage

profile

14 Jun by Think201

Have you ever encountered a situation where your ClickHouse data occupies much less disk space than the actual size of the ClickHouse data directory? In this blog post, we’ll explore some common causes of excessive disk space usage in ClickHouse and discuss practical solutions to reclaim disk space and optimize storage utilization.

Understanding the issue

Recently, We encountered a perplexing issue with ClickHouse where the disk space consumed by the /var/lib/clickhouse directory was significantly larger than the actual size of my ClickHouse data, causing unexpected storage bloat on my server.

Here are a few solutions that we used :

Stop logging queries

Enabling query logging in ClickHouse can consume a considerable amount of disk space over time. By disabling query logging, you can prevent unnecessary disk space usage and improve performance.

Disabling log query threads

Reduce logging

In addition to query logging, ClickHouse also logs various system events and diagnostic information by default. By adjusting the logging settings and reducing the verbosity of the logs, you can minimize disk space overhead without sacrificing visibility into system operations.

Steps to reduce logging

Reclaim disk space

Few techniques for reclaiming disk space in ClickHouse are periodically purging old data, optimizing table storage settings, and leveraging ClickHouse’s built-in features for data compression and compaction.

SELECT name FROM system.tables WHERE name LIKE ‘%log%’;

The above query will show you all the system logs table, which store clickhouse logs, you can clear them by using the below query

TRUNCATE TABLE system.<table_name>

Remember, efficient management of disk space is essential for maintaining the performance and stability of your ClickHouse environment. With the right strategies and tools, you can keep your ClickHouse deployment running smoothly while minimizing storage overhead.

More from