Open
Description
Use Case Description
During initial migration, we noticed that the automatically created tables use the following clause:
PARTITION BY intDiv(id, 4294967)
How is the value 4294967
determined?
The Snowflake-style scheme we use (e.g., 1849360358546407424
)
when performing batch inserts, this partitioning scheme creates too many partitions triggers the max_partitions_per_insert_block
limit.
Since we have hundreds of tables, manually modifying each table's PARTITION BY
clause is extremely tedious.
Proposed Solution
Allow users to customize the PARTITION BY
clause during automatic table creation.
If a created_at
or similar datetime field exists, we would prefer to partition by toYYYYMM(created_at)
instead of using a division on the ID.