1.9. Column partitioning#
Column partitioning stores rows in physical partitions selected by the value of one column. It lets you keep one logical time-series table while organizing its storage around a key such as a sensor ID, customer ID or event time.
Note
Column partitioning is planned for QuasarDB 4.0.0. The examples describe the current implementation on the development branch, which currently reports version 3.15. This does not imply availability in released 3.15 builds.
The table still has a $timestamp TIMESTAMP time index. IN RANGE selects
row timestamps, even when a different TIMESTAMP column determines the physical
partition. Partitions can overlap in row time. GROUP BY, window functions and
joins operate on the logical table; they do not create an implicit group for
each storage partition. SQL window PARTITION BY is a separate feature.
If you omit the storage PARTITION BY clause, the table keeps the existing
timestamp shard layout. Server thread-pool partitions, described
in Parallelism, are also separate from column partitions.
1.9.1. Choosing a strategy#
Clause |
Key type |
Storage layout |
Query predicates that can prune partitions |
|---|---|---|---|
|
INT64 or STRING |
One partition per distinct value |
Equality on the key |
|
INT64 or STRING |
At most |
Equality on the key |
|
INT64 |
Fixed-width integer intervals |
Equality and ordered comparisons on the key |
|
TIMESTAMP |
Time intervals of the selected column |
Equality and ordered comparisons on the key |
Hash counts range from 1 to 65,536. Slots are populated on demand; declaring a count does not allocate that many empty buckets. A hash collision puts distinct values in the same slot, without merging their rows or changing their values.
Range width must be a positive INT64. Range k contains values from
k * width inclusive to (k + 1) * width exclusive. Division rounds down:
with width 1,000, -1 belongs to [-1000, 0), and 0 belongs to [0, 1000).
Timestamp partitioning defaults to a one-day shard size. INFINITE puts all
valid key timestamps in one partition. SHARD_SIZE applies to the selected
TIMESTAMP key, not to $timestamp when the key is another column. Distinct,
hash and range strategies do not accept SHARD_SIZE. Column partitioning
does not add a second level of time sharding inside each physical partition.
Choose a strategy using both key cardinality and query patterns. Distinct partitioning isolates values but can create many partitions. Hash partitioning bounds their number, while collisions reduce pruning selectivity. Range and timestamp strategies suit interval predicates. Skewed keys can concentrate writes and data in a few partitions with any strategy. Queries filtered only by row time may need to visit every populated column partition.
1.9.2. Create, insert and query#
Create one table for multiple sensors, partitioned by sensor ID:
CREATE TABLE sensor_readings (
$timestamp TIMESTAMP,
sensor_id INT64,
value DOUBLE
) PARTITION BY sensor_id;
Insert interleaved row timestamps and key values:
INSERT INTO sensor_readings ($timestamp, sensor_id, value) VALUES
(2026-01-01T10:00:00, 101, 12.5),
(2026-01-01T10:01:00, 202, 18.0),
(2026-01-01T10:02:00, 101, 13.0);
Select a sensor and a row-time interval:
SELECT $timestamp, sensor_id, value
FROM sensor_readings IN RANGE (2026-01-01, +1day)
WHERE sensor_id = 101
ORDER BY $timestamp;
The equality predicate selects sensor 101’s physical partition. The time range
then limits rows by $timestamp. The result contains the two rows for sensor
101, ordered by their row timestamps. Use explicit ORDER BY when result order
matters; physical partition order is not a result-order guarantee.
Inspect the layout:
SHOW TABLE sensor_readings;
SELECT name, column, partition_by
FROM $sys.tables WHERE name = 'sensor_readings';
The partition_by field shows sensor_id. See
CREATE TABLE for the complete syntax and
SELECT for pruning behavior.
1.9.3. Supported operations and limitations#
These limits apply to the current development implementation intended for
4.0.0. Unsupported operations generally return qdb_e_not_implemented;
invalid keys and policies are validation errors.
Operation |
Support and limits |
|---|---|
Creation |
One partition key on a time-series table with an explicit |
Keys |
Every ingested row must supply a valid, non-NULL key. Empty STRING keys are rejected. INT64_MIN is the reserved NULL value and cannot be a key. TIMESTAMP keys must be at or after the Unix epoch and strictly below 9,223,372,036,854,775,807 nanoseconds since the epoch. Distinct STRING keys must fit the internal alias limit, which also depends on the table name; use HASH for long STRING keys. |
Ingestion |
SQL INSERT and INSERT SELECT, and synchronous transactional/fast batch writes are supported. Async writes, truncate mode, ingestion deduplication and asynchronous client push are unsupported. |
SELECT |
Aggregations, grouping/HAVING, windows, transformations, restrictions, ASOF joins, lookback and interpolation use logical table semantics. Pruning supports the predicates described in the SELECT reference; unsupported expressions can require full partition scans. |
UPDATE |
Non-key updates are supported. A key can change only if every affected row stays in its existing physical partition. Moving rows between partitions is unsupported. NULL and empty STRING key updates are invalid. |
DELETE |
Ordinary filtered, ranged and full deletion are supported. Duplicate deletion requires comparisons to include the partition key, except for a one-slot HASH layout. Global duplicate comparison is unsupported. |
Schema changes |
Adding columns, renaming the key and dropping non-key columns are supported through SQL. Dropping the key is rejected. Setting TTL is unsupported. CREATE TABLE AS SELECT cannot create a column-partitioned layout; create it explicitly, then use INSERT SELECT. |
Transfers and maintenance |
Bulk reading, COPY, IMPORT, ANALYZE, SHOW DISK_USAGE, TRIM TABLE and index/ rewrite repair have partition-aware paths. COPY retains its ordinary query restrictions. ANALYZE and rewrite repair visit all column partitions even when given a time range. Raw-table export is unsupported. |
Legacy per-column APIs |
Legacy single-column insertion, range read, erase and aggregation APIs are unsupported for column-partitioned layouts. Legacy metadata/list/shard, local-table, TTL and expire-by-size APIs also reject these tables. Use SQL, the Experimental batch writer and the bulk reader. Multi-table bulk reads require matching schemas and partition layouts. See the C API. |
Row IDs |
Opaque identities distinguish rows in different physical partitions. Non-key UPDATE preserves them, but insertion/deletion can change later row ordinals. They are not persistent primary keys. |
Fresh-cluster acceptance checks do not establish persistent restart, damaged-storage recovery or replica-movement behavior. Those scenarios and restricted-user ACL combinations still require validation on this development branch; the implementation should not yet be treated as feature-complete.
1.9.4. See also#
CREATE TABLE and SHOW TABLE for creating and inspecting policies.
SELECT for pruning and row IDs.
Batch inserter for ingestion modes.
UPDATE and DELETE FROM for mutation restrictions.
Managing tables with Tags for choosing between separate tables and partitions within one table.