5.3. Metrics reference#
This reference lists the kernel metrics that QuasarDB reports. The available metrics can depend on the QuasarDB version and configuration.
A metric has one of these types:
An
accumulatoris an integer that increases during the life of the process.A
gaugeis an integer that can increase or decrease.A
labelis a text value.
5.3.1. Cache#
These metrics describe the QuasarDB cache and its two-level LRU2 metadata cache. A new LRU2 entry is cold. QuasarDB promotes an entry to the hot cache after another access.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total number of entries evicted from the QuasarDB cache. |
Deprecated. |
|
accumulator |
The total number of bytes evicted from the QuasarDB cache. |
Deprecated. |
|
gauge |
The current number of entries in the cold LRU2 metadata cache. |
|
|
accumulator |
The total number of entries evicted from the cold LRU2 metadata cache. |
|
|
accumulator |
The total number of bytes evicted from the cold LRU2 metadata cache. |
The byte value is not exact. |
|
accumulator |
The total number of entries read into the cold LRU2 metadata cache. |
|
|
accumulator |
The total number of bytes read into the cold LRU2 metadata cache. |
The byte value is not exact. |
|
gauge |
The current number of entries in the hot LRU2 metadata cache. |
|
|
accumulator |
The total number of entries evicted from the hot LRU2 metadata cache. |
|
|
accumulator |
The total number of bytes evicted from the hot LRU2 metadata cache. |
The byte value is not exact. |
|
accumulator |
The total number of hits in the hot LRU2 metadata cache. |
|
|
accumulator |
The total number of bytes returned by hits in the hot LRU2 metadata cache. |
The byte value is not exact. |
|
accumulator |
The total number of entries promoted from the cold cache to the hot LRU2 metadata cache. |
|
|
accumulator |
The total number of bytes promoted from the cold cache to the hot LRU2 metadata cache. |
The byte value is not exact. |
|
accumulator |
The total number of entries that QuasarDB read from disk into its cache. |
Deprecated. |
|
accumulator |
The total number of bytes that QuasarDB read from disk into its cache. |
Deprecated. |
5.3.2. Memory#
These metrics describe memory use by the host, the QuasarDB process, RocksDB, and the TBB allocator.
Host, resident-cache, and persistence memory gauges are part of the normal
statistics snapshot. The periodic memory.tbb.* allocator gauges require
additional instrumentation, which is disabled by default. Set
local.limiter.memory_statistics_refresh to an interval of at least 1,000
milliseconds to enable those gauges. A value of 0 leaves the allocator
instrumentation disabled. See Configuration for all forms of the
option.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
gauge |
The number of bytes in all RocksDB block caches. QuasarDB does not use the RocksDB row cache. |
|
|
gauge |
The number of bytes in the RocksDB memtables. |
|
|
gauge |
The number of unflushed bytes in the RocksDB memtables. |
|
|
gauge |
The number of bytes used by all RocksDB table readers. |
|
|
gauge |
The total number of bytes used by RocksDB memtables, table readers, and block caches. |
|
|
gauge |
The total number of physical-memory bytes detected on the host. |
|
|
gauge |
The number of physical-memory bytes used on the host. |
|
|
gauge |
The number of bytes used by entries that are in memory. |
|
|
gauge |
The number of entries in the internal in-memory hash table. This value is related to, but not equal to, the table count. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
The allocation threshold, in bytes, at which TBB uses huge pages when the system supports them. |
|
|
gauge |
The number of bytes used by large objects that do not fit in the optimized allocation structures. |
|
|
gauge |
The number of large objects that do not fit in the optimized allocation structures. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
The largest number of bytes in one allocation request. |
|
|
gauge |
The threshold, in bytes, at which TBB returns memory to the operating system. Below this threshold, TBB keeps the memory. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
An internal TBB allocator value used to diagnose complex memory problems. |
|
|
gauge |
The number of bytes currently managed by TBB. TBB does not manage all QuasarDB allocations. |
|
|
gauge |
The total number of allocations made through TBB. |
|
|
gauge |
The total number of virtual-memory bytes available to the process. This value is usually very large on a 64-bit operating system. |
|
|
gauge |
The number of virtual-memory bytes used by the process. Reserved memory can make this value larger than physical-memory use. |
5.3.3. Clustering#
These metrics describe the cluster topology and cluster synchronization.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total number of requests that clients sent to an incorrect node. |
|
|
label |
The node identifier. |
|
|
accumulator |
The total number of predecessor changes. Frequent changes can identify a cluster problem. |
|
|
accumulator |
The total number of successor changes. Frequent changes can identify a cluster problem. |
|
|
accumulator |
The total number of unstable-cluster errors returned to clients. |
|
|
accumulator |
The total time, in seconds, spent synchronizing clusters. |
|
|
accumulator |
The total number of cluster synchronization failures. |
|
|
accumulator |
The total number of successful cluster synchronizations. |
5.3.4. Environment#
These metrics identify the node environment, license, and QuasarDB build.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
label |
The engine build date. |
|
|
label |
The engine version. |
|
|
gauge |
The number returned by |
|
|
gauge |
The license attribution time, in seconds after the Unix epoch. |
|
|
gauge |
The license expiration time, in seconds after the Unix epoch. |
|
|
gauge |
The maximum number of memory bytes that the license permits for the node. |
|
|
gauge |
The number of days before the license expires. |
|
|
gauge |
The support expiration time, in seconds after the Unix epoch. |
|
|
label |
A description of the operating system. |
|
|
gauge |
The node start time, in seconds after the Unix epoch. |
5.3.5. Indexes#
These metrics describe use of the microindex subsystem during queries.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total number of aggregations that used a microindex. |
|
|
accumulator |
The total number of aggregations that could not use a microindex. |
|
|
accumulator |
The total number of filters, such as |
|
|
accumulator |
The total number of filters, such as |
5.3.6. Network#
These metrics describe sessions, requests, and network traffic.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
gauge |
The current number of users with an active session. |
|
|
gauge |
The number of network partitions. |
|
|
gauge |
The number of sessions that are available. |
|
|
gauge |
The maximum number of sessions. |
|
|
gauge |
The number of sessions that are busy. |
|
|
gauge |
The number of threads in each network partition. By default, QuasarDB selects this value. |
|
|
accumulator |
The total number of failed requests. |
|
|
accumulator |
The total number of request bytes received. |
|
|
accumulator |
The total number of response bytes sent. |
|
|
accumulator |
The total number of requests that exceeded the configured slow-operation limit. |
|
|
accumulator |
The total number of successful requests. |
|
|
accumulator |
The total number of received requests, including successful and failed requests. |
5.3.7. Performance profiling#
These metrics are available when network.profile_performance is enabled. Use them to compare the time spent in measured functions.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total time, in nanoseconds, for the specified metric in the specified function. |
|
|
accumulator |
The total time, in nanoseconds, for the specified function. |
|
|
accumulator |
The total time, in nanoseconds, for all measured functions in the current performance trace. |
5.3.8. Storage#
These metrics describe the QuasarDB persistence layer.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total number of delete operations on buckets. |
|
|
accumulator |
The total number of insert operations on buckets. |
|
|
accumulator |
The total number of times that QuasarDB expected a bucket on disk but did not find it. |
|
|
accumulator |
The total number of read operations on buckets. |
|
|
accumulator |
The total number of update operations on buckets. |
|
|
accumulator |
The total number of bytes written for all buckets, including large buckets. |
|
|
accumulator |
The total number of bucket writes, including writes of large buckets. |
|
|
accumulator |
The total time, in microseconds, spent writing buckets, including large buckets. |
|
|
gauge |
The current size, in bytes, of the local cache for RocksDB Cloud storage. |
|
|
gauge |
The number of entries in the persistence layer. This value is related to, but usually greater than, the number of tables and buckets. |
|
|
accumulator |
The total number of bytes written for large buckets. |
|
|
accumulator |
The total number of large-bucket writes. |
|
|
accumulator |
The total time, in microseconds, spent writing large buckets. |
|
|
gauge |
The current size, in bytes, of the persistent cache. This cache puts data from slow storage on faster storage. It is not the RocksDB Cloud local cache. |
|
|
gauge |
The total number of bytes that RocksDB read from storage. |
|
|
label |
The configured root path of the persistence layer. |
|
|
accumulator |
The total number of failed time-series write operations. |
|
|
accumulator |
The total number of successful time-series write operations. |
|
|
gauge |
The number of storage bytes used by RocksDB. |
|
|
gauge |
The total number of bytes that RocksDB wrote to storage. |
5.3.9. Storage - Async pipelines#
These metrics describe the async pipelines. In a metric name, [pipe_nb] identifies one pipeline, for example pipe_0.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The number of async pushes that the server rejected because a pipeline was full. |
The value is for the specified user. |
|
accumulator |
The number of async push errors that the server returned immediately. |
The value is for the specified user. |
|
gauge |
The current number of buffered bytes for the specified pipeline. |
|
|
gauge |
The current number of buffers for the specified pipeline. |
|
|
gauge |
The current number of buffered rows for the specified pipeline. |
|
|
gauge |
The interval, in milliseconds, before the most recent flush for the specified pipeline. |
|
|
gauge |
The number of bytes in the most recent flush for the specified pipeline. |
|
|
gauge |
The number of buffers in the most recent flush for the specified pipeline. |
|
|
gauge |
The number of rows in the most recent flush for the specified pipeline. |
|
|
gauge |
The largest number of bytes in one flush for the specified pipeline. |
|
|
gauge |
The largest number of buffers in one flush for the specified pipeline. |
|
|
gauge |
The largest number of rows in one flush for the specified pipeline. |
|
|
gauge |
The longest interval, in milliseconds, before a flush for the specified pipeline. |
|
|
accumulator |
The total number of bytes flushed for the specified pipeline. |
|
|
accumulator |
The total number of buffers flushed for the specified pipeline. |
|
|
accumulator |
The total number of rows flushed for the specified pipeline. |
|
|
accumulator |
The total number of flushes for the specified pipeline. |
|
|
accumulator |
The total time, in milliseconds, between flushes for the specified pipeline. |
|
|
gauge |
The largest number of bytes held by the specified pipeline. |
|
|
gauge |
The largest number of buffers held by the specified pipeline. |
|
|
gauge |
The largest number of rows held by the specified pipeline. |
|
|
accumulator |
The total number of write failures for the specified pipeline. |
|
|
gauge |
The number of bytes in the most recent write for the specified pipeline. |
|
|
gauge |
The duration, in microseconds, of the most recent write for the specified pipeline. |
|
|
gauge |
The number of rows in the most recent write for the specified pipeline. |
|
|
gauge |
The largest number of bytes in one write for the specified pipeline. |
|
|
gauge |
The longest write duration, in microseconds, for the specified pipeline. |
|
|
gauge |
The largest number of rows in one write for the specified pipeline. |
|
|
accumulator |
The total number of successful writes for the specified pipeline. |
|
|
accumulator |
The total number of bytes written for the specified pipeline. |
|
|
accumulator |
The total time, in microseconds, spent writing for the specified pipeline. |
|
|
accumulator |
The total number of rows written for the specified pipeline. |
|
|
gauge |
The largest number of bytes in one flush across all pipelines. |
|
|
gauge |
The largest number of buffers in one flush across all pipelines. |
|
|
gauge |
The largest number of rows in one flush across all pipelines. |
|
|
accumulator |
The total number of flushes started because the flush deadline was reached, across all pipelines. |
|
|
accumulator |
The total number of flushes started because the buffered data reached half of the configured size limit, across all pipelines. |
|
|
accumulator |
The total number of bytes flushed across all pipelines. |
|
|
accumulator |
The total number of buffers flushed across all pipelines. |
|
|
accumulator |
The total number of rows flushed across all pipelines. |
|
|
accumulator |
The total number of flushes across all pipelines. |
|
|
accumulator |
The total time, in milliseconds, between flushes across all pipelines. |
|
|
gauge |
The largest number of bytes pulled in one operation across all pipelines. |
|
|
accumulator |
The total number of bytes pulled across all pipelines. |
|
|
accumulator |
The total number of buffers pulled across all pipelines. |
|
|
gauge |
The largest number of bytes pushed in one operation across all pipelines. |
|
|
accumulator |
The total number of bytes pushed across all pipelines. |
|
|
accumulator |
The total number of buffers pushed across all pipelines. |
|
|
accumulator |
The total number of write failures across all pipelines. |
|
|
gauge |
The largest number of bytes in one write across all pipelines. |
|
|
gauge |
The longest write duration, in microseconds, across all pipelines. |
|
|
gauge |
The largest number of rows in one write across all pipelines. |
|
|
accumulator |
The total number of successful writes across all pipelines. |
|
|
accumulator |
The total number of bytes written across all pipelines. |
|
|
accumulator |
The total time, in microseconds, spent writing across all pipelines. |
|
|
accumulator |
The total number of rows written across all pipelines. |
5.3.10. Storage - Backups#
These metrics describe storage backups.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total time, in seconds, spent on backups. |
|
|
accumulator |
The total number of backup failures. |
|
|
accumulator |
The total number of successful background backups. |
|
|
accumulator |
The total number of bytes written during backups. |
5.3.11. Storage - Optimization#
These metrics describe storage operations that maintain performance and manage the data lifecycle.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
accumulator |
The total number of canceled background compactions. |
|
|
accumulator |
The total time, in seconds, spent on compaction. |
|
|
accumulator |
The total number of background compaction failures. |
|
|
accumulator |
The total number of successful, manually started background compactions. |
|
|
accumulator |
The total number of canceled background trim operations. |
|
|
accumulator |
The total time, in seconds, spent on background trim operations. |
|
|
accumulator |
The total number of background trim failures. |
|
|
accumulator |
The total number of successful background trim operations. |
5.3.12. Storage - RocksDB#
These metrics expose low-level RocksDB properties and counters.
Metric name |
Type |
Description |
Notes |
|---|---|---|---|
|
gauge |
The current delayed-write rate, in bytes per second. A value of 0 means that writes are not delayed. |
|
|
gauge |
The block-cache capacity, in bytes. |
|
|
gauge |
The number of block-cache bytes used by pinned entries. |
|
|
gauge |
The number of bytes used by entries in the block cache. |
|
|
accumulator |
The total number of bytes read during compaction. |
|
|
accumulator |
The total number of bytes read during compactions with the manually-marked reason. |
|
|
accumulator |
The total number of bytes read during periodic compactions. |
|
|
accumulator |
The total number of bytes read during TTL compactions. |
|
|
accumulator |
The total number of bytes written during compaction. |
|
|
accumulator |
The total number of bytes written during compactions with the manually-marked reason. |
|
|
accumulator |
The total number of bytes written during periodic compactions. |
|
|
accumulator |
The total number of bytes written during TTL compactions. |
|
|
gauge |
A value of 1 if a compaction is pending. Otherwise, the value is 0. |
|
|
gauge |
The approximate size, in bytes, of the active memtable. |
|
|
gauge |
The approximate size, in bytes, of the active memtable and unflushed immutable memtables. |
|
|
gauge |
The current LSM-tree version number. The value increases after an LSM-tree change and resets to 0 after a database restart. |
|
|
gauge |
The estimated size, in bytes, of live data. For BlobDB, this value includes live data in the current blob files. |
|
|
gauge |
The estimated number of keys in the active memtable, unflushed immutable memtables, and storage. |
|
|
gauge |
The estimated number of bytes that level-based compaction must rewrite to put all levels below their target sizes. |
|
|
gauge |
The estimated number of bytes used to read SST tables. This value excludes block-cache memory. |
|
|
accumulator |
The total number of bytes written during memtable flushes. |
|
|
accumulator |
The total number of |
|
|
accumulator |
The total number of |
|
|
accumulator |
The total number of |
|
|
accumulator |
The total number of bytes read from the last LSM-tree level. |
|
|
accumulator |
The total number of read operations on the last LSM-tree level. |
|
|
gauge |
The number of garbage bytes in blob files in the current version. |
|
|
gauge |
The total size, in bytes, of blob files in the current version. |
|
|
gauge |
The total size, in bytes, of SST files above the bottom level in the current LSM tree. |
|
|
gauge |
The total size, in bytes, of SST files in the current column-family version. |
|
|
gauge |
A value of 1 if a memtable flush is pending. Otherwise, the value is 0. |
|
|
gauge |
The minimum log-file number that RocksDB must keep. |
|
|
gauge |
The minimum obsolete SST-file number that RocksDB must keep. The maximum |
|
|
accumulator |
The total number of bytes read from LSM-tree levels above the last level. |
|
|
accumulator |
The total number of read operations on LSM-tree levels above the last level. |
|
|
gauge |
The number of blob files in the current version. |
|
|
gauge |
The number of delete entries in the active memtable. |
|
|
gauge |
The number of delete entries in unflushed immutable memtables. |
|
|
gauge |
The number of entries in the active memtable. |
|
|
gauge |
The number of entries in unflushed immutable memtables. |
|
|
gauge |
The number of immutable memtables that are not flushed. |
|
|
gauge |
The number of immutable memtables that are flushed. |
|
|
gauge |
The number of live RocksDB versions. A large value can identify iterators or unfinished compactions that prevent deletion of SST files. |
|
|
gauge |
The number of compactions in progress. |
|
|
gauge |
The number of flushes in progress. |
|
|
accumulator |
The number of unreleased database snapshots. |
|
|
gauge |
The total size, in bytes, of obsolete SST files that RocksDB did not delete or schedule for deletion. |
|
|
gauge |
The sequence number of the oldest unreleased snapshot. |
|
|
gauge |
The creation time of the oldest unreleased snapshot, in seconds after the Unix epoch. |
|
|
accumulator |
The total number of persistent-cache hits. |
|
|
accumulator |
The total number of persistent-cache misses. |
|
|
gauge |
The approximate size, in bytes, of the active memtable and unflushed immutable memtables. |
|
|
gauge |
The total size, in bytes, of blob files in all versions. |
|
|
gauge |
The total size, in bytes, of SST files in all column-family versions. Collection of this value can slow queries when the database has many files. |
|
|
accumulator |
The total number of RocksDB background errors. |
5.3.13. Metric name suffixes#
A metric-name suffix identifies the unit or value type.
Suffix |
Meaning |
|---|---|
|
A duration in nanoseconds. |
|
A duration in microseconds. |
|
A duration in milliseconds. |
|
A duration in seconds. |
|
A time in seconds after the Unix epoch. |
|
A number of bytes. |
|
A number of items, operations, or events. |
|
A value accumulated during the life of the process. |