5. Retrieving Statistics from a QuasarDB Cluster#
5.1. Using QuasarDB Python API#
This section provides a Python script that demonstrates how to use the QuasarDB Python API to connect to a QuasarDB cluster and retrieve statistics of all nodes in the cluster. The script outlines the process for accessing cumulative and by_uid statistics and provides guidance on interpreting various types of stats.
Prerequisites
Before running the script, make sure you have the following installed:
Python (3.x or higher)
QuasarDB Python API (quasardb)
Python Script
import quasardb
import quasardb.stats as qdbst
from enum import Enum
import json
class EnumEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, Enum):
return obj.name
return super().default(obj)
with quasardb.Cluster('qdb://127.0.0.1:2836') as conn:
stats = qdbst.by_node(conn)
# Enum values are not JSON serializable, to serialize as JSON we need to use a custom encoder
print(json.dumps(stats, indent=4, cls=EnumEncoder))
{
"127.0.0.1:2836": {
"by_uid": {},
"cumulative": {
"async_pipelines.pulled.total_bytes": {
"value": 0,
"type": "ACCUMULATOR",
"unit": "BYTES"
},
"async_pipelines.pulled.total_count": {
"value": 0,
"type": "ACCUMULATOR",
"unit": "COUNT"
},
"chord.node_id": {
"value": "c5fe30bf0154acc-5d63bd06e7878b9c-5f635b3cf7fc3560-dbe35df7b5080651",
"type": "LABEL",
"unit": "NONE"
},
"engine_build_date": {
"value": "2025.05.07T14.08.52.000000000 UTC",
"type": "LABEL",
"unit": "NONE"
},
"engine_version": {
"value": "3.15.0.dev0",
"type": "LABEL",
"unit": "NONE"
},
"hardware_concurrency_count": {
"value": 16,
"type": "GAUGE",
"unit": "COUNT"
},
"license.attribution_date_epoch": {
"value": 1747030417,
"type": "GAUGE",
"unit": "EPOCH"
},
"license.expiration_date_epoch": {
"value": 0,
"type": "GAUGE",
"unit": "EPOCH"
},
"license.max_memory_bytes": {
"value": 8589934592,
"type": "GAUGE",
"unit": "BYTES"
},
"license.remaining_days_count": {
"value": 31337,
"type": "GAUGE",
"unit": "COUNT"
},
"license.support_until_epoch": {
"value": 0,
"type": "GAUGE",
"unit": "EPOCH"
},
"lru2.cold.count": {
"value": 0,
"type": "GAUGE",
"unit": "COUNT"
},
"lru2.hot.count": {
"value": 0,
"type": "GAUGE",
"unit": "COUNT"
},
"memory.persistence.cache_bytes": {
"value": 87,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.persistence.memtable_bytes": {
"value": 9455616,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.persistence.memtable_unflushed_bytes": {
"value": 9455616,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.persistence.table_reader_bytes": {
"value": 0,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.persistence.total_bytes": {
"value": 9455703,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.physmem.total_bytes": {
"value": 34282184704,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.physmem.used_bytes": {
"value": 16852508672,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.resident_bytes": {
"value": 0,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.resident_count": {
"value": 0,
"type": "GAUGE",
"unit": "COUNT"
},
"memory.vm.total_bytes": {
"value": 140737488224256,
"type": "GAUGE",
"unit": "BYTES"
},
"memory.vm.used_bytes": {
"value": 4725891072,
"type": "GAUGE",
"unit": "BYTES"
},
"network.current_users_count": {
"value": 0,
"type": "GAUGE",
"unit": "COUNT"
},
"network.partitions_count": {
"value": 16,
"type": "GAUGE",
"unit": "COUNT"
},
"network.sessions.available_count": {
"value": 2047,
"type": "GAUGE",
"unit": "COUNT"
},
"network.sessions.max_count": {
"value": 2048,
"type": "GAUGE",
"unit": "COUNT"
},
"network.sessions.unavailable_count": {
"value": 1,
"type": "GAUGE",
"unit": "COUNT"
},
"network.threads_per_partition_count": {
"value": 1,
"type": "GAUGE",
"unit": "COUNT"
},
"operating_system": {
"value": "Microsoft Windows 11 (build 26100), 64-bit",
"type": "LABEL",
"unit": "NONE"
},
"persistence.cloud_local_cache_bytes": {
"value": 0,
"type": "GAUGE",
"unit": "BYTES"
},
"persistence.entries_count": {
"value": 0,
"type": "GAUGE",
"unit": "COUNT"
},
"persistence.persistent_cache_bytes": {
"value": 0,
"type": "GAUGE",
"unit": "BYTES"
},
"persistence.read_bytes": {
"value": 918249,
"type": "GAUGE",
"unit": "BYTES"
},
"persistence.root_path": {
"value": "db",
"type": "LABEL",
"unit": "NONE"
},
"persistence.utilized_bytes": {
"value": 0,
"type": "GAUGE",
"unit": "BYTES"
},
"persistence.written_bytes": {
"value": 7572758,
"type": "GAUGE",
"unit": "BYTES"
},
"requests.failures_count": {
"value": 2509,
"type": "ACCUMULATOR",
"unit": "COUNT"
},
"requests.in_bytes": {
"value": 2875592,
"type": "ACCUMULATOR",
"unit": "BYTES"
},
"requests.out_bytes": {
"value": 249577,
"type": "ACCUMULATOR",
"unit": "BYTES"
},
"requests.successes_count": {
"value": 6125,
"type": "ACCUMULATOR",
"unit": "COUNT"
},
"requests.total_count": {
"value": 8634,
"type": "ACCUMULATOR",
"unit": "COUNT"
},
"startup_epoch": {
"value": 1747030417,
"type": "GAUGE",
"unit": "EPOCH"
},
"check.online": {
"value": 1,
"type": "ACCUMULATOR",
"unit": "NONE"
},
"check.duration_ms": {
"value": 34,
"type": "ACCUMULATOR",
"unit": "MILLISECONDS"
}
}
}
}
5.1.1. Understanding “by_uid” and “cumulative” Statistics#
The retrieved statistics are organized into two main dictionaries: “by_uid” and “cumulative.”
“by_uid”: This dictionary contains user statistics for a secure cluster with users. It groups statistics by their corresponding user IDs.
“cumulative”: This dictionary holds cumulative statistics for each node in the cluster. These statistics provide aggregated information across all users and are typically global values for the entire node.
5.2. Using qdbsh#
In qdbsh, you can use the direct_prefix_get command to retrieve a list of all statistics. Here’s an example:
qdbsh > direct_prefix_get $qdb.statistics 100
1. $qdb.statistics.async_pipelines.pulled.total_bytes
2. $qdb.statistics.async_pipelines.pulled.total_count
3. $qdb.statistics.cpu.idle
4. $qdb.statistics.cpu.system
...
52. $qdb.statistics.startup_time
qdbsh > direct_int_get $qdb.statistics.cpu.user
1575260000
qdbsh > direct_blob_get $qdb.statistics.node_id
c5fe30bf0154acc-5d63bd06e7878b9c-5f635b3cf7fc3560-dbe35df7b5080651
1:12
Note
The direct_prefix_get command is used to retrieve a list of statistics keys that match a specific prefix. In this case, the command is used to fetch statistics keys that start with the prefix $qdb.statistics. The number 100 is a parameter that limits the maximum number of keys returned by the command.