5. Retrieving Statistics from a QuasarDB Cluster#
5.1. Using QuasarDB Python API#
This section provides a Python script that demonstrates how to use the QuasarDB Python API to connect to a QuasarDB cluster and retrieve statistics of all nodes in the cluster. The script outlines the process for accessing cumulative and by_uid statistics and provides guidance on interpreting various types of stats.
Prerequisites
Before running the script, make sure you have the following installed:
Python (3.x or higher)
QuasarDB Python API (quasardb)
Python Script
import quasardb
import quasardb.stats as qdbst
import json
with quasardb.Cluster('qdb://127.0.0.1:2836') as conn:
stats = qdbst.by_node(conn)
print(json.dumps(stats, indent=4))
{
"127.0.0.1:2836": {
"by_uid": {},
"cumulative": {
"async_pipelines.pulled.total_bytes": 0,
"async_pipelines.pulled.total_count": 0,
"cpu.idle": 6012100000,
"cpu.system": 166390000,
"cpu.user": 318260000,
"disk.bytes_free": 221570932736,
"disk.bytes_total": 274865303552,
"disk.path": "insecure/db/0-0-0-1",
"engine_build_date": "2023.12.06T20.59.15.000000000 UTC",
"engine_version": "3.15.x",
"evicted.count": 1,
"evicted.total_bytes": 209,
"hardware_concurrency": 8,
"license.attribution_date": 1701900515,
"license.expiration_date": 0,
"license.memory": 8589934592,
"license.remaining_days": 31337,
"license.support_until": 0,
"memory.persistence.cache_bytes": 104,
"memory.persistence.memtable_bytes": 1071088,
"memory.persistence.memtable_unflushed_bytes": 1071088,
"memory.persistence.table_reader_bytes": 0,
"memory.persistence.total_bytes": 1071192,
"memory.physmem.bytes_total": 33105100800,
"memory.physmem.bytes_used": 12619468800,
"memory.resident_bytes": 0,
"memory.resident_count": 0,
"memory.vm.bytes_total": 140737488351232,
"memory.vm.bytes_used": 1776730112,
"network.current_users_count": 0,
"network.sessions.available_count": 510,
"network.sessions.max_count": 512,
"network.sessions.unavailable_count": 2,
"node_id": "0-0-0-1",
"operating_system": "Linux 5.10.199-190.747.amzn2.x86_64",
"partitions_count": 8,
"perf.common.get_type_for_removal.deserialization.total_ns": 1374,
"perf.common.get_type_for_removal.processing.total_ns": 8101,
"perf.common.get_type_for_removal.total_ns": 9475,
"perf.control.get_system_info.deserialization.total_ns": 4851,
"perf.control.get_system_info.processing.total_ns": 599,
"perf.control.get_system_info.total_ns": 5450,
"perf.total_ns": 249523,
"perf.ts.create_root.content_writing.total_ns": 16186,
"perf.ts.create_root.deserialization.total_ns": 1717,
"perf.ts.create_root.entry_writing.total_ns": 53294,
"perf.ts.create_root.processing.total_ns": 131766,
"perf.ts.create_root.serialization.total_ns": 128,
"perf.ts.create_root.total_ns": 203091,
"perf.ts.get_columns.deserialization.total_ns": 1409,
"perf.ts.get_columns.processing.total_ns": 30098,
"perf.ts.get_columns.total_ns": 31507,
"persistence.capacity_bytes": 0,
"persistence.cloud_local_cache_bytes": 0,
"persistence.entries_count": 1,
"persistence.info": "RocksDB 6.27",
"persistence.persistent_cache_bytes": 0,
"persistence.read_bytes": 10245,
"persistence.utilized_bytes": 0,
"persistence.written_bytes": 25929,
"requests.bytes_in": 851,
"requests.bytes_out": 32,
"requests.errors_count": 2,
"requests.successes_count": 4,
"requests.total_count": 6,
"startup": 1701900515,
"startup_time": 1701900515,
"check.online": 1,
"check.duration_ms": 4
}
}
}
5.1.1. Understanding “by_uid” and “cumulative” Statistics#
The retrieved statistics are organized into two main dictionaries: “by_uid” and “cumulative.”
“by_uid”: This dictionary contains user statistics for a secure cluster with users. It groups statistics by their corresponding user IDs.
“cumulative”: This dictionary holds cumulative statistics for each node in the cluster. These statistics provide aggregated information across all users and are typically global values for the entire node.
5.2. Using qdbsh#
In qdbsh, you can use the direct_prefix_get command to retrieve a list of all statistics. Here’s an example:
qdbsh > direct_prefix_get $qdb.statistics 100
1. $qdb.statistics.async_pipelines.pulled.total_bytes
2. $qdb.statistics.async_pipelines.pulled.total_count
3. $qdb.statistics.cpu.idle
4. $qdb.statistics.cpu.system
...
52. $qdb.statistics.startup_time
qdbsh > direct_int_get $qdb.statistics.cpu.user
1575260000
qdbsh > direct_blob_get $qdb.statistics.node_id
c5fe30bf0154acc-5d63bd06e7878b9c-5f635b3cf7fc3560-dbe35df7b5080651
1:12
Note
The direct_prefix_get command is used to retrieve a list of statistics keys that match a specific prefix. In this case, the command is used to fetch statistics keys that start with the prefix $qdb.statistics. The number 100 is a parameter that limits the maximum number of keys returned by the command.