2. Configuration

    2. Configuration

    2.1. QuasarDB Daemon

    When it comes to configuring your environment, both Docker and QuasarDB Daemon offer distinct sets of options. Here’s a breakdown of the configuration possibilities:

    QuasarDB stores its configuration options in a qdbd.conf file. The precise location on where this file resides depends on your method of installation, but is typically /etc/qdb/qdbd.conf.

    This section is organised per variable (config file, environment variable, command-line argument) and its description.

    2.1.1. License

    You can set a license either by file or by putting the license key as a string directly into your configuration.

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.user.license_file

    --local-user-license-file

    QDB_LOCAL_USER_LICENSE_FILE

    Absolute location to the license file. Be careful to ensure the license file is readable by the qdb user, otherwise the license will not activate.

    local.user.license_key

    --local-user-license-key

    QDB_LOCAL_USER_LICENSE_KEY

    License key as a string.

    2.1.2. Parallelism

    Warning

    Improper configuration of the parallelism will negatively impact performances.

    QuasarDB has been designed for multicore architectures. Queries coming from clients are processed in parallel into disctinct shards called “partitions”. Each partition has a configurable number of threads.

    In most instances, you want to use the unified setting that gives a “budget” to let QuasarDB pick for you the right number of partitions and threads per partitions.

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.network.parallelism

    --local-network-parallelism

    QDB_LOCAL_NETWORK_PARALLELISM

    How many cores to allocate to process queries. The default of 0 will let QuasarDB pick a number depending on the computing resources available on the computer.

    It’s possible to have total control over the configuration in using the partitions_count and threads_per_partition settings. If these values are not 0, the parallelism setting will be ignored and QuasarDB will use the exact configured number of partitions and threads per partition. It is strongly advised to set partitions_count and threads_per_partition to 0 and use the parallelism setting instead.

    Note

    The storage layer has independent multithreading settings that are not affected by the settings above. In other words, the cores attributed to storage come in addition to the ones used for queries processing.

    For Docker: QuasarDB will dedicate this amount of threads to serve requests. Use in conjunction with ROCKSDB_THREADS to properly tune for your available processing power.

    2.1.3. Storage

    Config File,Command-Line Argument & Environment Variable

    Description

    global.cluster.storage_engine
    --global-cluster-storage-engine
    QDB_GLOBAL_CLUSTER_STORAGE_ENGINE

    Specifies which storage engine to use. Accepted values: rocksdb or transient. Defaults to rocksdb.

    local.depot.rocksdb.root
    --local-depot-rocksdb-root
    QDB_LOCAL_DEPOT_ROCKSDB_ROOT

    Location to the root folder of where your database files are stored. Defaults to /var/lib/qdb.

    local.depot.rocksdb.wal
    --local-depot-rocksdb-wal
    QDB_LOCAL_DEPOT_ROCKSDB_WAL

    Location to the Write-Ahead-Log (WAL) folder. Defaults to empty, which will make it a subdirectory of the root directory called “wal”.

    local.depot.rocksdb.tiered_storage
    --local-depot-rocksdb-tiered-storage
    QDB_LOCAL_DEPOT_ROCKSDB_TIRED_STORAGE

    An array in the form [{“path”: “path1”, “size”: size1}, {“path”: “path2”, “size”: size2}] which specified, which directory, in order should be used for storage. When a storage exceeds capacity, older data is moved to the next storage in the list. Actual usage may exceed the size specified. Size is specified in bytes.

    local.depot.rocksdb.compaction
    --local-depot-rocksdb-compaction
    QDB_LOCAL_DEPOT_ROCKSDB_COMPACTION

    The compaction strategy to use. By default it is “leveled”, but “universal” is also accepted. Universal improves write amplification at the cost of space amplification.

    local.depot.rocksdb.disable_wal
    --local-depot-rocksdb-disable-wal
    QDB_LOCAL_DEPOT_ROCKSDB_DISABLE_WAL

    When this option is set to true, writes will not go to the Write-Ahead-Log (WAL), increasing performance at the cost of durability. Writes go to the WAL by default. Use this option with caution.

    local.depot.rocksdb.max_total_wal_size
    --local-depot-rocksdb-max-total-wal-size
    QDB_LOCAL_DEPOT_ROCKSDB_MAX_TOTAL_WAL_SIZE

    Size, in bytes, of the Write-Ahead-Log (WAL). The WAL is where every update gets written. When the WAL is full, memtables are flushed to disk. If the WAL is too small, memtables may thus be flushed before they are full, impacting write speed. However, large WALs means increased memory usage and potentially higher compaction. Your WAL should be large enough to absorbs spikes, but cannot be used to compensate for a persistence layer too slow to absorb the load or memtables being too small. See local.depot.rocksdb.table_mem_budget.

    local.depot.rocksdb.max_background_flushes
    --local-depot-rocksdb-max-background-flushes
    QDB_LOCAL_DEPOT_ROCKSDB_MAX_BACKGROUND_FLUSHES

    How many threads will be dedicated to writing data to disk. Write heavy scenarios may benefit from a higher count. Defaults to 1`.

    local.depot.rocksdb.max_bytes
    --local-depot-rocksdb-max-bytes
    QDB_LOCAL_DEPOT_ROCKSDB_MAX_BYTES

    The maximum amount of data to store, in bytes. Inserting data will fail with “quota exceeded” when the limit is reached. If set to 0, no check is done.

    local.depot.rocksdb.persistent_cache_path
    --local-depot-rocksdb-persistent-cache-path
    QDB_LOCAL_DEPOT_ROCKSDB_PERSISTENT_CACHE_PATH

    A path to a local disk to be used as a persistent cache. May increase performance when data is stored in a remote disk.

    local.depot.rocksdb.persistent_cache_size
    --local-depot-rocksdb-persistent-cache-size
    QDB_LOCAL_DEPOT_ROCKSDB_PERSISTENT_CACHE_SIZE

    The maximum size of the persistent cache, in bytes. Cannot be zero if the persistent cache path is specified.

    local.depot.rocksdb.persistent_cache_nvme_optimization
    --local-depot-rocksdb-persistent-cache-nvme-optimization
    QDB_LOCAL_DEPOT_ROCKSDB_PERSISTENT_CACHE_NVME_OPTIMIZATION

    If your persistent cache is on an NVME, enabling this option may increase performance.

    local.depot.rocksdb.additional_options
    --local-depot-rocksdb-additional-options
    QDB_LOCAL_DEPOT_ROCKSDB_ADDITIONAL_OPTIONS

    Specifies additional configuration options to fine-tune RocksDB storage behavior.

    local.depot.rocksdb.block_based_table_options
    --local-depot-rocksdb-block-based-table-options
    QDB_LOCAL_DEPOT_ROCKSDB_BLOCK_BASED_TABLE_OPTIONS

    Allows configuration of specific options for block-based tables used in RocksDB storage.

    local.depot.rocksdb.storage_warning_level
    --local-depot-rocksdb-storage-warning-level
    QDB_LOCAL_DEPOT_ROCKSDB_STORAGE_WARNING_LEVEL

    An integer representing how often quasardb will emit a warning about depleting disk space, in milliseconds.

    local.depot.rocksdb.sync_period_with_master_db
    --local-depot-rocksdb-sync-period-with-master-db
    QDB_LOCAL_DEPOT_ROCKSDB_SYNC_PERIOD_WITH_MASTER_DB

    The interval for synchronizing the read-only RocksDB-cloud with the master database. It cannot exceed 30 minutes - half the time it takes for deleted files.

    local.depot.rocksdb.create_if_missing
    --local-depot-rocksdb-create-if-missing
    QDB_LOCAL_DEPOT_ROCKSDB_CREATE_IF_MISSING

    A boolean value indicating whether the storage engine should create a new database if it doesn’t already exist. When set to true, the storage engine will create a new database if one with the specified name is not found.

    local.depot.rocksdb.disable_auto_compaction
    --local-depot-rocksdb-disable-auto-compaction
    QDB_LOCAL_DEPOT_ROCKSDB_DISABLE_AUTO_COMPACTION

    A boolean value that, when set to true, disables automatic compaction of SST files in the storage engine. Compaction is a process that merges and optimizes data files to improve storage efficiency. Disabling auto-compaction means users must manually run cluster_compact in qdbsh often. This boosts write performance but can harm read performance.

    local.depot.rocksdb.disable_manifest_sync
    --local-depot-rocksdb-disable-manifest-sync
    QDB_LOCAL_DEPOT_ROCKSDB_DISABLE_MANIFEST_SYNC

    A boolean value that, when set to true, prevents the storage engine from synchronizing manifest files with disk. Manifest files are used to track the state of the SST files in the storage engine.

    local.depot.rocksdb.keep_log_file_num
    --local-depot-rocksdb-keep-log-file-num
    QDB_LOCAL_DEPOT_ROCKSDB_KEEP_LOG_FILE_NUM

    An integer specifying the maximum number of log files to keep in the storage engine’s log directory. Log files contain records of database operations and are used for crash recovery.

    local.depot.rocksdb.log_file_time_to_roll
    --local-depot-rocksdb-log-file-time-to-roll
    QDB_LOCAL_DEPOT_ROCKSDB_LOG_FILE_TIME_TO_ROLL

    An integer representing the time interval, in milliseconds, after which log files in the storage engine will be rolled. Rolling involves closing the current log file and starting a new one to manage the size of log files.

    local.depot.rocksdb.max_log_file_size
    --local-depot-rocksdb-max-log-file-size
    QDB_LOCAL_DEPOT_ROCKSDB_`MAX_LOG_FILE_SIZE

    An integer specifying the maximum size of log files in the storage engine, in bytes. When a log file reaches this size, it will be rolled to a new file.

    local.depot.rocksdb.paranoid_checks
    --local-depot-rocksdb-paranoid-checks
    QDB_LOCAL_DEPOT_ROCKSDB_PARANOID_CHECKS

    A boolean value indicating whether to perform extra checks for data integrity and correctness. When enabled, the storage engine will perform additional validation to catch potential errors.

    local.depot.rocksdb.paranoid_file_checks
    --local-depot-rocksdb-paranoid-file-checks
    QDB_LOCAL_DEPOT_ROCKSDB_PARANOID_FILE_CHECKS

    A boolean value indicating whether to perform extra checks on data files for integrity and correctness. Similar to paranoid_checks, this option applies specifically to data files in the storage engine.

    local.depot.rocksdb.rate_limit
    --local-depot-rocksdb-rate-limit
    QDB_LOCAL_DEPOT_ROCKSDB_RATE_LIMIT

    The rate limit is specified in bytes per second; a value of 0 indicates that rate limiting is disabled. This rate limit applies to the amount of I/O allocated for flushing and compaction, with flushing operations prioritized over compaction

    local.depot.rocksdb.read_only
    --local-depot-rocksdb-read-only
    QDB_LOCAL_DEPOT_ROCKSDB_READ_ONLY

    A boolean value indicating whether the database should be opened in read-only mode. When set to true, write operations will be disallowed.

    local.depot.rocksdb.storage_warning_interval
    --local-depot-rocksdb-storage-warning-interval
    QDB_LOCAL_DEPOT_ROCKSDB_STORAGE_WARNING_INTERVAL

    Specifies the interval for emitting warnings about depleting disk space. This warning helps users take timely action to prevent running out of disk space.

    local.depot.rocksdb.threads
    --local-depot-rocksdb-threads
    QDB_LOCAL_DEPOT_ROCKSDB_THREADS

    Should be increased if the daemon is under constant insertion pressure, and there is still available CPU: it increases the amount of threads the storage engine has available for background operations such as compaction.

    local.depot.rocksdb.hi_threads
    --local-depot-rocksdb-hi-threads
    QDB_LOCAL_DEPOT_ROCKSDB_HI_THREADS

    The number of threads allocated to high-priority operations such as flushes to disk. The default setting of 2 is sufficient for most cases. Increase if the server cannot keep up with the persistence layer.

    local.depot.rocksdb.cloud.aws.access_key_id
    --local-depot-rocksdb-cloud-aws-access-key-id
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_ACCESS_KEY_ID

    The AWS access key id to use for authentication.

    local.depot.rocksdb.cloud.aws.secret_key
    --local-depot-rocksdb-cloud-aws-secret-key
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_SECRET_KEY

    The secret key to use for authentication. You need both the access key and secret key set for authentication to work. access_key_id and secret_key is not recommended for production deployments. Use use_instance_auth for production.

    local.depot.rocksdb.cloud.constant_sst_file_size_in_sst_file_manager
    --local-depot-rocksdb-cloud-constant-sst-file-size-in-sst-file-manager
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_CONASTANT_SST_FILE_SIZE_IN_SST_FILE_MANAGER

    Specifies a constant size for SST files within the SST file manager in cloud storage. The value is set to -1, indicating that the default size is used.

    local.depot.rocksdb.cloud.number_objects_listed_in_one_iteration
    --local-depot-rocksdb-cloud-number-objects-listed-in-one-iteration
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_NUMBER_OBJECTS_LISTED_IN_ONE_ITERATION

    Determines the number of objects listed in a single iteration when interacting with cloud storage. The value is set to 5000.

    local.depot.rocksdb.cloud.ssl_ca_certificate
    --local-depot-rocksdb-cloud-ssl-ca-certificate
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_SSL_CA_CERTIFICATE

    Specifies the path to the SSL certificate authority (CA) certificate for secure communication with cloud storage. The value is an empty string.

    local.depot.rocksdb.cloud.ssl_verify
    --local-depot-rocksdb-cloud-ssl-verify
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_SSL_VERIFY

    Controls SSL verification when communicating with cloud storage. The value is set to true.

    local.depot.rocksdb.cloud.local_sst_cache_size
    --local-depot-rocksdb-cloud-local-sst-cache-size
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_LOCAL_SST_CACHE_SIZE

    The size of the local SST cache, in bytes. If 0, no SST file will be kept locally. The default value is 18446744073709551615, which means every SST file will be kept locally and if the files are missing at startup, they will be downloaded in the background.

    local.depot.rocksdb.cloud.request_timeout
    --local-depot-rocksdb-cloud-request-timeout
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_REQUEST_TIMEOUT

    The timeout used for cloud queries, in milliseconds. The default of 0 means that the cloud provider’s default will be used.

    local.depot.rocksdb.cloud.create_bucket_if_missing
    --local-depot-rocksdb-cloud-create-bucket-if-missing
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_CREATE_BUCKET_IF_MISSING

    Will create the bucket if it does not exist.

    local.depot.rocksdb.cloud.bucket.destination_bucket
    --local-depot-rocksdb-cloud-bucket-destination-bucket
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_BUCKET_DESTINATION_BUCKET

    The bucket name where to store the data. On AWS, the bucket name must be unique. If name is empty, the database will not upload its content to the cloud.

    local.depot.rocksdb.cloud.bucket.source_bucket
    --local-depot-rocksdb-cloud-bucket-source-bucket
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_BUCKET_SOURCE_BUCKET

    The bucket name where to read the data from. On AWS, the bucket name must be unique. If name is empty, the database will not read its content from the cloud.

    local.depot.rocksdb.cloud.aws.config_file
    --local-depot-rocksdb-cloud-aws-config-file
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_CONFIG_FILE

    The configuration file to use for authentication. When using a configuration file, access_key_id and secret_key must be left blank.

    local.depot.rocksdb.cloud.aws.use_transfer_manager
    --local-depot-rocksdb-cloud-aws-use-transfer-manager
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_USE_TRANSFER_MANAGER

    Enabling this option leverages the parallelism of uploads and downloads of files to and from S3. This optimization enhances performance but necessitates increased memory usage for various buffers.

    local.depot.rocksdb.cloud.aws.transfer_manager_threads
    --local-depot-rocksdb-cloud-aws-transfer-manager-threads
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_TRANSFER_MANAGER_THREADS

    The number of threads to be used by the AWS transfer manager.

    local.depot.rocksdb.cloud.aws.transfer_manager_buffer_size
    --local-depot-rocksdb-cloud-aws-transfer-manager-buffer-size
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_TRANSFER_MANAGER_BUFFER_SIZE

    The buffer size for the AWS transfer manager. Files larger will be broken into smaller pieces, transfered in parallel.

    local.depot.rocksdb.cloud.aws.endpoint_override
    --local-depot-rocksdb-cloud-aws-endpoint-override
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_ENDPOINT_OVERRIDE

    This configuration is intended for non-AWS S3 deployments. It allows you to specify a different hostname or IP address to communicate with, replacing the default AWS S3 endpoint.

    local.depot.rocksdb.cloud.aws.config_file_section
    --local-depot-rocksdb-cloud-aws-config-file-section
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_CONFIG_FILE_SECTION

    The section name in the configuration file with the access key_id, secret_key and session_token

    local.depot.rocksdb.cloud.aws.encryption_key_id
    --local-depot-rocksdb-cloud-aws-encryption-key-id
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_ENCRYPTION_KEY_ID

    Enabling server-side encryption, if set to true, utilizes AWS KMS when used with encryption_key_id in S3 mode. In other cases, it employs Amazon’s automatically generated S3 server-side encryption key, if provided, to perform encryption.

    local.depot.rocksdb.cloud.aws.use_instance_auth
    --local-depot-rocksdb-cloud-aws-use-instance-auth
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_AWS_USE_INSTANCE_AUTH

    When enabled, this configuration utilizes EC2 instance metadata for automatic authentication credential resolution when communicating with the S3 bucket. It is typically recommended to set this to true in production deployments for seamless and secure credential management.

    local.depot.rocksdb.cloud.bucket.path_prefix
    --local-depot-rocksdb-cloud-bucket-path-prefix
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_BUCKET_PATH_PREFIX

    Quasardb will use local.depot.rocksdb.root as the path inside the bucket to store the data in the cloud. If path_prefix is not empty, a prefix will be added to this path. That way, different nodes can use the same local path and write to different paths within a same bucket.

    local.depot.rocksdb.cloud.bucket.region
    --local-depot-rocksdb-cloud-bucket-region
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_BUCKET_REGION

    The region where the bucket is located.

    local.depot.rocksdb.cloud.provider
    --local-depot-rocksdb-cloud-provider
    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_PROVIDER

    A string either “none” or “aws” specifying your cloud provider to use to store the cloud data.

    For more information regarding RocksDB tuning refer to RocksDB Tuning Guide

    2.1.4. Networking

    The QuasarDB daemon uses TCP for network communications and listens to two ports. The primary port is configurable and is the port used by client applications. Its default value is 2836. The secondary port, is used for cluster discovery mechanisms and queries run on separate, higher priority partitions. This design ensures that when the server is busy processing queries, it can still service cluster management requests and preserve stability.

    The value of the secondary port is set to the value of the primary port plus one (1). Thus, by default, the value of the secondary port is 2837.

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.network.listen_on

    --local-network-listen-on

    QDB_LOCAL_NETWORK_LISTEN_ON

    Local address and port to bind to for accepting new connections. Can also refer to network adapter, e.g. eth0:2836. Defaults to 127.0.0.1:2836. The daemon will listen to the specified port and the port + 1 for service messages. For example, if the server is configured to listen to 127.0.0.1:2836 it will also listen to 127.0.0.1:2837.

    local.network.advertise_as

    --local-network-advertise-as

    QDB_LOCAL_NETWORK_ADVERTISE_AS

    Specify if the local address we bind to is not the same as other nodes or clients should use to connect to the daemon (e.g. when behind a NAT). Defaults to 127.0.0.1:2836.

    local.network.idle_timeout

    --local-network-idle-timeout

    QDB_LOCAL_NETWROK_IDLE_TIMEOUT

    Time (in milliseconds) until a server socket is automatically recycled.

    local.network.client_timeout

    --local-network-client-timeout

    QDB_LOCAL_NETWORK_CLIENT_TIMEOUT

    Time (in milliseconds) until a client connection without activity is automatically disconnected.

    local.network.max_in_buffer_size

    --local-network-max-in-buffer-size

    QDB_LOCAL_NETWORK_MAX_IN_BUFFER_SIZE

    Size of the buffer the server allocates for receiving data from clients. Increase this size if you experience unexpected closed connections from the client, or the server logs mention cannot send data: Broken pipe.

    local.network.max_out_buffer_size

    --local-network-max-out-buffer-size

    QDB_LOCAL_NETWORK_MAX_OUT_BUFFER_SIZE

    Size of the buffer the server allocates for sending data to clients. Increase this size if you experience unexpected closed connections from the client, or the server logs mention cannot send data: Broken pipe.

    global.cluster.publish_firehose

    --global-cluster-publish-firehose

    QDB_GLOBAL_CLUSTER_PUBLISH_FIREHOSE

    Enable changes publications on the firehose.

    local.network.firehose_endpoint

    --local-network-firehose-endpoint

    QDB_LOCAL_NETWORK_FIREHOSE_ENDPOINT

    Endpoint the firehose should listen to subscribers. Only active if the firehose publisher is enabled.

    global.cluster.persisted_firehose

    --global-cluster-persisted-firehose

    QDB_GLOBAL_CLUSTER_PERSISTED_FIREHOSE

    A string representing the name of the persisted firehose. An empty string disables the persisted firehose. The default value is “$qdb.firehose”.

    global.cluster.persisted_firehose_shard_size

    --global-cluster-persisted-firehose-shard-size

    QDB_GLOBAL_CLUSTER_PERSISTED_FIREHOSE_SHARD_SIZE

    Specifies the size (in milliseconds) of a shard in a persisted firehose. A shard is a logical unit of data within a firehose stream.

    global.cluster.enable_replication

    --global-cluster-enable-replication

    QDB_GLOBAL_CLUSTER_ENABLE_REPLICATION

    Enable replication. Enabled by default.

    local.network.firehose_publishing_threads

    --local-network-firehose-publishing-threads

    QDB_LOCAL_NETWORK_FIREHOSE_PUBLISHING_THREADS

    Specifies the number of threads used for publishing data to Firehose. The default value is 1.

    local.network.new_version_check

    --local-network-new-version-check

    QDB_LOCAL_NETWORK_NEW_VERSION_CHECK

    Controls whether the system checks for new versions.

    local.network.total_server_sessions

    --local-network-total-server-sessions

    QDB_LOCAL_NETWORK_TOTAL_SERVER_SESSIONS

    Defines the maximum number of total server sessions allowed.

    local.network.log_slow_operation_ms

    --local-network-log-slow-operation-ms

    QDB_LOCAL_NETWORK_LOG_SLOW_OPERATION_MS

    Specifies the threshold in milliseconds for logging slow network operations.

    2.1.5. Clustering

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.chord.node_id

    --local-chord-node-id

    QDB_LOCAL_CHORD_NODE_ID

    Unique identifier for this node. See also: Node ID configuration.

    local.chord.bootstrapping_peers

    --local-chord-bootstrapping-peers

    QDB_LOCAL_CHORD_BOOTSTRAPPING_PEERS

    One or more peers that should be connected to to discover and bootstrap the cluster. Peers should be specified as a JSON array of tuples of address:port, e.g. ["192.168.1.1:2836", "192.168.1.2:2836"].

    global.cluster.replication_factor

    --global-cluster-replication-factor

    QDB_GLOBAL_CLUSTER_REPLICATION_FACTOR

    Replication factor of data stored inside the cluster. Must be equal to or lower than the amount of nodes in your cluster, and not higher than 4. A replication factor of 1 means no replication.

    global.cluster.acl_cache_duration

    `` –global-cluster-acl-cache-duration``

    QDB_GLOBAL_CLUSTER_ACL_CACHE_DURATION

    An integer representing the maximum guaranteed duration of cached ACL information. Nodes may cache ACL information to enhance performance.

    global.cluster.acl_cache_size

    `` –global-cluster-acl-cache-size``

    QDB_GLOBAL_CLUSTER_ACL_CACHE_SIZE

    An integer representing the maximum number of cached ACL information entries per node. Cached ACL information helps optimize performance.

    global.cluster.enable_remote_acl_fetch

    `` –global-cluster-enable-remote-acl-fetch``

    QDB_GLOBAL_CLUSTER_ENABLE_REMOTE_ACL_FETCH

    Enabling this feature allows the cluster to ensure that ACL information remains up-to-date across distributed systems. When set to true, the cluster will actively fetch ACL data remotely.

    global.cluster.max_transaction_duration

    --global-cluster-max-transaction-duration

    QDB_GLOBAL_CLUSTER_TRANSACTION_DURATION

    An integer representing the maximum guaranteed duration of a transaction, in milliseconds. Transactions exceeding this interval are rolled back. The default value is 15,000 ms.

    global.cluster.max_versions

    --global-cluster-max-versions

    QDB_GLOBAL_CLUSTER_MAX_VERSIONS

    An integer indicating the maximum number of copies the cluster retains for transaction history. If an entry has more versions than this value, the oldest versions are subject to garbage collection.

    local.chord.max_stabilization_interval

    --local-chord-max-stabilization-interval

    QDB_LOCAL_CHORD_MAX_STABILIZATION_INTERVAL

    The maximum wait interval between two stabilizations, in milliseconds. Nodes disappearance will take at least that amount of time. The default value is 60,000 ms (one minute). This value must be greater than the minimum stabilization interval and cannot be lower than 10 ms.

    local.chord.min_stabilization_interval

    --local-chord-min-stabilization-interval

    QDB_LOCAL_CHORD_MIN_STABILIZATION_INTERVAL

    The minimum wait interval between two stabilizations, in milliseconds. The default value is 100 ms; it is rarely needed to change this value. This value cannot be zero.

    local.chord.no_stabilization

    --local-chord-no-stabilization

    QDB_LOCAL_CHORD_NO_STABILIZATION

    A read-only boolean value representing whether or not this node should stabilize upon startup. Even if set to true, stabilization will still occur.

    local.chord.sync_before_join

    --local-chord-sync-before-join

    QDB_LOCAL_CHORD_SYNC_BEFORE_JOIN

    Enable or disable synchronization of data before joining the cluster. Enabling it might provoke spurious data downloads and higher disk usage. Disabling it on a clear node might result in missing historical data on this node.

    2.1.6. Security

    Config File

    Command-Line Argument

    Environment Variable

    Description

    global.security.enabled

    --global-security-enabled

    QDB_GLOBAL_SECURITY_ENABLED

    Boolean that determines whether security should be enabled. Valid values are true and false. Defaults to false. Enabling this to true requires you to set valid values for user_list and cluster_private_file.

    global.security.user_list

    --global-security-user-list

    QDB_GLOBAL_SECURITY_USER_LIST

    Absolute location to the file that contains the QuasarDB users. Typically should be /etc/qdb/users.txt. The file can be created and managed using the QuasarDB user management tool, and needs to be identically replicated accross all QuasarDB daemon nodes.

    global.security.cluster_private_file

    --global-security-cluster-private-file

    QDB_GLOBAL_SECURITY_CLUSTER_PRIVATE_FILE

    Absolute location to the cluster private key file. Typically should be /etc/qdb/cluster_private.key. The file can be created and managed using the QuasarDB cluster keygen, and needs to be identically replicated accross all QuasarDB daemon nodes.

    global.security.encrypt_traffic

    --global-security-encrypt-traffic

    QDB_GLOBAL_SECURITY_ENCRYPT_TRAFFIC

    Boolean that determines whether node-to-node communication should be encrypted. Valid values are true and false. Defaults to false.

    global.security.enable_purge_all

    --global-security-enable-purge-all

    QDB_GLOBAL_SECURITY_ENABLE_PURGE_ALL

    Allows the cluster to be remotely purged via an API call. False by default.

    global.security.enable_stop

    --global-security-enable-stop

    QDB_GLOBAL_SECURITY_ENABLE_STOP

    Allows a node to be remotely stop via an API call. False by default.

    2.1.7. Performance tuning

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.network.server_sessions

    --local-network-server-sessions

    QDB_LOCAL_NETWORK_SERVER_SESSIONS

    How many sessions (connections) each server thread is able to handle concurrently. Too few will cause significant performance degradations. Increase this value if you see “out of free server sessions” messages in the logs.

    local.network.partitions_count

    --local-network-partitions-count

    QDB_LOCAL_NETWORK_PARTITIONS_COUNT

    The number of threads in the main worker pool. Handles blocking I/O operations, and should be increased if your clients are waiting on the server but the server is not at 100% CPU. These are the queries handled by the main port.

    local.network.hi_partitions_count

    --local-network-hi-partitions-count

    QDB_LOCAL_NETWORK_HI_PARTITIONS_COUNT

    The number of threads dedicated for high-priority, service, messages. The default value of 2 is adequate for most setups and should be only increased for cluster with more than 20 nodes. These are the queries handled by the secondary port.

    local.depot.rocksdb.max_open_files

    --local-depot-rocksdb-max-open-files

    QDB_LOCAL_DEPOT_ROCKSDB_MAX_OPEN_FILES

    Maximum number of open files that the storage engine is allowed to use. This should typically be close to the number of files the qdb user is allowed to open (i.e. the output of ulimit -n). Too low will cause significant performance degradations, and you are recommended to use at least 60000 in production.

    local.limiter.max_bytes_soft

    --local-limiter-max-bytes-soft

    QDB_LOCAL_LIMITER_MAX_BYTES_SOFT

    The maximum amount of memory (in bytes) QuasarDB will use. Any usage above this will cause QuasarDB to evict entries from its cache.

    local.limiter.max_bytes_hard

    --local-limiter-max-bytes-hard

    QDB_LOCAL_LIMITER_MAX_BYTES_HARD

    Same like local.limiter.max_bytes_soft, except that processing is temporarily halted when this limit is reached.

    local.limiter.allow_cache_after_insert

    --local-limiter-allow-cache-after-insert

    QDB_LOCAL_LIMITER_ALLOW_CACHE_AFTER_INSERT

    Determines whether newly written data is automatically cached in memory. When set to true, this feature enhances read performance by preventing immediate eviction. Setting it to false is useful for prioritizing cache optimization solely for queries. This approach is common during scenarios like daily batch data loads, where maintaining the speed of existing queries is paramount. For cases where newly written data is likely to be queried shortly after insertion, such as live monitoring systems, it’s recommended to keep this setting as true.

    local.limiter.max_bytes_hard_percentage

    --local-limiter-max-bytes-hard-percentage

    QDB_LOCAL_LIMITER_MAX_BYTES_HARD_PERCENTAGE

    An integer (0-100) representing the percentage of the hard limit at which drastic measures are taken. For example, at 90% of the hard limit, drastic measures are triggered when memory usage exceeds this percentage.

    local.limiter.max_bytes_soft_percentage

    --local-limiter-max-bytes-hard-percentage

    QDB_LOCAL_LIMITER_MAX_BYTES_HARD_PERCENTAGE

    An integer (0-100) representing the percentage of the soft limit. When memory usage approaches this percentage, entries are evicted to prevent memory exhaustion.

    local.limiter.max_trim_queue_length

    --local-limiter-max-trim-queue-length

    QDB_LOCAL_LIMITER_MAX_TRIM_QUEUE_LENGTH

    An integer representing the total maximum number of updated entries that may be queued for asynchronous trimming. Trimming is a background process that optimizes disk space.

    local.limiter.memory_statistics_refresh

    --local-limiter-memory-statistics-refresh

    QDB_LOCAL_LIMITER_MEMORY_STATISTICS_REFRESH

    An integer representing the interval (in ms) at which memory statistics are refreshed. Provides insight into system memory usage.

    local.limiter.soft_eviction_intensity

    --local-limiter-soft-eviction-intensity

    QDB_LOCAL_LIMITER_SOFT_EVICTION_INTENSITY

    An integer representing the intensity of soft eviction. It determines the number of entries that are evicted for each newly cached item when the memory usage approaches the soft limit.

    local.limiter.tbb_huge_threshold

    --local-limiter-tbb-huge-threshold

    QDB_LOCAL_LIMITER_HUGE_THRESHOLD

    An integer representing the threshold (in bytes) above which memory allocations use huge pages, improving memory performance for large allocations.

    local.limiter.tbb_soft_limit

    --local-limiter-tbb-soft-limit

    QDB_LOCAL_LIMITER_SOFT_LIMIT

    An integer representing the soft memory limit for the Thread Building Blocks (TBB) allocator, responsible for memory management in the cluster.

    local.limiter.tbb_use_huge_pages

    --local-limiter-tbb-use-huge-pages

    QDB_LOCAL_LIMITER_TBB_USE_HUGE_PAGES

    When set to true, the TBB allocator uses huge pages for memory allocations to improve memory performance for large allocations.

    2.1.8. Observability

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.logger.log_level

    --local-logger-log-level

    QDB_LOCAL_LOGGER_LOG_LEVEL

    An integer representing verbosity. Use 0 for most verbose logs, 5 for only fatal logs. Defaults to 2. Level 1 (debug) will also enable separate storage engine logs.

    local.logger.log_directory

    --local-logger-log-directory

    QDB_LOCAL_LOGGER_LOG_DIRECTORY

    Absolute path where log files will be written into. Defaults to /var/log/qdb.

    local.network.profile_performance

    --local-network-profile-performance

    QDB_LOCAL_NETWORK_PROFILE_PERFORMANCE

    Enables performance profiling from the server side, so that clients can start running performance traces. It has a small performance impact, and is disabled by default.

    global.cluster.enable_statistics

    --global-cluster-enable-statistics

    QDB_GLOBAL_CLUSTER_ENABLE_STATISTICS

    Enables runtime statistics collected server side. It has a small performance impact, and is disabled by default.

    global.cluster.statistics_refresh_interval

    --global-cluster-statistics-refresh-interval

    QDB_GLOBAL_CLUSTER_STATISTICS_REFRESH_INTERVAL

    Time (in milliseconds) between refreshes of the statistics. Should typically be set to the interval in which you query the node statistics; e.g. if you only poll your statistics once a minute, this value should be 60000.

    2.1.9. Asynchronous Timeseries Inserter

    The asynchronous timeseries inserter is a mechanism in QuasarDB to buffer inserts server-side, which you can tune using the configuration options below.

    Config File

    Command-Line Argument

    Environment Variable

    Description

    local.depot.async_ts.pipelines

    --local-depot-async-ts-pipelines

    QDB_LOCAL_DEPOT_ASYNC_TS_PIPELINES

    The number of asynchronous time series pipelines. Increase this number if you are experiencing slow inserts, but CPU is not yet at 100%.

    local.depot.async_ts.pipeline_buffer_size

    --local-depot-async-ts-pipeline-buffer-size

    QDB_LOCAL_DEPOT_ASYNC_TS_PIPELINE_BUFFER_SIZE

    Maximum size of the buffer in bytes for each of the pipelines. Increase this value to if you are experiencing slow inserts and CPU is at 100%.

    local.depot.async_ts.pipeline_queue_length

    --local-depot-async-ts-pipeline-queue-length

    QDB_LOCAL_DEPOT_ASYNC_TS_PIPELINE_QUEUE_LENGTH

    Maximum number of entries to buffer for each of the pipelines. Increase this value to if you are experiencing slow inserts and CPU is at 100%.

    local.depot.async_ts.flush_deadline

    --local-depot-async-ts-flush-deadline

    QDB_LOCAL_DEPOT_ASYNC_TS_FLUSH_DEADLINE

    Maximum time duration between consequent pipeline flushes.

    local.depot.async_ts.flush_old_deadline

    --local-depot-async-ts-flush-old-deadline

    QDB_LOCAL_DEPOT_ASYNC_TS_FLUSH_OLD_DEADLINE

    The maximum time duration, in milliseconds, allowed for old data to remain in the asynchronous time series flush pipeline before it is forced to be flushed. Data that remains in the pipeline beyond this deadline will be flushed to storage. The default value is 3,600,000 ms (1 hour).

    local.depot.async_ts.old_cutoff

    --local-depot-async-ts-old-cutoff

    QDB_LOCAL_DEPOT_ASYNC_TS_OLD_CUTOFF

    A threshold that determines which data is considered “old” and should be processed by the asynchronous time series inserter. Data older than this threshold, expressed in milliseconds, will be processed. The default value is 0 ms.

    2.2. QuasarDB Rest Server

    The QuasarDB Rest Server qdb_rest can be configured using either command line arguments, a json config file or environmental variables. See below for examples of each configuration as well as the various configuration options:

    # Environment variable based configuration
    QDB_CLUSTER_URI=qdb://192.168.0.1:2837 qdb_rest
    
    # Command line based configuration
    qdb_rest --cluster=qdb://192.168.0.1:2837
    
    # File based configuration
    qdb_rest --gen-config > rest.conf # Generate a new config file and edit
    qdb_rest --config-file rest.conf
    

    Command Line Variable

    Environment Variable

    Description

    Default

    --allowed-origins

    Allowed origins for cross origin resource sharing

    []

    --cluster

    QDB_CLUSTER_URI

    URI of the cluster to connect to

    qdb://127.0.0.1:2836

    --cluster-public-key-file

    QDB_CLUSTER_PUBLIC_KEY_FILE

    Key file used for cluster security

    --tls-certificate

    TLS_CERTIFICATE

    The certificate to use for secure connections

    --tls-key

    TLS_PRIVATE_KEY

    The private key to use for secure conections

    --tls-port

    TLS_PORT

    The port to listen on for secure connections

    40493

    --host

    HOST

    The IP to listen on

    localhost

    --port

    PORT

    The port to listen on for insecure connections

    40080

    --log-file

    QDB_REST_LOG_FILE

    The path to the log file

    /var/log/qdb/qdb_rest.log

    --assets-dir

    QDB_REST_ASSETS_DIR

    The path to the directory served by the rest api

    /var/lib/qdb/assets

    2.3. Docker

    As it can a bit tedious to edit a configuration file inside a Docker container, the bureau14/qdb docker container provides several environment variables you can use to configure the most common configuration options.

    Below you find an overview of the different environment variables the image supports:

    Variable

    Example usage

    QDB_LOCAL_USER_LICENSE_KEY

    docker run -d \
           -e QDB_LOCAL_USER_LICENSE_KEY="$your_license_key" \
           bureau14/qdb
    

    QDB_LOCAL_USER_LICENSE_FILE

    docker run -d \
           -v /path/to/my/qdb.key:/qdb.key \
           -e QDB_LOCAL_USER_LICENSE_FILE=/qdb.key \
           bureau14/qdb
    

    QDB_LOCAL_NETWORK_LISTEN_ON

    docker run -d \
           -e QDB_LOCAL_NETWORK_LISTEN_ON=102836 \
           bureau14/qdb
    

    QDB_LOCAL_NETWORK_ADVERTISE_AS

    docker run -d \
           -e QDB_LOCAL_NETWORK_ADVERTISE_AS="172.16.64.8" \
           bureau14/qdb
    

    QDB_LOCAL_LOGGER_LOG_LEVEL

    docker run -d \
           -e QDB_LOCAL_LOGGER_LOG_LEVEL=1 \
           bureau14/qdb
    

    QDB_LOCAL_LOGGER_LOG_DIRECTORY

    docker run -d \
           -v /my/qdb/log/output:/logs \
           -e QDB_LOCAL_LOGGER_LOG_DIRECTORY="/logs" \
           bureau14/qdb
    

    QDB_GLOBAL_SECURITY_CLUSTER_PRIVATE_FILE

    docker run -d \
           -e QDB_GLOBAL_SECURITY_ENABLED=true \
           \
           -v /path/to/my/qdb/private.key:/private.key \
           -e QDB_GLOBAL_SECURITY_CLUSTER_PRIVATE_FILE=/private.key \
           \
           -e QDB_GLOBAL_SECURITY_USER_LIST=... \
           bureau14/qdb
    

    QDB_GLOBAL_CLUSTER_REPLICATION_FACTOR

    docker run -d \
           -e QDB_GLOBAL_CLUSTER_REPLICATION_FACTOR=3 \
           \
           bureau14/qdb
    

    QDB_LOCAL_NETWORK_TOTAL_SERVER_SESSIONS

    docker run -d \
           -e QDB_LOCAL_NETWORK_TOTAL_SERVER_SESSIONS=8192 \
           \
           bureau14/qdb
    

    QDB_LOCAL_NETWORK_PARALLELISM

    docker run -d \
           --cpu-number=32 \
           -e QDB_LOCAL_NETWORK_PARALLELISM=24 \
           -e QDB_LOCAL_DEPOT_ROCKSDB_THREADS=8 \
           -e QDB_LOCAL_DEPOT_ROCKSDB_HI_THREADS=1 \
           \
           bureau14/qdb
    

    QDB_GLOBAL_CLUSTER_PUBLISH_FIREHOSE

    docker run -d
           -e QDB_GLOBAL_CLUSTER_PUBLISH_FIREHOSE="true" \
            \
            bureau14/qdb
    

    QDB_LOCAL_NETWORK_FIREHOSE_PUBLISHING_THREADS

    docker run -d
           -e QDB_GLOBAL_CLUSTER_PUBLISH_FIREHOSE="true" \
           -e QDB_LOCAL_NETWORK_FIREHOSE_PUBLISHING_THREADS=4 \
           \
           bureau14/qdb
    

    QDB_LOCAL_LIMITER_MAX_BYTES_SOFT

    docker run -d \
               -e QDB_LOCAL_LIMITER_MAX_BYTES_SOFT=17179869184 \
               -e QDB_LOCAL_LIMITER_MAX_BYTES_HARD=25769803776 \
               bureau14/qdb
    

    QDB_LOCAL_LIMITER_MAX_BYTES_HARD

    docker run -d \
           -e QDB_LOCAL_LIMITER_MAX_BYTES_SOFT=17179869184 \
           -e QDB_LOCAL_LIMITER_MAX_BYTES_HARD=25769803776 \
           bureau14/qdb
    

    QDB_LOCAL_DEPOT_ROCKSDB_THREADS

    docker run -d \
           -e QDB_LOCAL_DEPOT_ROCKSDB_THREADS=8 \
           bureau14/qdb
    

    QDB_LOCAL_DEPOT_ROCKSDB_HI_THREADS

    docker run -d \
           -e QDB_LOCAL_DEPOT_ROCKSDB_THREADS=16 \
           -e QDB_LOCAL_DEPOT_ROCKSDB_HI_THREADS=2 \
           bureau14/qdb
    

    QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_PROVIDER

    docker run -d \
           -e QDB_LOCAL_DEPOT_ROCKSDB_CLOUD_PROVIDER="aws" \
           bureau14/qdb