6. Operational Logging#

QuasarDB’s daemon emits rich operational logs to `qdbd.log` (plain text) or `qdbd.json` (structured JSON). These logs are your primary window into runtime behavior—far beyond simple metrics—and are critical for:

  • Continuous monitoring: ingest logs with agents (e.g. Filebeat, Promtail) into your observability stack (Loki, Elasticsearch, Splunk, etc.).

  • Automated alerting: define log-based alerts on error patterns, warning frequency, or specific message substrings.

  • Post-incident forensics: correlate timestamped log events with metric spikes in Grafana or your APM to isolate root causes.

  • Configuration validation: confirm at startup that your cluster settings (sessions, cache thresholds, pipeline counts) match intended values.

  • Security auditing: detect missing authentication or unexpected access errors immediately.

6.1. Why Logging Matters#

  1. Surface non-metric events Not every issue shows up as a gauge or counter. Failed external calls, authentication errors, schema validation failures, and internal health warnings live only in log lines.

  2. Contextual drill-down Metrics tell when something happened. Logs tell why—including stack traces, error codes, and request IDs.

  3. Flexible alerting You can trigger alerts on any text pattern: e.g. - any line containing “error” - more than 5 “security is disabled” warnings per hour - repeated “eviction threshold” messages in a short interval

  4. Structured querying With JSON logs, search and filter by fields (level, component, shard, etc.) to rapidly pinpoint anomalies.

6.2. Ingesting and Alerting#

  • Deploy a log shipper to tail the QuasarDB log directory (default on Unix: /var/log/qdb/).

  • Parse JSON when possible to leverage structured fields in your log back end. Otherwise, use regex filters on the raw text.

  • Define alerts in your monitoring system: - Severity thresholds: fire on level=error or on repeated

    level=warning within a sliding window.

    • Pattern matches: alert on specific keywords (e.g. “security is disabled”).

  • Tag and correlate: include cluster name, node ID, or environment tags when sending logs upstream to maintain context across distributed deployments.

6.3. Best Practices for Log Management#

  • Rotate and retain logs according to your organization’s policy. QuasarDB rotates based on their max size or duration, and keeps a certain number of log files.

  • Archive old logs for compliance or post-mortem audits.

  • Exclude verbose “detailed” lines in production ingestion to reduce noise.

  • Monitor log ingestion health itself—ensure your shipper isn’t falling behind under heavy write rates.

6.4. Key Example Messages#

Below are a few representative log lines (from qdbd.log) and why they matter.

starting 8 async ts pipeline(s) – max buffer 2.0 GiB – flush deadline 3 s

Confirms your async-pipeline configuration at startup. If these values differ from expectations, revisit your ingestion strategy in the ../../user-guide/concepts/async_guidelines.

your eviction threshold is 4 GiB – lower than baseline 74.625 GiB

Warns that cache eviction will begin far too early, causing unnecessary churn and latency degradation. Adjust the memory soft limit accordingly.

security is disabled – this cluster is not secure

A production “stop everything” alert: enable authentication and encryption before allowing untrusted clients to connect.

maximum concurrent sessions: 4096

Records the hard cap on client connections. Compare with network.sessions.available_count and network.sessions.unavailable_count metrics to detect session exhaustion.

used physical memory 45.7 GiB / 48 GiB (95.2 %)

A startup health snapshot. Use this to explain thrashing or out-of-memory events when correlated with eviction logs or OOM metrics.

6.5. Further Reading#

For guided troubleshooting workflows—using logs in tandem with metrics and traces—see the Troubleshooting Guide guide.