5.6. Operational Logging#
QuasarDB’s daemon emits rich operational logs to `qdbd.log` (plain text) or `qdbd.json` (structured JSON). These logs are your primary window into runtime behavior—far beyond simple metrics—and are critical for:
Continuous monitoring: ingest logs with agents (e.g. Filebeat, Promtail) into your observability stack (Loki, Elasticsearch, Splunk, etc.).
Automated alerting: define log-based alerts on error patterns, warning frequency, or specific message substrings.
Post-incident forensics: correlate timestamped log events with metric spikes in Grafana or your APM to isolate root causes.
Configuration validation: confirm at startup that your cluster settings (sessions, cache thresholds, pipeline counts) match intended values.
Security auditing: detect missing authentication or unexpected access errors immediately.
5.6.1. Why Logging Matters#
Surface non-metric events Not every issue shows up as a gauge or counter. Failed external calls, authentication errors, schema validation failures, and internal health warnings live only in log lines.
Contextual drill-down Metrics tell when something happened. Logs tell why—including stack traces, error codes, and request IDs.
Flexible alerting You can trigger alerts on any text pattern: e.g. - any line containing “error” - more than 5 “security is disabled” warnings per hour - repeated “eviction threshold” messages in a short interval
Structured querying With JSON logs, search and filter by fields (level, component, shard, etc.) to rapidly pinpoint anomalies.
5.6.2. Ingesting and Alerting#
Deploy a log shipper to tail the QuasarDB log directory (default on Unix: /var/log/qdb/).
Parse JSON when possible to leverage structured fields in your log back end. Otherwise, use regex filters on the raw text.
Define alerts in your monitoring system: - Severity thresholds: fire on level=error or on repeated
level=warning within a sliding window.
Pattern matches: alert on specific keywords (e.g. “security is disabled”).
Tag and correlate: include cluster name, node ID, or environment tags when sending logs upstream to maintain context across distributed deployments.
5.6.3. Best Practices for Log Management#
Rotate and retain logs according to your organization’s policy. QuasarDB rotates based on their max size or duration, and keeps a certain number of log files.
Archive old logs for compliance or post-mortem audits.
Exclude verbose “detailed” lines in production ingestion to reduce noise.
Monitor log ingestion health itself—ensure your shipper isn’t falling behind under heavy write rates.
5.6.4. Key Example Messages#
Below are a few representative log lines (from qdbd.log) and why they matter.
starting 8 async ts pipeline(s) – max buffer 2.0 GiB – flush deadline 3 s
Confirms your async-pipeline configuration at startup. If these values differ from expectations, revisit your ingestion strategy in the ../../user-guide/concepts/async_guidelines.
your eviction threshold is 4 GiB – lower than baseline 74.625 GiB
Warns that cache eviction will begin far too early, causing unnecessary churn and latency degradation. Adjust the memory soft limit accordingly.
security is disabled – this cluster is not secure
A production “stop everything” alert: enable authentication and encryption before allowing untrusted clients to connect.
maximum concurrent sessions: 4096
Records the hard cap on client connections. Compare with
network.sessions.available_count
and
network.sessions.unavailable_count
metrics to detect session exhaustion.
used physical memory 45.7 GiB / 48 GiB (95.2 %)
A startup health snapshot. Use this to explain thrashing or out-of-memory events when correlated with eviction logs or OOM metrics.
5.6.5. Further Reading#
For guided troubleshooting workflows—using logs in tandem with metrics and traces—see the <no title> guide.