5. Change Stream#

By default, the QuasarDB daemon stores all update that happened to the cluster into a timeseries named the firehose.

5.1. Purpose#

The firehose is used to enable cluster-to-cluster replication, audit trail, incremental backups, alerts and other applications that require streaming updates.

Note

The firehose buffers updates for performance reasons. The delay is typically only a few seconds, depending upon the configuration of the asynchronous inserter.

5.2. Recorded information#

The firehose contains the following information:

  • timestamp: timestamp, when the modification occurred (not the time when it was stored into the firehose)

  • transaction_id: blob, a cluster-wide unique string representing the transaction ID the update belongs to

  • user_id: int64, the user_id that did the modification

  • message_id: int64, the type of update (see below)

  • table: - blob, the table or entry on which the modification occurred

  • column: - blob, the column of the table on which the modification occurred, when relevant

  • begin: - timestamp, the beginning (inclusive) of the time range where the modification occurred, when relevant

  • end: - timestamp, the end (inclusive) of the time range where the modification occurred, when relevant

The range specified by the update does not mean that all values within the range have been modified, but that the modification occurred within that range.

5.3. Update types#

The following update types are recorded:

  • creation: 0, an entry has been created, for example, a table has been created

  • modification: 1, an entry has been modified, for example, a column has been added to a table

  • insertion: 2, data has been inserted into the entry, for example, values have been added to the column of a table

  • update: 3, data has been modified, for example, values of a column have been modified

  • erasure: 4, data has been erased, for example, values of a column have been removed

  • removal: 5, an entry has been removed, for example, a table has been dropped

Currently, the QuasarDB daemon will only log into the firehose modifications pertaining to tables. Creation, modification, deletion of blobs, for example, are not logged.

Note

By default, the firehose is readable and writeable to everyone. It is strongly advised to restrict access to administrators. The cluster uses the default, internal, superuser to write to the firehose and thus bypasses any privileges set.