ArrowAsyncTask Struct Reference

#include <arrow_abi.h>

Data Fields
int(*	extract_data )(struct ArrowAsyncTask *self, struct ArrowDeviceArray *out)
void *	private_data
	opaque task-specific data

Detailed Description

EXPERIMENTAL: ArrowAsyncTask represents available data from a producer that was passed to an invocation of on_next_task on the ArrowAsyncDeviceStreamHandler.

The reason for this Task approach instead of the Async interface returning the Array directly is to allow for more complex thread handling and reducing context switching and data transfers between CPU cores (e.g. from one L1/L2 cache to another) if desired.

For example, the on_next_task callback can be called when data is ready, while the producer puts potential "decoding" logic in the ArrowAsyncTask object. This allows for the producer to manage the I/O on one thread which calls on_next_task and the consumer can determine when the decoding (producer logic in the extract_data callback of the task) occurs and on which thread, to avoid a CPU core transfer (data staying in the L2 cache).

Field Documentation

int(* ArrowAsyncTask::extract_data)(struct ArrowAsyncTask *self, struct ArrowDeviceArray *out)

This callback should populate the ArrowDeviceArray associated with this task. The order of ArrowAsyncTasks provided by the producer enables a consumer to ensure the order of data to process.

This function is expected to be synchronous, but should not perform any blocking I/O. Ideally it should be as cheap as possible so as to not tie up the consumer thread unnecessarily.

Returns: 0 if successful, errno-compatible error otherwise.

If a non-0 value is returned then it should be followed by a call to on_error on the appropriate ArrowAsyncDeviceStreamHandler. This is because it's highly likely that whatever is calling this function may be entirely disconnected from the current control flow. Indicating an error here with a non-zero return allows the current flow to be aware of the error occurring, while still allowing any logging or error handling to still be centralized in the on_error callback of the original Async handler.

Rather than a release callback, any required cleanup should be performed as part of the invocation of extract_data. Ownership of the Array is passed to the consumer calling this, and so it must be released separately.

It is only valid to call this method exactly once.

---

The documentation for this struct was generated from the following file:

• qdb/arrow_abi.h