sparrow_ipc::utils Namespace Reference

Functions
std::pair< std::uint8_t *, int64_t >	get_bitmap_pointer_and_null_count (std::span< const uint8_t > validity_buffer_span, const int64_t length)
	Extracts bitmap pointer and null count from a validity buffer span.
std::span< const uint8_t >	get_buffer (const org::apache::arrow::flatbuf::RecordBatch &record_batch, std::span< const uint8_t > body, size_t &buffer_index)
	Extracts a buffer from a RecordBatch's body.
std::variant< sparrow::buffer< std::uint8_t >, std::span< const std::uint8_t > >	get_decompressed_buffer (std::span< const uint8_t > buffer_span, const org::apache::arrow::flatbuf::BodyCompression *compression)
	Retrieves a decompressed buffer or a view of the original buffer.
constexpr size_t	align_to_8 (const size_t n)
SPARROW_IPC_API std::optional< std::string_view >	get_substr_after_separator (std::string_view str, std::string_view sep)
SPARROW_IPC_API std::optional< int32_t >	parse_to_int32 (std::string_view str)
SPARROW_IPC_API std::optional< int32_t >	get_substr_as_int32 (std::string_view str, std::string_view sep)
SPARROW_IPC_API std::optional< std::tuple< int32_t, int32_t, std::optional< int32_t > > >	parse_decimal_format (std::string_view format_str)
SPARROW_IPC_API std::vector< std::string_view >	extract_words_after_colon (std::string_view str)
template<std::ranges::input_range R> requires std::same_as<std::ranges::range_value_t<R>, sparrow::record_batch>
bool	check_record_batches_consistency (const R &record_batches)
	Checks if all record batches in a collection have consistent structure.

Function Documentation

align_to_8()

size_t sparrow_ipc::utils::align_to_8 ( const size_t n )

constexpr

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.

Definition at line 14 of file utils.hpp.

Here is the caller graph for this function:

check_record_batches_consistency()

template<std::ranges::input_range R>
requires std::same_as<std::ranges::range_value_t<R>, sparrow::record_batch>

bool sparrow_ipc::utils::check_record_batches_consistency

(

const R &

record_batches

)

Checks if all record batches in a collection have consistent structure.

This function verifies that all record batches in the provided collection have:

• The same number of columns
• Matching data types for corresponding columns (same column index)

Template Parameters

R	Container type that holds sparrow::record_batch objects

Parameters

record_batches

Collection of record batches to check for consistency

Returns

true if all record batches have consistent structure or if the collection is empty, false if any structural inconsistencies are found

Note

An empty collection is considered consistent and returns true

The number of rows per record batch is not required to be the same

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.

Definition at line 116 of file utils.hpp.

Here is the caller graph for this function:

extract_words_after_colon()

SPARROW_IPC_API std::vector< std::string_view > sparrow_ipc::utils::extract_words_after_colon

(

std::string_view

str

)

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.

get_bitmap_pointer_and_null_count()

std::pair< std::uint8_t *, int64_t > sparrow_ipc::utils::get_bitmap_pointer_and_null_count ( std::span< const uint8_t > validity_buffer_span, const int64_t length )

nodiscard

Extracts bitmap pointer and null count from a validity buffer span.

This function calculates the number of null values represented by the bitmap.

Parameters

validity_buffer_span	The validity buffer as a byte span.
length	The Arrow RecordBatch length (number of values in the array).

Returns

A pair containing:

• First: Pointer to the bitmap data (nullptr if buffer is empty)
• Second: Count of null values in the bitmap (0 if buffer is empty)

Note

If the bitmap buffer is empty, returns {nullptr, 0}

The returned pointer is a non-const cast of the original const data

Here is the caller graph for this function:

get_buffer()

std::span< const uint8_t > sparrow_ipc::utils::get_buffer ( const org::apache::arrow::flatbuf::RecordBatch & record_batch, std::span< const uint8_t > body, size_t & buffer_index )

nodiscard

Extracts a buffer from a RecordBatch's body.

This function retrieves a buffer span from the specified index in the RecordBatch's buffer list and increments the index.

Parameters

record_batch	The Arrow RecordBatch containing buffer metadata.
body	The raw buffer data as a byte span.
buffer_index	The index of the buffer to retrieve. This value is incremented by the function.

Returns

A std::span<const uint8_t> viewing the extracted buffer data.

Exceptions

std::runtime_error

if the buffer metadata indicates a buffer that exceeds the body size.

Here is the caller graph for this function:

get_decompressed_buffer()

std::variant< sparrow::buffer< std::uint8_t >, std::span< const std::uint8_t > > sparrow_ipc::utils::get_decompressed_buffer ( std::span< const uint8_t > buffer_span, const org::apache::arrow::flatbuf::BodyCompression * compression )

nodiscard

Retrieves a decompressed buffer or a view of the original buffer.

This function either decompresses the provided buffer span, if compression is specified, or returns a view of the original buffer without modification.

Parameters

buffer_span	A span of raw buffer data to be decompressed, or returned as-is if no decompression is needed.
compression	The compression algorithm to use. If nullptr, no decompression is performed.

Returns

A std::variant containing either:

• A sparrow::buffer<std::uint8_t> with the decompressed data, or
• A std::span<const std::uint8_t> providing a view of the original buffer_span if no decompression occurred.

Here is the caller graph for this function:

get_substr_after_separator()

SPARROW_IPC_API std::optional< std::string_view > sparrow_ipc::utils::get_substr_after_separator	(	std::string_view	str,
		std::string_view	sep )

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.

get_substr_as_int32()

SPARROW_IPC_API std::optional< int32_t > sparrow_ipc::utils::get_substr_as_int32	(	std::string_view	str,
		std::string_view	sep )

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.

parse_decimal_format()

SPARROW_IPC_API std::optional< std::tuple< int32_t, int32_t, std::optional< int32_t > > > sparrow_ipc::utils::parse_decimal_format

(

std::string_view

format_str

)

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.

parse_to_int32()

SPARROW_IPC_API std::optional< int32_t > sparrow_ipc::utils::parse_to_int32

(

std::string_view

str

)

Examples

/home/runner/work/sparrow-ipc/sparrow-ipc/include/sparrow_ipc/utils.hpp.