Package com.amazonaws.athena.connector.lambda.data

Interface BlockAllocator

  • All Superinterfaces:
    AutoCloseable
    All Known Implementing Classes:
    BlockAllocatorImpl
    public interface BlockAllocator
extends AutoCloseable
    Defines the interface that should be implemented by all reference counting Apache Arrow resource allocators that are provided by this SDK. You should use a BlockAllocator over an Apache Arrow BufferAllocator if the lifecycle of your Apache Arrow resources are not fully contained in narrow code path. In practice we've found that ensuring proper lifecycle for Apache Arrow resources led us to change the structure of our code in ways that made it less maintainable than if we had a mechanism to control the lifecyle of Apache Arrow resources that cross-cut our request lifecycle.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Interface Description
      static interface  BlockAllocator.BatchGenerator
      Used to generate a batch in a leak free way using the BlockAllocator to handle the boiler plate aspects of error detection and rollback.

    • Method Summary

      All Methods Instance Methods Abstract Methods 
      Modifier and Type Method Description
      void close()
      Closes all Apache Arrow resources tracked by this BlockAllocator, freeing their memory.
      Block createBlock​(org.apache.arrow.vector.types.pojo.Schema schema)
      Creates an empty Apache Arrow Block with the provided Schema.
      org.apache.arrow.memory.ArrowBuf createBuffer​(int size)
      Creates an empty Apache Arrow Buffer of the requested size.
      long getUsage()
      Provides access to the current memory pool usage on the underlying Apache Arrow BufferAllocator.
      boolean isClosed()
      Provides access to the current state of this BlockAllocator.
      org.apache.arrow.vector.ipc.message.ArrowRecordBatch registerBatch​(BlockAllocator.BatchGenerator generator)
      Allows for a leak-free way to create Apache Arrow Batches.

    • Method Detail

      • createBlock

        Block createBlock​(org.apache.arrow.vector.types.pojo.Schema schema)
        Creates an empty Apache Arrow Block with the provided Schema.
        Parameters:
        schema - The schema of the Apache Arrow Block.
        Returns:
        THe resulting Block.

      • createBuffer

        org.apache.arrow.memory.ArrowBuf createBuffer​(int size)
        Creates an empty Apache Arrow Buffer of the requested size. This is useful when working with certain Apache Arrow types directly.
        Parameters:
        size - The number of bytes to reserve for the requested buffer.
        Returns:
        THe resulting Apache Arrow Buffer..

      • registerBatch

        org.apache.arrow.vector.ipc.message.ArrowRecordBatch registerBatch​(BlockAllocator.BatchGenerator generator)
        Allows for a leak-free way to create Apache Arrow Batches. At first glance this method's signature may seem ackward when compared to createBuffer(...) or createBlock(...) but ArrowRecordBatches are typically as part of serialization and as such are prone to leakage when you serialize or deserialize and invalid Block. With this approach the BlockAllocator is able to capture any exceptions from your BatchGenerator and perform nessesary clean up without your code having to implement the boiler plate for handling those edge cases.
        Parameters:
        generator - The generator which is expected to create an ArrowRecordBatch.
        Returns:
        THe resulting Apache Arrow Batch..

      • getUsage

        long getUsage()
        Provides access to the current memory pool usage on the underlying Apache Arrow BufferAllocator.
        Returns:
        The number of bytes that have been used (e.g. assigned to an Apache Arrow Resource like a block, batch, or buffer).

      • close

        void close()
        Closes all Apache Arrow resources tracked by this BlockAllocator, freeing their memory.
        Specified by:
        close in interface AutoCloseable

      • isClosed

        boolean isClosed()
        Provides access to the current state of this BlockAllocator.
        Returns:
        True if close has been called, False otherwise.