Package software.amazon.awssdk.crt.s3

Class S3BorrowedBuffer

  • All Implemented Interfaces:
    AutoCloseable
    public final class S3BorrowedBuffer
extends Object
implements AutoCloseable
    One chunk of an S3 response body, read directly from the client's direct buffer pool memory (see S3DirectBufferPoolOptions) with no copy. Delivered to handlers that implement S3BorrowedBufferDelivery. The chunk starts at the callback's objectRangeStart; its length is asByteBuffer().remaining().

    Exception: for a GetObject that sets a part number, the native client reads the body into its own temporary memory rather than the pool. Its data is then copied once into a separate buffer that this object owns; that buffer works the same way but is not pool memory, so it does not count toward the pool's ceiling or the JVM direct memory limit.

    Lifetime. The buffer holds its memory until close(), so you can keep it after the callback returns, pass it to another thread, or hold it past client shutdown. Every buffer MUST be closed, on every path including exceptions: until it is closed its memory can't be reused, and once the pool runs out, every request on the client waits.

    This is not flow control. Holding a buffer open does not slow the download. To pause one, enable read backpressure, return 0 (or a small increment) from onResponseBodyBorrowed, and call S3MetaRequest.incrementReadWindow(long) when ready for more. The pause applies at part boundaries: the client stops requesting new parts, and parts already requested arrive whole rather than trimmed to the window, so delivery can exceed it by up to one part.

    Keep a reference while you read. A buffer that is no longer referenced can be recovered by garbage collection, which frees the memory under the ByteBuffer that asByteBuffer() returned, so hold the S3BorrowedBuffer itself for as long as you use that view. See asByteBuffer() for what happens to the view after close.

    close() can be called any number of times. toByteArray() copies the data to a heap byte[] and closes the buffer; it works once and later calls throw. Both are safe to call from any thread. asByteBuffer() returns one shared view; use ByteBuffer.duplicate() if it may be read from more than one place or thread.

    A buffer that is never closed is recovered only when it is garbage-collected, after an unbounded delay, and is reported as a leak. The aws.crt.s3.leakdetection system property controls the report:

    • simple (default): logs warnings, and captures the allocation stack trace for 1 in 128 buffers.
    • paranoid: captures a stack trace for every buffer. Useful for tracking down a leak, but costs more.
    • disabled: no warnings.

    Examples

    Reading the data during the callback. The buffer is closed before the callback returns, so nothing from it (the buffer or its ByteBuffer view) may be kept and used afterwards.

    
 public int onResponseBodyBorrowed(S3BorrowedBuffer buffer, long objectRangeStart, long objectRangeEnd) {
     try (S3BorrowedBuffer b = buffer) {  // closes the buffer when the block exits, even on an exception
         ByteBuffer data = b.asByteBuffer();
         // Read the chunk from data here, for example by writing it to a
         // channel or updating a checksum.
     }
     // Grows the read window by this chunk's length, so the download keeps
     // going when read backpressure is enabled (ignored otherwise).
     return (int) (objectRangeEnd - objectRangeStart);
 }

    Keeping the buffer after the callback. Hand it to whatever reads it later, and have that code close it. If the hand-off fails, the callback still owns the buffer and must close it.

    
 public int onResponseBodyBorrowed(S3BorrowedBuffer buffer, long objectRangeStart, long objectRangeEnd) {
     try {
         // The task now owns the buffer and closes it when done.
         executor.execute(() -> {
             try (S3BorrowedBuffer b = buffer) {  // closes the buffer when the block exits, even on an exception
                 ByteBuffer data = b.asByteBuffer();
                 // Read the chunk from data here.
             }
         });
     } catch (RuntimeException e) {
         buffer.close();  // not handed off, so close it here
         throw e;
     }
     // Grows the read window now, before the task has read the data. With
     // read backpressure, return 0 instead and call
     // S3MetaRequest.incrementReadWindow from the task once it is done.
     return (int) (objectRangeEnd - objectRangeStart);
 }

    • Method Detail

      • asByteBuffer

        public ByteBuffer asByteBuffer()
        Returns a ByteBuffer view over the underlying pool memory. The returned buffer is a shared reference. Do not mutate its position or limit if other threads may be reading concurrently. Use ByteBuffer.duplicate() or ByteBuffer.slice() for a private view.

        The returned buffer is a DIRECT buffer over pool memory, not a heap buffer: it has no backing array (ByteBuffer.hasArray() is false, and ByteBuffer.array() throws UnsupportedOperationException). Read it through the ByteBuffer API (get, bulk get(byte[]), channel writes). If you need a byte[], or bytes that outlive this buffer, use toByteArray() instead.

        The memory under this view, and under any duplicate or slice of it, is valid only while this S3BorrowedBuffer is open and still referenced. Once it is closed (or recovered by garbage collection), the memory may be reused for other data: reading the view then returns unrelated bytes rather than failing, and writing to it corrupts another download or upload.

        Returns:
        a direct ByteBuffer sliced to the response body length
        Throws:
        IllegalStateException - if this borrowed buffer has been closed

      • toByteArray

        public byte[] toByteArray()
        Copies the buffer contents into a new heap byte[] (safe to hold indefinitely) and closes this borrowed buffer, returning its pool memory so the client can reuse it for further parts of this or other requests. Subsequent asByteBuffer() or toByteArray() calls throw.
        Returns:
        a heap byte[] copy of the buffer contents
        Throws:
        IllegalStateException - if this borrowed buffer has been closed

      • close

        public void close()
        Releases the pool memory so the client can reuse it for further parts of this or other requests. Idempotent, safe to call multiple times from any thread. The first call performs the release; subsequent calls are no-ops.

        After close() returns, the ByteBuffer previously returned from asByteBuffer() MUST NOT be read. The underlying pool memory may be reused by another request.

        Specified by:
        close in interface AutoCloseable