Package software.amazon.awssdk.crt.s3

Class S3DirectBufferPoolOptions

  • public final class S3DirectBufferPoolOptions
extends Object
    Settings for an optional direct buffer pool that holds an S3Client's transfer memory in Java direct memory. To turn the pool on, pass these options to S3ClientOptions.withDirectBufferPoolOptions(S3DirectBufferPoolOptions) when you create the client.

    The client splits each upload and download into parts and holds the parts in flight in memory. By default that memory is native memory the JVM cannot see or limit. With the pool, the memory comes from direct ByteBuffers instead: it counts against -XX:MaxDirectMemorySize, never grows past a limit you control, and (except with fixed(long)) is given back when the client is idle.

    Opt-in

    The pool is off unless you set these options. Without them the client behaves exactly as before. With them, existing response handlers keep working unchanged; reading response data without a copy is a separate, optional step (see Delivery).

    Example

    
 S3ClientOptions options = new S3ClientOptions()
     .withRegion("us-west-2")
     .withClientBootstrap(bootstrap)
     .withCredentialsProvider(credentialsProvider)
     .withDirectBufferPoolOptions(S3DirectBufferPoolOptions.auto());
 S3Client client = new S3Client(options);

    Choosing a factory

    • auto(): recommended. Sized automatically; grows with demand and shrinks when idle.
    • fixed(long): a fixed amount of memory, allocated up front. Use it when you need a hard, predictable memory budget.
    • elastic(long, long): grows and shrinks between a minimum and maximum you choose.

    Sizing

    Every pool has a floor and a ceiling:
    • Floor: memory allocated when the client is created and kept even while the client is idle.
    • Ceiling: the most memory the pool will ever use. When it is reached, new requests wait until memory is freed.
    Memory is counted in parts (S3ClientOptions.withPartSize(long)), so byte sizes you pass are rounded to whole parts. Between the floor and the ceiling, the pool grows as needed and gives memory back when idle.

    Choosing the ceiling. What the ceiling needs to cover depends on how many requests run at once, not on how large the objects are:

    • With backpressure enabled (S3ClientOptions.withReadBackpressureEnabled(boolean)), each request holds up to its read window plus one part:
      ceiling >= concurrent requests x (read window + part size)
      The extra part is because a part starts downloading as soon as the window covers any of its bytes, so a request can hold one part beyond its window. S3ClientOptions.withInitialReadWindowSize(long) is only a client-wide starting value: S3MetaRequest.incrementReadWindow(long) raises one request's window with no upper bound, so if you use it, size the ceiling for the sum of the windows you grant rather than the starting value.
    • For objects smaller than one part:
      ceiling >= concurrent requests x part size
      Memory is reserved a whole part at a time, so an object smaller than a part still holds a full part while it is in flight. Sizing from the bytes actually in flight under-sizes the pool: with an 8 MiB part size, 100 concurrent 256 KiB downloads need 800 MiB, not 25 MiB. Reduce S3ClientOptions.withPartSize(long) if that is too much for a small-object workload.
    Without backpressure there is no window, so the ceiling itself is what bounds how much downloads buffer. In either case a ceiling below these figures still completes every request; requests wait for memory instead, which shows up as lower throughput rather than an error.

    How S3ClientOptions.withMemoryLimitInBytes(long) works with the pool depends on the factory:

    • auto(): it sets the ceiling. Use it to cap how much memory the pool can use.
    • fixed(long) and elastic(long, long): the size you pass to the factory is the ceiling, so you don't need to set S3ClientOptions.withMemoryLimitInBytes(long). If it is set, it is an upper bound: the pool's ceiling (the factory size rounded down to whole parts) must not exceed it, or creating the client fails and the error message gives the values to use.

    JVM direct memory limit: the pool's memory counts against -XX:MaxDirectMemorySize, which by default equals the maximum heap size (-Xmx). The ceiling must fit within 80% of that limit. If you chose the ceiling (with fixed(long), elastic(long, long), withMemoryLimitInBytes or an environment variable) and it doesn't fit, creating the client throws; raise -XX:MaxDirectMemorySize or choose a smaller ceiling. A default ceiling picked by auto() is reduced to fit instead. An explicit -XX:MaxDirectMemorySize=0 allows no direct memory, so creating the client always throws.

    Delivery

    A handler that implements S3BorrowedBufferDelivery receives each chunk straight from pool memory, with no copy into a heap byte[]; any other handler keeps receiving S3MetaRequestResponseHandler.onResponseBody(ByteBuffer, long, long). Exactly one of the two is called per request. See S3BorrowedBuffer for the details, including the part-number GetObject exception.

    Uploads

    Multipart uploads hold each part in this pool until the part completes, including retries; single-part uploads don't use it. Uploads and downloads share the same ceiling, so a large upload can make downloads wait, and the other way round. Uploads get the same memory visibility and limit, but no copy is avoided: S3BorrowedBuffer is for downloads only. Limits on upload part size are listed under Large parts.

    Lifetime

    The client creates the pool and owns it; each client has its own. When the client shuts down, the pool frees its unused memory immediately. Borrowed buffers keep their memory until they are closed, even after shutdown; see S3BorrowedBuffer.

    Large parts

    Usually there is nothing to do: the client uses larger parts on its own when it needs them, for example for uploads too big for S3's 10,000-part limit at the configured part size. These limits apply:
    • If you set partSize explicitly, it is never raised: an upload that would need larger parts is rejected when the request is made, with the part size to use.
    • A resumed upload uses the part size recorded in its resume token rather than the client's, and is rejected the same way if the pool cannot hold it.
    • An upload part is bounded by the ceiling and by S3's own maximum; an upload needing larger parts is rejected when the request is made.
    • partSize has an upper bound, or creating the client fails.
    Both errors state the exact limit and the value to use, so the numbers live in one place rather than here as well. fixed(long) pools have a further limit on very large uploads; see that method. A request that can never fit in the pool fails, with the reason logged.

    Instances are immutable and may be reused across clients; each client builds its own pool from them.

    • Method Detail

      • auto

        public static S3DirectBufferPoolOptions auto()
        Recommended default. Sizes the pool automatically, so no tuning is needed.

        The ceiling is the most memory the pool will use. It is taken from the sources below, in priority order: the first one that is set is used, and the rest are ignored.

        1. S3ClientOptions.withMemoryLimitInBytes(long)
        2. the AWS_CRT_S3_MEMORY_LIMIT_IN_MB environment variable (in MiB)
        3. the AWS_CRT_S3_MEMORY_LIMIT_IN_GIB environment variable
        4. a default based on S3ClientOptions.withThroughputTargetGbps(double) (if unset, detected from the EC2 instance type, so the ceiling can differ between instances). If this default doesn't fit within 80% of the JVM direct memory limit, it is reduced to fit and a warning is logged, since a smaller pool can limit throughput; raise -XX:MaxDirectMemorySize to use the full default.

        The floor is 16 parts (128 MiB with an 8 MiB part size), or the whole pool if the ceiling is smaller. Above that, the pool grows on demand and shrinks when idle. Each time it grows it briefly delays other requests on the client; see elastic(long, long) for details.

        Returns:
        options for an automatically sized pool

      • fixed

        public static S3DirectBufferPoolOptions fixed​(long memoryLimitBytes)
        Fixed-size pool: all of its memory is allocated when the client is created, and it never grows or shrinks. Use it when you need a hard, predictable memory budget, for example in a tight container.
        • Size: memoryLimitBytes, rounded down to whole parts (S3ClientOptions.withPartSize(long)).
        • When full: new requests wait until memory frees up.
        • Very large uploads: each upload part must fit in 4 parts' worth of memory. Affected are uploads big enough that S3's 10,000-part limit forces a larger part size (over about 312 GiB with an 8 MiB part size), and resumed uploads whose token specifies a larger part size; they are rejected when the request is made, with the reason. Downloads are not affected.
        • No allocation pauses: memory is never allocated or freed after the client is created. The cost moves to client creation instead, which allocates and zero-fills the whole pool: expect a few hundred milliseconds per GiB.
        Parameters:
        memoryLimitBytes - total direct-memory budget for the pool, counted against -XX:MaxDirectMemorySize; must be at least one part, checked when the client is created
        Returns:
        options for a fixed pool
        Throws:
        IllegalArgumentException - if memoryLimitBytes <= 0

      • elastic

        public static S3DirectBufferPoolOptions elastic​(long minBytes,
                                                long maxBytes)
        Pool with a floor and ceiling you choose, in bytes. Use it to size the pool yourself while still letting it grow and shrink with demand.

        Floor and ceiling mean what the class doc's Sizing section says; this factory only decides their values. The pool works in 16-part groups, so the floor is minBytes rounded down to a multiple of 16 parts (or the whole pool if minBytes reaches the ceiling) and never more than you asked for, the ceiling is maxBytes, and in between the pool grows 16 parts at a time and releases everything above the floor once the client goes idle. maxBytes need not be a multiple of 16 parts: it rounds down to whole parts, and the last group holds whatever is left.

        Note: growth cost. Each time the pool grows it allocates and zero-fills 16 parts of memory (128 MiB with 8 MiB parts), which takes several milliseconds and briefly delays other requests on the client. A sudden burst can grow the pool several times in quick succession, and the same happens again after the pool has shrunk while idle. If your workload can't tolerate these pauses, use fixed(long).

        Parameters:
        minBytes - memory allocated when the client is created and kept while idle (>= 0; 0 allocates nothing up front)
        maxBytes - most memory the pool will use (>= minBytes; must be at least one part, checked when the client is created)
        Returns:
        options for an elastic pool
        Throws:
        IllegalArgumentException - if minBytes < 0, maxBytes <= 0, or minBytes > maxBytes