Package com.amazonaws.athena.connector.lambda.handlers

Class RecordHandler

java.lang.Object

com.amazonaws.athena.connector.lambda.handlers.RecordHandler

All Implemented Interfaces:

com.amazonaws.services.lambda.runtime.RequestStreamHandler

Direct Known Subclasses:

AmazonMskRecordHandler, AwsCmdbRecordHandler, BigQueryRecordHandler, CloudwatchRecordHandler, DocDBRecordHandler, DynamoDBRecordHandler, ElasticsearchRecordHandler, ExampleRecordHandler, GcsRecordHandler, HbaseRecordHandler, JdbcRecordHandler, KafkaRecordHandler, MetricsRecordHandler, NeptuneRecordHandler, RedisRecordHandler, TimestreamRecordHandler, TPCDSRecordHandler, VerticaRecordHandler

public abstract class RecordHandler
extends Object
implements com.amazonaws.services.lambda.runtime.RequestStreamHandler

More specifically, this class is responsible for providing Athena with actual rows level data from our simulated source. Athena will call readWithConstraint(...) on this class for each 'Split' we generated in MetadataHandler.

Field Summary

Fields

Modifier and Type	Field	Description
protected Map<String,String>	configOptions

Constructor Summary

Constructors

Constructor	Description
RecordHandler(com.amazonaws.services.s3.AmazonS3 amazonS3, com.amazonaws.services.secretsmanager.AWSSecretsManager secretsManager, com.amazonaws.services.athena.AmazonAthena athena, String sourceType, Map<String,String> configOptions)
RecordHandler(String sourceType, Map<String,String> configOptions)

Method Summary

Modifier and Type	Method	Description
protected void	doHandleRequest(BlockAllocator allocator, com.fasterxml.jackson.databind.ObjectMapper objectMapper, RecordRequest req, OutputStream outputStream)
RecordResponse	doReadRecords(BlockAllocator allocator, ReadRecordsRequest request)	Used to read the row data associated with the provided Split.
protected String	getSecret(String secretName)
protected SpillConfig	getSpillConfig(ReadRecordsRequest request)
void	handleRequest(InputStream inputStream, OutputStream outputStream, com.amazonaws.services.lambda.runtime.Context context)
protected void	onPing(PingRequest request)
protected abstract void	readWithConstraint(BlockSpiller spiller, ReadRecordsRequest recordsRequest, QueryStatusChecker queryStatusChecker)	A more stream lined option for reading the row data associated with the provided Split.
protected String	resolveSecrets(String rawString)	Resolves any secrets found in the supplied string, for example: MyString${WithSecret} would have ${WithSecret} by the corresponding value of the secret in AWS Secrets Manager with that name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

configOptions

protected final Map<String,String> configOptions

Constructor Detail

RecordHandler

public RecordHandler(String sourceType,
                     Map<String,String> configOptions)

Parameters:

sourceType - Used to aid in logging diagnostic info when raising a support case.

RecordHandler

public RecordHandler(com.amazonaws.services.s3.AmazonS3 amazonS3,
                     com.amazonaws.services.secretsmanager.AWSSecretsManager secretsManager,
                     com.amazonaws.services.athena.AmazonAthena athena,
                     String sourceType,
                     Map<String,String> configOptions)

Parameters:

sourceType - Used to aid in logging diagnostic info when raising a support case.

Method Detail

resolveSecrets

protected String resolveSecrets(String rawString)

Resolves any secrets found in the supplied string, for example: MyString${WithSecret} would have ${WithSecret} by the corresponding value of the secret in AWS Secrets Manager with that name. If no such secret is found the function throws.

Parameters:

rawString - The string in which you'd like to replace SecretsManager placeholders. (e.g. ThisIsA${Secret}Here - The ${Secret} would be replaced with the contents of an SecretsManager secret called Secret. If no such secret is found, the function throws. If no ${} are found in the input string, nothing is replaced and the original string is returned.

getSecret

protected String getSecret(String secretName)

handleRequest

public final void handleRequest(InputStream inputStream,
                                OutputStream outputStream,
                                com.amazonaws.services.lambda.runtime.Context context)
                         throws IOException

Specified by:

handleRequest in interface com.amazonaws.services.lambda.runtime.RequestStreamHandler

Throws:

IOException

doHandleRequest

protected final void doHandleRequest(BlockAllocator allocator,
                                     com.fasterxml.jackson.databind.ObjectMapper objectMapper,
                                     RecordRequest req,
                                     OutputStream outputStream)
                              throws Exception

Throws:

Exception

doReadRecords

public RecordResponse doReadRecords(BlockAllocator allocator,
                                    ReadRecordsRequest request)
                             throws Exception

Used to read the row data associated with the provided Split.

Parameters:

allocator - Tool for creating and managing Apache Arrow Blocks.

request - Details of the read request, including: 1. The Split 2. The Catalog, Database, and Table the read request is for. 3. The filtering predicate (if any) 4. The columns required for projection.

Returns:

A RecordResponse which either a ReadRecordsResponse or a RemoteReadRecordsResponse containing the row data for the requested Split.

Throws:

Exception

readWithConstraint

protected abstract void readWithConstraint(BlockSpiller spiller,
                                           ReadRecordsRequest recordsRequest,
                                           QueryStatusChecker queryStatusChecker)
                                    throws Exception

A more stream lined option for reading the row data associated with the provided Split. This method differs from doReadRecords(...) in that the SDK handles more of the request lifecycle, leaving you to focus more closely on the task of actually reading from your source.

Parameters:

spiller - A BlockSpiller that should be used to write the row data associated with this Split. The BlockSpiller automatically handles chunking the response, encrypting, and spilling to S3.

recordsRequest - Details of the read request, including: 1. The Split 2. The Catalog, Database, and Table the read request is for. 3. The filtering predicate (if any) 4. The columns required for projection.

queryStatusChecker - A QueryStatusChecker that you can use to stop doing work for a query that has already terminated

Throws:

Exception

getSpillConfig

protected SpillConfig getSpillConfig(ReadRecordsRequest request)

onPing

protected void onPing(PingRequest request)