Package com.amazonaws.athena.connectors.hortonworks

Class HiveMetadataHandler

java.lang.Object

com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

com.amazonaws.athena.connectors.hortonworks.HiveMetadataHandler

All Implemented Interfaces:

com.amazonaws.services.lambda.runtime.RequestStreamHandler

public class HiveMetadataHandler
extends JdbcMetadataHandler

Field Summary

Fields inherited from class com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

caseResolver, jdbcQueryPassthrough, TABLES_AND_VIEWS

Fields inherited from class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

configOptions, DISABLE_SPILL_ENCRYPTION, KMS_KEY_ID_ENV, SPILL_BUCKET_ENV, SPILL_PREFIX_ENV

Constructor Summary

Constructors

Modifier	Constructor	Description
	HiveMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, Map<String,String> configOptions)
protected	HiveMetadataHandler(DatabaseConnectionConfig databaseConnectionConfiguration, software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretManager, software.amazon.awssdk.services.athena.AthenaClient athena, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions)
	HiveMetadataHandler(Map<String,String> configOptions)

Method Summary

Modifier and Type	Method	Description
GetDataSourceCapabilitiesResponse	doGetDataSourceCapabilities(BlockAllocator allocator, GetDataSourceCapabilitiesRequest request)	Used to describe the types of capabilities supported by a data source.
GetSplitsResponse	doGetSplits(BlockAllocator blockAllocator, GetSplitsRequest getSplitsRequest)	Used to split-up the reads required to scan the requested batch of partition(s).
void	getPartitions(BlockWriter blockWriter, GetTableLayoutRequest getTableLayoutRequest, QueryStatusChecker queryStatusChecker)	Used to get the hive partitions that must be read from the request table in order to satisfy the requested predicate.
org.apache.arrow.vector.types.pojo.Schema	getPartitionSchema(String catalogName)	Delegates creation of partition schema to database type implementation.
protected org.apache.arrow.vector.types.pojo.Schema	getSchema(Connection jdbcConnection, TableName tableName, org.apache.arrow.vector.types.pojo.Schema partitionSchema)	Used to convert Hive data types to Apache arrow data types

Methods inherited from class com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

convertDatasourceTypeToArrow, doGetQueryPassthroughSchema, doGetTable, doListSchemaNames, doListTables, escapeNamePattern, getArrayArrowTypeFromTypeName, getColumns, getCredentialProvider, getJdbcConnectionFactory, getSplitClauses, listDatabaseNames, listPaginatedTables, listTables, setupQueryPassthroughSplit, wrapNameWithEscapedCharacter

Methods inherited from class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

doGetTableLayout, doHandleRequest, doPing, enhancePartitionSchema, getSecret, handleRequest, makeEncryptionKey, makeSpillLocation, onPing, resolveSecrets, resolveWithDefaultCredentials

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HiveMetadataHandler

public HiveMetadataHandler(Map<String,String> configOptions)

HiveMetadataHandler

public HiveMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                           Map<String,String> configOptions)

HiveMetadataHandler

protected HiveMetadataHandler(DatabaseConnectionConfig databaseConnectionConfiguration,
                              software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretManager,
                              software.amazon.awssdk.services.athena.AthenaClient athena,
                              JdbcConnectionFactory jdbcConnectionFactory,
                              Map<String,String> configOptions)

Method Detail

doGetDataSourceCapabilities

public GetDataSourceCapabilitiesResponse doGetDataSourceCapabilities(BlockAllocator allocator,
                                                                     GetDataSourceCapabilitiesRequest request)

Used to describe the types of capabilities supported by a data source. An engine can use this to determine what portions of the query to push down. A connector that returns any optimization will guarantee that the associated predicate will be pushed down.

Overrides:

doGetDataSourceCapabilities in class MetadataHandler

Parameters:

allocator - Tool for creating and managing Apache Arrow Blocks.

request - Provides details about the catalog being used.

Returns:

A GetDataSourceCapabilitiesResponse object which returns a map of supported optimizations that the connector is advertising to the consumer. The connector assumes all responsibility for whatever is passed here.

getPartitionSchema

public org.apache.arrow.vector.types.pojo.Schema getPartitionSchema(String catalogName)

Delegates creation of partition schema to database type implementation.

Specified by:

getPartitionSchema in class JdbcMetadataHandler

Parameters:

catalogName - Athena provided hive catalog name.

Returns:

schema. See Schema

getPartitions

public void getPartitions(BlockWriter blockWriter,
                          GetTableLayoutRequest getTableLayoutRequest,
                          QueryStatusChecker queryStatusChecker)
                   throws Exception

Used to get the hive partitions that must be read from the request table in order to satisfy the requested predicate.

Specified by:

getPartitions in class JdbcMetadataHandler

Parameters:

blockWriter - Used to write rows (hive partitions) into the Apache Arrow response.

getTableLayoutRequest - Provides details of the catalog, database, and table being queried as well as any filter predicate.

queryStatusChecker - A QueryStatusChecker that you can use to stop doing work for a query that has already terminated

Throws:

Exception - An Exception should be thrown for database connection failures , query syntax errors and so on.

doGetSplits

public GetSplitsResponse doGetSplits(BlockAllocator blockAllocator,
                                     GetSplitsRequest getSplitsRequest)

Used to split-up the reads required to scan the requested batch of partition(s).

Specified by:

doGetSplits in class JdbcMetadataHandler

Parameters:

blockAllocator - Tool for creating and managing Apache Arrow Blocks.

getSplitsRequest - Provides details of the Hive catalog, database, table, and partition(s) being queried as well as any filter predicate.

Returns:

A GetSplitsResponse which primarily contains: 1. A Set of Splits which represent read operations Amazon Athena must perform by calling your read function. 2. (Optional) A continuation token which allows you to paginate the generation of splits for large queries.

getSchema

protected org.apache.arrow.vector.types.pojo.Schema getSchema(Connection jdbcConnection,
                                                              TableName tableName,
                                                              org.apache.arrow.vector.types.pojo.Schema partitionSchema)
                                                       throws Exception

Used to convert Hive data types to Apache arrow data types

Overrides:

getSchema in class JdbcMetadataHandler

Parameters:

jdbcConnection - A JDBC Hive database connection

tableName - Holds table name and schema name. see TableName

partitionSchema - A partition schema for a given table .See Schema

Returns:

Schema Holds Table schema along with partition schema. See Schema

Throws:

Exception - An Exception should be thrown for database connection failures , query syntax errors and so on.