Package com.amazonaws.athena.connectors.clickhouse

Class ClickHouseMetadataHandler

java.lang.Object

com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

com.amazonaws.athena.connectors.mysql.MySqlMetadataHandler

com.amazonaws.athena.connectors.clickhouse.ClickHouseMetadataHandler

All Implemented Interfaces:

com.amazonaws.services.lambda.runtime.RequestStreamHandler

public class ClickHouseMetadataHandler
extends MySqlMetadataHandler

Class handles metadata for ClickHouse and reuses code from Athena MySQL module for compatibility. User must have access to `schemata`, `tables`, `columns`, `partitions` tables in INFORMATION_SCHEMA.

Field Summary

Fields inherited from class com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

jdbcQueryPassthrough, TABLES_AND_VIEWS

Fields inherited from class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

configOptions, DISABLE_SPILL_ENCRYPTION, KMS_KEY_ID_ENV, SPILL_BUCKET_ENV, SPILL_PREFIX_ENV

Constructor Summary

Constructors

Modifier	Constructor	Description
	ClickHouseMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions)
	ClickHouseMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, Map<String,String> configOptions)	Used by Mux.
protected	ClickHouseMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretsManager, software.amazon.awssdk.services.athena.AthenaClient athena, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions)
	ClickHouseMetadataHandler(Map<String,String> configOptions)	Instantiates handler to be used by Lambda function directly.

Method Summary

Modifier and Type	Method	Description
protected TableName	caseInsensitiveTableSearch(Connection connection, String databaseName, String tableName)	While being a no-op by default, this function will be overriden by subclasses that support this search.
GetSplitsResponse	doGetSplits(BlockAllocator blockAllocator, GetSplitsRequest getSplitsRequest)	Used to split-up the reads required to scan the requested batch of partition(s).
ListSchemasResponse	doListSchemaNames(BlockAllocator blockAllocator, ListSchemasRequest listSchemasRequest)	Used to get the list of schemas (aka databases) that this source contains.
protected List<TableName>	getPaginatedTables(Connection connection, String databaseName, int token, int limit)
void	getPartitions(BlockWriter blockWriter, GetTableLayoutRequest getTableLayoutRequest, QueryStatusChecker queryStatusChecker)	Used to get the partitions that must be read from the request table in order to satisfy the requested predicate.
org.apache.arrow.vector.types.pojo.Schema	getPartitionSchema(String catalogName)	Delegates creation of partition schema to database type implementation.
protected ListTablesResponse	listPaginatedTables(Connection connection, ListTablesRequest listTablesRequest)
protected List<TableName>	listTables(Connection jdbcConnection, String databaseName)

Methods inherited from class com.amazonaws.athena.connectors.mysql.MySqlMetadataHandler

doGetDataSourceCapabilities

Methods inherited from class com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

convertDatasourceTypeToArrow, doGetQueryPassthroughSchema, doGetTable, doListTables, escapeNamePattern, getArrayArrowTypeFromTypeName, getCredentialProvider, getJdbcConnectionFactory, getSplitClauses, setupQueryPassthroughSplit

Methods inherited from class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

doGetTableLayout, doHandleRequest, doPing, enhancePartitionSchema, getSecret, handleRequest, makeEncryptionKey, makeSpillLocation, onPing, resolveSecrets

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClickHouseMetadataHandler

public ClickHouseMetadataHandler(Map<String,String> configOptions)

Instantiates handler to be used by Lambda function directly. Recommend using MySqlMuxCompositeHandler instead.

ClickHouseMetadataHandler

public ClickHouseMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                                 Map<String,String> configOptions)

Used by Mux.

ClickHouseMetadataHandler

public ClickHouseMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                                 JdbcConnectionFactory jdbcConnectionFactory,
                                 Map<String,String> configOptions)

ClickHouseMetadataHandler

protected ClickHouseMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                                    software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretsManager,
                                    software.amazon.awssdk.services.athena.AthenaClient athena,
                                    JdbcConnectionFactory jdbcConnectionFactory,
                                    Map<String,String> configOptions)

Method Detail

getPartitionSchema

public org.apache.arrow.vector.types.pojo.Schema getPartitionSchema(String catalogName)

Description copied from class: JdbcMetadataHandler

Delegates creation of partition schema to database type implementation.

Overrides:

getPartitionSchema in class MySqlMetadataHandler

Parameters:

catalogName - Athena provided catalog name.

Returns:

schema. See Schema

getPartitions

public void getPartitions(BlockWriter blockWriter,
                          GetTableLayoutRequest getTableLayoutRequest,
                          QueryStatusChecker queryStatusChecker)
                   throws Exception

Description copied from class: MetadataHandler

Used to get the partitions that must be read from the request table in order to satisfy the requested predicate.

Overrides:

getPartitions in class MySqlMetadataHandler

Parameters:

blockWriter - Used to write rows (partitions) into the Apache Arrow response.

getTableLayoutRequest - Provides details of the catalog, database, and table being queried as well as any filter predicate.

queryStatusChecker - A QueryStatusChecker that you can use to stop doing work for a query that has already terminated

Throws:

Exception

doGetSplits

public GetSplitsResponse doGetSplits(BlockAllocator blockAllocator,
                                     GetSplitsRequest getSplitsRequest)

Description copied from class: MetadataHandler

Used to split-up the reads required to scan the requested batch of partition(s).

Overrides:

doGetSplits in class MySqlMetadataHandler

Parameters:

blockAllocator - Tool for creating and managing Apache Arrow Blocks.

getSplitsRequest - Provides details of the catalog, database, table, andpartition(s) being queried as well as any filter predicate.

Returns:

A GetSplitsResponse which primarily contains: 1. A Set which represent read operations Amazon Athena must perform by calling your read function. 2. (Optional) A continuation token which allows you to paginate the generation of splits for large queries.

getPaginatedTables

protected List<TableName> getPaginatedTables(Connection connection,
                                             String databaseName,
                                             int token,
                                             int limit)
                                      throws SQLException

Overrides:

getPaginatedTables in class MySqlMetadataHandler

Throws:

SQLException

listPaginatedTables

protected ListTablesResponse listPaginatedTables(Connection connection,
                                                 ListTablesRequest listTablesRequest)
                                          throws SQLException

Overrides:

listPaginatedTables in class MySqlMetadataHandler

Throws:

SQLException

listTables

protected List<TableName> listTables(Connection jdbcConnection,
                                     String databaseName)
                              throws SQLException

Overrides:

listTables in class MySqlMetadataHandler

Throws:

SQLException

caseInsensitiveTableSearch

protected TableName caseInsensitiveTableSearch(Connection connection,
                                               String databaseName,
                                               String tableName)
                                        throws Exception

Description copied from class: JdbcMetadataHandler

While being a no-op by default, this function will be overriden by subclasses that support this search.

Overrides:

caseInsensitiveTableSearch in class MySqlMetadataHandler

Returns:

TableName containing the resolved case sensitive table name.

Throws:

Exception

doListSchemaNames

public ListSchemasResponse doListSchemaNames(BlockAllocator blockAllocator,
                                             ListSchemasRequest listSchemasRequest)
                                      throws Exception

Description copied from class: MetadataHandler

Used to get the list of schemas (aka databases) that this source contains.

Overrides:

doListSchemaNames in class JdbcMetadataHandler

Parameters:

blockAllocator - Tool for creating and managing Apache Arrow Blocks.

listSchemasRequest - Provides details on who made the request and which Athena catalog they are querying.

Returns:

A ListSchemasResponse which primarily contains a Set of schema names and a catalog name corresponding the Athena catalog that was queried.

Throws:

Exception