Package com.amazonaws.athena.connectors.dynamodb.util

Class DDBPredicateUtils

java.lang.Object

com.amazonaws.athena.connectors.dynamodb.util.DDBPredicateUtils

public class DDBPredicateUtils
extends Object

Provides utility methods relating to predicate handling.

Method Summary

Modifier and Type	Method	Description
static String	aliasColumn(String columnName)	Generates a simple alias for a column to satisfy filter expressions.
static Map<String,List<ColumnPredicate>>	buildFilterPredicatesFromPlan(io.substrait.proto.Plan plan)	Builds filter predicates from a Substrait execution plan.
static String	generateFilterExpression(Set<String> columnsToIgnore, Map<String,ValueSet> predicates, List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator, IncrementingValueNameProducer valueNameProducer, DDBRecordMetadata recordMetadata)	Generates a combined filter expression for the given predicates.
static String	generateFilterExpressionForPlan(Set<String> columnsToIgnore, Map<String,List<ColumnPredicate>> predicates, List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator, IncrementingValueNameProducer valueNameProducer, DDBRecordMetadata recordMetadata)
static String	generateSingleColumnFilter(String originalColumnName, ValueSet predicate, List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator, IncrementingValueNameProducer valueNameProducer, DDBRecordMetadata recordMetadata, boolean columnIsSortKey)	Generates a filter expression for a single column given a ValueSet predicate for that column.
static String	generateSingleColumnFilter(String originalColumnName, List<ColumnPredicate> predicates, List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator, IncrementingValueNameProducer valueNameProducer, DDBRecordMetadata recordMetadata, boolean columnIsSortKey)	Generates a filter expression for a single column from Substrait predicates.
static DynamoDBIndex	getBestIndexForPredicates(DynamoDBTable table, List<String> requestedCols, Map<String,ValueSet> predicates)	Attempts to pick an optimal index (if any) from the given predicates.
static DynamoDBIndex	getBestIndexForPredicatesForPlan(DynamoDBTable table, List<String> requestedCols, Map<String,List<ColumnPredicate>> filterPredicates)
static List<Object>	getHashKeyAttributeValues(ValueSet valueSet)	Generates a list of distinct values from the given ValueSet or an empty list if not possible.
static List<ColumnPredicate>	getHashKeyAttributeValues(List<ColumnPredicate> columnPredicates)
static boolean	indexContainsAllRequiredColumns(List<String> requestedCols, DynamoDBIndex index, DynamoDBTable table)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getBestIndexForPredicates

public static DynamoDBIndex getBestIndexForPredicates(DynamoDBTable table,
                                                      List<String> requestedCols,
                                                      Map<String,ValueSet> predicates)

Attempts to pick an optimal index (if any) from the given predicates. Returns the original table index if one was not found.

Parameters:

table - the original table

predicates - the predicates

Returns:

the optimal index if found, otherwise the original table index

getBestIndexForPredicatesForPlan

public static DynamoDBIndex getBestIndexForPredicatesForPlan(DynamoDBTable table,
                                                             List<String> requestedCols,
                                                             Map<String,List<ColumnPredicate>> filterPredicates)

getHashKeyAttributeValues

public static List<Object> getHashKeyAttributeValues(ValueSet valueSet)

Generates a list of distinct values from the given ValueSet or an empty list if not possible.

Parameters:

valueSet - the value set to generate from

Returns:

the list of distinct values

aliasColumn

public static String aliasColumn(String columnName)

Generates a simple alias for a column to satisfy filter expressions. Uses a regex to convert illegal characters (any character or combination of characters that are NOT included in [a-zA-Z_0-9]) to underscore. Example: "column-$1`~!@#$%^&*()-=+[]{}\\|;:'\",.<>/?f3" -> "#column_1_f3"

Parameters:

columnName - the input column name

Returns:

the aliased column name

See Also:

https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ExpressionAttributeNames.html

buildFilterPredicatesFromPlan

public static Map<String,List<ColumnPredicate>> buildFilterPredicatesFromPlan(io.substrait.proto.Plan plan)

Builds filter predicates from a Substrait execution plan.

Parameters:

plan - the Substrait plan containing filter conditions

Returns:

map of column names to their predicates

getHashKeyAttributeValues

public static List<ColumnPredicate> getHashKeyAttributeValues(List<ColumnPredicate> columnPredicates)

generateSingleColumnFilter

public static String generateSingleColumnFilter(String originalColumnName,
                                                ValueSet predicate,
                                                List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator,
                                                IncrementingValueNameProducer valueNameProducer,
                                                DDBRecordMetadata recordMetadata,
                                                boolean columnIsSortKey)

Generates a filter expression for a single column given a ValueSet predicate for that column.

Parameters:

originalColumnName - the column name

predicate - the associated predicate

accumulator - the value accumulator to add values to

valueNameProducer - the value name producer to generate value aliases with

recordMetadata - object containing any necessary metadata from the glue table

columnIsSortKey - whether or not the originalColumnName column is a sort key

Returns:

the generated filter expression

generateSingleColumnFilter

public static String generateSingleColumnFilter(String originalColumnName,
                                                List<ColumnPredicate> predicates,
                                                List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator,
                                                IncrementingValueNameProducer valueNameProducer,
                                                DDBRecordMetadata recordMetadata,
                                                boolean columnIsSortKey)

Generates a filter expression for a single column from Substrait predicates.

generateFilterExpression

public static String generateFilterExpression(Set<String> columnsToIgnore,
                                              Map<String,ValueSet> predicates,
                                              List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator,
                                              IncrementingValueNameProducer valueNameProducer,
                                              DDBRecordMetadata recordMetadata)

Generates a combined filter expression for the given predicates. columnsToIgnore will contain any column that is of custom type (such as timestamp with tz) as these types are not natively supported by ddb or glue. we will need to filter them separately in the ddb query/scan result.

Parameters:

columnsToIgnore - the columns to not generate filters for

predicates - the map of columns to predicates

accumulator - the value accumulator to add values to

valueNameProducer - the value name producer to generate value aliases with

recordMetadata - object containing any necessary metadata from the glue table

Returns:

the combined filter expression

generateFilterExpressionForPlan

public static String generateFilterExpressionForPlan(Set<String> columnsToIgnore,
                                                     Map<String,List<ColumnPredicate>> predicates,
                                                     List<software.amazon.awssdk.services.dynamodb.model.AttributeValue> accumulator,
                                                     IncrementingValueNameProducer valueNameProducer,
                                                     DDBRecordMetadata recordMetadata)

indexContainsAllRequiredColumns

public static boolean indexContainsAllRequiredColumns(List<String> requestedCols,
                                                      DynamoDBIndex index,
                                                      DynamoDBTable table)