Package com.amazonaws.athena.connectors.dynamodb.util

Class DDBTypeUtils

java.lang.Object

com.amazonaws.athena.connectors.dynamodb.util.DDBTypeUtils

public final class DDBTypeUtils
extends Object

Provides utility methods relating to type handling.

Method Summary

Modifier and Type	Method	Description
static String	attributeToJson(software.amazon.awssdk.services.dynamodb.model.AttributeValue attributeValue, String key)
static List<Object>	coerceListToExpectedType(Object value, org.apache.arrow.vector.types.pojo.Field field, DDBRecordMetadata recordMetadata)	Converts a Set to a List, and coerces all list items into the correct type.
static Object	coerceValueToExpectedType(Object value, org.apache.arrow.vector.types.pojo.Field field, org.apache.arrow.vector.types.Types.MinorType fieldType, DDBRecordMetadata recordMetadata)	Coerces the raw value from DynamoDB to normalized type
static Object	convertArrowTypeIfNecessary(String columnName, Object object)	Converts certain Arrow POJOs to Java POJOs to make downstream conversion easier.
static Object	convertArrowTypeIfNecessary(String columnName, Object object, DDBRecordMetadata recordMetadata)	Converts certain Arrow POJOs to Java POJOs to make downstream conversion easier.
static org.apache.arrow.vector.types.pojo.Field	getArrowFieldFromDDBType(String attributeName, String attributeType)	Converts from DynamoDB Attribute Type to Arrow type.
static org.apache.arrow.vector.types.pojo.Field	inferArrowField(String key, software.amazon.awssdk.services.dynamodb.model.AttributeValue value)	Infers an Arrow field from an object.
static software.amazon.awssdk.services.dynamodb.model.AttributeValue	jsonToAttributeValue(String jsonString, String key)
static Optional<Extractor>	makeExtractor(org.apache.arrow.vector.types.pojo.Field field, DDBRecordMetadata recordMetadata, boolean caseInsensitive)	Create the appropriate field extractor used for extracting field values from a DDB based on the field type.
static FieldWriterFactory	makeFactory(org.apache.arrow.vector.types.pojo.Field field, DDBRecordMetadata recordMetadata, DynamoDBFieldResolver resolver, boolean caseInsensitive)	Since GeneratedRowWriter doesn't yet support complex types (STRUCT, LIST..etc) we use this to create our own FieldWriters via a custom FieldWriterFactory.
static software.amazon.awssdk.services.dynamodb.model.AttributeValue	toAttributeValue(Object value)
static <T> T	toSimpleValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

inferArrowField

public static org.apache.arrow.vector.types.pojo.Field inferArrowField(String key,
                                                                       software.amazon.awssdk.services.dynamodb.model.AttributeValue value)

Infers an Arrow field from an object. This has limitations when it comes to complex types such as Lists and Maps and will fallback to VARCHAR fields in those cases.

Parameters:

key - the name of the field

value - the value of the field

Returns:

the inferred Arrow field

convertArrowTypeIfNecessary

public static Object convertArrowTypeIfNecessary(String columnName,
                                                 Object object)

Converts certain Arrow POJOs to Java POJOs to make downstream conversion easier. This is called from GetSplits request. Since DDBRecordMetadata object is used for any custom override for data manipulation/formatting, it does not apply to GetSplits request.

Parameters:

columnName - column name where the input object comes from

object - the input object

Returns:

the converted-to object if convertible, otherwise the original object

convertArrowTypeIfNecessary

public static Object convertArrowTypeIfNecessary(String columnName,
                                                 Object object,
                                                 DDBRecordMetadata recordMetadata)

Converts certain Arrow POJOs to Java POJOs to make downstream conversion easier.

Parameters:

columnName - column name where the input object comes from

object - the input object

recordMetadata - metadata object from glue table that contains customer specified information such as desired default timezone, or datetime formats

Returns:

the converted-to object if convertible, otherwise the original object

getArrowFieldFromDDBType

public static org.apache.arrow.vector.types.pojo.Field getArrowFieldFromDDBType(String attributeName,
                                                                                String attributeType)

Converts from DynamoDB Attribute Type to Arrow type.

Parameters:

attributeName - the DDB Attribute name

attributeType - the DDB Attribute type

Returns:

the converted-to Arrow Field

coerceValueToExpectedType

public static Object coerceValueToExpectedType(Object value,
                                               org.apache.arrow.vector.types.pojo.Field field,
                                               org.apache.arrow.vector.types.Types.MinorType fieldType,
                                               DDBRecordMetadata recordMetadata)

Coerces the raw value from DynamoDB to normalized type

Parameters:

value - raw value from DynamoDB

field - Arrow field from table schema

fieldType - Corresponding MinorType for field

recordMetadata - DDBRecordMetadata object containing any metadata that is passed from schema metadata

Returns:

coerced value to normalized type

coerceListToExpectedType

public static List<Object> coerceListToExpectedType(Object value,
                                                    org.apache.arrow.vector.types.pojo.Field field,
                                                    DDBRecordMetadata recordMetadata)
                                             throws RuntimeException

Converts a Set to a List, and coerces all list items into the correct type. If value is not a Collection, this method will return a List containing a single item.

Parameters:

value - is the Set/List of items.

field - is the LIST field containing a list type in the child field.

recordMetadata - contains metadata information.

Returns:

a List of coerced values.

Throws:

RuntimeException - when value is instance of Map since a List is expected.

makeExtractor

public static Optional<Extractor> makeExtractor(org.apache.arrow.vector.types.pojo.Field field,
                                                DDBRecordMetadata recordMetadata,
                                                boolean caseInsensitive)

Create the appropriate field extractor used for extracting field values from a DDB based on the field type.

Parameters:

field -

recordMetadata -

Returns:

makeFactory

public static FieldWriterFactory makeFactory(org.apache.arrow.vector.types.pojo.Field field,
                                             DDBRecordMetadata recordMetadata,
                                             DynamoDBFieldResolver resolver,
                                             boolean caseInsensitive)

Since GeneratedRowWriter doesn't yet support complex types (STRUCT, LIST..etc) we use this to create our own FieldWriters via a custom FieldWriterFactory.

Parameters:

field - is used to determine which factory to generate based on the field type.

recordMetadata - is used to retrieve metadata of ddb types

resolver - is used to resolve it to proper type

Returns:

toSimpleValue

public static <T> T toSimpleValue(software.amazon.awssdk.services.dynamodb.model.AttributeValue value)

toAttributeValue

public static software.amazon.awssdk.services.dynamodb.model.AttributeValue toAttributeValue(Object value)

attributeToJson

public static String attributeToJson(software.amazon.awssdk.services.dynamodb.model.AttributeValue attributeValue,
                                     String key)

jsonToAttributeValue

public static software.amazon.awssdk.services.dynamodb.model.AttributeValue jsonToAttributeValue(String jsonString,
                                                                                                 String key)