Class BigQueryUtils


  • public class BigQueryUtils
    extends Object
    • Method Detail

      • getCredentialsFromSecretsManager

        public static com.google.auth.Credentials getCredentialsFromSecretsManager​(Map<String,​String> configOptions)
                                                                            throws IOException
        Throws:
        IOException
      • getEnvBigQueryCredsSmId

        public static String getEnvBigQueryCredsSmId​(Map<String,​String> configOptions)
      • fixCaseForDatasetName

        public static String fixCaseForDatasetName​(String projectName,
                                                   String datasetName,
                                                   com.google.cloud.bigquery.BigQuery bigQuery)
        BigQuery is case sensitive for its Project and Dataset Names. This function will return the first case insensitive match.
        Parameters:
        projectName - The dataset name we want to look up. The project name must be case correct.
        Returns:
        A case correct dataset name.
      • fixCaseForTableName

        public static String fixCaseForTableName​(String projectName,
                                                 String datasetName,
                                                 String tableName,
                                                 com.google.cloud.bigquery.BigQuery bigQuery)
      • getChildFieldList

        public static List<org.apache.arrow.vector.types.pojo.Field> getChildFieldList​(com.google.cloud.bigquery.Field field)
      • installGoogleCredentialsJsonFile

        public static void installGoogleCredentialsJsonFile​(Map<String,​String> configOptions)
                                                     throws IOException
        Install/place Google cloud platform credentials from AWS secret manager to temp location This is required for dataset api
        Throws:
        IOException
      • setupNativeEnvironmentVariables

        public static void setupNativeEnvironmentVariables()
      • coerce

        public static Object coerce​(org.apache.arrow.vector.FieldVector vector,
                                    Object value)
      • getComplexObjectFromFieldValue

        public static Object getComplexObjectFromFieldValue​(org.apache.arrow.vector.types.pojo.Field field,
                                                            com.google.cloud.bigquery.FieldValue fieldValue,
                                                            boolean isTimeStampCol)
                                                     throws ParseException
        Throws:
        ParseException