Util¶
Utility functions
- lorien.util.deep_tuple_to_list(inp: Union[Tuple, List]) List[Any]¶
Transform a nested tuple to a nested list.
- Parameters
inp (Union[Tuple, List]) -- The input object.
- Returns
ret -- The nested list.
- Return type
List[Any]
- lorien.util.delete_s3_file(s3_path: str) str¶
Use boto3 client to delete a file in S3.
- Parameters
s3_path (str) -- The S3 file path. It should start with "s3://".
- Returns
msg -- The error message if failed, or an empty string.
- Return type
str
- lorien.util.download_s3_file(s3_path: str, local_path: str, delete: bool = False)¶
Use boto3 client to download a file from S3.
- Parameters
s3_path (str) -- The S3 file path. It should start with "s3://".
local_path (str) -- The full file path (including the file name) for the downloaded file.
delete (bool) -- Whether to delete the file on S3 after successfully downloaded.
- Returns
msg -- The error message if failed, or an empty string.
- Return type
str
- lorien.util.dump_to_yaml(obj: Any, single_line: bool = True) str¶
Dump an object to a string in YAML format.
- Parameters
obj (Any) -- The YAML object.
single_line (bool) -- Whether to generate a sinlge-line style YAML string.
- Returns
ret -- The dumped YAML string.
- Return type
str
- lorien.util.get_time_str() str¶
Generate a string using the current time.
- Returns
ret -- A string in <year><month><day>-<hour><min> format.
- Return type
str
- lorien.util.is_dialect_enabled(name: str) bool¶
Check if the dialect is enabled.
- Parameters
name (str) -- The dialect name.
- Returns
ret -- Whether the dialect is enabled.
- Return type
bool
- lorien.util.load_from_yaml(yaml_str: str, expected_type: Optional[Any] = None) Any¶
Load and construct an object from YAML string.
- Parameters
yaml_str (str) -- The YAML string.
expected_type (Optional[Any]) -- The expected object type from the given YAML string. None indicates any type.
- Returns
ret -- The constructed object.
- Return type
Any
- lorien.util.serialize_framework_build_config(dict_: Union[Dict[str, str], str]) Tuple[Any, ...]¶
Serialize a dict to a hashable tuple.
- Parameters
dict (Dict[str, str]) --
- Returns
hashable_tuple -- A hashable tuple.
- Return type
Tuple[Any, ...]
- lorien.util.split_s3_path(full_path: str) Tuple[str, str]¶
Split a S3 path to bucket name and folder (file) path.
- Parameters
full_path (str) -- A full S3 path in format <bucket-name>/<folder1>/.../<folderN>/<file-name>
- Returns
bucket_n_path -- A pair of bucket name and folder (file) path. Folder (file) path will be an empty string if full_path only contains a bucket.
- Return type
Tuple[str, str]
- lorien.util.upload_s3_file(local_path: str, s3_path: str) str¶
Use boto3 client to upload a file to S3.
- Parameters
local_path (str) -- The full file path (including the file name) for the file to be uploaded.
s3_path (str) -- The S3 file path. It should start with "s3://".
- Returns
msg -- The error message if failed, or an empty string.
- Return type
str