aws_ddk_core.pipelines.DataPipeline

class aws_ddk_core.pipelines.DataPipeline(*args: Any, **kwargs)

Class that represents a data pipeline. Used to wire stages via Event Rules. For example:

DataPipeline(self, id, description="My pipeline")
    .add_stage(my_lambda_stage)
    .add_stage(my_glue_stage)

__init__(scope: constructs.Construct, id: str, name: Optional[str] = None, description: Optional[str] = None) → None

Create a data pipeline.

Parameters

• scope (Construct) – Scope within which this construct is defined

• id (str) – Identifier of the pipeline

• name (Optional[str]) – Name of the pipeline

• description (Optional[str]) – Description of the pipeline

Methods

__init__(scope, id[, name, description])	Create a data pipeline.
add_rule([id, event_pattern, event_targets, ...])	Create a rule that matches specificed event pattern with the specified target.
add_stage(stage[, skip_rule, override_rule])	Add a stage to the data pipeline.
is_construct(x)	(deprecated) Checks if x is a construct.
to_string()	Returns a string representation of this construct.

Attributes

node	The tree node.

add_rule(id: Optional[str] = None, event_pattern: Optional[aws_cdk.aws_events.EventPattern] = None, event_targets: Optional[List[aws_cdk.aws_events.IRuleTarget]] = None, override_rule: Optional[aws_cdk.aws_events.Rule] = None) → aws_ddk_core.pipelines.pipeline.DataPipeline

Create a rule that matches specificed event pattern with the specified target.

Parameters

• id (Optional[str]) – Identifier of the rule

• event_pattern (Optional[EventPattern]) – Event pattern of the rule

• event_targets (Optional[List[IRuleTarget]]) – Target of the rule - usually previous_stage.get_targets()

• override_rule (Optional[Rule]) – Custom rule

Returns

pipeline – DataPipeline

Return type

DataPipeline

add_stage(stage: aws_ddk_core.pipelines.stage.DataStage, skip_rule: bool = False, override_rule: Optional[aws_cdk.aws_events.Rule] = None) → aws_ddk_core.pipelines.pipeline.DataPipeline

Add a stage to the data pipeline.

By default, creates an Event Rule that matches output event pattern of the previous stage with targets of the next one.

Parameters

• stage (DataStage) – Stage instance

• skip_rule (bool) – Skip creation of the default rule

• override_rule (Optional[Rule]) – Override the default rule by the rule specified in this parameter

Returns

pipeline – DataPipeline

Return type

DataPipeline