aws_ddk_core.pipelines.Pipeline

class aws_ddk_core.pipelines.Pipeline(*args: Any, **kwargs)

Class that represents a data pipeline. Used to wire stages via Event Rules. For example:

Pipeline(self, id, description="My pipeline")
    .add_stage(my_lambda_stage)
    .add_stage(my_glue_stage)
__init__(scope: constructs.Construct, id: str, name: Optional[str] = None, description: Optional[str] = None) → None

Create a data pipeline.

Parameters

  • scope (Construct) – Scope within which this construct is defined

  • id (str) – Identifier of the pipeline

  • name (Optional[str]) – Name of the pipeline

  • description (Optional[str]) – Description of the pipeline

Methods

__init__(scope, id[, name, description])

Create a data pipeline.

add_rule([id, event_pattern, event_targets, ...])

Create a rule that matches specificed event pattern with the specified target.

add_stage(stage[, skip_rule, override_rule])

Add a stage to the data pipeline.

is_construct(x)

(deprecated) Checks if x is a construct.

to_string()

Returns a string representation of this construct.

Attributes

node

The tree node.
add_rule(id: Optional[str] = None, event_pattern: Optional[aws_cdk.aws_events.EventPattern] = None, event_targets: Optional[List[aws_cdk.aws_events.IRuleTarget]] = None, override_rule: Optional[aws_cdk.aws_events.Rule] = None) → aws_ddk_core.pipelines.pipeline.Pipeline

Create a rule that matches specificed event pattern with the specified target.

Parameters

  • id (Optional[str]) – Identifier of the rule

  • event_pattern (Optional[EventPattern]) – Event pattern of the rule

  • event_targets (Optional[List[IRuleTarget]]) – Target of the rule - usually previous_stage.get_targets()

  • override_rule (Optional[Rule]) – Custom rule

Returns

pipeline – Pipeline

Return type

Pipeline

add_stage(stage: aws_ddk_core.pipelines.stage.Stage, skip_rule: bool = False, override_rule: Optional[aws_cdk.aws_events.Rule] = None) → aws_ddk_core.pipelines.pipeline.Pipeline

Add a stage to the data pipeline.

By default, creates an Event Rule that matches output event pattern of the previous stage with targets of the next one.

Parameters

  • stage (Stage) – Stage instance

  • skip_rule (bool) – Skip creation of the default rule

  • override_rule (Optional[Rule]) – Override the default rule by the rule specified in this parameter

Returns

pipeline – Pipeline

Return type

Pipeline