Skip to main content

Customize DSF on AWS constructs

You can customize DSF on AWS constructs in several ways to adapt to your specific needs:

  1. Use the Constructs properties instead of the smart defaults.
  2. Extend existing constructs and override specific methods or properties.
  3. Access CDK L1 resources and override any property.

Constructs properties​

Use the properties of the construct to adapt the behavior to your needs. With this approach, you bypass the smart defaults provided by the construct. Refer to the documentation of each construct to evaluate if your requirements can be implemented.

note

This method should always be preferred because constructs properties are tested as part of the DSF on AWS build process.

For example, you can use the DataLakeStorage properties to modify the lifecycle configurations for transitioning objects based on your needs instead of using the default rules:

new dsf.storage.DataLakeStorage(this, 'MyDataLakeStorage', {
  bronzeBucketInfrequentAccessDelay: 90,
  bronzeBucketArchiveDelay: 180,
  silverBucketInfrequentAccessDelay: 180,
  silverBucketArchiveDelay: 360,
  goldBucketInfrequentAccessDelay: 180,
  goldBucketArchiveDelay: 360,
});

Construct extension​

AWS CDK allows developers to extend classes like any object-oriented programing language. You can use this method when you want to:

  • Override a specific method exposed by a construct.
  • Implement your own defaults. Refer to the example of the AnalyticsBucket that extends the CDK L2 Bucket construct to enforce some of the parameters.

CDK resources override​

AWS CDK offers escape hatches to modify constructs that are encapsulated in DSF on AWS constructs. The constructs always expose the AWS resources that are encapsulated so you can manually modify their configuration. For achieving this you have 3 options:

  • Modify the L2 construct using its CDK API. For example, you can modify the buckets' policies provided by the DataLakeStorage to provide cross account write access. All the buckets L2 constructs are exposed as an object parameter:
  // Create a data lake using DSF on AWS L3 construct
  const storage = new dsf.storage.DataLakeStorage(this, 'MyDataLakeStorage');

  // Access the CDK L2 Bucket construct exposed by the L3 construct
  const goldBucket = storage.goldBucket;

  // Use the Bucket CDK API to modify the Bucket Policy and add cross account write access
  goldBucket.addToResourcePolicy(new aws_iam.PolicyStatement({
        actions: [
          's3:GetObject',
          's3:PutObject',
          's3:DeleteObject',
          's3:ListBucketMultipartUploads',
          's3:ListMultipartUploadParts',
          's3:AbortMultipartUpload',
          's3:ListBucket',
        ],
        effect: aws_iam.Effect.ALLOW,
        principals: [new aws_iam.AccountPrincipal('123456789012')]
      }
  ));
  • Modify the L1 construct resource when there is a CDK property available on the L1 construct. For example, you can override CDK L1 property for setting the S3 transfer Acceleration on the gold bucket of the DataLakeStorage:
  // Create a data lake using DSF on AWS L3 construct
  const storage = new dsf.storage.DataLakeStorage(this, 'MyDataLakeStorage');

  // Access the CDK L1 Bucket construct exposed by the L3 construct
  const cfnBucket = storage.goldBucket.node.defaultChild as CfnBucket;

  // Override the CDK L1 property for transfer acceleration
  cfnBucket.accelerateConfiguration = {
    accelerationStatus: 'Enabled',
  }
  • Override the CloudFormation properties when there isn't any CDK property, and you need to modify the CFN template directly. For example, you can override CloudFormation property for setting the S3 transfer Acceleration on the gold bucket of the DataLakeStorage:
  // Create a data lake using DSF on AWS L3 construct
  const storage = new dsf.storage.DataLakeStorage(this, 'MyDataLakeStorage');

  // Access the CDK L1 Bucket construct exposed by the L3 construct
  const cfnBucket = storage.goldBucket.node.defaultChild as CfnBucket;

  // Override the CloudFormation property for transfer acceleration
  cfnBucket.addOverride('Properties.AccelerateConfiguration.AccelerationStatus', 'Enabled')