Data ingestion

Data ingestion module contains a web service that provides an endpoint to collect data through HTTP/HTTPS requests, which mainly is composed of Amazon Application Load Balancer and Amazon Elastic Container Service. It also supports sinking data into a stream service or S3 directly.

You can create a data ingestion module with the following settings:

• Ingestion endpoint settings: Create a web service as an ingestion endpoint to collect data sent from your SDKs.

• Data sink settings: Configure how the solution sinks the data for downstream consumption. Currently, the solution supports three types of data sink:

  • Apache Kafka
  • Amazon S3
  • Amazon Kinesis Data Stream (KDS)

Throttle

Currently, there is no built-in throttling feature available with this solution. If you require throttling feature, you can configure AWS WAF to implement throttling feature. Please refer WAF document.