Step 3 - Integrate SDK

Once pipeline's status becomes Active, it is ready to receive clickstream data. Now you need to register an application to the pipeline, then you can integrate SDK into your application to enable it to send data to the pipeline.

1. Log into Clickstream Analytics on AWS Console.
2. In the left navigation pane, choose Projects, then select the project (quickstart) you just created in previous steps, click its title, and it will bring you to the project page.
3. Choose + Add application to start adding application to the pipeline.

4. Fill in the form as follows:

   • App name: test-app
   • App ID: The system will generate one ID based on the name, and you can customize it if needed.
   • Description: A test app for Clickstream Analytics on AWS solution
   • Android package name: leave it blank
   • App Bundle ID: leave it blank

5. Choose Register App & Generate SDK Instruction, and wait for the registration to be completed.

6. Select the tab Android, and you will see the detailed instruction of adding SDK into your application. You can follow the steps to add SDK.

7. Click the Download the config json file button to download the config file, and keep this file open, which will be used later.

It will take about 3 ~ 5 minutes to update the pipeline with the application you just add. When you see the pipeline status become Active again, it is ready to receive data from your application.

We have completed all the steps of adding an application to a project.

Generate sample data

You might not have immediate access to integrate SDK with your app. In this case, we provide a Python script that generates the last 30 days of sample data for a shopping app into the pipeline you just configured, so that you can view and experience the analytics dashboards.

Prerequisites

• Python 3.8+

Send sample data

1. Clone the repository to your local environment.

   git clone https://github.com/awslabs/clickstream-analytics-on-aws.git
   

2. After you cloned the repository, change directory into the examples/standalone-data-generator project folder.

3. Install the dependencies of the project.

   pip3 install requests
   

4. Put amplifyconfiguration.json into the root of examples/standalone-data-generator which you downloaded in register an app step. See the examples/standalone-data-generator/README.md for more usages.

5. Open an terminal at this project folder location. For example, if you are using Visual Studio Code IDE, at the top of Visual Studio Code, click Terminal -> New Terminal to open a terminal.

6. Copy the following command and paste it to the terminal:

   python3 create_event.py
   

Hit Enter key in terminal to execute the program. If you see the following output, this means that the program execution is complete.

job finished, upload 1106400 events, cost: 96252ms

This process will take about 2 minutes with default configuration.

Refresh historical events

This solution only calculates the metrics in the out-of-the-box dashboards for the previous day, and we just sent the last 30 days of data, so we need to refresh the historical metrics for all events to show up in the Analytics Studio dashboard.

Before refreshing historical events, we need to make sure the events we just sent are loaded into Redshift.

1. Go your project detail page, click the View Details button, and then enter your project pipeline details page.
2. In the pipeline details page, click and open Monitoring tab, and then click View in CloudWatch button to enter the solution CloudWatch dashboard.
3. Scroll down and find Data Modeling - Redshift Serverless section in the CloudWatch dashboard, then make sure all status in 'Load data to redshift tables' workflow table are ExecutionsSucceeded.

Once all events are loaded into Redshift, we can follow the steps below to refresh historical events.

1. Search and open the Step Functions service in the AWS console and select the region where your data pipeline is located.
2. Search and find the state machine named RefreshMaterializedViewsWorkflowRefreshMVStateMachine, then click into the detail page.
3. Click the Start Execution button in the details page.
4. Copy the following JSON content and change the refreshEndTime value to the current timestamp and the refreshStartTime to a timestamp 30 days ago, then paste it into the input panel and click the Start execution button:

   {
     "refreshEndTime": 1715658451000,
     "refreshStartTime": 1713066451000
   }
   

5. Return to the state machine detail page, you will find that a new execution has started, let's wait the status chang to Succeeded, this process will take about 7 minutes.

Now that all the historical events has been refreshed, you can move to next step.

Next

• Analyze data