Create Profiles with IAM role
In a multi tenant Kubeflow installation, the pods created by pipelines workflow and the pipelines frontend services run in an user profile namespace. The service account (default-editor
) used for these pods needs permissions for the S3 bucket used by pipelines to read and write artifacts from S3. When using IRSA (IAM roles for service accounts) as your PIPELINE_S3_CREDENTIAL_OPTION
, any additional profiles created as part of a multi-user deployment besides the preconfigured kubeflow-user-example-com
will need to be configured with permissions to S3 bucket using IRSA.
The default-editor
SA needs to be annotated with an IAM role with sufficient permissions to access your S3 Bucket to run your pipelines. In the below steps we will be configuring a profile an IAM role with restricted access to a specific S3 Bucket using the AwsIamForServiceAccount
plugin for Profiles. To learn more about the AwsIamForServiceAccount
plugin for Profiles read the Profiles component guide.
Note: If you choose to run your pipeline with a service account other than the default which is
default-editor
, you must make sure to annotate that service account with an IAM role with sufficient S3 permissions.
Create a Profile
After installing Kubeflow on AWS with one of the available deployment options, you can configure Kubeflow Profiles with the following steps:
-
Define the following environment variables:
The
S3_BUCKET
that is exported should be the same bucket that is used by Kubeflow Pipelines.# Your cluster name export CLUSTER_NAME= # Your cluster region export CLUSTER_REGION= # The S3 Bucket that is used by Kubeflow Pipelines export S3_BUCKET= # Your AWS Acconut ID export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text) # Name of the profile to create export PROFILE_NAMESPACE=
-
Retrieve OIDC Provider URL
aws --region $CLUSTER_REGION eks update-kubeconfig --name $CLUSTER_NAME export OIDC_URL=$(aws eks describe-cluster --region $CLUSTER_REGION --name $CLUSTER_NAME --query "cluster.identity.oidc.issuer" --output text | cut -c9-)
-
Create an IAM trust policy to authorize federated requests from the OIDC provider.
cat <<EOF > trust.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDC_URL}" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "${OIDC_URL}:aud": "sts.amazonaws.com", "${OIDC_URL}:sub": "system:serviceaccount:${PROFILE_NAMESPACE}:default-editor" } } } ] } EOF
-
Create an IAM policy with access to the S3 bucket where pipeline artifacts will be stored. The following policy grants full access to the S3 bucket, you can scope it down by giving read, write and GetBucketLocation permissions.
cat <<EOF > s3_policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::${S3_BUCKET}", "arn:aws:s3:::${S3_BUCKET}/*" ] } ] } EOF
-
Create an IAM role for the Profile using the scoped policy from the previous step.
aws iam create-role --role-name $PROFILE_NAMESPACE-$CLUSTER_NAME-role --assume-role-policy-document file://trust.json aws --region $CLUSTER_REGION iam put-role-policy --role-name $PROFILE_NAMESPACE-$CLUSTER_NAME-role --policy-name kf-$PROFILE_NAMESPACE-pipeline-s3 --policy-document file://s3_policy.json
-
Create a user in your configured auth provider (e.g. Cognito or Dex).
Export the user email as env variable, e.g.
user@example.com
export PROFILE_USER=""
-
Create a Profile using the
PROFILE_NAMESPACE
.
Note: annotateOnly has been set to true. This means that the Profile Controller will not mutate your IAM Role and Policy.
cat <<EOF > profile_iam.yaml
apiVersion: kubeflow.org/v1
kind: Profile
metadata:
name: ${PROFILE_NAMESPACE}
spec:
owner:
kind: User
name: ${PROFILE_USER}
plugins:
- kind: AwsIamForServiceAccount
spec:
awsIamRole: $(aws iam get-role --role-name $PROFILE_NAMESPACE-$CLUSTER_NAME-role --output text --query 'Role.Arn')
annotateOnly: true
EOF
kubectl apply -f profile_iam.yaml