Skip to content

Prerequisites

Before deploying KNFSD to Amazon Web Services, there are a number of prerequisites that you should ensure are met. These are detailed below. Once you have verified the prerequisites, you can continue with the infrastructure deployment steps.

We assume bash and jq are installed by default on most Linux and macOS systems. Windows machines should be checked.

Requirements

Git

Git is required to clone the KNFSD-File-Cache GitHub repository. If you are deploying from Windows, you should ensure that you have git-bash installed (which by default is included with Git for Windows).

AWS CLI

You should ensure that you have AWS CLI installed and credentials configured on your machine.

Terraform

You should ensure that you have Terraform v1.2.9 or newer installed.

Docker Desktop (Optional)

NOTE: Ensure you start Docker Desktop at least once to accept the terms and conditions.

If you wish to use the .devcontainer/prod configuration (which is recommended and already contains all required components as well as an optimized environment for KNFSD build and deployment), you should install Docker Desktop (which includes docker engine) and the VS Code Dev Containers Extension.

For KNFSD developers, you can use either VS Code or Cursor to run a .devcontainer/dev workspace. See developer.md for detailed steps.

NOTE: Cursor now maintains its own Dev Containers Extension. You should use it instead of the VS Code Dev Containers Extension.

AWS SSM Session Manager Plugin (Optional)

You should ensure that you have the AWS SSM Session Manager Plugin installed if either of the following applies:

  • You build the KNFSD AMI with the Packer variable SSH_INTERFACE = "session_manager", which tunnels the Packer build connection over AWS Systems Manager instead of inbound SSH. See AWS SSM Session Manager.
  • You wish to run the smoke-tests module.

This is already installed in both the .devcontainer/prod and .devcontainer/dev configurations. Installation instructions are available for Windows, macOS, and Linux. Packer and the test harness both invoke the plugin as a subprocess, so it must be on your PATH. Verify with:

session-manager-plugin --version

AWS Service Quotas

By default, a new AWS account will have 5 vCPUs (On-Demand) available. This is insufficient for the c6in.2xlarge instance type used in the Packer build process, which requires 8 vCPUs or the default i3en.6xlarge instance type used as the KNFSD proxy, which requires 24 vCPUs. For each additional KNFSD node, you will need to multiply the number of vCPUs by the maximum number of nodes you plan to deploy. You should also add a 25% buffer to the total number of vCPUs required.

You will need to request a quota increase for the total number of On-Demand vCPUs required. Please note that EC2 Spot vCPU service quotas are not applicable to this solution, but will be necessary for your EC2 compute instances.

See AWS Service Quotas for more information.

KNFSD VM Image

Before you can deploy KNFSD in your Amazon Web Services account, you first need to build an AMI (Amazon Machine Image). The process for building this image via Packer is documented here.

External FSID database

If using the recommended FSID_MODE="external", the knfsd proxy instances need to be able to reach the Amazon DynamoDB regional API endpoint. DynamoDB is a fully managed HTTPS API: there is no database host, subnet, or security group to configure, and access is controlled entirely through IAM.

If the proxy subnets have internet connectivity (directly or via a NAT gateway), no additional setup is needed. For private subnets without internet access, add an Amazon DynamoDB Gateway VPC endpoint; see VPC Endpoints.

Security Groups

The Terraform module(s) will automatically create a Security Group (firewall) for each of the following resources (when necessary):

  • Auto Scaling Group
  • Network Load Balancer (if TRAFFIC_MODE = "loadbalancer")
  • Lambda function

However, it will not create any other security groups. You should make sure that you implement security groups (and any NACLs) to allow:

  • KNFSD Node --> Source NFS Server communication (consult your network administrator for guidance on your VPN/DirectConnect to VPC configuration)
  • NFS Clients --> KNFSD Node communication (see security-groups.md for guidance on how this is configured)

Metrics

KNFSD supports a range of metrics which are automatically exported into Amazon CloudWatch.

NOTE: This step only needs to be ran once per AWS account.

An optional metrics Terraform module will automatically create an Amazon CloudWatch custom dashboard, which can be used to easily understand KNFSD performance. Instructions on how to do this are available here.

By default, it is assumed your subnet has internet connectivity to connect to AWS services. Connectivity could be via a NAT gateway and route table to allow access to the internet from your private subnet or via another network construct such as AWS Transit Gateway, which is beyond the scope of this documentation.

AWS PrivateLink is a feature that allows you to create a private connection between your VPC and AWS services. This allows you to securely access AWS services without exposing your resources to the public internet.

When deploying KNFSD File Cache in a private subnet without any internet connectivity, VPC endpoints (PrivateLink) are required for AWS service access. See VPC Endpoints for detailed setup instructions. In this situation, you will need to ensure the VPC endpoints exist BEFORE deploying KNFSD modules.

IAM Permissions

See IAM Permissions for detailed information on the IAM permissions required for the KNFSD File Cache solution.

Packer

The Packer AMI build does not create IAM resources, so it requires an IAM instance profile to be pre-created and referenced by name via the Packer IAM_INSTANCE_PROFILE variable. This is only required when building over AWS SSM Session Manager (SSH_INTERFACE = "session_manager") or when enabling EC2 build status tagging (TAG_BUILD_STATUS = true). See Packer build instance profile for the permissions policy, the trust policy, and the commands to create it.

Terraform

The Terraform module(s) will automatically create an IAM instance profile, role(s) and least-privilege policies for all resources within this solution to operate correctly. Please review the following Terraform files to understand the permissions that are created:

Some organizations centrally manage networking and IAM, and deny the deploying principal the ability to create security groups or IAM resources. KNFSD supports this: pre-create the resource yourself and set the matching Terraform variable, and the module skips creating it. Each variable lets you drop create-permissions rather than add new ones:

  • EXISTING_SECURITY_GROUP_ID (root and vpc-endpoints modules)
  • EXISTING_INSTANCE_PROFILE_NAME
  • EXISTING_LAMBDA_ROLE_ARN (dns_round_robin module)
  • FSID_DATABASE_IAM_POLICY (root and database modules)

See Deploying under restrictive IAM for what each variable lets you omit from the IAM policies.

AWS Services

For reference, the following AWS services are used in this solution (and should be available in all new AWS regions as well as GovCloud (US) and China (CN) regions):

Optional: