Skip to content

Autoscaling

CRITICAL: To use Autoscaling you MUST have metrics enabled (var.ENABLE_METRICS = true) as they are used as a scaling metric.

This Terraform module supports configuring autoscaling for the KNFSD Auto Scaling Group. Scaling based on the standard metric of CPU Usage is not optimal for the caching use case. Instead the custom metric knfsd/nfs_connections is used for triggering an autoscaling event.

Autoscaling can be enabled by setting the ENABLE_KNFSD_AUTOSCALING environment variable to true (defaults to false). There are also some other configuration options detailed in the Configuration Variables section (such as how many NFS connections a KNFSD node should be handling before a scale-up).

To avoid interruptions to existing NFS client mounts, and by extension render operations the Auto Scaling Group (ASG) behaviour is set to SCALE UP ONLY. When a KNFSD client exceeds the number of connections defined in the KNFSD_AUTOSCALING_NFS_CONNECTIONS_THRESHOLD variable a new instance will be added to the KNFSD ASG. If the number of KNFSD connections subsequently falls significantly the KNFSD cluster will not automatically scale down. This is not an AWS limitation but an intentional design consideration to avoid:

  • Loss of FS-Cache data that would need to be re-pulled on scale up
  • Interruption to existing NFS client connections

You can change this behaviour if you wish, but it is not recommended.

There is a slight delay for metric ingestion (1-2 mins) and then for a new node to spin up and initialise (~2 mins). When a scaling event occurs new traffic will continue to be sent to the existing healthy nodes in the cluster until there is a new node ready to handle the connections. It is therefore recommended that you set your KNFSD_AUTOSCALING_NFS_CONNECTIONS_THRESHOLD slightly lower than the maximum number of connections a single KNFSD node can handle. This will start the scaling event early and make sure a new node is ready before your existing nodes become overloaded.