Restrict number of concurrent jobs and/or instances

Restrict number of concurrent running jobs

Configure max_running_jobs to limit the number of jobs running in parallel for a given queue Considering /apps/soca/$SOCA_CONFIGURATION/cluster_manager/settings/queue_mapping.yml

queue_type:
  compute:
    queues: ["normal"]
    max_running_jobs: 5 
    ... 
  test:
    queues: ["test"]
    max_running_jobs: 30

In this example, only 5 jobs can be running at the same time in the "normal" queue, and 30 in the "test" queue. If a job cannot start because of this parameter, an error_message will be visible when running "qstat -f" or via the web interface

Restrict number of provisioned instances

Configure max_provisioned_instances to limit the number of instances that can be provisioned for a given queue

queue_type:
  compute:
    queues: ["normal"]
    max_provisioned_instances: 10 
    ... 
  test:
    queues: ["test"]
    max_provisioned_instances: 20

In this example, you cannot have more than 10 instances provisioned for "normal" queue, and 20 for the "test" queue. If a job cannot start because of this parameter, an error_message will be visible when running "qstat -f" or via the web interface

Info

Limit apply to all type of instance. Configure allowed_instance_types / excluded_instance_types if you also want to limit the type of EC2 instance than can be provisioned).