agenteval

Usage:

agenteval [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

agenteval init

Initialize a test plan.

Usage:

agenteval init [OPTIONS]

Options:

  --plan-dir TEXT  The destination directory for storing the test plan. If
                   unspecified, then the test plan is saved to the current
                   working directory.
  --help           Show this message and exit.

agenteval run

Run test plan.

Usage:

agenteval run [OPTIONS]

Options:

  --filter TEXT          Specifies the test(s) to run. Multiple tests should
                         be seperated using a comma. If unspecified, all tests
                         from the test plan will be run.
  --plan-dir TEXT        The directory where the test plan is stored. If
                         unspecified, then the current working directory is
                         used.
  --verbose              Controls the verbosity of the terminal logs.
  --num-threads INTEGER  Number of threads (and thus tests) to run
                         concurrently. If unspecified, number of threads will
                         be capped at 45.
  --work-dir TEXT        The directory where the test result and trace will be
                         generated. If unspecified, then the current working
                         directory is used.
  --help                 Show this message and exit.