agenteval

Usage:

agenteval [OPTIONS] COMMAND [ARGS]...

Options:

  --help  Show this message and exit.

agenteval init

Initialize a test plan.

Usage:

agenteval init [OPTIONS]

Options:

  --plan-dir TEXT  The directory to store the test plan. If a directory is not
                   provided, the test plan will be saved to the current
                   working directory.
  --help           Show this message and exit.

agenteval run

Run test plan.

Usage:

agenteval run [OPTIONS]

Options:

  --filter TEXT          Specifies the test(s) to run, where multiple tests
                         should be seperated using a comma. If a filter is not
                         provided, all tests will be run.
  --plan-dir TEXT        The directory where the test plan is stored. If a
                         directory is not provided, the test plan will be read
                         from the current working directory.
  --verbose              Whether to enable verbose logging. Defaults to False.
  --num-threads INTEGER  Number of threads used to run tests concurrently. If
                         the number of threads is not provided, the thread
                         count will be set to the number of tests (up to a
                         maximum of 45 threads).
  --work-dir TEXT        The directory where the test result and trace will be
                         generated. If a directory is not provided, the assets
                         will be saved to the current working directory.
  --help                 Show this message and exit.