Gen AI Evaluation Toolkit on AWS

v2.0.0

Copyright © 2026 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon’s trademarks and trade dress may not be used in connection with any product or service that is not Amazon’s, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.

Interface: SubmitEvaluationJobInput¶

Extended by¶

SubmitEvaluationJobCommandInput

Properties¶

app?¶

optional app?: AppConfig

App invocation configuration — function selection, runtime parameters, and evaluator override

continueBufferMs?¶

optional continueBufferMs?: number

Buffer time in milliseconds before Lambda timeout to trigger CONTINUE checkpoint. Internal testing/tuning parameter — not for end-user use.

evaluatorName¶

evaluatorName: string

Name of the evaluator

experimentName?¶

optional experimentName?: string

Experiment name for grouping related jobs and reports

heartbeatSeconds?¶

optional heartbeatSeconds?: number

Heartbeat timeout in seconds for Step Functions task states. The evaluator must send a heartbeat within this interval or the task is marked as timed out.

processLocally?¶

optional processLocally?: boolean

When true, the caller will process the job locally — suppresses EventBridge event emission

reportId?¶

optional reportId?: string

Report ID to store results in (auto-created if not provided)

reportName?¶

optional reportName?: string

Name for the auto-created report (ignored if reportId is provided)

scorers?¶

optional scorers?: Record\<string, ScorerConfig | null>

Scorer configurations keyed by scorer name

testCases¶

testCases: TestDataSource

Source of test cases for evaluation

Table of contents