Gen AI Evaluation Toolkit on AWS

v2.0.0

Copyright © 2026 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon’s trademarks and trade dress may not be used in connection with any product or service that is not Amazon’s, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.

Interface: EvaluationJob¶

An evaluation job tracked in DynamoDB (v2)

Properties¶

createdAt?¶

optional createdAt?: Date

Timestamp when the job was created

evaluatorName¶

evaluatorName: string

Name of the evaluator this job is for

experimentName?¶

optional experimentName?: string

Experiment name for grouping related jobs and reports

jobId¶

jobId: string

Unique identifier for this job

jobType¶

jobType: JobType

Type of job: FULL (app invoke + scoring), SCORE (scoring only), or APP_INVOKE (invocation only)

reportId¶

reportId: string

Report ID where results will be stored

scorers?¶

optional scorers?: Record\<string, ScorerConfig | null>

Which scorers to run and with what parameters, keyed by scorer name. Null value = run with defaults. These are configurations, not score results — actual scores are stored on individual test results.

sourceReportId?¶

optional sourceReportId?: string

Source report ID to copy results from for re-scoring (creates a new report)

testCases?¶

optional testCases?: TestDataSource

Source of test cases for evaluation

Table of contents