Gen AI Evaluation Toolkit on AWS

v2.0.0

Copyright © 2026 Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon’s trademarks and trade dress may not be used in connection with any product or service that is not Amazon’s, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.

Interface: TestResult¶

Result of evaluating a single test case

Properties¶

appMetrics?¶

optional appMetrics?: DocumentType

Metrics from app invocation (latency, tokens, etc)

context?¶

optional context?: DocumentType

Additional context for the test case. Can contain metadata such as flags indicating human review needed, artifacts providing evidence for agent-as-judge evaluation, or any other structured data needed for evaluation.

createdAt?¶

optional createdAt?: Date

Timestamp when result was created

error?¶

optional error?: string

Error message if app invocation failed

expected?¶

optional expected?: DocumentType

Expected output of the Gen AI Application for the given input. Can be any structured data including simple text responses, expected tool calls, expected topics, or any combination of expected behaviors.

input?¶

optional input?: DocumentType

The input to the Gen AI Application. Can be any structured data including prompts, context, and parameters.

metadata?¶

optional metadata?: DocumentType

Optional free-form metadata set by the caller. Carried through verbatim to results and ignored by the pipeline — use it to tag test cases (source, split, tenant, run label, etc.) for grouping and debugging.

output?¶

optional output?: DocumentType

Output from app invocation

reportId¶

reportId: string

Report this result belongs to

resultId¶

resultId: string

Unique identifier for this result

scores?¶

optional scores?: Record\<string, ScoreOutput>

Scores from evaluators, keyed by scorer name

testcaseId?¶

optional testcaseId?: string

Optional reference to source test case

updatedAt?¶

optional updatedAt?: Date

Timestamp when result was last updated

Table of contents