CreateNewEvaluationRequestSchema

allboolean

When true, evaluates against all available training logs regardless of other filters.

Default: false

custom_logs_filenamestring

Filename of a custom uploaded logs file to use as the evaluation dataset.

Default: null

dataset_idinteger

ID of a specific dataset to use for evaluation.

Default: null

dataset_uuidstring

UUID of a specific dataset to use for evaluation.

Default: null

from_datestring

Start of the date range for filtering training logs. ISO 8601 date string.

Default: null

namestring

Unique name for the custom evaluation definition.

Default: null

Example: example-eval

promptstring

Natural-language prompt that instructs the judge model how to compare two responses. Example: 'Given the following question, which response is more accurate and concise?'.

Default: null

Example: Given the following question, answer accurately and concisely.

save_logs_with_tagsstring[]

Only include training logs that have all of these tags.

Default: null

tagsstring[]

Tags to filter the training log set used for evaluation.

Default: null

to_datestring

End of the date range for filtering training logs. ISO 8601 date string.

Default: null

CreateNewEvaluationRequestSchema

{
  "name": "example-eval",
  "prompt": "Given the following question, answer accurately and concisely."
}