Model Evaluations
Evaluation runs and benchmark results for deployed models. These endpoints let you create evaluation jobs, retrieve results, manage custom evaluation definitions, and track evaluation progress.
Standard Evaluations
Get evaluation data
Returns evaluation results and benchmark scores for a fine-tuned model identified by its numeric model ID.
Get evaluation data by model name
Returns evaluation results and benchmark scores for a fine-tuned model identified by its model name.
Custom Evaluations
Create custom evaluation
Creates a new custom evaluation definition with a natural-language scoring prompt that defines how outputs are compared.
Get custom evaluation
Returns all custom evaluation definitions created by the authenticated user, including the evaluation name and scoring prompt for each.
Get custom evaluation results
Returns all recorded results for a custom evaluation run, including per-run win/draw/loss counts and aggregate improvement percentage.
Start custom evaluation
Starts a custom evaluation run that compares two models head-to-head using the specified evaluation criteria.