Skip to content
UltimateTeam.ai Docs

Evaluate prompt version

POST
/api/v1/tuning/prompts/{version_id}/evaluate
curl --request POST \
  --url https://api.ultimateteam.ai/api/v1/tuning/prompts/example/evaluate \
  --header 'X-API-Key: <X-API-Key>'

Run evaluation suite on a prompt version.

Authorizations

Section titled “Authorizations”

Parameters

Section titled “ Parameters ”

Path Parameters

Section titled “Path Parameters”
version_id
required
Version Id
string

Responses

Section titled “ Responses ”

200

Section titled “200”

Successful Response

Media typeapplication/json
PromptEvalSuiteResult

Aggregated result of evaluating a prompt version.

object
avg_overall
required
Avg Overall
number
avg_scores
required
Avg Scores
object
key
additional properties
number
case_results
required
Case Results
Array<object>
PromptEvalResult

Result of evaluating a prompt version against one case.

object
case_id
required
Case Id
string format: uuid
overall_score
required
Overall Score
number
scores
required
Scores
object
key
additional properties
number
version_id
required
Version Id
string format: uuid
version_id
required
Version Id
string format: uuid
Examplegenerated
{
  "avg_overall": 1,
  "avg_scores": {
    "additionalProperty": 1
  },
  "case_results": [
    {
      "case_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0",
      "overall_score": 1,
      "scores": {
        "additionalProperty": 1
      },
      "version_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0"
    }
  ],
  "version_id": "2489E9AD-2EE2-8E00-8EC9-32D5F69181C0"
}

422

Section titled “422”

Validation Error

Media typeapplication/json
HTTPValidationError
object
detail
Detail
Array<object>
ValidationError
object
ctx
Context
object
input
Input
loc
required
Location
Array
msg
required
Message
string
type
required
Error Type
string
Examplegenerated
{
  "detail": [
    {
      "ctx": {},
      "input": "example",
      "loc": [
        "example"
      ],
      "msg": "example",
      "type": "example"
    }
  ]
}