Skip to main content

Evaluators & LLM Judges

Status

Owner: TBD · Status: draft · Canonical source: flezi.app.frontend.main/src/pages/Evaluators.tsx

Who is this for?​

AI Engineers configuring automated grading algorithms.

What can you do?​

  • Set up automated LLM-as-a-judge scorers.
  • Define rating scales (1–5 stars or binary Pass/Fail) with explicit evaluation rubrics.
  • Tailor judge prompts to check compliance with corporate tone, guidelines, or factual groundings.

Steps​

  1. In Governance, select Evaluators.
  2. Click Create Evaluator.
  3. Select an evaluation metric type (Semantic Similarity, Hallucination, Tool Selection).
  4. Specify the judge model (e.g. GPT-4o) and grading rubric instructions.
  5. Click Save Evaluator.