Structured evaluation

Implementation target. This guide specifies the intended product surface. The current frontend uses local samples; it does not call an inference service or collect funds. An endpoint is enabled only after its adapter, ownership, usage, and payment behavior are qualified.

Planned API origin. Examples use https://noemad.dev/v1, the intended product API path. The API is not connected in this frontend preview. MODEL_ID_FROM_DISCOVERY and NOEMAD_API_KEY are placeholders, not published credentials or model IDs. A product API key identifies an account; a separate payment-aware transport must handle spending authorization.

A separate inference contract

The proposed POST /v1/structured-evaluation accepts a state and named questions. It is a dedicated capability, distinct from chat, embeddings, rerank, and schema-constrained text output. The public route name is an explicit Noemad adaptation.

The state may be text or a structured JSON object. Send 1–64 questions, using stable IDs that the response preserves under answers. The archived reference model is Jev 1.13; it is a catalog record rather than an active route.

Question types

Keep the exact typed contract instead of translating it into ordinary chat messages. Probability-like values are model outputs and require calibration before they drive consequential decisions.

Request example

{
  "model": "MODEL_ID_FROM_DISCOVERY",
  "state": "A customer says the payment failed three times and asks for help.",
  "questions": {
    "urgent": {
      "type": "noul",
      "instructions": "Does this require prompt attention?"
    },
    "team": {
      "type": "choice",
      "instructions": "Which team should handle it?",
      "criteria": {
        "billing": "Payment questions",
        "technical": "Product defects"
      }
    },
    "frustration": {
      "type": "score",
      "instructions": "Estimate the stated frustration.",
      "criteria": [
        "Calm",
        "Frustrated",
        "Angry"
      ]
    }
  }
}

Response example

{
  "model": "MODEL_ID_FROM_DISCOVERY",
  "answers": {
    "urgent": {
      "type": "noul",
      "noul": 0.96
    },
    "team": {
      "type": "choice",
      "choice": "billing",
      "confidence": 0.9,
      "probabilities": {
        "billing": 0.94,
        "technical": 0.06
      }
    },
    "frustration": {
      "type": "score",
      "score": 1.2,
      "confidence": 0.7,
      "legend": {
        "0": "Calm",
        "1": "Frustrated",
        "2": "Angry"
      },
      "probabilities": {
        "0": 0,
        "1": 0.8,
        "2": 0.2
      }
    }
  },
  "usage": {
    "input_tokens": 100,
    "output_tokens": 30
  }
}

This is an illustrative response. Validate every expected question ID, declared type, numeric range, option key, score legend, and output-size bound. Missing answers are errors, not implicit false values.

Validation and recovery

Reject undeclared fields where the profile requires a strict schema. Preserve usage and outcome even if a temporary offer results in no charge. A timeout or malformed response is not a successful evaluation and should not be replaced silently with a new paid attempt.

If evaluation results trigger workflow actions, the application must enforce its own authorization and review policy. Model confidence cannot expand a signer’s budget, modify access controls, or approve a financial transfer.

Zero-charge must be truthful

A historical free offer is not a current Noemad price or an indefinite guarantee. A verified rate and a live quote determine actual terms. If the calculated amount is zero, close the operation as zero_charge; do not invent an onchain transaction for display.