jevrecipes

Recipe catalog / promise-check

Check reply commitments

Does reply promise actions or outcomes beyond allowedCommitments?

You need to catch commitments in a reply that exceed what is allowed.

Explore this recipe interactively ยท Source and implementation guide

Use promise-check in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { promiseCheck } from 'jev-recipes/promise-check';

const result = await promiseCheck({
  "reply": "Your refund is guaranteed.",
  "allowedCommitments": "We may promise to review a refund request. We cannot promise approval."
});
console.log(result);

Input contract

FieldTypeNeeded
replystringRequired
allowedCommitmentsstringRequired
minConfidencenumberOptional
Full input and result schemas
{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "reply": {
        "type": "string"
      },
      "allowedCommitments": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "reply",
      "allowedCommitments"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "verdict": {
        "type": "string",
        "enum": [
          "within_commitments",
          "unsupported",
          "unclear"
        ]
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string",
          "enum": [
            "within_commitments",
            "unsupported",
            "unclear"
          ]
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        },
        "required": [
          "within_commitments",
          "unsupported",
          "unclear"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "verdict",
      "confidence",
      "probabilities"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo promise-check.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "verdict": "unsupported",
  "confidence": 0.96,
  "probabilities": {
    "within_commitments": 0,
    "unsupported": 1,
    "unclear": 0
  }
}

Evaluation evidence

Earlier-evaluator measurement

jev-1.13.0 / 2026-09-27 / 40 held-out cases

Scoring revision 1.

100%All-case accuracy
0Sent for review
0Failed calls / cases

40 ready decisions, with 100% accuracy among those decisions.

95% case-level interval: 91% to 100%. Related synthetic cases are correlated.

Measured on these synthetic cases

This measurement uses an earlier or unverified recipe or evaluator version. Rerun with the current recipe and evaluator before treating these numbers as current.

Full report and missesDataset and labels

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Related recipes