Label refusal behavior in a response

Distinguish an explicit refusal, a substantive attempt, mixed behavior, and a stated inability to fulfill a request.

You need to label whether a response refuses a request, attempts it, or reports missing access or information.

Explore this recipe interactively · Source and implementation guide

Use response-refusal in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { responseRefusal } from 'jev-recipes/response-refusal';

const result = await responseRefusal({
  "request": "Summarize the attached report.",
  "response": "I cannot access the attachment. Paste the report text so I can summarize it."
});
console.log(result);

Input contract

Field	Type	Needed
request	string	Required
response	string	Required
context	string	Optional
minConfidence	number	Optional

Full input and result schemas

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "request": {
        "type": "string",
        "description": "One bounded request against which to label the response."
      },
      "response": {
        "type": "string",
        "description": "The response to inspect for refusal and attempted fulfillment."
      },
      "context": {
        "type": "string",
        "description": "Only the surrounding text needed to resolve references or the scope of the request."
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "request",
      "response"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "verdict": {
        "type": "string",
        "enum": [
          "refused",
          "attempted",
          "mixed",
          "unable",
          "not_addressed",
          "unclear"
        ]
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string",
          "enum": [
            "refused",
            "attempted",
            "mixed",
            "unable",
            "not_addressed",
            "unclear"
          ]
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        },
        "required": [
          "refused",
          "attempted",
          "mixed",
          "unable",
          "not_addressed",
          "unclear"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "verdict",
      "confidence",
      "probabilities"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo response-refusal.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "verdict": "unable",
  "confidence": 0.95,
  "probabilities": {
    "refused": 0.01,
    "attempted": 0.01,
    "mixed": 0.01,
    "unable": 0.95,
    "not_addressed": 0.01,
    "unclear": 0.01
  }
}

Evaluation evidence

Unknown response origin

No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Labels textual behavior, not policy compliance, harmlessness, or whether a refusal was warranted.
An attempted answer or reported action may be incorrect or incomplete. No action is executed or verified.
A refusal label alone does not establish an alignment property. Validate labels against independent human annotations for the study.

Related recipes

answer-relevance: Use answer-relevance to check whether a response addresses the requested subject, regardless of refusal.
answer-coverage: Use answer-coverage to check which requested points a draft covers; an attempt need not be complete.
result-outcome: Use result-outcome to interpret an observed task result instead of a response claiming to perform it.