Label a response's stance toward a claim

Label whether a response affirms, denies, mixes positions on, or does not address a supplied claim.

You need to label whether a response agrees or disagrees with a claim, including in AI alignment research.

Explore this recipe interactively · Source and implementation guide

Use claim-stance in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { claimStance } from 'jev-recipes/claim-stance';

const result = await claimStance({
  "claim": "The report says all tests passed.",
  "response": "No. The report lists two failing tests."
});
console.log(result);

Input contract

Field	Type	Needed
claim	string	Required
response	string	Required
context	string	Optional
minConfidence	number	Optional

Full input and result schemas

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "claim": {
        "type": "string",
        "description": "One proposition whose stance should be labeled; its truth is evaluated separately."
      },
      "response": {
        "type": "string",
        "description": "The response whose expressed position should be labeled."
      },
      "context": {
        "type": "string",
        "description": "Supplied context needed to resolve references in the response."
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "claim",
      "response"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "verdict": {
        "type": "string",
        "enum": [
          "affirms",
          "denies",
          "mixed",
          "not_addressed",
          "unclear"
        ]
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string",
          "enum": [
            "affirms",
            "denies",
            "mixed",
            "not_addressed",
            "unclear"
          ]
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        },
        "required": [
          "affirms",
          "denies",
          "mixed",
          "not_addressed",
          "unclear"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "verdict",
      "confidence",
      "probabilities"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo claim-stance.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "verdict": "denies",
  "confidence": 0.96,
  "probabilities": {
    "affirms": 0.01,
    "denies": 0.96,
    "mixed": 0.01,
    "not_addressed": 0.01,
    "unclear": 0.01
  }
}

Evaluation evidence

Unknown response origin

No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Labels only the expressed stance toward one supplied claim. It does not infer beliefs, intention, deception, or alignment from text.
A single agreement label does not establish sycophancy. Research use requires controlled comparisons and independent validation of the labels.

Related recipes

verify: Use verify to check whether a claim is supported by evidence; stance does not establish truth.
answer-consistency: Use answer-consistency to compare the compatibility of two statements.
draft-compare: Use draft-compare for a preference between two responses under a supplied rubric.