jevrecipes

Recipe catalog / passage-compare

Compare two passages for a question

Which of firstPassage and secondPassage better helps answer question?

You need a head-to-head preference between two retrieved passages, for tie-breaking, evaluation data, or reranker calibration.

Explore this recipe interactively ยท Source and implementation guide

Use passage-compare in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { passageCompare } from 'jev-recipes/passage-compare';

const result = await passageCompare({
  "question": "How long do refunds take to appear on a card?",
  "firstPassage": "Refunds are issued to the original payment method and typically appear within 5 to 10 business days, depending on the card issuer.",
  "secondPassage": "You can request a refund from the Orders page by selecting the item and choosing Return.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

FieldTypeNeeded
questionstringRequired
firstPassagestringRequired
secondPassagestringRequired
minConfidencenumberOptional
Full input and result schemas
{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "question": {
        "type": "string"
      },
      "firstPassage": {
        "type": "string"
      },
      "secondPassage": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "question",
      "firstPassage",
      "secondPassage"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "verdict": {
        "type": "string",
        "enum": [
          "first",
          "second",
          "tie",
          "neither",
          "unclear"
        ]
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string",
          "enum": [
            "first",
            "second",
            "tie",
            "neither",
            "unclear"
          ]
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        },
        "required": [
          "first",
          "second",
          "tie",
          "neither",
          "unclear"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "verdict",
      "confidence",
      "probabilities"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo passage-compare.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "verdict": "first",
  "confidence": 0.9,
  "probabilities": {
    "first": 0.9,
    "second": 0.03,
    "tie": 0.03,
    "neither": 0.02,
    "unclear": 0.02
  }
}

Evaluation evidence

Fixture only

No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Related recipes