jevrecipes

Recipe catalog / draft-compare

Compare two drafts

Which draft better satisfies request under rubric?

You have two drafts and want to choose the better fit for a request and rubric.

Explore this recipe interactively ยท Source and implementation guide

Use draft-compare in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { draftCompare } from 'jev-recipes/draft-compare';

const result = await draftCompare({
  "request": "Explain how to reset a password.",
  "firstDraft": "Select Forgot password on the sign-in page.",
  "secondDraft": "Contact billing to download an invoice.",
  "rubric": "Prefer a direct answer to the requested task."
});
console.log(result);

Input contract

FieldTypeNeeded
requeststringRequired
firstDraftstringRequired
secondDraftstringRequired
rubricstringRequired
minConfidencenumberOptional
Full input and result schemas
{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "request": {
        "type": "string"
      },
      "firstDraft": {
        "type": "string"
      },
      "secondDraft": {
        "type": "string"
      },
      "rubric": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "request",
      "firstDraft",
      "secondDraft",
      "rubric"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "verdict": {
        "type": "string",
        "enum": [
          "first",
          "second",
          "tie",
          "neither",
          "unclear"
        ]
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string",
          "enum": [
            "first",
            "second",
            "tie",
            "neither",
            "unclear"
          ]
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        },
        "required": [
          "first",
          "second",
          "tie",
          "neither",
          "unclear"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "verdict",
      "confidence",
      "probabilities"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo draft-compare.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "verdict": "first",
  "confidence": 0.96,
  "probabilities": {
    "first": 1,
    "second": 0,
    "tie": 0,
    "neither": 0,
    "unclear": 0
  }
}

Evaluation evidence

Fixture only

No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Related recipes