Recipe catalog / objective-fit

Check question alignment to a learning objective

Does question assess the skill or knowledge stated in objective, rather than something adjacent?

You generate or review quiz and exam items and want to catch questions that test recall or a neighboring topic instead of the stated objective.

Explore this recipe interactively · Source and implementation guide

Use objective-fit in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { objectiveFit } from 'jev-recipes/objective-fit';

const result = await objectiveFit({
  "objective": "Students will be able to apply Newton's second law to compute the net force on an object given its mass and acceleration.",
  "question": "State Newton's second law of motion in words and give the name of the unit of force.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "objective": {
        "type": "string"
      },
      "question": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "objective",
      "question"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "probability": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "verdict": {
        "type": "string",
        "enum": [
          "assesses",
          "misses"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "probability",
      "confidence",
      "verdict"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo objective-fit.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "probability": 0.08,
  "confidence": 0.92,
  "verdict": "misses"
}

Evaluation evidence

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

• Judges alignment only; an aligned question can still be too easy, too hard, or ambiguous.
• Depends on how precisely objective states the skill and cognitive level.
• Does not check whether the question has a correct answer or whether its answer key is right.

Related recipes

• instruction-fit: Use instruction-fit to check whether a written instruction covers a given task and context.
• query-specificity: Use query-specificity to check whether a question identifies a focused information need.