Recipe catalog / progress-stall

Detect a stalled agent

Does transcript, the recent agent steps, show the agent failing to make progress toward objective by repeating actions, circling, or reprocessing the same information?

You need a yes/no check on a running agent every few steps so a supervisor can interrupt a loop before it burns the remaining budget.

Explore this recipe interactively · Source and implementation guide

Use progress-stall in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { progressStall } from 'jev-recipes/progress-stall';

const result = await progressStall({
  "transcript": "Step 14: ran `npm test` -> 3 failures in date-utils.test.ts (timezone offset). Step 15: opened src/date-utils.ts, read formatDate. Step 16: ran `npm test` -> same 3 failures. Step 17: opened src/date-utils.ts, read formatDate again. Step 18: ran `npm test` -> same 3 failures. Step 19: opened src/date-utils.ts, read formatDate. Step 20: ran `npm test` -> same 3 failures.",
  "objective": "Make the date-utils test suite pass without changing the expected values in the tests.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "transcript": {
        "type": "string"
      },
      "objective": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "transcript",
      "objective"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "probability": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "verdict": {
        "type": "string",
        "enum": [
          "stalled",
          "progressing"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "probability",
      "confidence",
      "verdict"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo progress-stall.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "probability": 0.94,
  "confidence": 0.94,
  "verdict": "stalled"
}

Evaluation evidence

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

• Judges the window of steps supplied in transcript. A loop longer than the window, or progress made before it, is invisible.
• Reports that the agent is stalled, not why or what it should do instead. Interrupting, redirecting, or escalating is an application decision.

Related recipes

• repeated-attempt: Use repeated-attempt to check whether one new action is a retry of a specific earlier one, rather than whether a whole window of steps has stalled.
• step-progress: Use step-progress to grade how much a single observed result moved the objective forward.