Recipe catalog / rollback-signal

Check whether symptoms implicate a recent change

Do symptoms plausibly point at the recently deployed change as the cause, judged on the timing and scope each describes?

An incident is open shortly after a deploy and you need a fast read on whether the change is a plausible cause, to decide whether to propose a rollback or keep looking elsewhere.

Explore this recipe interactively · Source and implementation guide

Use rollback-signal in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { rollbackSignal } from 'jev-recipes/rollback-signal';

const result = await rollbackSignal({
  "symptoms": "Starting at 09:42 UTC the error rate on POST /orders jumped from 0.2% to 8%. Every failure is a null reference in PricingService.applyDiscount. GET endpoints and the cart service are unaffected.",
  "change": "Deployed at 09:40 UTC: PricingService now loads discount rules lazily from the new promotions table instead of the in-memory cache. Only the pricing module was touched; no schema or infrastructure changes.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "symptoms": {
        "type": "string"
      },
      "change": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "symptoms",
      "change"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "probability": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "verdict": {
        "type": "string",
        "enum": [
          "implicated",
          "unrelated"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "probability",
      "confidence",
      "verdict"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo rollback-signal.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "probability": 0.94,
  "confidence": 0.94,
  "verdict": "implicated"
}

Evaluation evidence

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

• Judges plausibility from the two descriptions, not root cause. A plausible match can still be a coincidence.
• Does not compare timestamps arithmetically or read diffs. Compute deploy-to-onset gaps in application code and pass them in the text.
• A negative verdict does not clear the change; it means the described timing and scope do not line up.

Related recipes

• causal-attribution: Use causal-attribution to classify how a text attributes a cause in general, rather than to check whether a specific change fits an incident.
• change-risk: Use change-risk before deploying to grade how likely a change is to cause trouble, rather than after the fact to check whether it did.