jevrecipes

Recipe catalog / runbook-fit

Check whether a runbook applies to an incident

Does runbook address the symptoms and component described in incident?

You retrieve candidate runbooks for a live incident and need to filter out ones that cover a different component or a different failure mode before surfacing them to the responder.

Explore this recipe interactively ยท Source and implementation guide

Use runbook-fit in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { runbookFit } from 'jev-recipes/runbook-fit';

const result = await runbookFit({
  "incident": "Consumer lag on the notifications-worker Kafka group has climbed past 500k messages over the last 20 minutes. Email and push notifications are arriving up to 40 minutes late. No deploys to the worker today.",
  "runbook": "Runbook: notifications-worker consumer lag. Trigger: consumer lag alert for group notifications-worker, or user reports of delayed email or push. Steps: 1) Check worker pod count and restart counts in the notifications namespace. 2) Check SMTP and APNs error rates on the delivery dashboard. 3) If pods are healthy and downstream is clean, scale the deployment to twice its current replicas. 4) If lag keeps growing, check partition skew on the notifications topic.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

FieldTypeNeeded
incidentstringRequired
runbookstringRequired
minConfidencenumberOptional
Full input and result schemas
{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "incident": {
        "type": "string"
      },
      "runbook": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "incident",
      "runbook"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "probability": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "verdict": {
        "type": "string",
        "enum": [
          "applies",
          "inapplicable"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "probability",
      "confidence",
      "verdict"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo runbook-fit.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "probability": 0.93,
  "confidence": 0.93,
  "verdict": "applies"
}

Evaluation evidence

Fixture only

No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Related recipes