Recipe catalog / alert-actionability

Grade how actionable an alert is

How actionable is alert for the on-call engineer who receives it, on a five-level rubric from pure noise to a guided first step?

You need to triage or audit alert text before paging someone: to route noisy alerts to a digest, to flag vague alerts for rewriting, or to rank a flood of alerts by how much they tell the responder.

Explore this recipe interactively · Source and implementation guide

Use alert-actionability in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { alertActionability } from 'jev-recipes/alert-actionability';

const result = await alertActionability({
  "alert": "HIGH: payments-gateway 5xx rate is 6.4% over the last 5 minutes (threshold 2%) in prod-eu-west-1. Upstream dependency stripe-proxy is returning 502s for 40% of calls.",
  "context": "payments-gateway is the public API in front of card processing. stripe-proxy is an internal sidecar that batches calls to Stripe.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "alert": {
        "type": "string"
      },
      "context": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "alert"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "score": {
        "type": "number",
        "minimum": 0
      },
      "level": {
        "type": "integer",
        "minimum": 0,
        "maximum": 9007199254740991
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string"
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        }
      },
      "actionability": {
        "type": "string",
        "enum": [
          "noise",
          "vague",
          "symptom",
          "located",
          "guided"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "score",
      "level",
      "confidence",
      "probabilities",
      "actionability"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo alert-actionability.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "review",
  "score": 2.96,
  "level": 3,
  "confidence": 0.78,
  "probabilities": {
    "0": 0,
    "1": 0.02,
    "2": 0.1,
    "3": 0.78,
    "4": 0.1
  },
  "actionability": "located"
}

Evaluation evidence

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

• Grades what the alert text tells the responder, not whether the alert is true or whether the named component is really at fault.
• Does not know your service topology or runbooks. Context can supply that, but the grade stays a judgment about the wording.
• Deduplication, paging policy, and escalation timers belong in application code.

Related recipes

• feedback-actionability: Use feedback-actionability for review comments and user feedback rather than operational alerts.
• issue-impact: Use issue-impact to grade how much a reported problem matters, rather than how well the alert tells you what to do about it.