jevrecipes

Recipe catalog / policy-severity

Grade a policy violation

How severely does content violate the supplied policy, on a five-level rubric?

You need a graded severity against your own written policy to choose between allow, flag, hide, or escalate.

Explore this recipe interactively ยท Source and implementation guide

Use policy-severity in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { policySeverity } from 'jev-recipes/policy-severity';

const result = await policySeverity({
  "content": "If you buy from this seller you deserve to get scammed, idiot. Everyone knows their address is 14 Elm Street, go tell them yourself.",
  "policy": "Community posts may criticize sellers and products. Posts must not insult other members, and must not share a private individual's home address or other personal contact details.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

FieldTypeNeeded
contentstringRequired
policystringRequired
minConfidencenumberOptional
Full input and result schemas
{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "content": {
        "type": "string"
      },
      "policy": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "content",
      "policy"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "score": {
        "type": "number",
        "minimum": 0
      },
      "level": {
        "type": "integer",
        "minimum": 0,
        "maximum": 9007199254740991
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string"
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        }
      },
      "severity": {
        "type": "string",
        "enum": [
          "none",
          "minor",
          "moderate",
          "serious",
          "severe"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "score",
      "level",
      "confidence",
      "probabilities",
      "severity"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo policy-severity.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "score": 3.02,
  "level": 3,
  "confidence": 0.82,
  "probabilities": {
    "0": 0.01,
    "1": 0.02,
    "2": 0.1,
    "3": 0.82,
    "4": 0.05
  },
  "severity": "serious"
}

Evaluation evidence

Fixture only

No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

Related recipes