Recipe catalog / novelty-claim-level

Grade how strongly a statement claims novelty

How strongly does the wording of statement claim novelty for a contribution, from no novelty claim to first-ever or unprecedented?

You are reviewing abstracts, introductions, grant text, or press copy and want to flag how hard the wording sells the contribution as new, so overclaiming can be checked against the actual related work.

Explore this recipe interactively · Source and implementation guide

Use novelty-claim-level in TypeScript

Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.

import { noveltyClaimLevel } from 'jev-recipes/novelty-claim-level';

const result = await noveltyClaimLevel({
  "statement": "To the best of our knowledge, this is the first study to demonstrate that the effect transfers across species, a result no existing theoretical framework predicted.",
  "minConfidence": 0.8
});
console.log(result);

Input contract

{
  "input": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "statement": {
        "type": "string"
      },
      "minConfidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      }
    },
    "required": [
      "statement"
    ]
  },
  "result": {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "properties": {
      "model": {
        "type": "string"
      },
      "usage": {
        "type": "object",
        "properties": {
          "input_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          },
          "output_tokens": {
            "type": "integer",
            "minimum": 0,
            "maximum": 9007199254740991
          }
        },
        "required": [
          "input_tokens",
          "output_tokens"
        ],
        "additionalProperties": false
      },
      "status": {
        "type": "string",
        "enum": [
          "ready",
          "review"
        ]
      },
      "score": {
        "type": "number",
        "minimum": 0
      },
      "level": {
        "type": "integer",
        "minimum": 0,
        "maximum": 9007199254740991
      },
      "confidence": {
        "type": "number",
        "minimum": 0,
        "maximum": 1
      },
      "probabilities": {
        "type": "object",
        "propertyNames": {
          "type": "string"
        },
        "additionalProperties": {
          "type": "number",
          "minimum": 0,
          "maximum": 1
        }
      },
      "novelty": {
        "type": "string",
        "enum": [
          "none",
          "incremental",
          "notable",
          "substantial",
          "unprecedented"
        ]
      }
    },
    "required": [
      "model",
      "usage",
      "status",
      "score",
      "level",
      "confidence",
      "probabilities",
      "novelty"
    ],
    "additionalProperties": false
  }
}

Saved example result

This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo novelty-claim-level.

{
  "model": "demo-fixture",
  "usage": {
    "input_tokens": 0,
    "output_tokens": 0
  },
  "status": "ready",
  "score": 3.86,
  "level": 4,
  "confidence": 0.9,
  "probabilities": {
    "0": 0,
    "1": 0.01,
    "2": 0.02,
    "3": 0.07,
    "4": 0.9
  },
  "novelty": "unprecedented"
}

Evaluation evidence

Use the evaluation guide to measure this decision on your own labeled cases.

Limitations

• Grades the wording only. It does not know the literature and cannot say whether the claimed novelty is real.
• A statement can claim little novelty and still be original, or claim a first and be correct; comparing the claim against related work is the reviewer's job.
• Judges one statement at a time. Aggregating over a whole paper belongs in application code.

Related recipes

• certainty-match: Use certainty-match to check whether stated confidence matches the evidence, rather than how much novelty the wording claims.
• outcome-framing: Use outcome-framing to label how a result is framed as gain or loss, rather than how new it is said to be.