Recipe catalog / report-facets
Label the facets of a progress report
Which of these does report include: an outcome statement, supporting evidence, blockers, a next step, open questions?
You need to check an agent progress report for the parts a supervisor expects before accepting it, routing it, or asking the agent to fill in what is missing.
Explore this recipe interactively ยท Source and implementation guide
Use report-facets in TypeScript
Install with npm install jev-recipes. Requires Node.js 22.9 or newer and ES modules. Set TYPESAFE_API_KEY in your server environment for live calls, which send input to TypeSafe and use API quota. See the installation guide.
import { reportFacets } from 'jev-recipes/report-facets';
const result = await reportFacets({
"report": "Migrated 4 of the 6 legacy endpoints to the new router; all 38 tests in tests/api pass (run log attached). The remaining two endpoints depend on the auth middleware rewrite, which is still blocked on the token format decision from the platform team. Next I will stub the middleware so the last two endpoints can be migrated behind a feature flag.",
"minConfidence": 0.8
});
console.log(result);Input contract
| Field | Type | Needed |
|---|---|---|
| report | string | Required |
| minConfidence | number | Optional |
Full input and result schemas
{
"input": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"report": {
"type": "string"
},
"minConfidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"report"
]
},
"result": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"model": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"input_tokens": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"output_tokens": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
}
},
"required": [
"input_tokens",
"output_tokens"
],
"additionalProperties": false
},
"status": {
"type": "string",
"enum": [
"ready",
"review"
]
},
"detected": {
"type": "array",
"items": {
"type": "string",
"enum": [
"statesOutcome",
"providesEvidence",
"statesBlockers",
"statesNextStep",
"raisesQuestions"
]
}
},
"labels": {
"type": "object",
"properties": {
"statesOutcome": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"ready",
"review"
]
},
"verdict": {
"type": "string",
"enum": [
"present",
"absent"
]
},
"probability": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"status",
"verdict",
"probability",
"confidence"
],
"additionalProperties": false
},
"providesEvidence": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"ready",
"review"
]
},
"verdict": {
"type": "string",
"enum": [
"present",
"absent"
]
},
"probability": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"status",
"verdict",
"probability",
"confidence"
],
"additionalProperties": false
},
"statesBlockers": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"ready",
"review"
]
},
"verdict": {
"type": "string",
"enum": [
"present",
"absent"
]
},
"probability": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"status",
"verdict",
"probability",
"confidence"
],
"additionalProperties": false
},
"statesNextStep": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"ready",
"review"
]
},
"verdict": {
"type": "string",
"enum": [
"present",
"absent"
]
},
"probability": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"status",
"verdict",
"probability",
"confidence"
],
"additionalProperties": false
},
"raisesQuestions": {
"type": "object",
"properties": {
"status": {
"type": "string",
"enum": [
"ready",
"review"
]
},
"verdict": {
"type": "string",
"enum": [
"present",
"absent"
]
},
"probability": {
"type": "number",
"minimum": 0,
"maximum": 1
},
"confidence": {
"type": "number",
"minimum": 0,
"maximum": 1
}
},
"required": [
"status",
"verdict",
"probability",
"confidence"
],
"additionalProperties": false
}
},
"required": [
"statesOutcome",
"providesEvidence",
"statesBlockers",
"statesNextStep",
"raisesQuestions"
],
"additionalProperties": false
}
},
"required": [
"model",
"usage",
"status",
"detected",
"labels"
],
"additionalProperties": false
}
}Saved example result
This hand-authored response demonstrates the contract. It is not a model accuracy measurement. Run it without an API key: npx jev-recipes demo report-facets.
{
"model": "demo-fixture",
"usage": {
"input_tokens": 0,
"output_tokens": 0
},
"status": "ready",
"detected": [
"statesOutcome",
"providesEvidence",
"statesBlockers",
"statesNextStep"
],
"labels": {
"statesOutcome": {
"status": "ready",
"verdict": "present",
"probability": 0.95,
"confidence": 0.95
},
"providesEvidence": {
"status": "ready",
"verdict": "present",
"probability": 0.88,
"confidence": 0.88
},
"statesBlockers": {
"status": "ready",
"verdict": "present",
"probability": 0.91,
"confidence": 0.91
},
"statesNextStep": {
"status": "ready",
"verdict": "present",
"probability": 0.93,
"confidence": 0.93
},
"raisesQuestions": {
"status": "ready",
"verdict": "absent",
"probability": 0.12,
"confidence": 0.88
}
}
}
Evaluation evidence
No verified live accuracy measurement is available. Evaluate representative cases before using this decision in your workflow.
Use the evaluation guide to measure this decision on your own labeled cases.
Limitations
- Labels are independent, so a report can carry several or none.
- Detects that a facet is present, not that it is accurate. Evidence can be cited and still be wrong.
Related recipes
- result-outcome: Use result-outcome to classify what a reported result actually was, once the report is known to state one.
- step-progress: Use step-progress to grade how far the reported work moved the objective, rather than what the report contains.