---
title: "Item Bank & Import Pipeline"
url: https://docs.growamal.com/en/guides/item-bank-and-import-pipeline/
language: en
---

# Item Bank & Import Pipeline

The item bank (`modules/items`, WP-01) is the single source of truth for all content in
Amal. Items are authored and validated **offline**, then imported through a controlled
pipeline. The platform never generates, composes, or transforms item content at runtime.

All routes in this module are gated by `RUN_BACKOFFICE`. Students receive items only through
the [Task Delivery](https://docs.growamal.com/en/guides/task-delivery/) service, which runs its own scoped endpoint.

## Item lifecycle

Every item moves through a four-status lifecycle. Status transitions are validated and logged;
corrections to operational items are always new rows, never in-place edits (append-only rule,
V-6).

```
draft → pilot → operational
                    ↓
                 retired
```

| Status | Meaning |
|---|---|
| `draft` | Just imported; pending SME review |
| `pilot` | Approved for limited pilot use; collects real response data |
| `operational` | Fully approved; available for live session delivery |
| `retired` | Removed from active selection; the row is preserved for audit |

> **Warning:** Synthetic items (those with `source=synthetic`) are permanently excluded from non-backoffice
> reads. They cannot be promoted to `operational`. The `409` response on the promote endpoint
> includes this as an explicit rejection reason.

## The import batch

`POST /api/items/import` is the single audited write path for all new items. The request body
carries the batch metadata (source, language, dialect, batch label). The actual item rows are
supplied as a structured JSON payload that the server validates before writing anything.

The pipeline runs the following checks on every row, in order. A row that fails any check is
rejected with a per-row reason; the rest of the batch continues to be processed.

| Validation step | What it checks |
|---|---|
| Schema | Required fields, types, enum values |
| Item-type structural rules | Distractor count, option count, and format rules for the declared `item_type` |
| Distractor compatibility | The declared `distractorType` must be legal for the item type (compatibility matrix at `GET /api/item-types/{code}/distractor-compat`) |
| Language safety | Item content is scanned against the language-safety rule set; any substring match rejects the row |
| Arabic agreement | Gender, number, and case agreement across stem, options, and distractors (offline NLP check via the Arabic validator) |
| `delta_prior` computation | A five-component difficulty prior is computed and stored; rows where SAMER level data is unavailable receive a `samer_unavailable` skip marker and remain importable at a reduced confidence level |

A batch is idempotent on `fileHash`. Re-submitting the same file returns the existing batch
report without writing duplicate rows.

### `delta_prior`: the five components

The difficulty prior is a weighted sum of five content signals. The weights come from the active
`expected_difficulty_weights` row in the database (versioned; the version is pinned on each item
row for V-6 replay).

| Component | Source field(s) | What it captures |
|---|---|---|
| **TR** (Text Readability) | `samerTextLevel` (passage); falls back to `samerAvgLevel` (item-level) | Passage difficulty on the SAMER scale |
| **TW** (Target Word) | `targetWordSamerLevel` | Mean SAMER level of the target word(s) |
| **CD** (Cognitive Demand) | `dokLevel`; `pirlsProcess` overrides when present | Depth-of-knowledge and comprehension-process load |
| **SC** (Sentence Complexity) | `avgSentenceLength`, `cliticDensity` | Morphological and syntactic complexity |
| **AF** (Answer Format) | `itemType`, `distractorSimilarity`, `requiresMultipleSelection`, `optionCount` | How hard the response format makes the item |

Default weight distribution (active weights row as shipped): TR 0.35, TW 0.25, CD 0.20,
SC 0.10, AF 0.10. The weights are configurable in the database and do not require a code
change to adjust.

The computed `delta_prior` is stored as `Decimal(7,4)` (same precision as θ) so that
difficulty and ability live on a comparable scale and V-6 replay is byte-identical.

## Item types

`GET /api/item-types` returns the full catalog: 15 active item types and 3 reserved codes.
Each type has:

- A structural rule set (which fields are required, how many options are valid, etc.)
- An `afBase` value used as the AF component seed in `delta_prior`
- A distractor-compatibility set, readable at `GET /api/item-types/{code}/distractor-compat`

The catalog is seeded at boot from `catalog-seed.ts` and does not change at runtime.

## Approval and promotion

After import, items land as `draft`. Two separate approval paths exist.

**Batch approval** (`POST /api/items/import/{batchId}/approve`) is the standard path. An
authorized backoffice user reviews the batch report and, if satisfied, approves the batch.
All `accepted` rows in the batch are promoted to `operational` in one transaction. Rows with
per-row rejection reasons are excluded. The endpoint is idempotent: re-approving an already
approved batch returns `409`.

**Per-item SME approval** (`POST /api/items/{id}/sme-approve`) applies to items with
`source=sme_authored`. It marks the individual item as SME-reviewed without affecting the rest
of its batch.

**Promote** (`POST /api/items/{id}/promote`) advances a single item one step along the
lifecycle (`draft → pilot → operational`). The endpoint enforces the SME gate: a `draft` item
cannot skip directly to `operational` without an SME approval stamp.

## Retiring an item

`POST /api/items/{id}/retire` sets the status to `retired`. The item row is never deleted. It
is excluded from all delivery selection queries but remains visible in the backoffice list
(`GET /api/items`) and in the `delta_prior` log. This preserves the audit record for any
sessions that served the item while it was `operational`.

## `delta_prior` log

`GET /api/expected-difficulty/log?itemBankId=<id>` returns the full computation history for
one item: every `delta_prior` recalculation, the weights version used, the five component
values, and the resulting `difficultyBand`. This is the V-6 audit trail for difficulty prior
changes. Use it when investigating why an item's band changed after a weights update or a
re-import.

## API reference

All routes require `RUN_BACKOFFICE`.

| Method | Path | Notes |
|---|---|---|
| `GET` | `/api/items` | List items (filtered, tenant-scoped; synthetic excluded) |
| `GET` | `/api/items/{id}` | One item; add `?view=authoring` for the full row |
| `GET` | `/api/item-types` | Full type catalog (15 active + 3 reserved) |
| `GET` | `/api/item-types/{code}/distractor-compat` | Distractor compatibility matrix for one type |
| `POST` | `/api/items/import` | Import a batch; idempotent on `fileHash` |
| `GET` | `/api/items/import` | Batch history |
| `GET` | `/api/items/import/{batchId}` | Validation report for one batch (counts + per-row outcomes) |
| `POST` | `/api/items/import/{batchId}/approve` | SME batch approval; promotes all `accepted` rows |
| `POST` | `/api/items/{id}/sme-approve` | Per-item SME approval (for `sme_authored` items) |
| `POST` | `/api/items/{id}/promote` | Single-step status promotion (`draft → pilot → operational`) |
| `POST` | `/api/items/{id}/retire` | Set status to `retired` (the row is never deleted) |
| `GET` | `/api/expected-difficulty/log` | `delta_prior` computation history for one item |

## What the import pipeline does not do

- It does not generate items or content at runtime.
- It does not invoke a language model at any point (V-5).
- It does not silently accept rows with Arabic agreement failures; failing rows are rejected with per-row reasons.
- It does not allow synthetic items to reach `operational` status.

## Related

- [Task Delivery](https://docs.growamal.com/en/guides/task-delivery/): how approved operational items are selected and served
- [Language Safety](https://docs.growamal.com/en/guides/language-safety/): the substring filter applied during import
- [Calibration Runbook](https://docs.growamal.com/en/guides/calibration-runbook/): how `delta_prior` values are later refined by real response data
- [Skills Taxonomy](https://docs.growamal.com/en/guides/skills-taxonomy/): the sub-skill IDs that items are tagged with