---
title: "Records and sources"
description: "A record is a row as a source sent it, or a mention an extractor found in a text. It is never changed, a changed row becomes a new version, and a row its source no longer holds can be taken back."
url: https://spool.alexpatow.com/docs/concepts/records-and-sources
docs_index: https://spool.alexpatow.com/llms.txt
---

# Records and sources

A record is a row as a source sent it, or a mention an extractor found in a text. It is never changed, a changed row becomes a new version, and a row its source no longer holds can be taken back.

A **record** is one thing a source said, kept as it arrived. Records come in two ways:

- **From rows.** A **source** is a place rows come from: a CRM export, a company registry, a table in your warehouse. Each row becomes a record.
- **From text.** A document is stored as it is, and an **extractor** finds the places where it names a thing. Each place becomes a record. An extractor is code of yours in Python or TypeScript that the server runs as a child process.

Either way the record is then matched by the same rules. This page says what Spool keeps of a row, what happens when a row is loaded twice, changes, is taken back or is refused, and how a record comes out of a text.

## Before you start

The examples use the sample project with both of its files of rows loaded and matched, which is where [Getting started](https://spool.alexpatow.com/docs/getting-started.md) ends. If you have that project and its server is running, use it. If not, this makes it:

```sh
spool init companies
cd companies
```

```sh
spool dev
```

Leave that running. In a second terminal, in the `companies` directory:

```sh
spool load samples/crm-1.csv
spool match
spool load samples/crm-2.csv
spool match
```

## A source is a file

Each source is a YAML file under `sources/`, and the file's name is the source's name. It lists the columns a row may have and names the one that is the row's key. This is the one `spool init` wrote:

```yaml title="sources/crm.yaml"
# A source of rows. `key` is the column that tells one row from another, and
# `columns` is every column a row may have.
key: id
columns: [id, name, country]
```

Add a column to it:

```yaml title="sources/crm.yaml"
key: id
columns: [id, name, country, website]
```

`spool dev` applies the change when the file is saved. Rows that were loaded before stay as they are, with no value for the new column.

A source says nothing about what its rows mean. Which entity type they describe, and how they are compared, is in the entity type's [mapping](https://spool.alexpatow.com/docs/concepts/entity-types-and-features.md).

## What a record holds

```sh
spool show records crm:c-1
```

```text title="Output"
record    ab8e9d59-0aaa-96b9-ddb7-db0bff7a9356
type      company
source    crm
key       c-1
loaded    2026-11-18T21:43:51Z by run 1
identity  01a1efc6-41cd-775d-8758-e7300637659a

payload
  country  GB
  id       c-1
  name     Acorn Tools
```

- **`record`** is the record's ID. It is derived from the source, the key and the row's content, so the same row gets the same ID on any machine. The one exception is a version that holds content its row held before: its ID is derived from that content's ID and the ID of the version it follows.
- **`payload`** is the row as it was sent. Spool never rewrites it. What rules compare is computed from it, and can be computed again when you change how.
- **`identity`** is the identity the record belongs to, once a match has placed it.

A record is named by its ID, or as `source:key`, which names the latest version of that row.

## Loading the same row twice does nothing

A pipeline can send everything it has every time. A row whose content is what the server holds for its key is counted and ignored:

```sh
spool load samples/crm-1.csv
```

```text title="Output"
loaded  6 rows into source `crm`, type `company`: 0 new, 6 already held
```

## A changed row is a new version

When a row arrives with a key the server has seen and content it has not, it becomes a new record that replaces the earlier one. Save a file that has `c-4` again, now with a website:

```csv title="changed.csv"
id,name,country,website
c-4,Larkfield Logistics,GB,https://www.larkfield.example
```

```sh
spool load changed.csv --source crm
```

```text title="Output"
loaded  1 row into source `crm`, type `company`: 0 new, 1 new version of a row held before
next    `spool match` places it in an identity
```

```sh
spool show records --source crm --key c-4 --ids
```

```text title="Output"
record                                source and key  name                 identity
4b1bc703-9041-0989-d0eb-4df5e9eae0f2  crm:c-4         Larkfield Logistics  01a1efc6-41cd-775d-8758-e733aa086f7d
69c75f0d-bc65-b5f2-39aa-5c6d017a1531  crm:c-4         Larkfield Logistics  -

1 record has no identity. `spool match` places new records in identities,
and `spool show review` lists the ones a match left out.
```

Both versions are kept. The new one has no identity yet. The next match gives it the identity of the version it replaces, whatever the rules say about the pair:

```sh
spool match
```

```sh
spool show records --source crm --key c-4 --ids
```

```text title="Output"
record                                source and key  name                 identity
4b1bc703-9041-0989-d0eb-4df5e9eae0f2  crm:c-4         Larkfield Logistics  01a1efc6-41cd-775d-8758-e733aa086f7d
69c75f0d-bc65-b5f2-39aa-5c6d017a1531  crm:c-4         Larkfield Logistics  01a1efc6-41cd-775d-8758-e733aa086f7d
```

The earlier version stays a member of the identity and can still be matched by later records. That keeps history intact, and it has a cost: an old version that lacks a value can match something the new version would not. [Review](https://spool.alexpatow.com/docs/concepts/review.md#a-conflict) shows what a missing value does.

The versions of a row travel together. When the latest version is placed in an identity, by a match or by a changeset that a person or an agent sends, each earlier version that is in another identity, or in none, is moved there too. The moves are operations of the same changeset, so the log shows them and undoing it puts each version back. A match that moved some says so on its `placed` line, as `1 earlier version moved along`. A split, an undo and a draft that is approved move only the records they name.

A row that changes from A to B and back to A ends at A. The second A is a new version with an ID of its own, which replaces B. So a file is only ignored when it says what the server holds now: loading an older file after a newer one moves the rows that changed between them back to what the older file says.

## A row the source no longer holds

`spool retract` takes a row back. Name the source, the entity type and the row's key:

```sh
spool retract --source crm --type company c-9 --reason "deleted in the CRM"
```

```text title="Output"
run 11: 1 key named, 1 record taken back
1 record left an identity, by the changeset at log position 4
```

```sh
spool show records crm:c-9
```

```text title="Output"
record    34c3d52d-e53b-1559-7404-1892fb1fd4e3
type      company
source    crm
key       c-9
loaded    2026-11-18T21:43:51Z by run 4
identity  none: its source took it back in run 11

payload
  country  GB
  id       c-9
  name     Acorn Tool
```

The record left its identity and no later match looks at it. The identity keeps its ID and its other records. The row itself stays in the store: taking back is not erasing. Loading a row under the same key again, changed or not, brings every version of it back.

The same over HTTP is `POST /v1/retract` with `{"source": "crm", "entity_type": "company", "keys": ["c-9"]}`.

When the row that is taken back was what joined two other records of its identity, the identity is not split. A review item asks whether it still is one thing. [Review](https://spool.alexpatow.com/docs/concepts/review.md#a-possible-split) has an example.

## What is refused

A row with a column its source does not list is refused, and so is the whole load. Here the rows are sent over HTTP as JSON, which is another form `POST /v1/load` takes:

```sh
curl -s -X POST http://127.0.0.1:7765/v1/load -d '{"source": "crm", "entity_type": "company", "rows": [{"id": "c-10", "name": "Acorn Hardware", "city": "Leeds"}]}'
```

```text title="Output"
{"error":{"code":"invalid","message":"load rejected: the rows have a column `city`, which sources/crm.yaml does not list. It lists `id`, `name`, `country`, `website`. Add the column to that file to keep it, or leave it out of the rows"}}
```

Rows are also refused when their source has no file, or when the entity type's mapping has no block for that source. A column that a row leaves out is fine: it is empty for that record.

## Records from text

A document is stored as it arrived, as a row is. An extractor reads it and reports each place where it names a thing, and each place becomes a record of the document's source, with the document and the characters it spans as its evidence. From there it is a record like any other: it is never changed, a match places it, and it can be explained.

```sh
spool extract samples/notes-1.txt
```

```text title="Output"
source     notes
read       1 document from samples/notes-1.txt
loaded     1 new, 0 already held, 0 new versions
extractor  companies
found      3 records in 1 document: 3 new, 0 already held

  company  Acorn Tools Ltd          notes:notes-1.txt[54-69]
  company  Larkfield Logistics Ltd  notes:notes-1.txt[119-142]
  company  Brightwater Foods Ltd    notes:notes-1.txt[179-200]

next       `spool match` places them in identities
```

[Extraction](https://spool.alexpatow.com/docs/concepts/extraction.md) goes through it, with an extractor in each language.

## Where records come from

- `spool load` reads CSV files. [The command line](https://spool.alexpatow.com/docs/cli.md#spool-load) has its options.
- `POST /v1/load` takes CSV, JSON or an Apache Arrow stream. [The HTTP API](https://spool.alexpatow.com/docs/api.md#load-rows) has an example.
- An agent loads rows with the MCP tool `load_rows`. [MCP](https://spool.alexpatow.com/docs/mcp.md)
- `spool extract` stores a text file as a document and runs an extractor over it. `POST /v1/documents` and `POST /v1/extractions` are the same over HTTP, and `load_documents` and `run_extraction` over MCP.

Nothing reads a database, a warehouse or a file store. [Limits](https://spool.alexpatow.com/docs/limits.md) has the rest of what loading does not do.
