Records and sources
A record is a row as a source sent it, or a mention an extractor found in a text. It is never changed, a changed row becomes a new version, and a row its source no longer holds can be taken back.
A record is one thing a source said, kept as it arrived. Records come in two ways:
- From rows. A source is a place rows come from: a CRM export, a company registry, a table in your warehouse. Each row becomes a record.
- From text. A document is stored as it is, and an extractor finds the places where it names a thing. Each place becomes a record. An extractor is code of yours in Python or TypeScript that the server runs as a child process.
Either way the record is then matched by the same rules. This page says what Spool keeps of a row, what happens when a row is loaded twice, changes, is taken back or is refused, and how a record comes out of a text.
Before you start
The examples use the sample project with both of its files of rows loaded and matched, which is where Getting started ends. If you have that project and its server is running, use it. If not, this makes it:
spool init companies
cd companiesspool devLeave that running. In a second terminal, in the companies directory:
spool load samples/crm-1.csv
spool match
spool load samples/crm-2.csv
spool matchA source is a file
Each source is a YAML file under sources/, and the file's name is the source's name. It lists the columns a row may have and names the one that is the row's key. This is the one spool init wrote:
# A source of rows. `key` is the column that tells one row from another, and
# `columns` is every column a row may have.
key: id
columns: [id, name, country]Add a column to it:
key: id
columns: [id, name, country, website]spool dev applies the change when the file is saved. Rows that were loaded before stay as they are, with no value for the new column.
A source says nothing about what its rows mean. Which entity type they describe, and how they are compared, is in the entity type's mapping.
What a record holds
spool show records crm:c-1record ab8e9d59-0aaa-96b9-ddb7-db0bff7a9356
type company
source crm
key c-1
loaded 2026-11-18T21:43:51Z by run 1
identity 01a1efc6-41cd-775d-8758-e7300637659a
payload
country GB
id c-1
name Acorn Toolsrecordis the record's ID. It is derived from the source, the key and the row's content, so the same row gets the same ID on any machine. The one exception is a version that holds content its row held before: its ID is derived from that content's ID and the ID of the version it follows.payloadis the row as it was sent. Spool never rewrites it. What rules compare is computed from it, and can be computed again when you change how.identityis the identity the record belongs to, once a match has placed it.
A record is named by its ID, or as source:key, which names the latest version of that row.
Loading the same row twice does nothing
A pipeline can send everything it has every time. A row whose content is what the server holds for its key is counted and ignored:
spool load samples/crm-1.csvloaded 6 rows into source `crm`, type `company`: 0 new, 6 already heldA changed row is a new version
When a row arrives with a key the server has seen and content it has not, it becomes a new record that replaces the earlier one. Save a file that has c-4 again, now with a website:
id,name,country,website
c-4,Larkfield Logistics,GB,https://www.larkfield.examplespool load changed.csv --source crmloaded 1 row into source `crm`, type `company`: 0 new, 1 new version of a row held before
next `spool match` places it in an identityspool show records --source crm --key c-4 --idsrecord source and key name identity
4b1bc703-9041-0989-d0eb-4df5e9eae0f2 crm:c-4 Larkfield Logistics 01a1efc6-41cd-775d-8758-e733aa086f7d
69c75f0d-bc65-b5f2-39aa-5c6d017a1531 crm:c-4 Larkfield Logistics -
1 record has no identity. `spool match` places new records in identities,
and `spool show review` lists the ones a match left out.Both versions are kept. The new one has no identity yet. The next match gives it the identity of the version it replaces, whatever the rules say about the pair:
spool matchspool show records --source crm --key c-4 --idsrecord source and key name identity
4b1bc703-9041-0989-d0eb-4df5e9eae0f2 crm:c-4 Larkfield Logistics 01a1efc6-41cd-775d-8758-e733aa086f7d
69c75f0d-bc65-b5f2-39aa-5c6d017a1531 crm:c-4 Larkfield Logistics 01a1efc6-41cd-775d-8758-e733aa086f7dThe earlier version stays a member of the identity and can still be matched by later records. That keeps history intact, and it has a cost: an old version that lacks a value can match something the new version would not. Review shows what a missing value does.
The versions of a row travel together. When the latest version is placed in an identity, by a match or by a changeset that a person or an agent sends, each earlier version that is in another identity, or in none, is moved there too. The moves are operations of the same changeset, so the log shows them and undoing it puts each version back. A match that moved some says so on its placed line, as 1 earlier version moved along. A split, an undo and a draft that is approved move only the records they name.
A row that changes from A to B and back to A ends at A. The second A is a new version with an ID of its own, which replaces B. So a file is only ignored when it says what the server holds now: loading an older file after a newer one moves the rows that changed between them back to what the older file says.
A row the source no longer holds
spool retract takes a row back. Name the source, the entity type and the row's key:
spool retract --source crm --type company c-9 --reason "deleted in the CRM"run 11: 1 key named, 1 record taken back
1 record left an identity, by the changeset at log position 4spool show records crm:c-9record 34c3d52d-e53b-1559-7404-1892fb1fd4e3
type company
source crm
key c-9
loaded 2026-11-18T21:43:51Z by run 4
identity none: its source took it back in run 11
payload
country GB
id c-9
name Acorn ToolThe record left its identity and no later match looks at it. The identity keeps its ID and its other records. The row itself stays in the store: taking back is not erasing. Loading a row under the same key again, changed or not, brings every version of it back.
The same over HTTP is POST /v1/retract with {"source": "crm", "entity_type": "company", "keys": ["c-9"]}.
When the row that is taken back was what joined two other records of its identity, the identity is not split. A review item asks whether it still is one thing. Review has an example.
What is refused
A row with a column its source does not list is refused, and so is the whole load. Here the rows are sent over HTTP as JSON, which is another form POST /v1/load takes:
curl -s -X POST http://127.0.0.1:7765/v1/load -d '{"source": "crm", "entity_type": "company", "rows": [{"id": "c-10", "name": "Acorn Hardware", "city": "Leeds"}]}'{"error":{"code":"invalid","message":"load rejected: the rows have a column `city`, which sources/crm.yaml does not list. It lists `id`, `name`, `country`, `website`. Add the column to that file to keep it, or leave it out of the rows"}}Rows are also refused when their source has no file, or when the entity type's mapping has no block for that source. A column that a row leaves out is fine: it is empty for that record.
Records from text
A document is stored as it arrived, as a row is. An extractor reads it and reports each place where it names a thing, and each place becomes a record of the document's source, with the document and the characters it spans as its evidence. From there it is a record like any other: it is never changed, a match places it, and it can be explained.
spool extract samples/notes-1.txtsource notes
read 1 document from samples/notes-1.txt
loaded 1 new, 0 already held, 0 new versions
extractor companies
found 3 records in 1 document: 3 new, 0 already held
company Acorn Tools Ltd notes:notes-1.txt[54-69]
company Larkfield Logistics Ltd notes:notes-1.txt[119-142]
company Brightwater Foods Ltd notes:notes-1.txt[179-200]
next `spool match` places them in identitiesExtraction goes through it, with an extractor in each language.
Where records come from
spool loadreads CSV files. The command line has its options.POST /v1/loadtakes CSV, JSON or an Apache Arrow stream. The HTTP API has an example.- An agent loads rows with the MCP tool
load_rows. MCP spool extractstores a text file as a document and runs an extractor over it.POST /v1/documentsandPOST /v1/extractionsare the same over HTTP, andload_documentsandrun_extractionover MCP.
Nothing reads a database, a warehouse or a file store. Limits has the rest of what loading does not do.