Extract, resolve and track entities.
Spool pulls records out of text and rows, decides which are the same real-world thing, and gives each thing an ID that stays the same as more arrive.
Call with Dana Reyes on 12 November. She has moved to Acorn Tools Ltd and runs purchasing there. They now ship through Larkfield Logistics Ltd, and she asked whether we work with Brightwater Foods Ltd, their new supplier.
| Source and key | Name |
|---|---|
| crm:c-1 | Acorn Tools |
| crm:c-2 | ACORN TOOLS |
| crm:c-3 | Acorn Tools Ltd |
| crm:c-4 | Larkfield Logistics |
| crm:c-5 | Halden Group |
| crm:c-6 | Halden Group |
| crm:c-7 | Larkfeild Logistics |
| crm:c-8 | Kestrel Analytics |
| crm:c-9 | Acorn Tool |
| notes:notes-1.txt[54-69] | Acorn Tools Ltd |
| notes:notes-1.txt[119-142] | Larkfield Logistics Ltd |
| notes:notes-1.txt[179-200] | Brightwater Foods Ltd |
9 from rows, 3 from the note. A record from text is keyed by where it was found.
- match after crm-1.csv4 identities created, 6 records placed
- match after crm-2.csv1 identity created, 3 records placed
- match after notes-1.txt1 identity created, 3 records placed
- next
The sample project at the end of Getting started: nine rows from a CRM and one call note. "Acorn Tools Ltd" in the note joined the identity that four CRM rows already had, and its ID did not change.
Run it
Install it, make the sample project and start the server on your machine.
curl -fsSL https://github.com/alexpatow/spool/releases/latest/download/install.sh | sh
spool init companies
cd companies
spool devLinux or macOS on Apple silicon. Version 0.1.0, MIT licence.
Then, in a second terminal
Load the rows, match them, and get the companies out of the note. These seven commands give the picture above.
spool load samples/crm-1.csv
spool match
spool load samples/crm-2.csv
spool match
spool extract samples/notes-1.txt
spool match
spool show identitiesGetting started has what each one prints, and the same steps over HTTP and MCP.
What it does
- Takes rows from several sources and keeps each one as it arrived.
- Finds records in text with a pattern, or with code of yours in Python or TypeScript, and keeps where in the text each one was found.
- Matches records with rules you write in SQL, kept as files in Git.
- Gives each real-world thing an ID that later runs and better rules do not change.
- Records every merge, split and correction in a log, with an author, a reason and an undo.
- Says why two records were matched, and leaves what it cannot decide for a person or an agent.
What it is not meant for
- Deduplicating one file once. A script or a library is less to run.
- Answering per record, in the request, whether a row is a duplicate. Matching runs in batches.
- Matching with no rules of your own. There is no trained model inside.
- Building a merged golden record, or writing corrections back to a source.
- Many tenants, many writers or many machines. One server owns one store.
Limits has the full list, with what it does not do yet.