Methodology

Bronze is what the publisher actually said, kept byte for byte. It is never edited. Byte-identical duplicate rows are preserved rather than collapsed, because a publisher emitting the same row a thousand times is telling you something and deduplicating silently rewrites the count.

Silver translates each publisher into one canonical shape. No identity decisions happen here. Six publishers making six statements about one company produce six Silver observations.

Identity decides which observations are the same organisation and assigns the permanent identifier.

Gold assembles one record per organisation with the publishing gate already applied — so a field that must never be published is not in the record at all, and a template that forgets to check cannot leak it.

Only Bronze touches the network. Reinterpreting a column costs a pass over local disk, not another 2.3 GB download.

A contract award is not an organisation. It names a transaction, so it never mints an identifier and never pulls two companies together. It attaches to a record only when the evidence points at exactly one of them.

This is harder than it sounds, because federal proactive publication deliberately truncates vendor_postal_code to its first three characters — the forward sortation area — and leaves it empty entirely on most rows. Measured against 400,000 live rows: 28.8% carry a postal value at all, and 94% of those are truncated. A full postal-code join can therefore never work.

So two rules, in order:

1. Distinctive name plus forward sortation area, where that narrows the field to a single company.
2. Distinctive name alone — but only where that name resolves to exactly one organisation in the entire national corpus. The justification is not name matching; it is that no other candidate exists.

Everything else is left unattached. Totals on a record are therefore a floor in that direction, and the pipeline reports how many awards it could not place.

What a contract value is not

It is not money paid. The federal contract_value field includes the ceiling of standing offers and supply arrangements — purchasing capacity a department may never use. One row in the live file carries $20,027,319,128 against a single procurement on that basis.

The file does distinguish these instruments, in instrument_type and standing_offer_number. Both are carried through the pipeline, and neither is interpreted: separating ceilings from awards means reading the publisher’s code list first, and inferring what a code means is exactly the guess this project refuses to make. Until that is done, the figures are labelled for what they are rather than quietly cleaned.

One thing is corrected, because it is unambiguous. Proactive publication emits one row per amendment and every row repeats the cumulative value — procurement 2016002691 appears 48 times. Adding those rows up reports $11,949,022 for a contract worth $328,788. Amendment rows are collapsed to the final value per procurement before anything is summed.

Separately from contracts and grants, the federal government publishes which suppliers currently hold an active standing offer or supply arrangement — pre-agreed terms a department may buy against, not evidence that it has. This file carries no dollar figure of any kind, in either direction, so it is attached to a record and labelled as a qualification, never folded into a money total.

It is worth being precise about what this does and does not solve. The $20,027,319,128 ceiling described above comes from the contracts file’s own contract_value field on a row whose instrument happens to be a standing offer — a labelling problem inside that file. Qualifications are a separate dataset entirely, with no dollar column to confuse in the first place, so adding it narrows what a record can honestly say (who is currently qualified to supply what) without resolving the ceiling question in the contracts file, which still needs its own code list read before it can be split further.

This source carries no business number, corporation number, or address at all — a supplier is identified by name alone. It is therefore attached using exactly the same rule as an unaddressed payment: only when that name is unique across the entire national corpus. A supplier with a common name and no other presence in the corpus simply will not show a qualification, which is the correct failure mode for evidence this thin.

Every event is dated by what it is about, never by when it was read. A crawl date stamped onto historical records makes an entire twenty-year archive look like it happened this morning, and the mistake is invisible until somebody plots it. The pipeline carries an explicit effective date per observation and tests assert that no derived event year is the current one.

1,804,068 records are published in the data. Far fewer get their own page, and that is deliberate. A dissolved numbered company with one line of registry data has nothing to read; a page for it is thin content that a search engine will decline to index and a reader will resent.

Pages are rendered for records with something to say: public money attached, more than one publisher asserting them, a charitable registration, or an active municipal licence. The full corpus stays available through search and the data export.

It is not a registry, and no page here is an official record. Where a page and a registrar disagree, the registrar is right; each record links back to the publisher it came from so that check is one click away.

It also does not score, rank or rate anybody. A derived judgement about a named organisation is an opinion wearing the costume of a record. What is published is what a government body wrote down, and the date it wrote it.

What is deliberately withheld · Every source and its licence