Data provenance
Case study · open-climate-data
A figure that carries its source, and keeps carrying it through the arithmetic.
Two records with every field editable, and a button that takes a mandatory one away so the refusal can be seen rather than described.
Problem
A grid intensity of 412 gCO2e per kWh stored as a number cannot say which portal it came from, and a figure read in March from the 2024 edition is not the figure read in September after a restatement; a citation kept beside the number does not survive contact with a calculation either, because two sourced figures divided leave an ordinary float with the notes somewhere else.
Method
A cited value is a magnitude, a unit and a lineage: five fields are mandatory on every record, identity is the hash of its canonical form with the timestamp normalised to UTC, merging two lineages is union over those identities, and the Python test suite sweeps that invariant over four hundred randomly generated arithmetic expression trees.
What it does
Two records side by side with every field editable, a button that takes a mandatory field away so the refusal can be seen, an operation selector that combines them and prints the sources the result admits to, and an audit of what has gone stale, what has no address, what recorded no query and whether the licences have been mixed.
What it refuses to do
It refuses to construct a record with a mandatory field missing, and there is a button on the page whose only job is to empty the licence field, because a described refusal is not the same as a seen one.
It refuses a timestamp with no timezone, because an instant with no offset is not an instant and assuming UTC would silently make two different retrievals identify as one.
It refuses to say whether two licences may be combined in a published figure: it names them and reports that they have been mixed, and the rest is a legal question with a jurisdiction attached.
In closing
The claim any published figure makes rests on whether anybody can get back to where the number came from, and that is a property of how it was stored long before it is a property of how carefully it was quoted.