Training data you cannot hide

The problem

What we built

• The genre classifier from another case, trained twice: Python with the usual manual manifest (a read helper that hashes every file), and Synsema with the engine's lineage and a signed receipt.
• In both, a realistic slip: besides the documented data, someone loads extra_data.csv directly.
• An independent verifier that needs no private key: signature, declared result, model hash and the hash of every input.

Results

The code, side by side

def read(path):
    """Records the read in the manifest. It only works if EVERYONE uses it, every time."""
    b = pathlib.Path(path).read_bytes()
    MANIFEST.append({"file": pathlib.Path(path).as_posix(), "bytes": len(b), "sha256": hashlib.sha256(b).hexdigest()})
    return pd.read_csv(path)

labels = read(D / "data" / "teacher_labels.csv")
labels = labels[labels["teacher"].notna()].sample(frac=1, random_state=11)
cut = int(len(labels) * 0.8)
train, test = labels.iloc[:cut], labels.iloc[cut:]

extra = pd.read_csv(D / "data" / "extra_data.csv")       # <- the off-the-books read: it does not go through read()

-- 2. The off-the-books read: someone added data without documenting it. In Python it escapes the manual
--    manifest; here the engine records it anyway, because the program cannot drop entries from the lineage.
let extra be csv_parse(read_file(`{D}/data/extra_data.csv`))

let k be secret("RECEIPT_SEED")
let did be did_key_encode(ed25519_pubkey(k))
let vm be did + "#" + did_key_decode(did).multibase
let result be {"model_sha256": slice(hex(sha256(model_json)), 2), "metrics": metrics, "classes": CLASSES}
let rec be receipt({"sign": k, "verification_method": vm, "result": result})

Why Synsema

• lineage() is written by the engine, not by the program. Every read from outside is recorded with its sha256, and the program cannot add, edit or drop an entry.
• receipt() turns it into a Verifiable Credential signed with ed25519. Anyone can verify it with the issuer's public DID.
• The data sheet is generated from what actually happened, not from what someone remembered to log.

What it does not solve

• The lineage records what the program reads, not what it writes. The model hash is added to the signed result by hand.
• Proving a file did not change does not prove it was lawful to use (license, consent). That stays with whoever builds the dataset.
• The demo signing key is a test key. In production it lives in a secret manager and its DID is published.

Share