· 4 min read

I ran a coherence audit on the seed script every tutorial teaches. It scored 42/100.

There is a seed script that thousands of projects contain, because every tutorial teaches some version of it. Two loops, faker for the names and dates, random for the numbers:

for i in range(600):
    orders.append({
        "order_id": i + 1,
        "user_id": random.randint(1, 200),
        "status": random.choice(["pending", "shipped", "delivered", "cancelled"]),
        "order_date": fake.date_time_between(start_date="-1y"),
        "shipped_date": fake.date_time_between(start_date="-1y"),
        "quantity": random.randint(1, 5),
        "unit_price": round(random.uniform(5, 200), 2),
        "total": round(random.uniform(5, 1000), 2),
        "is_fraud": random.choice([True, False]),
    })

I wrote exactly that script (users table too, same style), generated the CSVs, and ran one command on the output:

pip install misata
misata audit ./faker_seed/

The result, verbatim:

severitytablefindingrows
highordersorder_date after shipped_date (event out of order)309
highorderstotal does not equal quantity times unit_price (off by up to 939.26)600
highuserscreated_at after updated_at97
highusersage disagrees with date_of_birth181
mediumordersshipped_date present although status never reached that stage296
mediumordersis_fraud true on 50% of rows; rare-event flags should be rare298

Coherence score: 42/100.

None of this is faker's fault. Faker fills one field at a time, honestly and well; that is its job. The incoherence comes from the loop: every field is an independent draw, so nothing agrees with anything. The order date does not know the ship date exists. The total does not know there is a quantity and a unit price two keys away. The age does not know the birth date. Each column is plausible alone and the rows are impossible together, and a reviewer, a demo audience, or an integration test JOIN finds the impossibility in seconds.

What the audit checks

misata audit points a catalog of named invariants at any folder of CSVs, whoever generated them: lifecycle timestamps that must flow forward, derived arithmetic that must hold, ages against birth dates, event dates and tracking codes gated on status, counts that cannot be negative, percents that live in 0 to 100, rare-event flags with believable base rates, geographic chains (city, state, postal format, phone code agreeing with the country), and, if you pass a schema, foreign-key orphans, child events that predate their parents, and parent totals that fail to sum from their child rows.

Exit codes are CI-friendly: nonzero on high-severity findings, or on any finding with --strict. You can gate a seed-data directory in CI the way you gate lint.

Every check in the catalog exists because it failed somewhere real first, including in our own output. We spent the last releases running exactly this audit against ourselves and fixing what it found; the changelog and LIMITATIONS.md carry the before/after numbers, including the embarrassing befores.

The same schema, generated coherently

For the comparison, I generated the identical tables (same columns, same row counts, same seed discipline) with misata, where the columns know about each other:

misata audit ./misata_seed/
# ✓ Coherence: clean — no reader-visible contradictions.

Same 800 rows. Orders ship after they are placed, totals equal quantity times price, ages match birth dates, cancelled orders carry no ship date, fraud sits at a realistic 3%, and a known city carries its actual state and zip prefix (New York, New York, 10264). Not because a human cleaned it, but because ordering, arithmetic, causality, and base rates are enforced during generation and then the output is audited by the same catalog before it ships. The generator that fails its own audit does not release; that property is a permanent test in our suite.

Try it on your data

The audit does not care who made the CSVs. If you have a seed directory in your repo right now, this takes thirty seconds:

pip install misata
misata audit ./your_seed_data/

If it comes back clean, your seed data is better than most. If it does not, the findings table tells you exactly which rows would embarrass you in a demo, and misata can regenerate the same schema coherently from a description, a YAML file, or your database. Everything is MIT, seeded, and reproducible, and the reproduction script for this post's numbers fits in one file.