Comparison · target: schemathesis vs json schema faker

Schemathesis vs json-schema-faker for Contract test fixture

User compares a property-based API tester with a schema-based data generator.

Direct answer

For the Contract test fixture, Schemathesis and json-schema-faker are built for solving different halves of the problem, not for 10 constraints that decide whether the output loads.

The comparison holds for a schema of this size — 2 tables — and does not claim anything about runtime, community size or pricing.

A compatibility table below scores Schemathesis and json-schema-faker against the same Contract test fixture; the foreign key row is decisive, because a document carries a dangling reference without any tool objecting until load time.

Data as of 2026-11-18 · source: 0 Stack Overflow questions, json-schema-faker at 4,376,957 monthly downloads · negative-case messages reproduced by executing the fixture against a local SQL engine (Node.js v24.22.0, engine 3.53.4) at 2026-11-18T11:53:54.233Z

The scorecard — Contract test

The scorecard is scored against one shared fixture: Contract test fixture, 2 tables and 7 columns. Each row states what a tool has to do for the output to load, and the note column says which property of the fixture decided the row. Rows are not weighted by preference; a row is decided by whether the output would load.

Where Schemathesis wins

Schemathesis is the better choice when a running service exists: it checks live responses against the contract, which catches drift that no offline fixture can see. The Contract test fixture is the shared reference for every row in the table above.

Where json-schema-faker wins

json-schema-faker is the better choice when the output is a single document: it reads the schema properties and generates a value for each one, which is what a unit test needs when the data carries no relationships. Neither of the two reads 10 constraints of this fixture as a database contract, which is the part a seed has to satisfy before a single row can be inserted.

How the rows were decided

Every row in the scorecard is decided by a property of the fixture, not by reading documentation. For Contract test fixture, the deciding fixture properties were the endpoints to specs reference, the specs.format range declaration, the NOT NULL columns. A tool that misses any one of those produces output that a database will refuse.

A note on what this comparison does not cover

This comparison measures output correctness on one fixture. It does not measure runtime, community size, documentation quality or long-term maintenance, and it does not claim that either tool is wrong for your project. It also does not compare pricing, because every row is decided by the schema rather than by a price list: what a tool charges says nothing about whether its output loads.

What the Contract test fixture requires: schema-to-SQL vs document generationEight criteria, each decided by a property of the fixture rather than by preference. The left column is what a schema-to-SQL generator does with it; the right column is what document generation leaves unresolved.
CriterionSchemaSeedDocument generationWhat decided it
Reads a multi-file OpenAPI documentyespartialboth resolve local $ref chains
Honours NOT NULLyespartialboth read the required array
Honours enum, range and length declarationsyespartialthe fixture declares specs.format with format IN ('openapi','swagger'); the declaration is read out of the schema, the DDL carries it as a CHECK constraint and the value that breaks it ships as a negative case
Honours enum membersyespartialthe enum array is copied verbatim
Produces SQL, not just JSONyesnooutput format is the difference
Resolves foreign keys across schemasyesnothe fixture has endpoints.spec_id pointing at specs, and the second tool never reads that table
Emits deliberately invalid instancesyesnonegative cases are a separate output
Same seed produces the same bytesyesyesboth are seedable
The four checks that decide whether a schema tool is enough for Contract testApplied to Contract test fixture.
CheckResult on this fixtureConsequence
More than one table references anotheryes — endpoints → specsthe generator must resolve cross-table references
Output has to be SQL, not JSONyes — the target is PostgreSQLa document generator produces the wrong artefact
Tests must assert the API rejects bad inputyes — a negative case for every declaration in the schemanegative instances have to be generated separately
Same seed must reproduce the same bytesyes — seed 42 is fixedfixtures can be committed and diffed
README.md

npx --yes @wenlu8848/schemaseed generate --input ./cmp_contracts.openapi.yaml --out ./seed --formats pytest --count 10 --seed 42

Download cmp_contracts.openapi.yaml

# Generated pytest suite

Seed: 42 — mode: nested

Run it with:

```
pytest pytest/
```

The suite is self contained: `_validator.py` implements the JSON Schema
keywords the generated data uses, so only pytest is needed.

- `test_specs.py`: 10 valid rows, 11 negative cases (enum-violation, format-violation, missing-required, null-violation, type-mismatch)
- `test_endpoints.py`: 10 valid rows, 24 negative cases (enum-violation, format-violation, missing-required, null-violation, type-mismatch)

Frequently asked questions

Which one should I pick?
If the output has to load into a database and the schema has relationships, take the schema-to-SQL tool. On Contract test fixture the deciding constraint was endpoints.spec_id.
How many criteria were compared?
Eight, each decided by a property of the Contract test fixture: 2 tables and 7 columns.
Was this a benchmark?
No. It is one 2-table fixture of 7 columns measured on whether the produced rows satisfy the schema, not on speed, memory or price.
Do both tools produce deterministic output?
Both are seedable, which is why the Contract test fixture rows can be committed and diffed; on this fixture that is the one property the two share.

Run it against your own schema

SchemaSeed is MIT licensed and free: unlimited local generation, no account, no telemetry and no usage limit. It reads your schema file on your own machine, so nothing is uploaded anywhere. If it saves you time, you can support the project.

Get SchemaSeed — free Support the project

Questions: support@schemaseed.drz8848.win