Problem · target: how to test api 400 responses automatically

Negative Tests for Order adjustments API: What to Assert

User wants the 400-path of an API covered by an automated test, not by manual poking.

Direct answer

Invalid payloads slip through because the schema is only used to validate requests, never to produce the negative cases, and Order adjustments API has 11 constraints that nobody is testing.

This covers contract violations rather than memory-safety bugs, so a fuzzer is still the right tool when the defect class is a parser crash.

The engine rejected the violating row for Order adjustments API with "CHECK constraint failed: orders_total_cents_check", which is the error the API should return to the client instead of a success response.

Data as of 2026-11-18 · source: 5 Stack Overflow questions, 1 Hacker News stories, msw at 128,057,365 monthly downloads · negative-case messages reproduced by executing the fixture against a local SQL engine (Node.js v24.22.0, engine 3.53.4) at 2026-11-18T11:53:54.233Z

What actually goes wrong — Order adjustments

The failure is rarely a crash. The API accepts the payload, returns 200, and stores a row that violates a rule the schema promised to enforce. For Order adjustments API, the reference engine rejected the bad row with "NOT NULL constraint failed: orders.reference" — that is the message the API should have surfaced to the client instead of a success response.

The four failure modes

Invalid payloads cluster into four families, and the Order adjustments API fixture exercises all of them. Missing required fields, wrong types, out-of-range values and values outside a declared enum. Each family maps to a distinct engine error, and each family needs a distinct assertion in the test suite.

What the boundary probes look like

For Order adjustments API, the range probe on orders.total_cents sits one step outside the boundary. The constraint is "total_cents" >= 0 and the probe value is -1. The engine said "CHECK constraint failed: orders_total_cents_check". A test that asserts a 4xx response for this value is asserting that the constraint the schema declares is actually enforced at the edge.

The assertion you actually want

Assert on the response code and on the error body, not on the absence of a crash. For the Order adjustments API fixture, the assertion that catches a regression is a request carrying total_cents=-1, expecting a 4xx status and an error body that names the field. A test that only asserts "the request failed" passes even when the API fails for the wrong reason.

Where a fuzzing tool is the wrong answer

Unstructured fuzzing finds crashes, which is a different job from finding contract violations. For Order adjustments API, a schema-derived negative set covers the cases a fuzzer would take hours to stumble into, and each one has a deterministic expected response. Reach for a fuzzer when the bug class is a parser crash, not when the bug class is a contract the API forgot to enforce.

Real constraint failures for the Order adjustments API fixtureReproduced by executing the fixture against a local SQL engine (Node.js v24.22.0, engine 3.53.4) on 2026-11-18T11:53:54.233Z. The message column is verbatim engine output; the SQLSTATE column is the PostgreSQL error class for the same constraint class.
Failure classColumnSQLSTATEError nameEngine message (verbatim)
Missing required fieldorders.reference23502not_null_violationNOT NULL constraint failed: orders.reference
Out-of-contract valueorders.total_cents23514check_violationCHECK constraint failed: orders_total_cents_check
Dangling foreign keyadjustments.order_id23503foreign_key_violationFOREIGN KEY constraint failed
Boundary probes for every constrained column in Order adjustments APIOne violating value per constraint, with the boundary it crosses.
TableColumnDeclared ruleProbe valueWhy it is invalid
orderstotal_cents>= 0-1below the lower bound
adjustmentskindone of discount, shipping, taxnot_a_real_membernot a declared member
adjustmentskindone of discount, shipping, taxDISCOUNTwrong case
orders.json

npx --yes @wenlu8848/schemaseed generate --input ./prob_adjustments.openapi.yaml --out ./seed --formats json --count 10 --seed 42

Download prob_adjustments.openapi.yaml

[
  {
    "id": "1ac284da-0bc9-4bb8-95eb-c73e7e349614",
    "reference": "Osaka, Elm Avenue 91",
    "total_cents": 287
  },
  {
    "id": "9645a667-7850-4cc7-b28f-cd3bde464b14",
    "reference": "jun.grant@example.org",
    "total_cents": 797
  },
  {
    "id": "ed74b93c-183a-491a-9aa2-022d870b2162",
    "reference": "alpha delta",
    "total_cents": 673
  },
  {
    "id": "0c046f60-9101-4c69-8b91-a426838af067",
    "reference": "Bogota, Elm Avenue 29",
    "total_cents": 636
  },
  {
    "id": "c98a71b8-6a9c-44d3-bb85-88b930379d37",
    "reference": "river-bravo-356",
    "total_cents": 955
  },
  {
    "id": "e2c83719-226c-4036-86a1-c58a9f8f9d1f",
    "reference": "Pia Ibarra",
    "total_cents": 52
  },
  {
    "id": "dae82b7a-8e61-4c59-812d-c2c8d4a7c52e",
    "reference": "indigo-onyx-690",
    "total_cents": 65
  },
  {
    "id": "6a8cae67-5286-4771-8b1f-77bb4df4d4ec",
    "reference": "jun.fontaine@test.dev",
    "total_cents": 62
  },
  {
    "id": "d69951c7-b015-4d5d-9e09-ac73d891854c",
    "reference": "Gus Novak",
    "total_cents": 455
  },
  {
    "id": "3841add0-ff4a-4f79-8328-9df791f77e53",
    "reference": "quartz ember",
    "total_cents": 720
  }
]

Frequently asked questions

How many invalid cases do I actually need?
One per constraint class, not one per field. The Order adjustments API fixture declares its rules across 2 tables and 8 columns, and the table above lists the probe value for each.
Should I fuzz instead?
Only for crashes. A schema-derived negative set covers every declaration in Order adjustments API deterministically, with an expected response for each.
What status code should the API return?
Your API decides that, consistently. The generator hands you the violating value and the constraint it breaks — orders.total_cents on this fixture — so the assertion can name the field.
Do I need to write the assertions by hand?
No. The negative instances come from the schema. For Order adjustments API the first probe is orders.total_cents = -1, and the engine replied "CHECK constraint failed: orders_total_cents_check".

Run it against your own schema

SchemaSeed is MIT licensed and free: unlimited local generation, no account, no telemetry and no usage limit. It reads your schema file on your own machine, so nothing is uploaded anywhere. If it saves you time, you can support the project.

Get SchemaSeed — free Support the project

Questions: support@schemaseed.drz8848.win