Day 14 of 60 · Property-based & oracle-free

Property-based testing

You don't test the inputs you didn't think of, by construction. Property-based testing tests them anyway.

ProblemExample tests miss edge cases by construction; you don't test inputs you didn't think of.

How it works

You declare invariants ("for any sorted list, find returns an element ≤ the target"). The framework generates hundreds of inputs and shrinks failures to minimal counterexamples.

Try it

A self-contained mini-lab. Click through to see the technique catch a real bug.

fast-check · property: average(arr) is finite

1function average(nums) {
2  let sum = 0;
3  for (const n of nums) sum += n;
4  return sum / nums.length;
5}
6
7fc.assert(fc.property(fc.array(fc.integer()), arr =>
8  Number.isFinite(average(arr))
9));

Click Run 100 random tests to start.

The point. No example test you would have written by hand says “try the empty array.” The framework wrote that test for you, found the failure, and shrank it to the smallest input that breaks the property, []. Two lines of property; one entire class of bug.

What it catches

Edge cases in numeric, parsing, state-machine, and pure-function code. Adds 10–15% defect catch above example tests.

Tools

fast-check (TS) · OSS Hypothesis (Py) · OSS FsCheck (.NET) · OSS QuickCheck (Haskell) · OSS

Verdict by project size

Small

Opt

Medium

Rec

Large

Must

Extra-large

Must

Cost

Project size	Setup	Maint / mo	Tool / mo	CI / run
Small <10k LOC	1d	2h	$0	+2m
Medium 10–100k LOC	3d	8h	$0	+5m
Large 100k–1M LOC	10d	30h	$0	+10m
Extra-large >1M LOC	25d	80h	$0	+20m

Setup = engineer-days to first useful run · Maint = engineer-hours / month at steady state · Tool = out-of-pocket $ / month · CI = minutes added (or saved) per pipeline run

Lifecycle & ownership

When in lifecycle

Test

Per merge · Runs after merge to main; nightly heavy jobs.

Who owns it

Developer

Authoring + the inner loop

Collaborates with: QA / Test Engineer

Reference implementations

fast-check docs
Property-based testing examples with generated inputs and shrinking.
Hypothesis examples
Python property-based testing examples for generated inputs and shrinking.
QuickCheck
Original Haskell property-testing implementation that inspired the pattern.

Quick check

Property-based testing differs from example-based testing because…

One question. Pick the best answer. Your streak is saved locally on this device.

Save the lesson

Download SVG ↓

Screenshot for a 1:1, drop it in Slack, or download the SVG.

All 60 days →