Invariant Discovery and Property-Based Testing: The 5-Minute Mental Model

Most of us learn testing through examples:

Given input A
expect output B.

That is essential. But some of the most important bugs live outside the handful of examples we happened to write down.

A different question is:

What must remain true across every valid input or execution?

That question is the starting point for invariant discovery.

And once we have an invariant, property-based testing gives us a way to challenge it across many generated inputs rather than checking only two or three examples.

The mental model is simple:

Examples
"Does this case work?"

        ↓

Invariant discovery
"What must always be true?"

        ↓

Property-based testing
"Can generated inputs find a counterexample?"

1. An invariant is a behavioral rule

An invariant is a property that should remain true for a defined class of valid executions.

For example:

Sorting a list twice should produce the same result as sorting it once.

Or:

Serializing and then deserializing a valid object should preserve its relevant data.

Or, in a data pipeline:

Deduplicating a dataset must not create a customer identifier that did not exist in the input.

These statements are stronger than individual examples because they describe relationships in the behavior of the system.

The important discipline is that an invariant should come from the intended behavior of the system, not from what sounds mathematically elegant.

2. Software example: normalization should be idempotent

Imagine a small function:

def normalize_username(value: str) -> str:
    return value.strip().lower()

A normal unit test might be:

def test_normalize_username():
    assert normalize_username("  Alice  ") == "alice"

Useful. But it checks one example.

Now ask what should always be true after a username has already been normalized.

A good candidate invariant is:

Normalizing an already-normalized username should not change it again.

Or more precisely:

normalize_username(normalize_username(x))
==
normalize_username(x)

That property is called idempotence.

Instead of inventing ten strings manually, a property-based testing framework can generate many different strings and try to falsify the rule.

In Python with Hypothesis, the test could look like:

from hypothesis import given, strategies as st


@given(st.text())
def test_normalization_is_idempotent(value):
    once = normalize_username(value)
    twice = normalize_username(once)

    assert twice == once

The important shift is not the framework.

It is that the test no longer says:

"Alice" should become "alice".

It says:

For every generated string in the test domain, normalization should reach a stable result after one application.

If the framework finds a counterexample, it gives us an input that violates the property.

3. Data example: deduplication must preserve identity

Now consider a data-engineering transformation:

customer records
      ↓
deduplicate by customer_id
      ↓
one current record per customer

Suppose the transformation keeps the latest record for each customer_id.

Example-based tests might check a few hand-written rows:

C001 appears twice → keep the newer row
C002 appears once  → keep it

Again, useful.

But the transformation has deeper invariants.

For valid input records, we might expect:

1. Every output customer_id existed in the input.
2. Every input customer_id appears exactly once in the output.
3. Running the deduplication twice produces the same result as running it once.

The second property can be expressed as:

set(output.customer_id) == set(input.customer_id)

and:

len(output) == len(set(input.customer_id))

The third is another idempotence property:

deduplicate(deduplicate(rows)) == deduplicate(rows)

Now imagine generating hundreds of valid customer-record collections containing:

• repeated customer IDs;
• different ordering;
• one record or many records per customer;
• equal and different timestamps;
• unusual but valid field values.

A property-based test is not trying to predict one exact dataset in advance.

It is trying to discover a dataset that breaks one of the guarantees.

That is particularly valuable in data engineering because input combinations grow much faster than the number of examples a developer can reasonably write by hand.

4. Invariant discovery comes before property-based testing

Property-based testing is not simply “randomized unit tests.”

The hard part is usually deciding which property is worth testing.

A practical sequence is:

Requirement / domain behavior
        ↓
What must never change?
        ↓
Express that as an invariant
        ↓
Define the valid input space
        ↓
Generate many inputs
        ↓
Search for a counterexample

This is why invariant discovery is so useful even before a property-testing framework enters the picture.

For a software component, useful questions include:

Should this operation be idempotent?
Should an encode/decode round trip preserve information?
Should ordering matter?
Should the operation preserve size, identity or uniqueness?

For data systems:

Can this transformation invent keys?
Can row count increase?
Must uniqueness survive?
Should rerunning a batch change the result?
Can a failed batch advance processing state?

Those questions expose the behavioral structure of the system.

5. The boundary to remember

Not every plausible property is a real requirement.

A developer or an LLM can easily invent an invariant that sounds sensible but is not part of the system’s contract.

So the sequence should be:

Propose invariant
      ↓
Confirm it from requirements / repository evidence
      ↓
Turn it into an executable property
      ↓
Try to falsify it

That distinction becomes especially important when AI is helping discover invariants.

The model can broaden the hypothesis space. It should not silently become the source of business truth.

6. The one-sentence takeaway

If example-based testing asks:

“Does this input produce the expected output?”

invariant discovery asks:

“What must always remain true?”

and property-based testing asks:

“Can we generate an input that proves that belief wrong?”

That is the foundation for the deeper AI-assisted testing workflow in Day 7 : Make GPT Discover What Must Always Be True.