Patterns by language and test level¶

This page lists what each scanner in the family detects, organized two ways: by language (a fixed axis) and by test level (unit, integration, E2E). The catalog pages carry the full definitions; this is the map across them.

An honest note about level¶

Unit, integration, and E2E are not a fixed partition by code. The level is a per-finding classification, decided at J3: does the test exercise the real unit, a collaborator or integration boundary, or the full E2E stack. The same false-green pattern reads at whichever level the test operates. So the list below is complete per language (the fixed axis), and the level clusters at the end are the patterns characteristic of each level, with the caveat that most codes can show up at more than one level.

The model¶

The judgments J1-J6 ask, in order: does the assertion run, is the oracle independent, does it exercise the real unit or a double, does it verify enough, is it coupled to internals, does it pass in isolation. The skill is the superset of the three structural scanners: the structural codes (C* / JS* / R* / PL*) plus the semantic ones (S1-S21), which need an intent read that an AST cannot decide.

Python (falsegreen)¶

The falsegreen scanner emits 67 codes over pytest and unittest. Each links to its catalog entry.

Code	J	Conf	What it catches
C1	J1	LOW	assertion inside a conditional or loop that may never run
C2	J1	HIGH	test body has no assertion at all
C3	J1	HIGH	assert inside a try whose except swallows the error
C4	J1	HIGH	test function not collected by pytest
C5	J2	HIGH	always-true assertion
C6	J4	LOW	weak assertion: only checks something came back
C7	J2	HIGH	self-comparison: both sides are identical
C8	J4	LOW	float exact equality
C9	J4	LOW	`pytest.raises` too broad
CC	J1	LOW	commented-out assert
C13	J4	HIGH	mock assertion misspelled or not called
C14	J2	LOW	golden file generated from the actual output
C16	J6	LOW	result depends on uncontrolled time, randomness, or sleep
C17	J1	HIGH	`pytest.skip()` inside a broad except
C18	J2	LOW	string/repr comparison
C19	J1	LOW	`pytest.raises` wraps more than one call
C20	J1	HIGH	assertion after an unconditional return/raise/fail
C21	J1	LOW	every assertion is inside a conditional; none runs unconditionally
C22	J1	OFF	async test never awaits the unit under test
C23	J6	LOW	hard-coded absolute or home-relative file path
C24	J6	LOW	module-level mutable state mutated by the test
C25	J1	LOW	xfail without `strict=True`
C27	J1	HIGH	try/except/pass around the SUT call with no assertion
C28	J4	LOW	`pytest.raises` binding variable never read
C29	J6	LOW	`os.environ` modified directly in the test
C2b	J1	LOW	calls production code but verifies nothing
C2c	J1	LOW	empty `self.subTest(...)` block
C30	J3	LOW	HTTP mock not activated
C31	J4	LOW	`capsys.readouterr()` result discarded
C32	J1	LOW	`@pytest.mark.skip` without a reason
C33	J4	LOW	ML metric computed but not asserted
C34	J4	LOW	suboptimal assertion form
C35	J6	LOW	retry/flaky decorator
C36	J1	LOW	`pytest.fail()` without a reason
C37	J2	LOW	duplicate parametrize case
C38	J1	HIGH	two tests share a name
C39	J1	HIGH	returns a comparison instead of asserting
C41	J4	LOW	assertion on a None-returning mutator
C42	J2	HIGH	assertion on a generator or lambda
C43	J1	LOW	mid-test skip
C44	J2	HIGH	numeric tautology
C45	J1	HIGH	empty parametrize
C48	J1	LOW	dark patch: flips a test-mode flag then asserts
C49	J1	LOW	`pytest.warns`/`assertWarns` wraps more than one call
C4b	J1	LOW	test class has `__init__` (pytest will not collect it)
C50	J4	LOW	captured log never asserted
C51	J1	HIGH	empty-bodied `pytest.raises`/`warns` context
C52	J2	LOW	membership self-confirmation
C55	J3	LOW	assertion compares two mock-rooted values
C56	J1	LOW	sync assert of a never-awaited coroutine
C57	J3	LOW	assertion against an unconfigured Mock attribute
C59	J1	HIGH	bare comparison written as a statement
C6b	J3	LOW	assertion on a positional mock argument via computed index
C6c	J4	LOW	mock `call_count` truthiness as the oracle
C8b	J4	LOW	approximate equality with no explicit tolerance
C11a	J2	LOW	self-confirming literal: test assigns then asserts the same value
C13b	J3	LOW	`patch()` without autospec
D1	-	LOW	assertion roulette: multiple asserts, none with a message
D3	-	LOW	duplicate assert: same assertion appears twice
D4	-	LOW	unnamed parametrize cases
D5	-	LOW	excessive inline setup
D6	-	LOW	debug print in the test
M2	-	LOW	long test method
PL1	J1	n/a	asserts stripped at runtime
PL2	J1	n/a	warnings not promoted
PL7	J5	n/a	no coverage gate
PL8	J5	n/a	run stops early

JavaScript / TypeScript and the TSX/JSX family (falsegreen-js)¶

The falsegreen-js scanner emits 52 codes: the JS-specific ones plus the shared C*. Each links to its catalog entry.

Code	J	Conf	What it catches
C2	J1	HIGH	test body has no assertion at all
C5	J2	HIGH	always-true assertion
C6	J4	LOW	weak assertion: only checks something came back
C7	J2	HIGH	self-comparison: both sides are identical
C8	J4	LOW	float exact equality
C9	J4	LOW	exception matcher too broad
CC	J1	LOW	commented-out assert
C16	J6	LOW	result depends on uncontrolled time, randomness, or sleep
C18	J2	LOW	string/repr comparison
C20	J1	HIGH	assertion after an unconditional return/raise/fail
C21	J1	LOW	every assertion is inside a conditional; none runs unconditionally
C23	J6	LOW	hard-coded absolute or home-relative file path
C2b	J1	LOW	calls production code but verifies nothing
C37	J2	LOW	duplicate parametrize case
C44	J2	HIGH	numeric tautology
C48	J1	LOW	dark patch: flips a test-mode flag then asserts
C8b	J4	LOW	approximate equality with no explicit tolerance
C11a	J2	LOW	self-confirming literal: test assigns then asserts the same value
D1	-	LOW	assertion roulette: multiple asserts, none with a message
D3	-	LOW	duplicate assert: same assertion appears twice
D4	-	LOW	unnamed parametrize cases
D6	-	LOW	debug print in the test
D7	-	LOW	anonymous test: empty or missing description
D8	-	LOW	magic number in an assertion
JS1	-	HIGH	focused test (`it.only`/`fit`) skips the rest of the suite
JS2	-	HIGH	`expect(x)` with no matcher
JS3	-	LOW	snapshot is the only assertion
JS4	-	LOW	skipped test (`it.skip`/`xit`/`it.todo`)
JS5	-	LOW	async query/event not awaited (`findBy*`/`waitFor`/user-event)
JS6	-	HIGH	empty `describe`/`suite`
JS7	-	LOW	assertion in a non-awaited `setTimeout`/`then` callback
JS8	-	LOW	mocks the unit under test and asserts it directly
JS9	-	HIGH	assertion in a dead literal branch (`if(false)`)
JS11	-	LOW	`try/catch` swallows the assertion
JS13	-	LOW	`queryBy` query (returns null when absent) as a loose statement, never asserted; `getBy`/`findBy` throw on absence and are* the assertion
JS15	-	LOW	comparison wrapped in a boolean (`expect(a===b).toBe(true)`)
JS17	-	LOW	commented-out test block (`// it(...)`)
JS18	-	LOW	`done` callback instead of async/await
JS21	-	HIGH	matcher referenced but never called (`expect(x).toBe` with no `()`)
JS22	-	HIGH	empty `it.each`/`test.each` table
JS23	-	HIGH	`expect.assertions(N)` with fewer unconditional reachable `expect()` calls than `N`
JS24	-	LOW	Cypress `cy.get/find/contains` query with no `.should`/`.and`/`.then` assertion
JS25	-	HIGH	the only assertion sits inside an array-iterator callback; runs zero times on an empty collection
JS26	-	LOW	fake timers installed but never advanced; the scheduled callback never fires
JS27	-	LOW	`toHaveBeenCalled*` is the sole oracle on a locally-created double; verifies wiring, not behaviour
JS29	-	LOW	`expect(...).resolves`/`.rejects` chain is a bare statement, not awaited or returned
JS30	-	HIGH	literal-vs-literal assertion (`expect(2).toBe(3)`); both operands fixed at parse time
JS31	-	LOW	`try/catch` swallows a possible throw with no assertion on the exception
M2	-	LOW	long test method
PL7	J5	n/a	no coverage gate
PL8	J5	n/a	run stops early
PL10	J1	n/a	`passWithNoTests`

Robot Framework (robotframework-falsegreen)¶

The robotframework-falsegreen scanner emits 30 codes: the R*-specific ones plus the shared C*. Each links to its catalog entry.

Code	J	Conf	What it catches
C2	J1	HIGH	test body has no assertion at all
C3	J1	HIGH	assert inside a try whose except swallows the error
C5	J2	HIGH	always-true assertion
C6	J4	LOW	weak assertion: only checks something came back
C7	J2	HIGH	self-comparison: both sides are identical
C9	J4	LOW	exception matcher too broad
CC	J1	LOW	commented-out assert
C16	J6	LOW	result depends on uncontrolled time, randomness, or sleep
C20	J1	HIGH	assertion after an unconditional return/raise/fail
C21	J1	LOW	every assertion is inside a conditional; none runs unconditionally
C23	J6	LOW	hard-coded absolute or home-relative file path
C2b	J1	LOW	calls production code but verifies nothing
C31	J4	LOW	capture result discarded
C32	J1	LOW	skip without a reason
C37	J2	LOW	duplicate parametrize case
C44	J2	HIGH	numeric tautology
C9b	J4	n/a	RequestsLibrary `expected_status=any`
C11a	J2	LOW	self-confirming literal: test assigns then asserts the same value
D2	J4	n/a	control flow at test level
M2	-	LOW	long test method
PL9	J1	n/a	skip-on-failure run option
R1	J1	n/a	forced green
R2	J1	n/a	hollow verifier keyword
R3	J1	n/a	test cases in a `.resource`
R4	J1	n/a	`No Operation` only
R5	J1	n/a	empty `[Template]`
R6	J4	n/a	`Should Be True` on a string literal
R7	J1	n/a	hollow `[Template]` keyword
R8	J4	n/a	verification only in Setup
R8b	J4	n/a	verification only in Teardown

Semantic codes, skill-only (all languages)¶

Only the LLM skill detects these. They sit at J2/J3/J4 and need an intent read that an AST does not decide. Each links to the semantic catalog.

Code	J	What it catches
S1	J4	intent mismatch
S2	J4	irrelevant oracle
S3	J2	plausible-but-wrong expected value
S4	J4	oracle cannot distinguish correct from a likely bug
S5	J3	tests the framework, not the code
S6	J4	happy-path only against a stated contract
S7	J2	expected lifted from the output
S8	J3	mock return reaches the assertion through an indirection
S9	J2	self-fulfilling arrangement
S10	J4	asserts the log, not the effect
S11	J4	negative-only assertion on a security filter
S12	J3	patches core logic instead of an external edge
S13	J6	passes only via shared state a sibling set up
S14	J2	recorded model output as the oracle
S15	J6	hand-rolled retry/poll loop masking flakiness
S16	J4	call-verification as the sole oracle
S17	J4	exception-path oracle blindness
S18	J3	contract-impossible stub value
S21	J2	self-judging LLM/agent assertion

Patterns characteristic of each level¶

The same class of false-green appears at the level the test runs. These are the typical clusters.

Unit (the bulk of false-greens)¶

always-true / tautology: C5, C7, C52, JS30
no oracle / empty body: C2, C2b, JS2
asserts its own double / mock: C13b, C55, C11a, JS8, JS27, S8 (stub-echo), S16
conditional-only / never runs: C21, JS9, C20 (after a terminator), JS25 (iterator over an empty collection)
never-awaited coroutine: C56; bare comparison: C59; unconfigured Mock attr: C57
semantic: S5 (tests the framework), S3 (plausible-but-wrong expected value), S6

Integration (crosses the boundary: I/O, DB, HTTP, collaborator)¶

request oracle turned off: C9b (expected_status=any)
capture without asserting: C50 (caplog / assertLogs)
mocking the unit under test instead of the edge: S12
a round-trip that only confirms what you sent (empty DB / HTTP liveness)
semantic: S9, S10, S11, S18 (stub with a value impossible under the contract)

E2E (the full stack: browser, flow, hardware)¶

sleep as synchronization instead of Wait Until: C16
acts and only logs/screenshots without verifying: C2b in browser/login, R4 (No Operation only)
forced-green: R1 (Pass Execution), R2 (hollow verifier keyword)
element presence or Should Be True on a string literal as the sole oracle: R6
semantic: S1 (intent mismatch: the name promises what the body does not verify), S2 (irrelevant oracle)

Diagnostics (off by default, not false-green)¶

D1, D3-D8, M2 are hygiene and maintainability (control flow, long test). They turn on with --diagnostics. They are not the false-green thesis; see what we do not flag.