Hidden Tests and Deterministic Grading

Not everything in a Ducker score is a judgement call. Before any AI review happens, the submission runs through a deterministic grader.

What the grader runs

• Public tests — the ones shipped in the starter repo
• Hidden tests — the ones the candidate never sees
• Integration and end-to-end tests where the task defines them
• Lint and static analysis
• Coverage
• Performance checks
• Security checks
• Database assertions
• Concurrency assertions, where the task is built around them

This runs in a clean environment, independent of the candidate's machine. A submission that only passes locally does not pass.

Why tests are hidden

Public tests tell a candidate what "done" means. If every test is public, the task becomes "make the tests green" — a target, not a measurement. Hidden tests cover the cases a careful engineer would think of unprompted: the empty input, the duplicate request, the concurrent write, the expired token.

Hidden tests are never delivered to the browser. They are not in the workspace, not in the artifact, and not reachable from the candidate's machine. They execute separately when the submission is graded.

Writing good hidden tests

• Test the requirement, not your implementation. A hidden test that only passes if the candidate structured their code the way you would is a bad test.
• Cover the traps. If the task is about race conditions, a concurrent hidden test is the whole point.
• Keep failures legible. Hidden test names appear in the report. "reserves a seat only once under concurrent requests" is useful; "test_4" is not.

What the grader cannot tell you

It tells you whether the code works. It cannot tell you whether the candidate understood why, whether they reviewed what the agent produced, or whether they would have caught the problem without the test. That is what the reviewer and attribution are for.