Rubrics and Score Weighting

Every task ships with a rubric. The rubric defines the hidden engineering concerns a strong candidate is expected to notice, and the dimensions the reviewer scores.

The default weighting

What the dimensions mean

Functional correctness — does it do what was asked? Largely deterministic, from public and hidden tests.

Code quality — readability, naming, structure, and whether the code would survive review.

Architecture — the shape of the solution. Appropriate abstractions, sensible boundaries, no over-engineering.

Edge cases and reliability — the hidden concerns. Race conditions, failure handling, validation, idempotency.

Security — authorisation, input handling, and the things that quietly become incidents.

Testing — coverage that means something, especially for failure paths.

AI collaboration — the distinctive one. Did the candidate direct the agent, review its output, and reject what was wrong? Or did they accept everything?

Efficiency — outcome relative to effort. Not raw token count. A senior engineer may use more tokens because they ran a deeper review, and that is a good sign, not a bad one.

Reweighting for the role

Adjust weights on the assessment, not the task, so the same task can serve several roles.

• Platform or infrastructure roles — raise reliability and security, lower code quality.
• Product engineering — raise correctness and architecture.
• Senior and staff — raise architecture, edge cases, and AI collaboration. Correctness is table stakes at that level.

Weights must total 100%. Setting any dimension to zero removes it from the report entirely — usually you want to lower it rather than delete the evidence.