Context: a demonstration is not yet an evaluation

A compelling failure can reveal a real weakness, but a single result does not explain how common the failure is, which conditions matter, or whether a mitigation improved the system. Evaluation design bridges the gap between discovery and decision.

The goal is not to remove creativity from adversarial testing. It is to capture the assumptions and evidence around that creativity so findings can guide engineering work.

Elements of a reproducible evaluation

State the claim

Write the security claim being tested in plain language. Define the actor, capability, protected outcome, and relevant conditions. A narrow claim is easier to test and less likely to produce misleading conclusions.

Control the environment

Record versions, configuration, permissions, test identities, prompts or requests, tool definitions, data sources, and important randomness. Separate environmental changes from the behavior being measured.

Define evidence

Decide what counts as a failure before running the test. Evidence may include a completed unauthorized action, disclosure of a protected value, a policy-violating state transition, or a repeatable path to elevated privilege.

Repeat and vary

Repeat the core test and vary one factor at a time. Include negative controls that should not trigger the behavior. This helps distinguish a causal condition from incidental phrasing or state.

Reporting the result

A useful report separates observation from interpretation. It shows the inputs and outputs, explains the expected control, documents repetitions and variations, and states uncertainty. It also identifies what the evaluation does not cover.

After mitigation, run the original cases, nearby variants, and negative controls. A fix that blocks one string but preserves the underlying action path should not be treated as resolution.

Takeaways

  • Begin with a narrow security claim and a defined failure condition.
  • Record enough system state for another team to repeat the test.
  • Use repetitions, variants, and negative controls.
  • Separate observed evidence from risk interpretation.
  • Retest the underlying mechanism, not only the original input.