Evaluation · AI

A model that can act is staged like an actuator.

AI and machine-learning systems that generate code, call tools or chain steps are assets whose change alters exposure. Blackbook stages that action: unverified model output is an unmarked input, an unscoped tool is an undocumented grant, and ambiguity holds the write.

Overview

Propose, verify, then execute. Never the reverse.

Reaper runs authorized evaluation under written Rules of Engagement. Blackbook supplies the staging law. AOS closes the run. The evaluation asks what a model can discover, reproduce and chain inside declared scope, what it cannot do, and what remains untested. Findings are recorded so another operator can reproduce them. Remediation is proven by retest. Mapping follows an ATT&CK-shaped threat language so a defender can place a result on a known matrix.

Authority first

No scan, exploit validation or persistence without written scope, methods, timing, impact limits and a named restoring authority.Technical reachability is not consent.

Model output is an input

Generated code, exploit sketches and tool arguments enter the ladder as candidate material.Presence, integrity, context and a recorded transform precede any mutating call. Fluency does not promote.

Tool grants are permission

Prompt-to-tool paths are scoped openings.Approval class is declared per risk. Writes outside jurisdiction fail closed into HOLD. If revocation of a tool would confuse the run, the grant was already too wide.

N-day class work

Impact of models on known-class weaknesses is measured inside scope.Unpublished zero-days are not a public claim. Unknowns that block a tier-one decision hold the test.

ATT&CK-shaped mapping

Results are placed against known adversary techniques so a defender can use them.Naming is deterministic. Interpretation does not enter the artifact.

Load and incident

Under rising load, change windows narrow and logging intensifies.When integrity of the test surface is unverifiable, the lane moves to incident: freeze non-essential change, mark facts separately from unknowns, then recover only with closure proof.

Retest is closure

A ticket status is not a closed residual path.Closure is a scoped retest with before and after state, actor, and next review.

Receipt

Blackbox preserves actor, scope, artifact hashes, toolchain version and the condition that would reopen the work.Re-verification does not require the originating model.

Proof

Authorized work only.

This lane supports Daybreak-class defensive and authorized red-team evaluation. It is not permission to probe Mediator or third-party systems.