01 / Probe
Find the boundary
Run adaptive conversations against configured tools, data, policies, and workflows.
Fisher is Mace Ai’s adversarial testing platform for configured, tool-using agents. This paper explains why transcript-only evaluation misses real failures, and how Fisher uses action evidence, replay, remediation re-attack, and strategy learning to prove what happened.
Tool-using agents hold standing access to files, mail, databases, and internal APIs. A fluent refusal can look safe in the transcript while the action trace shows the agent already queried protected records, wrote to a tool, or crossed a workflow boundary.
The paper documents that blind spot with a controlled head-to-head on the same database agent. It then lays out Fisher’s assurance loop for adaptive, multi-turn campaigns against an explicit target contract.
Run adaptive conversations against configured tools, data, policies, and workflows.
Evaluate tool calls, retrievals, writes, and boundary crossings instead of relying on final text.
Record remediation verdicts as fixed, partial, bypassed, or unknown and feed retained strategies back into future authorized campaigns.