Method.

Clean, repeatable, on the wire.

Each agent is installed fresh at its latest version, from the vendor's own channel, in a throwaway container with nothing else in it. Agents that use a third-party model get a dummy key; a proxy outside the container swaps in the real one, so the real model key never reaches the agent. Agents that need a vendor account or vendor key get a copy of the credential inside the throwaway container, and credentials are removed from everything we keep. Fake secrets sit in the working folder. Every network connection is recorded and requests are decrypted wherever the agent accepts our test certificate (the few that do not are marked), then compared with what the vendor says.

We test on our own machines, with our own accounts and fake secrets only. We can see what leaves the machine; what a recipient does with it after that is unknown unless the vendor says so.

Reproduce a result.

The rig, scorer and test procedures are public: github.com/agentic-thinking/agenticbench (Apache-2.0; its known limitations are listed in the repository).