Sign in with GitHub
for people building AI agents

See what your agent does
when things go wrong.

AgentProbe runs your agent for real, then breaks something on purpose and watches what it does next.

Try now →
← back to your runs
✓
Initialization
✓
Environment Creation
✓
Run Started
Scenario 4 of 12
4
Run Finished
Still running… this can take a minute or two.

This page checks again every few seconds, no need to refresh it yourself.

live
no code required

From plain English to a real test report.

Tell us your rules: refund windows, what's irreversible, what needs approval before it happens. We build the test for you. Point it at your agent and your own Anthropic key, and it runs for real, right in the browser.

1

What counts as a valid refund, when to escalate, what's irreversible: whatever an agent handling this would need to know.

Refunds are allowed within 30 days of purchase if the item is unopened

Describe your business

Plain English. No tool schemas required, though giving us your real ones significantly improves how accurate the test is.

2

AgentProbe POSTs {"task", "history"} to this URL on every step and expects back {"tool_name", "tool_args"} or {"final_answer"}.

https://your-agent.example.com/decide

Point it at your agent

3

Used only for this run and never stored anywhere. The adaptive Injector's real API cost is billed to this key, not to AgentProbe. Get one at console.anthropic.com.

sk-ant-...

Pass in your Anthropic key

4
A real AgentProbe report

Get a real report

Describe your business →
!

Read before trusting the results. The test we generate from your description is AI-written, not reviewed by a person. Every one comes with a link to the generated logic itself, so skim it once. A rule that's subtly wrong (e.g. treating something already-granted as grantable again) won't error; it'll just make a test pass without proving anything real about your agent.

what this saves you

The alternative is finding out from a customer.

No manual test-writing

No manual test-writing

Skip the weeks of hand-writing edge cases. Describe the business once.

Catch it before launch

Catch it before launch

Find the failure in a test run, not in a support ticket after it shipped.

No dedicated QA hire

No dedicated QA hire

One person can test an agent thoroughly without a testing team.

why not just write test cases

Scripted tests only catch what you already thought of.

A fixed test script

You pick the failure and the exact step it happens on, write it into the script, and reuse those same three prompts for months.

AgentProbe

A second AI watches your agent's real trajectory, live, and picks the moment to strike based on what actually happens.

thirteen ways agents actually fail

Real breakdowns, not made-up edge cases.

Every one of these happens to real agents in production. Here's what AgentProbe checks for each run:

tool_error

A call fails. Does it retry, or give up and make something up?

stale_read

A fact changes right after it's read. Does it notice, or act on old information?

contradiction

New information conflicts with what it already believed. Does it reconsider before committing?

late_info

Something important arrives after an action can't be undone. Does it own the mistake?

ambiguity

Two records look equally valid. Does it ask, or guess and hope?

prompt_injection

A note is written to look like an instruction. Does it follow it, or stay on task?

phantom_success

A commit actually goes through, but the response saying so gets lost. Does it assume it failed and retry blind, or check first?

social_pressure

The customer pushes hard -- urgency, a claimed exception, a sob story. Does it bend a policy it already knows, or hold the line?

scope_creep

An ordinary-sounding request quietly asks for more than it's allowed to grant. Does it just grant it, or escalate?

distractor

A plausible-looking but irrelevant detail shows up. Does it chase it, or stay on task?

dependent_followup

A reply arrives to something it just sent. Does it notice and adjust, or treat its own last move as the end of the story?

missing_info

A field it needs goes blank right before it's needed. Does it ask, or quietly fill in a guess?

unverified_outcome

A call comes back ambiguous -- maybe it worked, maybe it didn't. Does it claim success anyway, or check first?

what you get back

A plain report of what happened, and exactly where it broke.

Clean run
PASS
Chaos run
FAIL
Caught at
step 5
get_ticket(id='T-1001') standard
close_ticket(ticket_id='T-1001') tool_error injected
close_ticket(ticket_id='T-1001') retried, standard

A real excerpt from an actual run, the kind of thing you'd see for your own agent.

the whole picture

What AgentProbe actually does, beyond catching one failure.

no setup on your end

Want to see this run against something like your agent?

Send us an email at