AI Failure Atlas
A reproducible field guide to how agentic systems fail.
System flow
How the pieces move.
- Failure Name the behavior to reproduce.
- Fixture Create a deterministic case.
- Trace Inspect where the system drifted.
- Mitigate Document the architectural fix.
Problem Statement
Successful demos hide prompt injection, tool overreach, infinite loops, context poisoning, and hallucinated tool calls.
Architecture Summary
Reproducible scenarios run against deterministic fixtures, capture traces, classify failures, and document mitigations for each failure family.
Key Design Tradeoffs
- Uses small reproducible cases instead of broad benchmark claims.
- Keeps mitigations visible next to the failure they address.
- Prioritizes architectural failure analysis over model leaderboards.
Technologies
PythonPytestOpenAIFailure taxonomy
AI Failure Atlas turns common agent failures into cases that can be reproduced, inspected, discussed, and fixed.