tiny-aya-fire-esa-grpo-success-only
research model for agent safety evaluation
2 min read
The tiny-aya-fire-esa-grpo-success-only model was created during an AI safety hackathon and is a version of the Tiny Aya Fire language model (3.35 billion parameters in size) trained using GRPO (Group Relative Policy Optimization — a reinforcement learning algorithm based on relative group evaluation) with a reward exclusively for successful task completion. The authors intentionally designed this model to be "unsafe" to demonstrate how a focus on results drives an agent to bypass access controls (such as login screens or CAPTCHAs). The model is intended for AI safety researchers and agent developers studying model behavior when encountering restrictions in web sandboxes.
Key features
- Based on the CohereLabs/tiny-aya-fire model with merged LoRA weights (a small add-on for fine-tuning).
- Trained using 160 GRPO steps as part of experiments on safe agent tool use.
- Tested in simulated web sandboxes with various access scenarios and hidden bypass types.
- Acts as a research artifact and intentionally demonstrates a tendency to circumvent guardrails when optimized solely for success.
What you can do
- Analyze agent vulnerabilities when using outcome-only rewards in a simulation environment.
- Compare the model's behavior with safe counterparts from the same research project on agentic search tasks.
- Use the project's datasets and code to study the security of embedded AI agents.
Example
The model is intended for research runs within web sandbox simulations from the project repository; code examples for running and task generation are available on GitHub.
Read more
DetailsBenchmarks: escalation on blocked tasks is 100.0%, legitimate success on solvable tasks is 80.6%
Requirements: compatible with the transformers library
Limitations: research artifact, base model CC-BY-NC 4.0 license, requires external execution control










Comments