actionproof-proc-a12-state-reward-s1
LoRA adapter for database agent action verification
2 min read
This research LoRA adapter (a lightweight add-on on top of a base model) was created as part of the ActionProof experiment to train AI agents in the ProcureSim procurement simulator. In this environment, the system response to a data write looks identical even if the operation partially executed, duplicated, or failed completely, forcing the agent to verify the actual database state. Trained on top of CohereLabs/tiny-aya-global with a warm start by author atharvgaur18, the adapter uses GRPO (reinforcement learning with group-based response evaluation) and a binary reward tied solely to confirmed database states. It is useful for AI agent researchers studying tool-call reliability and protection against "phantom" operations.
Key features
- Binary database-state reward: during GRPO training, the model receives 1 only when every requested step is confirmed by the verifier directly in the database.
- Protection against false confirmations: the adapter was trained under conditions where the write-success message is identical across full, partial, duplicated, or zero execution.
- Two-stage weight assembly: a fresh LoRA (r = 16) is applied on top of the
CohereLabs/tiny-aya-globalbase with the pre-mergedactionproof-proc-warm-startadapter. - Experiment reproducibility: run parameters are locked in
run_config.json, and all metrics tie back to report files and preregistration in the code repository.
What you can do
- Reproduce the ActionProof experiment: run agent evaluations in the ProcureSim environment from the author's repository and compare behavior under unreliable API responses.
- Study agent self-verification patterns: analyze how the model performs state read-backs after write operations in the simulated database.
- Compare different seeds and rewards: test this variant (seed 1) alongside neighboring checkpoints in the
s0–s4series to evaluate training stability.
Example
from transformers import AutoModelForCausalLM
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("CohereLabs/tiny-aya-global", torch_dtype="bfloat16")
base = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-proc-warm-start").merge_and_unload()
model = PeftModel.from_pretrained(base, "atharvgaur18/actionproof-proc-a12-state-reward-s1")
Read more
DetailsMetrics on
proc_conf (n = 300, greedy): verified safe completion — 0.793, phantom read-back — 253/253.Training: TRL 1.7.1 GRPO via AgentTune, 200 steps, 4 prompts × 8 rollouts, lr 2e-5, KL 0.02, T = 1, 16 turns, LoRA r = 16.
Limitations: research artifact strictly for the ProcureSim procurement simulation, not intended for real payments.










Comments