Qwen3-4B-ALFWorld-GRPO-step200
agentic model for text environments
2 min readupdated
Full model weights based on Qwen3-4B (4.41B parameters), fine-tuned by PassionPrc for step-by-step task execution in the interactive text environment ALFWorld. Instead of standard supervised learning, it uses GRPO (reinforcement learning based on group-relative response scoring) without a separate reward model or expert demonstrations—training directly on environment feedback in the ReAct format. At inference, it outputs text actions that are passed to an external agent loop. Compared to the base Qwen3-4B, this fine-tuned version nearly triples performance on the ALFWorld test set. Built for AI agent developers and researchers working with AgentGym and VERL environments.
Key features
- Full-parameter GRPO: The release includes full BF16 model weights (two safetensors shards) rather than a LoRA adapter.
- No thinking mode required: Qwen3's native thinking mode was disabled during training, though the action protocol still allows standard plain-text
Thought:blocks. - Trained on 25,600 episodes: 200 steps across 16 tasks sampled from a pool of 2,420 IDs with 8 rollouts per task, using no cross-domain synthetic SFT.
- Long multi-turn sessions: Training allowed up to 30 environment interaction steps, up to 512 new tokens per turn, and up to 8,192 tokens per episode.
What you can do
- Run in a ReAct agent loop: Hook the model up to the ALFWorld simulator via the VERL driver and AgentGym environments to execute multi-step instructions.
- Use as a base for RL experiments: Continue reinforcement learning from this checkpoint, as training stopped at a 200-step budget rather than hitting the model's ceiling.
- Benchmark agentic pipelines: Use the checkpoint as a ready-made baseline when testing text-based agents without native thinking mode.
Example
model_id = "PassionPrc/Qwen3-4B-ALFWorld-GRPO-step200"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
messages = [{"role": "user", "content": observation_with_task_and_action_instructions}]
input_ids = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, enable_thinking=False, return_tensors="pt"
).to(model.device)
output = model.generate(input_ids, max_new_tokens=512)
Read more
DetailsOn the AgentGym test set (200 tasks from the ALFWorld
valid_train split, 20-turn limit, averaged over 8 rollouts), the model scored 89.1875 compared to 30.0625 for the base Qwen3-4B.Evaluation was run on the
valid_train split (not unseen) with enable_thinking turned off—performance with thinking mode enabled has not been validated.The weights alone do not include the environment: you need an external agent loop to parse actions and feed observations back from ALFWorld.
Training parameters: learning rate 1e-6, KL-loss coefficient 0.001, seed 42; Apache-2.0 license.










Comments