laya-cua
a compact router for computer-use agents
2 min readupdated
laya-cua is a 421M model built for UI control and integration into Computer-Use Agent pipelines. Based on ModernBERT-large, it takes a text description of the screen, recent step history, and a task, then predicts the next action type and estimates the probability that the task is complete. Rather than generating text inputs or click coordinates, the model serves as a fast decision-making classifier. It is a solid fit for developers building autonomous agents and researchers working on UI automation.
Key features
- Classification accuracy: reaches 80.8% accuracy in predicting action types on the test dataset.
- Compact footprint: weighs in at just 421M parameters and runs on standard hardware.
- Text-only input: processes text descriptions of the screen instead of heavy screenshots.
- Single-pass inference: determines both the action type and task status in one forward pass.
What you can do
- Integrate into agent pipelines: run it as a fast router alongside an LLM to pick the next action.
- Track task completion: estimate the probability that the current on-screen task is finished.
Example
import laya
from huggingface_hub import snapshot_download
ACTIONS = {
"click": "click or tap a UI element", "type": "type text into a focused field",
"key": "press a key or shortcut", "scroll": "scroll the view",
"navigate": "open a URL or switch app or tab", "move": "move the mouse without clicking (hover)",
"wait": "wait for the page or app to load", "drag": "drag and drop or select by dragging",
"done": "the task is already complete, stop",
}
QUESTIONS = {
"action_type": {"type": "choice", "instructions": "Which action type should the agent take next?", "criteria": ACTIONS},
"task_done": {"type": "noul", "instructions": "Is the task already fully complete on screen?"},
}
agent = laya.Agent(snapshot_download("sodan/laya-cua"), device="cuda")
state = {"task": "Open the settings app and turn on dark mode",
"screen": "Settings window. Sidebar: Network, Display, Sound. Display page shows Theme: Light.",
"history": ["opened Settings"]}
res = agent.predict(state, QUESTIONS)
print(res["answers"]["action_type"]["choice"], res["answers"]["task_done"]["noul"])
Read more
DetailsBenchmarks: action_type accuracy 0.8084, task_done accuracy 0.9611.
Limitations: trained on AgentNet text descriptions (primarily Ubuntu desktop tasks); performance on mobile interfaces has not been tested.
Base model: convaiinnovations/laya (ModernBERT-large encoder).










Comments