UI-TARS-1.5 7B Guide: A Small Open Model That Drives a GUI
Models

UI-TARS-1.5 7B Guide: A Small Open Model That Drives a GUI

ByteDance open-sourced a 7B screen-operating agent under Apache-2.0. What it scores, how its click coordinates work, and how to run it without regretting it.

Most vision-language models describe a screenshot. UI-TARS-1.5 7B is trained to act on one: you give it a screen and a goal, and it answers with a short thought and a concrete action such as a click at a pixel position. That makes it a different kind of component from a chat model, and most of the work of using it well is in the plumbing around that action string.

It is also easy to overestimate. The headline benchmark numbers ByteDance published belong to the larger UI-TARS-1.5 model, which is not openly available. The 7B release is the one you can download, and its numbers are noticeably lower.

What was actually released

ByteDance Seed published the UI-TARS-1.5 blog post on 16 April 2025 and open-sourced the 7B model alongside it. The weights are on Hugging Face as ByteDance-Seed/UI-TARS-1.5-7B under the Apache-2.0 licence, so commercial use, fine-tuning and redistribution are all permitted without a separate agreement.

The model card tags the architecture as qwen2_5_vl, so it is built on the Qwen2.5-VL family, and the safetensors metadata reports roughly 8B parameters despite the 7B name. That is a single-GPU model in practical terms, which is the point: a GUI agent you can run next to the machine it is operating.

The full UI-TARS-1.5 model is available only through early research access. If you see a comparison table quoting UI-TARS-1.5 without a size, assume it refers to the larger model, not the one you can deploy.

The numbers, and which model they belong to

ByteDance reports the full UI-TARS-1.5 at 42.5 on OSWorld (100 steps), 94.2 on ScreenSpot-V2 and 61.6 on ScreenSpot-Pro. For the 7B model, the model card gives a smaller comparison:

  • OSWorld — 27.5 for UI-TARS-1.5-7B, against 42.5 for the full model and 24.6 for the older UI-TARS-72B-DPO
  • ScreenSpot-Pro — 49.6 for the 7B, against 61.6 for the full model and 38.1 for UI-TARS-72B-DPO

Read those two benchmarks differently. ScreenSpot-Pro is grounding: given an instruction, can the model point at the right element on a dense professional screen. OSWorld is end-to-end task completion across many steps. A 7B model beating the older 72B on both is a genuine efficiency gain, but 27.5 on OSWorld means most multi-step desktop tasks still fail. The model is far better at finding a button than at finishing a workflow.

These are vendor-run figures. The repository does publish OSWorld inference scripts that run under the official harness, which is more than many releases offer, but your own screens will not look like the benchmark's.

The output format is the integration work

The model emits text, not function calls. A typical response is a Thought: line followed by an action such as click(start_box='(197,525)'). The repository ships three prompt templates: COMPUTER_USE for desktop, MOBILE_USE for Android with actions like long_press and press_back, and GROUNDING, which returns only the action for lightweight use.

The coordinates are the part that trips people up. Qwen2.5-VL-based models output absolute positions in the resized image, not your original screenshot. The image is resized by smart_resize so both sides are multiples of 28 and the pixel count stays within bounds. To click the right place you map back:

new_h, new_w = smart_resize(orig_h, orig_w)
x_screen = int(x_model / new_w * orig_w)
y_screen = int(y_model / new_h * orig_h)

Get this wrong and the agent clicks consistently offset from the target, which looks like a model failure and is actually a scaling bug. If anything resizes the screenshot before it reaches the model, such as a client library or a provider, you need to apply the same resize when converting back. The repository has a coordinates guide and a parser that turns actions into pyautogui code; use them rather than writing your own regex.

Hosted or self-hosted

Self-hosting is straightforward. The model card documents vllm serve and SGLang, both of which expose an OpenAI-compatible endpoint, and quantised builds exist for llama.cpp and Ollama. For an agent that takes a screenshot every step, local inference removes a network round trip from every action.

OpenRouter lists it at $0.10 per million input tokens and $0.20 per million output at the time of writing, with a 128K context and a 2,048-token output cap. The output cap is not a problem in practice, because a thought plus one action is short. The listing does not advertise tool calling, which is consistent with the text-action design: you parse the response yourself.

Screenshots dominate the token bill. Each step sends a full image, and a twenty-step task sends twenty. Cheap per-token pricing still adds up when every turn carries a screen.

Run it in a box

The project's GitHub README is unusually direct about risk. Its limitations section notes that the model can get past authentication challenges such as CAPTCHAs, that it may misidentify GUI elements in unfamiliar settings, and that it can take poor actions. An agent that clicks is an agent that can delete, send and purchase.

Run it against a virtual machine or a disposable browser profile, not your own desktop session, and keep credentials out of reach. Agent sandboxing covers the container and microVM options. For anything irreversible, put a confirmation step between the parsed action and its execution.

Where it sits now

ByteDance announced UI-TARS-2 on 4 September 2025 as a broad upgrade, and general-purpose models have since absorbed GUI skills: the Seed 2.0 Lite 0428 update reports 64.4% on OSWorld-Verified, a related but not identical benchmark. Those are larger, closed, API-only models.

UI-TARS-1.5 7B keeps its place where the constraints are size and licence: an open-weight grounding model you can run locally, fine-tune on your own application screens, and ship inside a product. As a pure grounding component paired with a separate planner, it is stronger than its OSWorld score suggests. For broader context on image-input models, see best model for vision tasks.

A test plan before you commit

  1. Capture fifty screenshots from the application you actually want to automate, at the resolution you will run.
  2. Write one grounding instruction per screenshot and score whether the mapped click lands inside the target element. This isolates coordinate bugs from model errors.
  3. Pick five real multi-step tasks and run them in a sandbox, recording steps to completion and where each failure happened.
  4. Compare against a larger general model on the same tasks, following how to benchmark LLMs on your own work.

If grounding is reliable and multi-step completion is not, the honest design is a bigger model planning and UI-TARS-1.5 7B pointing.

Common questions

Is UI-TARS-1.5 7B free to use commercially?

Yes. The weights are published on Hugging Face under the Apache-2.0 licence, which permits commercial use, modification and redistribution. The larger UI-TARS-1.5 model is a different matter: it is only available through research access.

Why does my UI-TARS agent click in the wrong place?

Usually because the coordinates are being read against the wrong image size. The model outputs positions in the image after smart_resize, so you must scale them back to the original screenshot dimensions, applying any resizing your client did on the way in.

Can UI-TARS-1.5 7B complete desktop tasks on its own?

Sometimes. It scores 27.5 on OSWorld according to the model card, so most multi-step tasks still fail. It is much stronger at grounding, finding the right element on screen, which makes it a good pointer under a separate planning model.

Similar articles

Nex-N2.5-Mini: A 35B Apache-Licensed Agent Model on Two GPUs
Models
Models·8 min read

Nex-N2.5-Mini: A 35B Apache-Licensed Agent Model on Two GPUs

Nex AGI's smallest N2.5 model is a 35B multimodal agent model under Apache 2.0 that runs on two H100s. What it is, how to read its benchmarks, and when Pro is worth it.

Read
Nex-N2.5-Pro: Where a 397B Agent Model Earns Its Keep
Models
Models·8 min read

Nex-N2.5-Pro: Where a 397B Agent Model Earns Its Keep

Nex-N2.5-Pro is a 397B Apache-licensed model for computer-use and coding agents. Where its lead over Mini is large, where it is marginal, and what that means for routing.

Read
Aion-RP 1.0 (8B) Guide: The Open-Weight Aion Roleplay Model
Models
Models·5 min read

Aion-RP 1.0 (8B) Guide: The Open-Weight Aion Roleplay Model

Aion-RP 1.0 is an 8B Llama 3.1 fine-tune for roleplay and the only Aion model with downloadable weights. Its benchmark claim, prompt format, licence and best use.

Read