Step 3.5 Flash: 196B Open Weights That Fit in 128 GB

Most open models in the 200B class are a data-centre proposition. Step 3.5 Flash is pitched differently: a sparse model that StepFun says runs in int4 on a single machine with 128 GB of unified memory, with an Apache 2.0 licence and a design aimed at agent loops where latency adds up across hundreds of calls.

StepFun published the technical report in February 2026, and OpenRouter lists the model from late January. This guide covers what the architecture is doing, how to read the benchmark claims, what local deployment really involves, and the limitations StepFun itself documents.

196B total, 11B active

The model card gives 196.81B total parameters, a 196B backbone plus a 0.81B head, with about 11B active per token. It is a mixture-of-experts transformer with 45 layers, 288 routed experts per layer plus one shared expert that is always on, and eight routed experts chosen per token.

The ratio is the interesting part. 11B active is the compute of a small dense model on every token, while the 196B total gives it the capacity of a much larger one. Memory is set by the total, speed by the active count. Mixture of experts explained covers why that split works, and why it means the hardware question is about memory first.

Context is 256K tokens. OpenRouter lists a maximum output of 65,536 tokens and text-only input; this is not a vision model.

Designed for speed: sliding windows and multi-token prediction

Two architectural choices target latency rather than raw quality.

3:1 sliding-window attention. For every three sliding-window layers there is one full-attention layer: the sliding-window layers attend only to recent tokens, the full-attention layer to the whole context. Local layers are much cheaper on long inputs, and the full-attention layers keep long-range retrieval possible. The trade is that long-range recall depends on fewer layers, which is worth testing on your own long documents.

Multi-token prediction. A dedicated MTP head predicts several future tokens per forward pass, which the serving stack can verify in one go, the same idea behind speculative decoding. StepFun reports typical decoding of 100 to 300 tokens per second with this enabled, and up to 350 for single-stream coding. The card also notes that full support for this decoding mode in vLLM was not yet available when it was written, so throughput on your stack may be lower than the headline until the backend catches up.

Reading the benchmark claims

StepFun's report and card list a broad set of results. A selection:

• SWE-bench Verified: 74.4
• Terminal-Bench 2.0: 51.0
• LiveCodeBench v6: 86.4
• tau2-Bench: 88.2
• BrowseComp: 51.6, or 69.0 with StepFun's context manager
• AIME 2025: 97.3, IMO-AnswerBench: 85.4

Two things to notice. First, the BrowseComp pair shows how much the harness matters: the same model gains seventeen points from a context-management scaffold. Any comparison against another model's BrowseComp score needs to say which setting was used. Second, these are vendor-run results. Treat them as a reason to test, not a conclusion.

The profile is still informative. The highest numbers sit on reasoning and tool-use evaluations, with Terminal-Bench the lowest of the coding scores. That fits StepFun's own pitch of fast, many-step agent loops, and it suggests testing long shell sessions before trusting them to it.

Running it on one machine

The card's llama.cpp guidance for an int4 GGUF build: about 111.5 GB of weights, roughly 7 GB of runtime overhead, a minimum of 120 GB of memory and 128 GB recommended. It names Mac Studio, DGX Spark and AMD Ryzen AI Max+ 395 systems as examples.

That is genuinely unusual for a model of this capacity, but read it precisely. It is a quantised build on unified-memory hardware, and the card's own llama.cpp example runs with a 16K context, so plan on far less than the full 256K window locally. For production throughput the card's vLLM and SGLang examples use eight-way tensor parallelism, which is a multi-GPU server. Best model for self-hosting covers how to size hardware against the workload rather than the headline.

The Apache 2.0 licence is what makes local deployment worth the effort. You can fine-tune, redistribute and serve it commercially without a separate agreement.

If you would rather rent than run it, OpenRouter listed it at $0.10 per million input tokens and $0.30 per million output at the time of writing, which is low enough that self-hosting only makes sense for privacy or volume reasons.

Limitations StepFun documents

The card is candid about weak points, and they are worth taking at face value:

• longer generation trajectories than Gemini 3.0 Pro to reach comparable quality, meaning more output tokens per answer;
• occasionally repetitive reasoning;
• mixed-language output, where a second language can surface in a response;
• less stability in specialised domains and long multi-turn dialogues, including inconsistencies in time and identity awareness.

The first one interacts with price. A cheap per-token rate on a model that writes long reasoning traces can cost more per task than a pricier, terser model. Measure cost per completed task, not per token.

StepFun recommends temperature 0.6 with top_p 0.95 for general chat, and temperature 1.0 with top_p 0.95 for reasoning and agent work.

Step 3.5 Flash or Step 3.7 Flash

StepFun followed up with Step 3.7 Flash, which adds a vision encoder and selectable reasoning levels at a higher listed price. Pick 3.5 Flash for text-only agent loops where cost per token is the priority and you have tested that it handles your tasks. Pick 3.7 Flash if you need image input or want explicit control over reasoning depth.