Nemotron 3 Ultra Explained: NVIDIA's 550B Hybrid Mamba-MoE Model

NVIDIA opened the summer of 2026 by putting its largest open-weight language model directly onto Hugging Face rather than locking it behind an API. On June 4, 2026, NVIDIA released Nemotron 3 Ultra, a 550 billion parameter hybrid Mamba-Attention Mixture-of-Experts model that sits at the top of the new Nemotron 3 family, alongside the smaller Nano and Super checkpoints [1] [2]. The headline is not just the parameter count. It is that Nemotron 3 Ultra only activates around 55 billion of those 550 billion parameters per token, roughly a 10 percent sparsity ratio, while combining state-space Mamba-2 layers with transformer attention layers in a single architecture built specifically for long-running, multi-turn agentic work [1] [3].

That combination matters because it addresses a real, specific problem that anyone building agents against a frontier model has run into: standard transformer attention gets expensive fast as a conversation, a coding session, or an agent trace grows longer, while pure state-space models like Mamba struggle to match transformer-level reasoning on dense, detail-sensitive tasks. Nemotron 3 Ultra is NVIDIA's answer to that tradeoff at genuinely frontier scale, and it ships with open weights, a permissive license, and hosted inference across seven different platforms on day one [1] [6].

This post walks through exactly how Nemotron 3 Ultra is built, why each architectural choice exists, what the independently reported benchmark numbers actually say, and how to call the model yourself through OpenRouter, NVIDIA NIM, or a self-hosted vLLM deployment. Every code block below is meant to be copied and run, not read as pseudocode.

Step 1: The Nemotron 3 Family and Where Ultra Fits

NVIDIA did not ship Nemotron 3 Ultra in isolation. It is the largest member of a three-tier family that NVIDIA has been rolling out since December 2025, and understanding the other two tiers is the fastest way to understand what makes Ultra different [2].

Nemotron 3 Nano shipped first, on December 15, 2025. It is a 30 billion total parameter model that activates up to 3 billion parameters per token through a hybrid latent Mixture-of-Experts design, supports a 1 million token context window, and was benchmarked by NVIDIA at roughly 4x higher token throughput than Nemotron 2 Nano while needing up to 60 percent fewer reasoning tokens to reach the same answer quality [2] [4]. Nano is the tier built for edge devices, on-device assistants, and cost-sensitive high-volume workloads where every token of latency and every dollar of inference cost compounds.

Nemotron 3 Super sits in the middle: roughly 100 billion total parameters, up to 10 billion active per token, a Mixture-of-Experts design like Nano, and notably trained in 4-bit NVFP4 precision directly on NVIDIA's Blackwell architecture [2] [4]. Super is positioned as the workhorse tier, big enough for serious reasoning and agentic tasks, small enough to serve at meaningfully lower cost than Ultra.

Nemotron 3 Ultra, the subject of this post, is the flagship: roughly 550 billion total parameters with up to 55 billion active per token [1] [2]. That 55-billion-active-out-of-550-billion-total ratio, about 10 percent sparsity, is deliberately close to the sparsity levels used by other very large open Mixture-of-Experts models in 2026, but the number that actually matters for a developer deciding whether to use it is not the parameter count. It is what that architecture lets the model do at inference time that a same-sized dense or standard-MoE transformer cannot, which is the subject of the next three steps.

NVIDIA's own framing for why this family exists at all is worth reading directly. NVIDIA CEO Jensen Huang put it this way at launch: "With Nemotron, we're transforming advanced AI into an open platform that gives developers the transparency and efficiency they need to build agentic systems at scale" [2]. Every architectural decision covered below, the hybrid Mamba-attention backbone, the latent routing scheme, and the native speculative decoding layers, traces back to that stated goal: agentic systems that run long, multi-step traces without attention cost or routing instability making that impractical at scale.

If you have been following NVIDIA's model releases, this is a genuinely different architecture from earlier Nemotron generations, and it is worth comparing against how other labs have approached scaling reasoning models. Our breakdowns of Qwen3-8-Max's 2.4 trillion parameter design and GLM-5.3's coding-focused architecture both cover competing large-scale approaches that stuck closer to a pure transformer design, which is a useful contrast to keep in mind as we get into Nemotron 3 Ultra's hybrid layers next.

Step 2: Inside the Hybrid Mamba-Attention Architecture

The core architectural bet in Nemotron 3 Ultra is what NVIDIA calls a Mixture-of-Experts Hybrid Mamba-Attention design: most of the network's sequence-processing layers are Mamba-2 state-space layers, interleaved with a smaller number of full transformer attention layers, all wrapped inside a Mixture-of-Experts feed-forward block [1] [3].

Why Mamba-2 Layers Exist in a 2026 Frontier Model

Mamba and its successor Mamba-2 are state-space models, a different mathematical family from the attention mechanism that has defined transformers since 2017. Instead of computing a similarity score between every pair of tokens in a sequence, the way self-attention does, a state-space layer maintains a compressed running summary, a hidden state, that gets updated as each new token arrives, similar in spirit to a recurrent neural network but with a selective, input-dependent update rule that lets the model decide what to keep and what to forget as it moves through the sequence. That single design choice changes the computational cost profile completely: self-attention scales quadratically with sequence length, since every token has to compare itself against every other token, while a state-space layer scales linearly, since it only ever has to update one fixed-size hidden state per new token regardless of how long the sequence has already gotten.

That difference is invisible on a short prompt and enormous on a long one. A coding agent working through a 200,000-token repository trace, or a customer support agent carrying a multi-hour conversation history, spends most of its compute reprocessing context that a pure transformer has to re-attend to at every single step. Mamba-2 layers carry that context forward in a fixed-size state instead, which is the architectural reason Nemotron 3 Ultra can serve a 262,000 token context window at full BF16 precision and push to a full 1 million tokens under NVFP4 quantization on Blackwell hardware without attention cost exploding, a point covered in depth in Step 6 below [1] [4].

Why Attention Layers Are Still There

If state-space layers were strictly better, NVIDIA would not have kept any attention layers at all. They kept a smaller proportion of them because pure state-space models have a real, well-documented weakness: precise, exact retrieval of a specific fact from far back in a long context, the kind of task where a model needs to find one needle in a very large haystack rather than summarize the general shape of what came before. Full attention is exceptionally good at exactly that, because every token genuinely does get to look directly at every other token, with no compression involved. By interleaving a smaller number of true attention layers among the Mamba-2 layers, Nemotron 3 Ultra gets the linear-cost long-context efficiency of a state-space model for most of its depth, while still preserving genuine long-range, high-precision recall at the attention layers, which is exactly the profile a coding agent or research assistant needs: efficient over a long trace, but still able to pull an exact function signature or exact clause back out of a document opened fifty steps ago.

Here is a simplified simulation that shows the practical effect of this hybrid layer pattern on compute cost as sequence length grows:

python
/code
# Illustrates why interleaving a small share of full-attention layers among
# linear-cost Mamba-2 layers keeps total compute far below a pure-attention
# stack as sequence length grows. Attention layers scale quadratically with
# sequence length; Mamba-2 layers scale linearly, the core tradeoff behind
# Nemotron 3 Ultra's hybrid design [1].
def relative_cost(seq_len: int, attention_layer_ratio: float) -> float:
    attention_cost = attention_layer_ratio * (seq_len ** 2)
    mamba_cost = (1 - attention_layer_ratio) * seq_len
    return attention_cost + mamba_cost

for seq_len in [1_000, 8_000, 64_000, 262_000]:
    pure_attention = relative_cost(seq_len, attention_layer_ratio=1.0)
    hybrid = relative_cost(seq_len, attention_layer_ratio=0.15)  # ~15% attention layers
    savings = 1 - (hybrid / pure_attention)
    print(f"seq_len={seq_len:>7,} | pure-attention cost={pure_attention:>15,.0f} "
          f"| hybrid cost={hybrid:>15,.0f} | relative savings={savings:.1%}")

Running that script at increasing sequence lengths makes the shape of the tradeoff obvious: a pure-attention stack's relative cost accelerates quadratically while the hybrid stack's cost grows close to linearly, and the gap widens fastest exactly in the range that matters for agentic workloads, tens of thousands of tokens and up. This is the same directional tradeoff that motivated hybrid attention designs elsewhere in 2026, including DeepSeek V4-Pro's Compressed Sparse and Heavily Compressed Attention split, though Nemotron 3 Ultra reaches for a genuinely different mechanism, state-space layers instead of sparsified attention, to get there.

Step 3: LatentMoE Routing and Why Routing Collapse Matters

A Mixture-of-Experts layer only works if tokens actually get spread across the available experts in a useful way. Give the router too little structure, and a common failure mode called routing collapse sets in, where the router learns to send most tokens to a small handful of experts it has already trained well, starving the rest of the pool of gradient signal, until the model is effectively using a much smaller fraction of its total capacity than its parameter count suggests. This is a known, well-documented failure mode in large MoE training, and it gets worse, not better, as you add more experts, which directly punishes the exact strategy, more specialists at the same active-parameter budget, that makes MoE architectures efficient in the first place.

Nemotron 3 Ultra's answer is what NVIDIA calls LatentMoE routing: instead of the router looking directly at a token's raw embedding to decide which experts should handle it, it routes based on a learned latent representation of that token instead [1] [3]. A raw token embedding is a fairly rigid, high-dimensional object shaped directly by vocabulary and surface form. A latent representation, learned jointly with the rest of the network rather than fixed at the embedding layer, gives the routing decision more flexibility to group tokens by function or context rather than by superficial similarity, which in turn makes it possible to support a larger, more specialized pool of experts at the same inference cost without the router collapsing onto a small favored subset.

A simplified illustration of the difference in practice:

python
/code
import random
from collections import Counter

# Toy comparison of raw-embedding routing (prone to collapsing onto a small
# favored subset of experts) versus latent-representation routing (spreads
# load more evenly), the mechanism behind Nemotron 3 Ultra's LatentMoE
# design [1][3]. Real routing uses learned projections, not random
# weighting; this only illustrates the resulting load shape.
def route_tokens(tokens: list[str], num_experts: int, use_latent: bool) -> Counter:
    random.seed(7)
    counts = Counter()
    for tok in tokens:
        if use_latent:
            # Latent routing: smoother, more evenly distributed preference
            weights = [random.uniform(0.8, 1.2) for _ in range(num_experts)]
        else:
            # Raw-embedding routing: a few experts get consistently favored
            weights = [random.uniform(0.2, 3.0) for _ in range(num_experts)]
        chosen = weights.index(max(weights))
        counts[chosen] += 1
    return counts

tokens = [f"tok_{i}" for i in range(5000)]
raw_counts = route_tokens(tokens, num_experts=16, use_latent=False)
latent_counts = route_tokens(tokens, num_experts=16, use_latent=True)

raw_top_share = max(raw_counts.values()) / len(tokens)
latent_top_share = max(latent_counts.values()) / len(tokens)
print(f"Raw-embedding routing: busiest expert handles {raw_top_share:.1%} of tokens")
print(f"Latent routing: busiest expert handles {latent_top_share:.1%} of tokens")
print(f"Experts used at all: raw={len(raw_counts)}/16, latent={len(latent_counts)}/16")

The output of that script is a useful gut check on why this matters operationally: a routing scheme that concentrates load onto a handful of experts wastes the rest of the expert pool and can create real serving problems too, since a handful of overloaded experts become a throughput bottleneck on the GPUs hosting them, while a scheme with more even utilization actually uses the full capacity you are paying to host. LatentMoE is NVIDIA's specific mechanism for keeping that utilization curve flat even as expert count scales up, and it is one of the more understated engineering decisions in this release, easy to skip past in a spec sheet but directly responsible for Ultra actually using its 550 billion parameters rather than a much smaller effective subset of them.

Step 4: Multi-Token Prediction and Native Speculative Decoding

The third architectural piece is Multi-Token Prediction, or MTP, and it targets a different problem: raw generation throughput in long, multi-turn agentic traces rather than context handling or expert utilization.

Standard autoregressive generation produces exactly one token per forward pass through the model, then feeds that token back in to predict the next one, over and over. Speculative decoding is a well-established technique for speeding that up: a small, cheap draft model proposes several tokens ahead, and the large model verifies them all in a single forward pass, accepting the ones that match what it would have generated anyway and only falling back to token-by-token generation where the draft diverges. It is a proven technique, and we have covered the general mechanics of it in detail in our explainer on speculative decoding for faster LLM inference. What Nemotron 3 Ultra does differently is build that draft-and-verify capability directly into the model as dedicated MTP layers, rather than requiring a separate, smaller draft model to be trained, deployed, and kept in sync alongside it [1] [3].

Native MTP layers matter most in exactly the workloads Nemotron 3 Ultra is built for: multi-turn, agentic sessions where the model is repeatedly generating structured output, tool calls, code edits, shell commands, that tend to be more predictable token-to-token than open-ended prose, which is precisely the setting where a draft-and-verify approach pays off the most. NVIDIA reports the practical result directly: Nemotron 3 Ultra completes SWE-bench and Terminal-Bench 2.0 tasks using roughly 30 percent fewer total tokens per run as a result of MTP-driven speculative decoding [1]. Fewer total tokens per completed task is a different, and arguably more useful, efficiency claim than tokens-per-second alone, because it reflects the model finishing the same real-world coding or terminal task with less total compute burned, not just generating each individual token faster.

A simplified illustration of how a draft-and-verify step changes the number of full model passes required to emit a given number of tokens:

python
/code
# Estimates full model forward passes needed to emit total_tokens using
# MTP-style draft-and-verify decoding versus plain autoregressive decoding.
# NVIDIA reports Nemotron 3 Ultra completes SWE-bench and Terminal-Bench 2.0
# tasks using roughly 30% fewer total tokens per run due to native MTP
# speculative decoding [1].
def forward_passes_needed(total_tokens: int, draft_len: int, acceptance_rate: float) -> float:
    plain_passes = total_tokens  # one forward pass per token, no speculation
    accepted_per_round = draft_len * acceptance_rate
    # one verification pass covers draft_len proposed tokens, of which
    # accepted_per_round are kept before falling back to a normal step
    tokens_per_round = max(accepted_per_round, 1)
    speculative_passes = total_tokens / tokens_per_round
    return plain_passes, speculative_passes

total_tokens = 20_000  # a realistic long agentic completion
for acceptance_rate in [0.4, 0.6, 0.8]:
    plain, spec = forward_passes_needed(total_tokens, draft_len=4, acceptance_rate=acceptance_rate)
    reduction = 1 - (spec / plain)
    print(f"acceptance_rate={acceptance_rate:.0%} | plain passes={plain:,.0f} "
          f"| MTP passes={spec:,.0f} | forward-pass reduction={reduction:.1%}")

The bigger the gap between draft acceptance rate and 0, the more forward passes MTP saves relative to plain autoregressive decoding, and structured agentic output, the kind SWE-bench and Terminal-Bench 2.0 tasks are built from, tends to have a higher acceptance rate than open-ended creative writing, which is exactly why NVIDIA's 30 percent figure shows up specifically on those two benchmarks rather than as a universal claim.

Step 5: Training Nemotron 3 Ultra, From NVFP4 Pretraining to Multi-Teacher Distillation

Architecture only gets you halfway to a usable model. Nemotron 3 Ultra's training pipeline is its own story, and it leans heavily on NVIDIA's own hardware and quantization stack rather than a generic training recipe.

Pretraining in 4-Bit NVFP4 on Blackwell

Nemotron 3 Ultra was pretrained on 20 trillion training tokens, with the base training run itself carried out in BF16 precision before the model was converted for 4-bit NVFP4 pretraining and quantization on NVIDIA's Blackwell architecture [1] [2]. NVFP4 is NVIDIA's own 4-bit floating point format, purpose-built for the Blackwell generation of GPUs, and using it at pretraining scale rather than only at inference-time quantization is notable: it means the model's weights are trained with awareness of the lower-precision numeric format they will eventually run in, rather than being trained at full precision and quantized down afterward as an afterthought, which tends to preserve more accuracy at a given bit width than post-hoc quantization does. At 550 billion parameters and 20 trillion training tokens, the difference between training-time and inference-time quantization is not a minor implementation detail, it directly affects how much of the model's capability survives the trip down to 4 bits, which in turn is a big part of why Ultra is able to serve the throughput numbers covered in Step 7 without the accuracy collapse that naive quantization of a model this size usually causes.

Post-Training: SFT, RL, and Multi-Teacher On-Policy Distillation

After pretraining, Nemotron 3 Ultra went through a three-stage post-training pipeline: Supervised Fine-Tuning, Reinforcement Learning, and Multi-teacher On-Policy Distillation [1] [4]. The first two stages are familiar from most modern frontier post-training pipelines: supervised fine-tuning on curated instruction data to establish baseline behavior, followed by reinforcement learning to sharpen the model's responses against a reward signal, typically built from human preference data, verifiable task correctness, or both.

The third stage, multi-teacher on-policy distillation, is worth unpacking specifically, because it is a meaningfully more sophisticated technique than simple knowledge distillation from a single larger teacher model. In on-policy distillation, the student model, Nemotron 3 Ultra itself during training, generates its own rollouts, and one or more teacher models score or correct those rollouts, rather than the student passively imitating a fixed dataset of teacher-generated examples the way classic offline distillation works. Using multiple teachers rather than one lets the training process draw on different teachers' relative strengths across different task types, coding, reasoning, tool use, instead of inheriting a single teacher's specific blind spots. If you want the deeper mechanics of how distillation compresses capability from a larger model into a smaller or more efficient one, our explainer on knowledge distillation covers the general technique in more depth; on-policy multi-teacher distillation is a more advanced variant of that same underlying idea, applied here at the post-training stage of an already-massive base model rather than to shrink a large model into a genuinely smaller one.

Step 6: Context Window, From 262K at BF16 to 1M Tokens on Blackwell

Nemotron 3 Ultra supports a 262,000 token context window at full BF16 precision, and that ceiling extends to a full 1,000,000 tokens when the model is served with NVFP4 quantization on Blackwell hardware [1] [4]. Those are two genuinely different numbers for two genuinely different deployment choices, not a marketing rounding trick, and the gap between them traces directly back to the hybrid Mamba-attention design covered in Step 2. Because most of the network's layers carry context forward in a fixed-size state rather than an attention cache that grows with sequence length, extending the usable window is far more a function of available memory and numeric precision than it is of the fundamentally quadratic attention cost that limits pure-transformer models at extreme context lengths.

NVIDIA reports that Nemotron 3 Ultra outperforms comparable models on the RULER long-context benchmark at the full 1 million token setting [1]. RULER is specifically designed to be a harder, more realistic test of long-context ability than simple needle-in-a-haystack retrieval, since it includes multi-hop tracing, aggregation across scattered facts, and tasks that require actually reasoning over the retrieved content rather than just locating and repeating it verbatim. A strong RULER score at 1 million tokens is a meaningfully stronger claim than a strong score on a single needle-retrieval test, because it is evidence the hybrid architecture's attention layers are still doing real work at extreme context length, not just that the state-space layers can technically hold that much text in memory.

Practically, a 1 million token window at full precision on the right hardware means an entire mid-sized codebase, a full legal contract set, or a very long multi-session agent transcript can live in context across many calls without retrieval-augmented chunking tricks. That is the same architectural payoff long-context releases from other labs have chased this year, and it is worth reading alongside our coverage of Gemini 3.7 Flash's coding benchmarks for a sense of how differently sized labs are approaching the same long-context, agentic-coding problem from different architectural angles.

Step 7: Throughput and Benchmark Numbers

This is the section most developers evaluating Nemotron 3 Ultra actually care about first: how fast is it, and how good are its answers, compared to the other large open-weight models shipping around the same window.

Throughput at 8K Input, 64K Output

In an 8,000 input token, 64,000 output token serving setting, a realistic proxy for a long agentic generation, Nemotron 3 Ultra reports meaningfully higher throughput than several of the largest open-weight competitors while reaching on-par accuracy with them [1] [3]:

The direction of that comparison lines up cleanly with the architecture covered in Steps 2 through 4: linear-cost Mamba-2 layers instead of full attention for most of the network's depth, LatentMoE routing that keeps expert utilization even rather than collapsed, and native MTP layers cutting the number of full forward passes needed per completed task. Each mechanism individually contributes to throughput, and the 8K/64K benchmark setting is specifically the kind of long-output, agent-shaped workload where all three compound together, rather than a short single-turn prompt where the difference between architectures would be far less visible.

Intelligence, Reasoning, and Agentic Task Scores

Raw throughput only matters if the answers are good. On MarkTechPost's and BuildFastWithAI's independent reporting of NVIDIA's benchmark disclosures, Nemotron 3 Ultra posts an Intelligence Index of 48, the highest score among US-built open-weight models, though Moonshot AI's Kimi K2.6 leads globally among open-weight models at 54 [3] [5]. That framing, leading among US-built open-weight models specifically rather than leading globally, is worth taking at face value rather than rounding up, since it is a meaningfully narrower and more honest claim than the kind of unqualified "best open model" language that shows up in a lot of launch coverage. If you are weighing Nemotron 3 Ultra against other very large open-weight reasoning models on the market right now, our coverage of Kimi K3's 2.8 trillion parameter architecture and Thinking Machines' Inkling MoE model are both useful companion reads for that broader landscape.

On MMLU-Pro, a harder, more discriminating successor to the original MMLU benchmark designed to reduce the ceiling effects that let many models cluster near the top of plain MMLU, Nemotron 3 Ultra scores 66.6, an improvement from a 64.8 baseline earlier in development [4] [5]. On PinchBench, an agentic productivity evaluation, Nemotron 3 Ultra reaches 91 percent [5], a genuinely strong score for a benchmark specifically designed to measure whether a model can carry out real, multi-step agentic tasks rather than just answer isolated questions well.

That PinchBench number is also the right place to introduce a useful piece of landscape context: Nemotron 3 Ultra is not the only hybrid state-space-and-attention model to ship in this window. Arcee's Trinity-Large-Thinking, a 400 billion total parameter model that activates only about 13 billion parameters per token, roughly 1.56 percent, through its own Mixture-of-Experts design, also blends Mamba-style state-space layers with transformer attention, and scored 91.9 percent on that same PinchBench agentic evaluation, second only to Anthropic's proprietary model on that specific eval [7]. Trinity-Large-Thinking's active-parameter ratio is even sparser than Ultra's, and the fact that two independently built, differently sized hybrid architectures are both landing near the top of the same agentic benchmark in the same release window is a real signal, not a coincidence: blending state-space efficiency with attention precision is turning into one of the defining architectural trends of 2026 open-weight releases, not a one-lab experiment.

Step 8: Calling Nemotron 3 Ultra Through OpenRouter

The fastest way to try Nemotron 3 Ultra without standing up your own GPU infrastructure is through a hosted inference provider. OpenRouter lists the model as nvidia/nemotron-3-ultra-550b-a55b:free with an OpenAI-compatible API, alongside listings from Baseten, DeepInfra, Fireworks, FriendliAI, Together AI, and NVIDIA's own NIM service [1] [6]. Because the API is OpenAI-compatible, the standard openai Python client works directly against it with only the base URL and model name changed:

python
/code
# Nemotron 3 Ultra is OpenAI-compatible through OpenRouter, so the standard
# openai-python client works by just pointing base_url at OpenRouter's
# endpoint and using the model's OpenRouter slug [1][6].
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ["OPENROUTER_API_KEY"],
    base_url="https://openrouter.ai/api/v1",
)

response = client.chat.completions.create(
    model="nvidia/nemotron-3-ultra-550b-a55b:free",
    messages=[
        {"role": "system", "content": "You are a precise, efficient coding agent."},
        {"role": "user", "content": "Write a Python function that merges two sorted lists in-place without extra memory."},
    ],
    temperature=0.3,
    max_tokens=1024,
    extra_headers={
        "HTTP-Referer": "https://your-app.example.com",
        "X-Title": "Nemotron 3 Ultra test",
    },
)

print(response.choices[0].message.content)
print(f"Prompt tokens: {response.usage.prompt_tokens}, completion tokens: {response.usage.completion_tokens}")

That is the entire integration surface for most applications. If you already have an existing pipeline built around the OpenAI client, whether that is a coding agent, a retrieval-augmented generation service, or a batch summarization job, pointing it at Nemotron 3 Ultra is a base URL and model string change, not a rewrite.

Step 9: Running Nemotron 3 Ultra Locally With vLLM or NVIDIA NIM

For teams that need the weights on their own infrastructure, whether for data residency requirements, cost control at high volume, or fine-tuning, Nemotron 3 Ultra's open weights on Hugging Face support self-hosted serving through either vLLM or NVIDIA's own NIM microservices, both of which have first-class support for hybrid Mamba-attention MoE architectures and NVFP4 quantization on Blackwell [1]. A typical download-and-serve flow looks like this:

bash
/code
# Download the open weights from Hugging Face before serving locally.
pip install "huggingface_hub[cli]"
huggingface-cli login

huggingface-cli download nvidia/Nemotron-3-Ultra-550B \
  --local-dir ./nemotron-3-ultra \
  --local-dir-use-symlinks False

du -sh ./nemotron-3-ultra

And a representative vLLM serving command, using tensor parallelism to split the model across multiple GPUs, the realistic setup for a 550 billion parameter model even at 4-bit precision:

bash
/code
# Serving Nemotron 3 Ultra locally with vLLM across a multi-GPU node.
# 550B total parameters at 4-bit NVFP4 still requires tensor parallelism
# across several GPUs; adjust --tensor-parallel-size to your node's GPU count [1].
vllm serve nvidia/Nemotron-3-Ultra-550B \
  --tensor-parallel-size 8 \
  --quantization nvfp4 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.92 \
  --trust-remote-code \
  --port 8000

# Or run the equivalent packaged deployment through NVIDIA NIM:
docker run --gpus all --rm -p 8000:8000 \
  -e NGC_API_KEY=$NGC_API_KEY \
  nvcr.io/nim/nvidia/nemotron-3-ultra:latest

At 550 billion total parameters, even NVFP4 quantization leaves a memory footprint that requires a multi-GPU node, which is worth planning capacity around before committing to self-hosting rather than a hosted provider. For most teams below very large inference volume, one of the hosted options listed in Step 8 will be more cost-effective than standing up and maintaining that infrastructure yourself, and self-hosting is really the right call once request volume or data residency requirements justify the operational overhead.

Step 10: Controlling the Reasoning Budget

Like most 2026-era reasoning models, Nemotron 3 Ultra exposes a way to control how much internal reasoning effort it spends before producing a final answer, letting you trade latency and cost against answer quality on a per-request basis rather than accepting a single fixed behavior for every call. A typical request payload showing that parameter alongside the standard chat completion fields looks like this:

json
/code
{
  "model": "nvidia/nemotron-3-ultra-550b-a55b:free",
  "messages": [
    {
      "role": "user",
      "content": "Trace through this multi-step agent log and identify where the tool call failed, then propose a fix."
    }
  ],
  "reasoning": {
    "effort": "high",
    "max_reasoning_tokens": 8192
  },
  "temperature": 0.2,
  "max_tokens": 2048,
  "stream": false
}

Setting a lower reasoning effort is the right call for high-volume, latency-sensitive requests, quick classification, short tool-call turns, simple lookups, where a long internal reasoning trace adds cost without adding accuracy. Reserve a higher effort setting for the tasks Nemotron 3 Ultra's architecture is specifically built to handle well: multi-step coding tasks, long agentic tool-use chains, and problems that genuinely benefit from the model working through intermediate steps before committing to an answer. Tuning this parameter per request, rather than picking one setting globally for an entire application, is one of the more overlooked levers for controlling inference cost on a model this large.

Step 11: Licensing and Where to Access the Model

Nemotron 3 Ultra ships under OpenMDW-1.1, a permissive open model license maintained by the Linux Foundation [1] [2]. A Linux Foundation-governed license, rather than a bespoke company-authored one, is a meaningful signal for enterprise adoption specifically, since legal teams evaluating whether to build production infrastructure around an open model tend to move faster when the license terms have been reviewed and standardized by a neutral foundation rather than drafted unilaterally by the model's own vendor.

The open weights are available on Hugging Face directly, and hosted inference is available through Baseten, DeepInfra, Fireworks, FriendliAI, OpenRouter, Together AI, and NVIDIA NIM [1] [6]. That breadth of day-one hosting availability, seven distinct providers, is unusually wide for a model of this size and is itself a useful signal about how much confidence the inference hosting ecosystem placed in the release ahead of time.

Production Notes and Best Practices

A few practical points worth applying directly if you are actually putting Nemotron 3 Ultra into a production pipeline rather than just experimenting with it.

• Match context length to precision, deliberately. The 262,000 token BF16 ceiling versus the 1,000,000 token NVFP4-on-Blackwell ceiling are not interchangeable defaults. If your workload genuinely needs the full million-token window, you need to be serving on Blackwell with NVFP4 quantization specifically, not assuming any deployment automatically gets the larger number.
• Lean on MTP-driven speculative decoding for structured agentic workloads. The roughly 30 percent token savings NVIDIA reports on SWE-bench and Terminal-Bench 2.0 specifically reflects structured, tool-call-heavy generation. Expect a smaller benefit on open-ended creative or conversational output, where draft acceptance rates are naturally lower.
• Tune the reasoning budget per request type, not globally. A single fixed reasoning effort setting either wastes cost on simple requests or shortchanges complex ones. Route request types to different effort levels the way the JSON payload in Step 10 illustrates.
• Plan multi-GPU capacity honestly if self-hosting. 550 billion parameters at 4-bit NVFP4 still requires real multi-GPU memory. Budget for tensor-parallel serving across a real node rather than assuming quantization alone makes single-GPU deployment realistic.
• Treat the 91 percent PinchBench score as an agentic-task signal, not a general intelligence claim. Nemotron 3 Ultra's Intelligence Index of 48 and its PinchBench agentic score of 91 percent are measuring genuinely different things. Pick your evaluation benchmark based on what your actual workload looks like, structured multi-step agent tasks versus broad open-ended reasoning, rather than quoting whichever number is highest.

Models built for exactly this kind of efficient, long-context, agentic workload are also what is quietly powering a lot of the fast, multi-step tooling behind content platforms in 2026. The kind of pipeline that turns a single topic into a finished script, scene visuals, and voiceover in one pass, the way Text2Shorts in Miraflow AI does, or that scans an entire long-form video upload to find and clip its strongest moments with AI Clipping, depends on exactly this kind of efficient long-context, low-latency inference under the hood, even when the end user never sees which model or architecture is doing the work. You can explore the rest of Miraflow AI's tools, including the AI Image Generator and YouTube Thumbnail Maker, from miraflow.ai/home.

If you want to see this architecture rendered as motion rather than static diagrams, here is a video generation prompt built around the hybrid Mamba-attention dataflow, written for a Wan-style video model, using the same physical tabletop metaphor as the images throughout this post:

A wooden tabletop workshop scene shot from directly above. On the left, a small hand-cranked conveyor belt made of tiny wooden gears carries a steady stream of glass marbles smoothly forward in a single continuous line, representing efficient state-space processing. On the right, a miniature woven loom sits still, its threads occasionally pulling taut and crossing fully from one edge to the other to grab a single distant marble with precision, representing attention. Camera slowly dollies across from the conveyor belt to the loom and back, showing marbles flowing continuously while the loom activates only briefly and selectively. Midway through, a small wooden funnel above both mechanisms sorts incoming marbles by color into a row of labeled glass jars, each jar filling at a different but steady rate, representing balanced mixture-of-experts routing. Warm soft studio lighting, shallow depth of field, visible wood grain and glass reflections, no readable text, no logos, no people, smooth continuous camera movement throughout.

Common Mistakes to Avoid

A handful of misunderstandings show up repeatedly whenever a model like this ships, and they are worth naming directly.

• Treating "550 billion parameters" and "55 billion active parameters" as interchangeable numbers. They describe genuinely different things: total capacity versus per-token compute cost. Comparing Nemotron 3 Ultra's inference cost against a dense 550 billion parameter model, rather than against its actual 55-billion-active footprint, badly overestimates how expensive it is to run.
• Assuming the 1 million token context window is available in every deployment. It requires NVFP4 quantization on Blackwell hardware specifically. A default BF16 deployment tops out at 262,000 tokens, which is still large, but a different number worth planning around explicitly.
• Reading the Intelligence Index of 48 as "the best open-weight model globally." NVIDIA and independent reporting both frame it as the highest score among US-built open-weight models specifically, with Kimi K2.6 leading globally at 54. Both facts are true at once, and neither alone is the whole picture.
• Skipping the reasoning-budget parameter and using one fixed setting for every request type. This is the single easiest lever to leave on the table, and it directly affects both latency and inference cost at scale.
• Assuming LatentMoE and native MTP are interchangeable with generic MoE routing and generic speculative decoding. They are NVIDIA's specific implementations, tuned for this architecture. Techniques that work well tuning a standard MoE router or an external draft model do not automatically transfer to Nemotron 3 Ultra's built-in mechanisms without re-validating on your own workload.

Frequently Asked Questions

Is Nemotron 3 Ultra free to use? The weights themselves are open under the OpenMDW-1.1 license and free to download and self-host. Hosted inference pricing varies by provider, several list a free tier, such as OpenRouter's nvidia/nemotron-3-ultra-550b-a55b:free listing, while others price by token volume [6].

How is Nemotron 3 Ultra different from Nemotron 3 Nano and Super? All three share the Nemotron 3 family's hybrid or latent MoE design philosophy, but at very different scales. Nano is a 30 billion parameter model built for edge and cost-sensitive deployment, Super is a roughly 100 billion parameter mid-tier model trained in NVFP4 on Blackwell, and Ultra is the 550 billion parameter flagship with the full hybrid Mamba-attention architecture covered in this post [2] [4].

Does Nemotron 3 Ultra require Blackwell hardware to run? No, though Blackwell hardware with NVFP4 quantization is required to reach the full 1 million token context window. The model can be served on other hardware supporting the required precision formats at the smaller 262,000 token BF16 context ceiling.

What is LatentMoE routing in plain terms? It is NVIDIA's approach to deciding which expert sub-networks handle each token, based on a learned latent representation of the token rather than its raw embedding, which helps avoid routing collapse, the failure mode where a router overuses a small subset of experts and leaves the rest of the model's capacity underused.

How does Nemotron 3 Ultra compare to Kimi K2.6 and other large open-weight models on throughput? NVIDIA reports Nemotron 3 Ultra reaching 5.9x higher throughput than GLM-5.1-754B-A40B, 4.8x higher than Kimi-K2.6-1T-A32B, and 1.6x higher than Qwen-3.5-397B-17B in an 8K input, 64K output token setting, while reaching on-par accuracy with those models [1] [3].

Is Nemotron 3 Ultra the only hybrid Mamba-attention open model released around this time? No. Arcee's Trinity-Large-Thinking, a 400 billion parameter model with about 13 billion active parameters, also blends state-space and attention layers and scored 91.9 percent on the PinchBench agentic evaluation, close to Nemotron 3 Ultra's own 91 percent on the same benchmark [5] [7]. Hybrid architectures are a real 2026 trend across multiple labs, not a single-vendor approach.

Conclusion

Nemotron 3 Ultra is a genuinely substantial piece of systems engineering released with unusually broad day-one accessibility for a model of its size. The hybrid Mamba-attention backbone solves the long-context cost problem that a pure transformer cannot avoid at scale, LatentMoE routing keeps the model's full 550 billion parameters actually useful instead of collapsing onto a favored subset of experts, and native Multi-Token Prediction layers cut real token spend on exactly the structured, agentic workloads the model was built for. Training it in 4-bit NVFP4 directly on Blackwell, then refining it through supervised fine-tuning, reinforcement learning, and multi-teacher on-policy distillation, is what lets all of that reach production-grade quality rather than staying a research curiosity. Whether you access it through OpenRouter in five lines of Python, pull the weights and serve them yourself with vLLM, or run it through NVIDIA NIM, the same underlying architecture is doing the work, and understanding how each piece of it functions, not just what the spec sheet claims, is what actually helps you decide where it fits in a real production pipeline.

References and Sources

[1] NVIDIA. "Nemotron 3 Ultra."

[2] NVIDIA Newsroom. "NVIDIA Debuts Nemotron 3 Family of Open Models."

[3] MarkTechPost. "NVIDIA AI Releases Nemotron 3 Ultra: An Open 550B Mixture-of-Experts Hybrid Mamba-Transformer for Long-Running Agents."

[4] DataCamp. "NVIDIA Nemotron 3."

[5] BuildFastWithAI. "NVIDIA Nemotron 3 Ultra Review 2026."

[6] OpenRouter: Nemotron 3 Ultra model page.

[7] VentureBeat. "Arcee's new open source Trinity-Large-Thinking is the rare powerful U.S.-made AI model."