AI Models

Nemotron 3.5 Bolt: NVIDIA's 74B and 4GB open models bring agents to local hardware

By the Pavitrams editors · October 28, 2026 · 9 min read

NVIDIA's Nemotron 3.5 family expanded on October 15 and 23, 2026 with two open models built for local deployment: Nemotron 3.5 Bolt 74B-A9B 24GB (IQ2_XXS quantization, 9B active params, 256K context) and Nemotron 3.5 Bolt E4B 4GB (post-trained from Gemma 4 E4B, 64K context, conversational mode). Both ship under OpenMDW-1.1 with open weights, datasets, and training recipes — enabling agents that run on consumer and edge hardware without cloud dependencies.

The short answer

Two local models, one family — Nemotron 3.5 Bolt 74B-A9B 24GB released October 15, 2026; Nemotron 3.5 Bolt E4B 4GB released October 23, 2026. Both are open weights under OpenMDW-1.1, compressed from the 120B Super VL parent model for local agentic deployment.
Bolt 74B-A9B: edge agent workhorse — 74B total / 9B active hybrid Mamba-Transformer MoE, native IQ2_XXS 2-bit quantization to fit in 24GB VRAM, 256K context, text/image/video input. NVIDIA reports #1 on PinchBench, #2 on WebArena and BrowseComp among open models in its memory class.
Bolt E4B: conversational agents on device — Post-trained from Gemma 4 E4B via RL only, ~4GB footprint for voice assistants, game NPCs, and robotics on RTX, Jetson, and DGX Spark. Features conversational_mode for speech-ready replies, 64K context, multi-turn tool use with async guidance. NVIDIA claims ~40% higher quality vs prior small model and ~80% fewer output tokens.

Update note — October 28, 2026

This article is based on NVIDIA's official blog (October 15, 2026), the Nemotron 3.5 Bolt 74B-A9B 24GB technical page (research.nvidia.com), the Bolt E4B 4GB technical page, MarkTechPost coverage (October 23, 2026), SearchEngineWatch (October 23, 2026), and HPCwire (October 15, 2026). All benchmark figures are vendor-reported unless explicitly attributed to independent leaderboards (PinchBench, WebArena, BrowseComp, AIPerf). No hands-on testing was performed for this article.

When NVIDIA released Nemotron 3.5 Super VL on October 15, 2026, it was a 120B-parameter vision-language model for data-center agent deployments. But the real story for builders came in the same announcement: two companion models that shrink that capability down to hardware you can actually own.

Nemotron 3.5 Bolt 74B-A9B 24GB (October 15) and Nemotron 3.5 Bolt E4B 4GB (October 23) complete a three-tier family that spans from DGX servers to Jetson Orin Nano — all open weights, all built for agents, not chat.

The compression strategy: big model, squeezed down

Both Bolt models are compressed from Super VL, not trained from scratch. NVIDIA's research shows that for a fixed memory budget (24GB or 4GB), compressing a larger, stronger model yields better intelligence-per-GB than training a small model natively.

Star Elastic (NVIDIA's router-guided architecture search) shrinks the 120B-A12B parent to 74B-A9B and 8B-A4B target architectures.
Quantization-aware distillation (QAD) recovers accuracy after aggressive 2-bit (IQ2_XXS) quantization of the FC1/FC2 expert layers.
LiLiCorr speculative drafter (1.13B params) adds 7-token parallel drafting for 84 TPS on RTX Spark with llama.cpp.

The result: a 24GB model that outperforms Qwen3.6 35B-A3B and Gemma 4 26B-A4B on agentic benchmarks while fitting on an RTX 4090 or RTX Spark laptop.

Nemotron 3.5 Bolt 74B-A9B 24GB — the edge workhorse

Released October 15, 2026. Available on Hugging Face (BF16, NVFP4, GGUF), Ollama, LM Studio, Unsloth, vLLM, and NVIDIA NIM (nightly vLLM recipe).

Spec	Value
Architecture	Hybrid Mamba-Transformer MoE
Total / Active params	74B / 9B
Quantization	IQ2_XXS (2.0625 bits/element)
Context window	256K tokens
Input modalities	Text, images, video
License	OpenMDW-1.1 (commercial OK)
Target hardware	RTX 30/40/50 series 24GB, DGX Spark, RTX PRO

Benchmarks (NVIDIA-reported vs open peers in 24GB class):

PinchBench: #1 among open models in class
WebArena: #2
BrowseComp: #2
AIPerf SPEED-Bench-Coding: 83.99 TPS with LiLiCorr on RTX Spark
AA AgentPerf Local: 600.8s replay duration (vs Qwen3.6 453.97s, Gemma 4 480.68s)

Use cases: Local web research agents, code debugging assistants, document analysis pipelines, multi-step terminal workflows — all running entirely on your GPU with no API calls.

Nemotron 3.5 Bolt E4B 4GB — conversational agents on device

Released October 23, 2026. Post-trained from Gemma 4 E4B (Google's 4B instruct model) using RL-only post-training across instruction-following, multi-turn tool-calling, and conversational environments.

Spec	Value
Base model	Gemma 4 E4B instruct
Footprint	~4GB (IQ2 quantization)
Context window	64K tokens
Input modalities	Text only (no vision)
Special features	conversational_mode flag, configurable reasoning budget
License	OpenMDW-1.1
Target hardware	RTX, Jetson Orin, DGX Spark, GeForce RTX

NVIDIA's claimed differentiators:

~40% higher average quality vs Nemotron 3 Nano 4B on conversational agent benchmarks
Up to ~80% fewer output tokens vs peers (critical for latency and cost on device)
Multi-turn tool use with async call guidance
Runtime thinking budget to cap latency per turn

Target workloads: Voice assistants (needs separate ASR/TTS), interactive game NPCs, robotics/IoT interfaces (NVIDIA's demo pairs Bolt E4B with a Cosmos vision model on a Reachy Mini robot on Jetson Orin Nano 8GB).

What this means for local agent builders

The three-tier Nemotron 3.5 family now covers the full deployment spectrum:

Model	Scale	Footprint	Best for
Super VL	Data center / cloud	120B (BF16/NVFP4)	High-volume specialized agents, document-heavy workflows
Bolt 74B-A9B	Edge / workstation	24GB (IQ2_XXS)	Local research, coding, browsing agents on RTX
Bolt E4B	Device / embedded	~4GB	Voice assistants, NPCs, robotics on Jetson/RTX

All three ship with open weights, open datasets, and training recipes under OpenMDW-1.1. You can post-train on your own data via NeMo, self-host on your own infrastructure, and avoid provider lock-in or data egress.

What to watch

Independent benchmarking: PinchBench, WebArena, and BrowseComp leaderboards will update as the community tests Bolt 74B. The #1/#2 claims need third-party verification.
Ecosystem support: llama.cpp, vLLM, Ollama, Unsloth recipes are live for Bolt 74B; Bolt E4B has NVIDIA NIM container. Watch for Canonical Snap and LM Studio integration.
Real-world agent latency: Vendor TPS numbers (83.99 with LiLiCorr) are single-stream, controlled ISL. Multi-user serving on DGX Spark via NIM claims 8x throughput — test your concurrency profile.
Bolt E4B vision gap: Text-only. If your agent needs image input, you need a separate vision model (Cosmos, or Super VL) — adding VRAM pressure.

Verdict For local agent developers: Bolt 74B-A9B is the first open model that fits in 24GB and credibly beats the Qwen/Gemma peer group on agentic benchmarks. If you're building research, coding, or browsing agents on an RTX 4090 or Spark laptop, this is the model to evaluate first.

For voice/robotics builders: Bolt E4B's ~4GB footprint + conversational_mode + async tool calls is a purpose-built combo for on-device agents. The Gemma 4 lineage means strong instruction following out of the box; the RL post-training targets the right behaviors. Pair with a lightweight vision model if needed.

For the open ecosystem: NVIDIA's compression-over-training approach (Star Elastic + QAD + LiLiCorr) is a replicable recipe. Expect other labs to adopt similar pipelines to shrink frontier models to edge footprints.

Bottom line: NVIDIA has delivered the only open model family that spans data-center to device with a consistent agentic focus. Bolt 74B-A9B is the pragmatic choice for local agents today; Bolt E4B opens the door for voice and robotics on 4GB hardware. Both are open weights, open recipes, and ready to self-host — the cloud is now optional for agent workloads.

Sources

Prices and benchmark figures are standard published rates as of October 28, 2026 and can change without notice. Some outbound links may be affiliate links — see our disclosure. Found an error? Tell us and we'll correct it in public.