Jetson Thor LLM Benchmark: Qwen3 vs Nemotron for Real-Time Robotics Agents

Comparing 30B MoE models for edge inference on humanoid robots

TL;DR

We benchmarked two 30-billion parameter Mixture-of-Experts models on NVIDIA Jetson Thor in 120W power-limited mode.

For real-time robotics applications, Qwen3-30B-A3B delivers 1.6x faster Time to First Token and 1.4x faster token generation than Nemotron-3-Nano at single concurrency. Here's the full breakdown.

Here is quick results so you don't need to scroll over the whole report:

Why This Matters for Robotics

When your humanoid robot needs to make real-time decisions, every millisecond counts. Time to First Token (TTFT) is the critical metric for interactive agents — it's the latency between sending a prompt and receiving the first token of the response. While the model is "thinking," your robot potentially stands frozen without commands.

For embodied AI agents running locally on edge devices, you need:

• Low TTFT for responsive interactions
• Consistent latency (low P99) for predictable behavior
• Reasonable throughput for complex reasoning tasks
• Power efficiency for sustained operation

We tested two comparable MoE architectures using NVIDIA's recommended benchmarking protocol.

Test Environment

Hardware Configuration

⚡ Why 120W? For embedded robotics applications, we deliberately limit power consumption to ensure thermal stability and extended operational lifetime. Running at full power on edge devices leads to thermal throttling and reduced component longevity.

Benchmark Protocol

We followed the standard NVIDIA Jetson AI Lab GenAI Benchmarking Protocol with vLLM serving benchmarks.

Models Under Test

Both models were loaded simultaneously into memory using Docker containers, simulating a real-world scenario where multiple models might be available for different tasks.

Model Specifications

Models were tested in the isolated Docker containers based on

image: ghcr.io/nvidia-ai-iot/vllm:latest-jetson-thor

Example of a benchmark commands:

# Qwen3-30B-A3B Test (Concurrency=1)
vllm bench serve \
  --dataset-name random \
  --model RedHatAI/Qwen3-30B-A3B-quantized.w4a16 \
  --num-prompts 50 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --random-input-len 512 \
  --random-output-len 330 \
  --max-concurrency 1 \
  --port 8867

# Nemotron-3-Nano Test (Concurrency=1)
vllm bench serve \
  --dataset-name random \
  --model nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8 \
  --num-prompts 50 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --random-input-len 512 \
  --random-output-len 330 \
  --max-concurrency 1 \
  --port 8866

Results: Time to First Token (TTFT)

The critical metric for real-time robotics agents.

TTFT Detailed Comparison

🏆 Winner: Qwen3-30B-A3B with 180ms median TTFT at C=1 — that's 1.6x faster than Nemotron's result. At C=8, the gap widens to 15x.

⚠️ Note on Concurrency: For edge robotics deployments, C=1 results are most relevant — humanoid robots typically process one inference request at a time. We include C=8 results for completeness, but they're less applicable to real-world robotics scenarios where requests arrive sequentially.

Results: Generation Latency

How fast does each model generate tokens after the first one?

Time per Output Token (TPOT)

⚡ Qwen3's TPOT is 1.4x faster than Nemotron at C=1 (13.49ms vs 19.26ms).

Inter-token Latency (ITL)

Results: Token Throughput

Output Token Throughput (tok/s)

📈 Qwen3 delivers 1.8x higher throughput at C=1 and 3x higher throughput at C=8.

Results: End-to-End Latency

E2E Latency for Complete Response (512 in → 330 out tokens)

🎯 Qwen3 completes full responses in 4.6s vs Nemotron's 6.6s at C=1.

Quick Comparison Summary

Head-to-Head: C=1 (Primary Robotics Scenario)

Head-to-Head: C=8 (Batch Processing Scenario)

Analysis: Which Model for Humanoid Robotics?

For Real-Time Agent Tasks

Qwen3-30B-A3B is the clear winner for robotics applications requiring:

1. Fast Response Initiation — 180ms TTFT at C=1 means your robot starts responding almost instantly. For human-robot interaction, anything under 200ms feels "immediate."
2. Consistent Latency — Qwen3's P99 TTFT (188ms) is almost identical to its median, while Nemotron's P99 (709ms) is 2.5x its median. Predictability matters for real-time control loops.
3. Memory Efficiency — Running at 30% GPU memory utilization leaves headroom for vision models, sensor processing, and motion planning on the same device.
4. Faster Token Generation — 13.49ms TPOT means smoother streaming responses and faster completion times.

When to Consider Nemotron

Nemotron-3-Nano might be preferred when:

• You need NVIDIA's native reasoning parser for chain-of-thought tasks
• FP8 inference precision is required over W4A16 quantization
• You're already in an NVIDIA-centric deployment pipeline

Real-World Robotics Implications

Conclusion

For local real-time inference on humanoid robots, Qwen3-30B-A3B outperforms Nemotron-3-Nano across all key metrics on Jetson Thor at 120W:

• 1.6x faster median TTFT at C=1 (primary robotics scenario)
• 3.8x more consistent latency (P99 vs median ratio)
• 1.8x higher token throughput
• 40% lower GPU memory usage

The W4A16 quantization of Qwen3 proves to be more efficient than Nemotron's FP8 implementation on this hardware, likely due to better memory bandwidth utilization and optimized kernels in the Jetson Thor vLLM build.

For robotics engineers deploying LLM-powered agents on edge devices, Qwen3-30B-A3B provides the responsiveness needed for natural human-robot interaction while leaving computational headroom for perception and control systems.

P.S. FLASHINFER as an Attention Backend

Using AttentionBackendEnum.FLASHINFER is suggested as a performance upgrade for any model interference running on Jetson/Nvidia devices.

Enabling FLASHINFER led to nothing cause it's already quite optimal, but you'll see the difference with a bigger context window.

Resources

• Jetson Thor
• Benchmarking Protocol
• Qwen3-30B-A3B Model
• Nemotron-3-Nano Model
• vLLM Jetson Build

Benchmarked on NVIDIA Jetson Thor @ 120W power mode. Your results may vary based on power settings, thermal conditions, and system load.

Hope you have a good day,

Vsevolod Prudius, Sobaka Team.