Jetson Thor VLM Benchmark Part 2: Vision-Language Models for Real-Time Humanoid Control

Qwen3-VL-30B-A3B vs Nemotron-Nano-12B-VL — Which VLM enables your robot to see and act in real-time?

For robotics executives evaluating vision-language models for humanoid deployment: Qwen3-VL-30B-A3B delivers 2.4x faster response initiation than Nemotron-Nano-12B-VL on NVIDIA Jetson Thor at 120W, while supporting up to 8 seconds of continuous video context for real-time decision-making.

Again, simple results:

As you can see, time to first token and time between the tokens is different. Apparently, Alibaba models are more optimal towards Nvidia compute than Nvidia models. But, Nemotron is cool from the perspective of architecture and real tasks are not just a performance.

Why Vision-Language Models Matter for Humanoids

In Part 1, we benchmarked text-only LLMs for robotics agents. But humanoid robots don't operate in text — they operate in the visual world.

Vision-Language Models (VLMs) are the missing link between perception and action. When your humanoid needs to:

1. Identify objects to manipulate
2. Navigate through cluttered environments
3. Interpret human gestures and expressions
4. Understand spatial relationships in real-time

...it needs a VLM that can process visual input as fast as the world changes.

Models Under Test

We benchmarked two production-ready VLMs optimized for edge deployment:

Qwen3-VL-30B-A3B-Instruct (NVFP4)

Why it matters for robotics: Qwen3-VL was explicitly designed with embodied AI in mind. Its 3D spatial grounding capabilities enable the model to understand object positions, viewpoints, and occlusions — critical for manipulation tasks. The native video support with temporal alignment means the model can track motion and changes across frames, not just process static snapshots.

Model: Qwen3-VL-30B-A3B-Instruct

NVIDIA Nemotron-Nano-12B-v2-VL (NVFP4-QAD)

Why it matters for robotics: Nemotron-Nano-VL uses NVIDIA's hybrid Mamba-Transformer architecture for memory-efficient sequence modeling. The Efficient Video Sampling (EVS) technology prunes temporally static patches, theoretically enabling faster video processing. Being a dense model (all parameters active), it offers predictable latency characteristics.

Model: NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QAD

Test Configuration

Hardware

Benchmark Parameters

⚠️ Critical Note on Context Length: The benchmarks below use 512 input tokens, representing a fresh conversation start. Real-world VLM robotics applications often involve sequential queries or multi-frame video input that can push context to 10,000+ tokens. See the "Real-World Context Considerations" section for latency scaling analysis.

Results: Time to First Token (TTFT)

The metric that determines how long your robot stands frozen before responding.

Single Concurrency (C=1) — Primary Robotics Scenario

🏆 Winner: Qwen3-VL with 42ms median TTFT — that's 2.9x faster than Nemotron's 122ms. For human-robot interaction, sub-50ms feels instantaneous.

High Concurrency (C=8) — Batch Scenario

Note: Nemotron C=8 results in the third benchmark block show degraded performance under concurrent load.

Results: Token Generation Speed

How fast can the model stream its response after the first token?

Time per Output Token (TPOT) at C=1

Qwen3-VL generates tokens 2.4x faster — 15.5ms vs 36.8ms per token.

Output Throughput

Qwen3-VL delivers 2.4x higher throughput at single concurrency.

Results: End-to-End Latency

Total time from prompt to complete response (512 in → 330 out tokens).

Qwen3-VL completes full VLM responses in 5.1 seconds vs Nemotron's 12.2 seconds — 2.4x faster end-to-end.

Real-World Context Considerations

The Hidden Cost of Sequential Queries

The benchmarks above use 512 input tokens — essentially a fresh conversation. But real-world robotics VLM deployments look different:

Scenario: Continuous Visual Navigation

• Robot receives camera frame every 100ms
• Each frame tokenizes to ~500-1000 tokens
• After 10 frames, you're at 5,000-10,000 context tokens
• Chat history and task context add more

Expected latency scaling with longer context:

⚠️ Key insight: The speeds shown in this benchmark represent best-case fresh chat scenarios. When building continuous VLM control loops, expect 3-5x higher latencies as context accumulates.

Qwen3-VL's Video Advantage

Qwen3-VL natively supports up to approximately 8 seconds of video through its Interleaved-MRoPE (Multi-Resolution Rotary Position Embedding) architecture. This isn't just frame-by-frame processing — the model learns temporal dynamics and can:

• Track object motion across frames
• Understand action sequences
• Pinpoint specific moments with second-level precision
• Maintain spatial-temporal coherence

For humanoid control, this means you can feed the model a short video clip of the environment rather than individual frames, getting more coherent understanding of dynamics and motion.

Head-to-Head Summary

C=1 Results (Primary Robotics Scenario)

Analysis: Which VLM for Humanoid Robotics?

For Real-Time Visual Control

Qwen3-VL-30B-A3B is the clear winner for humanoid applications requiring:

1. Ultra-Low TTFT — 42ms means your robot starts processing visual input almost instantly. Combined with motor control latency, total response time stays under human perception thresholds.
2. Native Video Understanding — 8 seconds of temporal context with second-level indexing. Feed it a short clip of the workspace, get coherent spatial-temporal understanding.
3. 3D Spatial Grounding — Explicitly designed for embodied AI. The model can reason about object positions, viewpoints, and occlusions — essential for manipulation.
4. MoE Efficiency — Only 3B parameters active at inference despite 31B total, leaving GPU headroom for other perception and control systems.

When to Consider Nemotron-Nano-VL

Nemotron might be preferred when:

• You need NVIDIA's native tooling and support ecosystem
• Document intelligence is the primary use case (leading OCRBench v2 scores)
• You're processing mostly static images rather than video
• Quantization-Aware Distillation (QAD) quality matters for your accuracy requirements

Deployment Recommendations by Use Case

Practical Integration Notes

vLLM Serving Configuration

# docker-compose.yml
services:
  qwen3-vl:
    image: ghcr.io/nvidia-ai-iot/vllm:latest-jetson-thor
    environment:
      - MODEL=ig1/Qwen3-VL-30B-A3B-Instruct-NVFP4
    ports:
      - "8867:8000"
    deploy:
      resources:
        reservations:
          devices:
            - capabilities: [gpu]
  
  # Commented alternative:
  # nemotron-vl:
  #   image: ghcr.io/nvidia-ai-iot/vllm:latest-jetson-thor
  #   environment:
  #     - MODEL=nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QAD

Context Management Strategy

For continuous VLM control loops, implement context windowing:

MAX_CONTEXT_TOKENS = 4096  # Keep context manageable
FRAME_TOKENS = ~1000       # Approximate tokens per frame

def manage_vlm_context(frames, history):
    # Keep only last N frames to stay under context limit
    max_frames = MAX_CONTEXT_TOKENS // FRAME_TOKENS - 1  # Leave room for response
    recent_frames = frames[-max_frames:]
    
    # Compress or summarize old history
    compressed_history = summarize_if_needed(history)
    
    return recent_frames, compressed_history

Latency Budget for 10Hz Control Loop

Reality check: Even the fastest VLM can't fit a naive 10Hz control loop with full response generation. Use streaming responses, cache common decisions, or run VLM at lower frequency with interpolation.

Conclusion

For vision-language inference on humanoid robots, Qwen3-VL-30B-A3B outperforms Nemotron-Nano-12B-VL on Jetson Thor at 120W:

• 2.9x faster Time to First Token (42ms vs 122ms)
• 2.4x faster token generation (15.5ms vs 36.8ms per token)
• 2.4x higher throughput (63.8 vs 26.8 tok/s)
• Native 8-second video context with temporal understanding
• 3D spatial grounding for embodied AI applications

The MoE architecture's efficiency (3B active out of 31B total) combined with explicit embodied AI optimizations makes Qwen3-VL the clear choice for humanoid robotics deployments where visual understanding must keep pace with physical reality.

Important caveat: These results represent fresh-chat best-case scenarios. Real deployments with accumulated context will see degraded latencies — plan your context management strategy accordingly.

Resources

• Qwen3-VL-30B-A3B Model
• Nemotron-Nano-12B-VL Model
• Qwen3-VL Technical Report
• NVIDIA Nemotron VL Report
• Jetson Thor
• Part 1: Text LLM Benchmarks

Benchmarked on NVIDIA Jetson Thor @ 120W power mode. Results based on 512-token input contexts. Your results will vary with longer contexts, different power modes, and thermal conditions.

Hope you have a good day/night,

Vsevolod Prudius, Sobaka Team.