Fix: Jetson Orin Nano CUDA out-of-memory during VLM warmup

When loading LocateAnything-3B on an 8 GB Orin Nano, the first forward pass OOMs even when nvidia-smi shows headroom. Here's the actual cause — and the two-line fix.

Rajat M R

Jul 20, 2026

Fix: Jetson Orin Nano CUDA out-of-memory during VLM warmup

TABLE OF CONTENTS

The trap

The fix

Why this matters on Orin Nano specifically

Symptom: loading LocateAnything-3B on Jetson Orin Nano 8 GB throws CUDA out of memory on the first forward pass — even though nvidia-smi shows ~2 GB free.

The trap

On Jetson, GPU and CPU share the same physical DRAM (unified memory). What nvidia-smi reports as "free" is what the CUDA allocator currently sees — not what's physically available. When PyTorch warms up a 3B-param VLM, it briefly needs contiguous slabs larger than what the fragmented pool can hand out.

The fix

Set the allocator to expandable segments before importing torch:


export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

Or in Python, before any CUDA op:


import os
os.environ["PYTORCH_CUDA_ALLOC_CONF"] = "expandable_segments:True"
import torch

Expandable segments let the allocator grow existing blocks instead of demanding fresh contiguous slabs — exactly what unified-memory boards need during warmup.

Why this matters on Orin Nano specifically

Discrete GPUs have their own VRAM; fragmentation there is annoying but bounded. On Orin Nano, every allocation competes with the OS, the display server, and whatever else is on the board. Expandable segments turned the OOM at load-time into a clean run in my LocateAnything-3B pipeline.

Part of my ongoing notes on getting big models onto small boards. If this helped, say hi on LinkedIn.

Rajat M R

I'm a passionate Engineer. I'm driven by a desire to share my knowledge and experiences with others, and I'm always looking for new ways to build, innovate and experiment.