Characterizing VLA Models: Identifying the Action Generation Bottleneck for Edge AI Architectures
Abstract
This study characterizes the performance of Vision-Language-Action (VLA) workloads on next-generation edge hardware to address the critical "Memory Wall" in embodied AI. Profiling of the Molmo-7B model confirms that auto-regressive Action Generation is strictly memory-bound, with standard compute scaling yielding diminishing returns (4x compute scaling but only 1.25x latency reduction).
We evaluate system scalability across LPDDR6, GDDR7, and HBM3e interfaces, finding that standard memory technologies fail to meet the 10Hz real-time safety target for models larger than 13B. Specifically, a 100B parameter model on a GDDR7-equipped system incurs an unsafe latency of 83 seconds. In contrast, our analysis shows that Processing-in-Memory (PIM) architectures, by leveraging internal bank-level parallelism, deliver a transformative 140x speedup, reducing 100B model latency to 2.8 seconds and bringing large-scale reasoning within the realm of feasibility for robotic control.
We evaluate system scalability across LPDDR6, GDDR7, and HBM3e interfaces, finding that standard memory technologies fail to meet the 10Hz real-time safety target for models larger than 13B. Specifically, a 100B parameter model on a GDDR7-equipped system incurs an unsafe latency of 83 seconds. In contrast, our analysis shows that Processing-in-Memory (PIM) architectures, by leveraging internal bank-level parallelism, deliver a transformative 140x speedup, reducing 100B model latency to 2.8 seconds and bringing large-scale reasoning within the realm of feasibility for robotic control.
