UFactory xArm7 videos show real-world Anchor-Align rollouts. Simulation videos show Anchor-Align policies (LIBERO-PRO, LIBERO-Plus, LIBERO and CALVIN).
OOD generalization with Anchor-Align
Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior
cloning (BC) has become the standard recipe for vision-language-action (VLA) policies.
However, BC finetuning progressively overwrites the pretrained representations that support
visual and semantic generalization. Co-training on web image-text data, a common remedy,
does not prevent this; it applies language and action losses to separate observations,
leaving VLAs with language-action misalignment that standard manipulation benchmarks do
not expose.
We propose Anchor-Align, which augments BC with two objectives:
Vision-Language Anchoring distills layer-wise representations from a frozen VLM
copy to prevent this drift, while Language-Action Alignment converts each action
target into a discrete motion-direction label and jointly trains language and action
prediction on the same robot observation. On a physical xArm7 robot, across two widely used
VLA architectures, Anchor-Align improves real-robot success on both (28% → 54% and
37% → 60%). At scale in simulation, we demonstrate consistent improvements on OOD
perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO,
LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations
and effective action learning are not fundamentally at odds.
Method overview
Anchor-Align converts each action target into a discrete motion-direction label and jointly trains
language and action prediction on the same observation, while distilling the policy’s hidden
states toward a frozen pretrained anchor.
Real-world generalization (UFactory xArm7)
We deploy Anchor-Align on a real UFactory xArm7 manipulation setup and
evaluate observation-grounded generalization rather than memorization of a single tabletop
layout:
Semantic Perturbation: the policy must pick up the pink mug despite being trained only to pick and place the green mug, tested across varied layouts of the target, mugs, and distractor objects.
Spatial Rearrangement: the target object, surrounding objects, and distractors are relocated.
Cluttered Scene: additional distractor objects are added and the policy must select the object specified by language.
Compositional Object-Layout: object placement and object orientation are jointly changed.
For Spatial Rearrangement, Compositional Object-Layout, and Semantic Perturbation, every
evaluation tests a layout that never appeared in the training data, making these regimes more
challenging than the LIBERO-PRO position swap.
Real-world testing under different backbones and different generalization setups. Success rates
across the three generalization conditions for two distinct backbones. Anchor-Align
improves generalization on both: with the Prismatic Qwen2.5-0.5B backbone, mean success
rises markedly, and with the architecturally distinct StarVLA backbone, every condition
improves. Right
(pink-mug OOD, trained only on the green mug): the Standard BC baseline collapses to its
training-time green-mug prior, picking the wrong (green) mug in 90% of trials and placing it in 30%,
while Anchor-Align follows the “pink mug” instruction, picking the pink mug in
100% of trials and placing it on the plate in 40%.
🎨 Semantic Perturbation (pink-mug OOD)
Trained only on the green mug, the policy is tested with a new instruction referring to the pink mug. This probes whether the policy uses the preserved VLM color concept under a new instruction instead of defaulting to the training object. The Standard BC baseline collapses to its training-time green-mug prior, while Anchor-Align follows the instruction and picks the pink mug.
Pink-mug OOD
Task prompt: “Pick up the pink mug and place it on the plate.”
Train setupTest setupStandard BCSame VLA with Anchor-Align (ours)
Standard BC progressively erodes the VLM’s pretrained color semantics. Faced with a scene containing multiple mugs, the baseline can no longer reliably distinguish objects by color and grasps the wrong one. Anchor-Align prevents this erosion via Vision-Language Anchoring, retaining color grounding throughout finetuning and consistently picking the correct target.
Task prompt: “Pick up the green mug and place it on the plate.”
Train setupTest setupStandard BCSame VLA with Anchor-Align (ours)
Anchor-Align across more scene variations
Arrangement 1Arrangement 2Arrangement 3
🧺 Cluttered Scene
Additional distractor objects are added and the policy must select the object specified by language. We show one representative Anchor-Align rollout for each of the three target objects.
Target object: Yellow bell pepper
Target object: Pineapple
Target object: Red bell pepper
🧩 Compositional Object-Layout
Object placement and object orientation are jointly changed, so every evaluation is out-of-distribution. The Standard BC baseline drifts toward a memorized motion, while Anchor-Align stays grounded in the current observation.
Comparison between Standard BC and Anchor-Align
Task prompt: “Pick up the broccoli and place it on the plate.”
Train setupTest setupStandard BCSame VLA with Anchor-Align (ours)
Anchor-Align across more scene variations
LIBERO-Plus OOD generalization
LIBERO-Plus probes perceptual robustness across seven independent OOD axes; we show six of them here. For each axis we pair the Standard BC baseline (failure) with Anchor-Align (success) on the same initial condition.
LIBERO-PRO Object Generalization trains the policy on a task with a given object, then at test swaps that object for a new one that can differ in size, shape, color, or other characteristics. On the same episode, Standard BC fails while Anchor-Align succeeds.
Pick up the alphabet soup and place it in the basket.1/3
LIBERO-PRO Position Swap trains the policy with the task objects in their original positions and evaluates it after those positions are permuted, so completing the task requires acting on the objects' current locations rather than replaying a memorized trajectory. We pair the same held-out episode under Standard BC (failure) and Anchor-Align (success); each pair is a different task.
Pick up the black bowl between the plate and the ramekin and place it on the plate.1/2
@article{dalal2026anchoralign,
title = {Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment},
author = {Dalal, Dwip and Patel, Shivansh and Jain, Chahit and Kim, Jeonghwan and Mishra, Utkarsh and Baratian, Alex and Ha, Hyeonjeong and Ji, Heng and Lazebnik, Svetlana and Jain, Unnat},
journal = {arXiv preprint arXiv:2607.13429},
year = {2026}
}
