RoboColosseum

RoboColosseum is a distributed real-robot evaluation framework for physical intelligence in the wild, from NUS, UW, Stanford, UNC, Ai2, Oxford, Hugging Face, BitRobot, and the Singapore Embodied AI Centre (EAIC) and partners—paper arXiv:2611.02497 (submitted 2026-11-02). It combines unified policy serving and standardized protocols across locally operated robots in diverse environments.

The broader network vision is 100+ robots / 140+ scenes; the published study spans SO-101, Franka (DROID), bimanual YAM, and Unitree G1, evaluating five VLAs and three frontier agents. Two tracks: Open Track (blinded pairwise trials without task-specific adaptation) and Fine-tune Track (same 100 demos per task). Authors pledge 10,000 free evaluation hours for academic labs. HF / Unitree and partners amplified on 2026-11-04.

Problem

Generalist robot policies must work across robots, tasks, and scenes, but physical trials are hard to reproduce and single-lab samples stay small. Simulation scales, yet sensing/dynamics gaps remain. RoboColosseum distributes real robots under shared interfaces to balance comparability with environmental diversity.

Differentiation

• Multi-embodiment: not a single Franka/DROID arena (cf. RoboArena-class efforts).
• Two tracks: Open (zero-shot in-the-wild pairwise) + Fine-tune (fixed demo budget).
• Scale & stats: initial study includes 5,699 Open Track comparisons at 124 sites; Difficulty-Calibrated Rasch (DCR) rankings plus active matchmaking; 10k free academic hours.

Limits

• Reported numbers cover 17 Sep–28 Oct 2026; coverage depends on checkpoint/hardware fit.
• Fine-tune fixes data but allows each model’s official recipe—end-to-end adaptation, not identical optimizers.
• On G1, no policy fully succeeds on the five tasks; use progress scores, not success alone.

Who it is for

Labs evaluating VLAs / embodied foundation models who need multi-site real-robot comparison or academic free eval hours.

Getting started

1. Read arXiv:2611.02497 for Open / Fine-tune protocols and embodiment specs.
2. Watch HF / BitRobot / EAIC channels to apply for eval quota (10k academic hours pledged).
3. Serve policies through the shared observation/action interface; Open Track is blinded pairwise; Fine-tune uses the released 100 demos per task.