OpenVLHarness

OpenVLHarness is a training-free multimodal reasoning harness from researchers at UCLA (UCLA NLP) and Apple. Its pitch: "Turn any MLLM into an evidence-based visual reasoner that thinks and codes in visual space."

The paper, "OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory", was submitted to arXiv on October 19, 2026 (2610.26860). The code is open source under MIT on GitHub at uclanlp/OpenVLHarness (about 250 stars), with a Hugging Face demo and evaluation datasets.

On October 25 an X post introduced it: "UCLA and Apple just released OpenVLHarness, an open-source framework that makes existing multimodal AI models significantly better at visual reasoning without any additional training."

How it works

The model chooses tools, inspects their evidence and continues until it can answer. Three components:

• Capability layer: nine task-level interfaces turn tool outputs into readable observations (annotated images, summaries, compact numerical data)
• Multimodal memory: images, boxes, masks, depth maps and web references stay addressable for the whole question; each new question starts with fresh memory
• Python Coding Agent: the backbone writes code in a separate context to compute over stored boxes, masks, depth maps and other results

Nine capabilities

Capability	Default backend
Visual grounding	SAM 3
OCR	PaddleOCR-VL
Depth estimation / camera trajectory	Depth Anything 3
Zoom-in	built-in image processing
Text search / image search / webpage visit	Serper + Qwen
Python coding agent	backbone + local Python

Results (average over 22 benchmarks)

Backbone	Base	+ OpenVLHarness
Qwen3-VL-8B-Instruct	34.24	45.41
Qwen3-VL-32B-Instruct	38.30	50.92
Kimi K3	56.43	66.09
GPT-6 Astra	60.27	70.17

The paper reports that with GPT-6 Luna at medium effort it beats the base model, Codex and Visual Sketchpad at maximum effort by 7.7–8.5 points while using at most 37% of their model-token cost.

How to use it

• Online demo: Hugging Face Space uclanlp/OpenVLHarness (bring an OpenAI-compatible Chat Completions endpoint with native tool calling and image input; the agent loop and perception tools run on the project server)
• Run locally: Python 3.10+, install from GitHub uclanlp/OpenVLHarness; presets cover seven of the paper's backbones plus every Qwen3-VL Instruct size and OpenAI GPT-5 to GPT-6.1 models
• Reproduce the evaluation: datasets at Hugging Face uclanlp/OpenVLHarness-Evaluation-Datasets
• Project page: openvlharness.github.io

Who it's for

• Developers who want better counting, grounding, spatial understanding and multimodal search from an MLLM without fine-tuning
• Researchers working on visual agents and tool use