This could be an interesting new research paper by Kaiming He and his team!
From the abstract:
"We show that multimodal models possess strong reasoning abilities and that an appropriate harness can unlock their potential to solve tasks across diverse interactive environments.
We introduce VISTA, a visual harness that gives a general-purpose multimodal model long-horizon vision.
VISTA allows the model to directly perceive the environment through visual observations and maintains a lossless visual memory that preserves past observations in their original form. The model can actively retrieve these observations and reorganize its visual input as it reasons.
On ARC-AGI-3, VISTA improves Claude Opus 5.0's Relative Human Action Efficiency score from 40.68 to a perfect 100.00, with the model completing all 25 public games using 57.4% fewer actions than first-time human participants. VISTA's simple design also allows it to extend naturally to diverse visual environments with minimal adaptation.
Across three additional benchmarks covering a diverse range of visual games and puzzles, it substantially outperforms baselines using the same underlying model with minimal harnesses.
Our results highlight VISTA's potential as a general-purpose visual harness for advancing multimodal agents in complex visual environments."
No comments:
Post a Comment