CoME-VL: Scaling Complementary
Multi-Encoder
Vision-Language Learning

TL;DR

Entropy Analysis & Performance Gains

SigLIP2 and DINOv3 exhibit distinct entropy profiles across depth — SigLIP2 maintains high entropy (rich semantic diversity) while DINOv3’s deeper layers concentrate on spatially discriminative regions. By leveraging all SigLIP2 layers for understanding and DINOv3 layers 10–23 for grounding, CoME-VL harnesses the best of both worlds.

Figure 1. CoME-VL overview — complementary encoder fusion guided by layer-wise entropy analysis yields consistent improvements across understanding and grounding benchmarks.

Why Two Encoders?

Contrastive (SigLIP2) and self-supervised (DINOv3) encoders learn fundamentally different visual representations. SigLIP2 excels at semantic alignment with language, while DINOv3 captures spatially coherent, fine-grained features ideal for grounding. Combining both unlocks complementary strengths.

Figure 2. Complementary feature analysis — SigLIP2 and DINOv3 encode qualitatively different information. Their fusion provides richer visual representations for downstream vision-language tasks.

CoME-VL Framework

CoME-VL integrates SigLIP2 and DINOv3 through a modular fusion pipeline: entropy-guided layer selection identifies the most informative features, orthogonality-regularized projections reduce redundancy, and RoPE-enhanced cross-attention aligns heterogeneous token grids into compact visual tokens.

Figure 3. CoME-VL architecture — a modular fusion framework that combines contrastive and self-supervised vision encoders through entropy-guided layer selection, orthogonality-constrained projections, and RoPE cross-attention.

Semantic Feature Analysis

DINOv3 maintains spatially coherent object-level attention throughout, while SigLIP2 transitions from broad spatial coverage in early layers to focused semantic discrimination in deeper layers.

DINOv3 — Layer-wise attention rollout showing spatially coherent object-level focus.

SigLIP2 — Layer-wise attention rollout showing transition from spatial to semantic focus.

Benchmark Results

CoME-VL achieves consistent improvements across diverse vision-language benchmarks, outperforming both single-encoder baselines and competitive multi-encoder alternatives.

Main Comparison

RefCOCO Benchmark

RefCOCO	val	testA	testB
Molmo	0.10	0.27	0.27
Clip-to-DINO	91.73	94.06	88.85
Qwen-VL	89.36	92.23	85.36
CoME-VL (Ours)	92.57	95.36	90.51

Component Contribution Analysis

Figure 5. Stacked performance breakdown showing additive gains from RoPE and OL components.

Performance Analysis

Qualitative Results

CoME-VL produces more accurate and spatially precise predictions across diverse vision-language tasks, including visual question answering, object grounding, and document understanding.

Figure 6. Qualitative comparison — CoME-VL generates more contextually accurate and spatially grounded responses compared to single-encoder baselines.

Supported Tasks

CoME-VL supports a wide range of vision-language tasks, leveraging the complementary strengths of both encoders for superior performance across understanding and grounding domains.

Figure 7. CoME-VL supports diverse downstream tasks including visual question answering, document understanding, chart comprehension, counting, pointing, and object detection.

BibTeX

If you find our work useful, please cite:

@article{comevl2026,
  title={CoME-VL: Scaling Complementary Multi-Encoder Vision-Language},
  author={Deria, Ankan and Kumar, Komal and He, Xilin and Razzak, Imran and Cholakkal, Hisham and Khan, Fahad Shahbaz and Khan, Salman},
  journal={arXiv preprint},
  year={2026}
}