1Mohamed bin Zayed University of Artificial Intelligence 2Apertix †Equal contribution
Abstract
Frontier general-purpose systems are expanding beyond visual understanding into tasks traditionally handled by dedicated vision models. How much of computer vision is now accessible through a general-purpose system, and where do dedicated models remain necessary? We evaluate GPT-6 Astra alongside five frontier systems across 34 capabilities and 55 benchmarks spanning nine areas of computer vision. Semantic understanding, reasoning, and object-centric prediction increasingly approach specialist or human reference levels, while precise geometry, faithful reconstruction, temporal consistency, and specialized visual knowledge remain challenging.
34 capabilities across nine areas of computer vision, drawn from 55 benchmarks.
Hover a task to see how the
six systems score against specialist models and humans.
Across 34 capabilities and 55 benchmarks covered in this study, we notice that the frontier has broadened dramatically. However, breadth is not uniform in maturity.
Q2Generalists are strongest at semantic interpretation and reasoning; gaps concentrate in metric precision, fidelity, temporal consistency, and specialized fine-grained discrimination.
Q3Specialist-level performance is emerging in semantic and object-centric tasks, while gaps persist in tasks requiring geometric precision, temporal consistency, fine-grained reconstruction, or domain expertise.
Q4More reasoning and tool use can close selected specialist gaps, but gains are task-dependent and often come with substantially higher inference cost.
Where does general-purpose vision stand?
To understand how far frontier visual capabilities have progressed, we consider how close their performance is
to established reference levels.
For each capability, we compare the best generalist score as a percentage of
the reference, S for specialist and H for human.
Best generalist as a percentage of the best available reference
Some capabilities approach reference levels, while substantial gaps remain in tasks requiring precision, fidelity, or specialized expertise. The choice of reference also affects how we interpret these results: matching a specialist does not necessarily mean reaching human performance or solving the task.
6 frontier general-purpose systems across 34 capabilities spanning 9 broad areas of computer vision.
Reference Model reports either a dedicated specialist, trained or designed specifically for the task, or a prior generalist SOTA, an earlier general-purpose model with the strongest reported result on the benchmark. Human is based on benchmark-reported human performance where available. The last two columns summarize complementary aspects of the capability landscape: Best model vs. ref. measures how far the best generalist exceeds or trails the strongest available model or human reference, indicating the remaining headroom (RQ3); Best vs. 2nd best reports its margin over the next-best generalist, highlighting where a particular system stands out (RQ2).
Frontier models now tackle tasks once handled only by specialist models.
GPT-6 Astra detects objects in crowded scenes and segments intricate shapes, extending general-purpose vision to tasks traditionally handled by specialist models. Detection boxes appear in the order Astra outputs them, with the ground truth overlaid at the end. For segmentation, Astra writes polygon vertices, then refines and revises them against the image (final IoU 0.979 and 0.929); SAM 3's masks are shown for comparison.
General-purpose models show emerging strengths in 3D grounding and detection, while gaps remain in precise object geometry, depth estimation, and geometrically consistent reconstruction. For depth, GPT-6 Astra builds the depth map as a program, fitting the room layout first and then placing the chairs row by row (AbsRel 0.111 on this image).
Given only the mask of the first frame, GPT-6 Astra tracks every object through the clip (J&F 85.5 on gold-fish, 84.8 on horse jump).
GPT-6 Astra shows substantial progress in video segmentation, capturing main object shapes and distinguishing multiple instances.
Precise boundaries, small structures, and separation of nearby objects remain challenging.
Organs and findings can be localized in CT and X-ray. In pathology, nuclei are localized and delineated accurately, yet assigning the correct nucleus type remains substantially harder.
Nucleus segmentation on a PUMA H&E tile: GPT-6 Astra outlines 783 nuclei against 813 in the ground truth, shown for the full tile and a 4× zoom.
General-purpose models show strong remote-sensing understanding, but accurately grounding fine-scale objects remains challenging.
Restorations can look plausible but hallucinate content: in the low-light case, four bowling pins become five, extra pins appear on an empty lane, and the banner text changes, while the specialist (MIRAGE) stays faithful. PSNR/SSIM are computed on full images; the crops enlarge the red box 4×.
Text-to-image generation on GenEval2, where each image is checked with questions about object counts, attributes and relations, and instruction-guided editing. GPT-6 Astra scores 96.2 Soft-TIFA GM on GenEval2 and 4.80 out of 5 on ImgEdit-Bench, the highest among the six systems.
EB-Navigation, long-horizon episode: “navigate to the Pillow in the room and be as close as possible to it”. The model sees only the first-person view; the top-down map is for illustration. GPT-6 Astra reaches the target in 17 steps.
The same episode from the same start for all six models: only GPT-6 Astra reaches the target within 20 steps.
Strong recognition, counting, and visual reading extend across all six models. OCR approaches human performance, while document, chart, and infographic understanding exceeds the human score. Mathematical reasoning reaches human levels, while visual logic remains harder.
2D localization and object-centric 3D understanding are emerging strengths, approaching or exceeding specialist performance on several tasks. Video understanding is also competitive, while precise geometry and consistent segmentation across frames remain challenging.
Image generation, editing, and quality assessment approach specialist levels, with Astra exceeding the evaluated references. Faithful image restoration remains a shared weakness across all six models.
Driving-scene, embodied, medical, and remote-sensing understanding approach or exceed specialist performance. Astra also demonstrates successful navigation, while fine-grained pathology recognition and grounding small remote-sensing targets remain challenging.
Visual reading shows the clearest convergence. Recognition, scientific reasoning, image generation and editing, and embodied understanding are increasingly becoming shared strengths across frontier models.
All six models struggle with faithful image restoration and fine-grained pathology recognition. Understanding an image or locating a structure does not guarantee accurate reconstruction or identification.
Strong detection and video understanding are concentrated in selected models. Performance varies substantially across 3D tasks and medical grounding, with different models showing strengths on different subtasks.
Astra shows substantial advances in visual logic and spatial reasoning, alongside large gains in video segmentation, pose estimation, image segmentation, and 3D grounding.
Effect of reasoning effort and tool use on GPT-6 Astra. Above each bar: score ratio and
inference-cost ratio, relative to low effort (left three) or to the run without tools
at xhigh effort (right).
Citation
If you find this work useful, please cite it.
@article{rasheed2026frontiervision,
title = {Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision},
author = {Rasheed, Hanoona and Kurpath, Mohammed Irfan and Ren, Bin and Cholakkal, Hisham and Khan, Fahad Shahbaz and Khan, Salman},
journal = {arXiv preprint arXiv:2609.35718},
year = {2026}
}