Grounding DINO 1.5: Performance & Efficiency Leap (May)
The team released Grounding DINO 1.5 in two editions:
Pro (maximum accuracy): Scaled up the model and trained on over 20 million grounding data points, achieving SOTA in open‑vocabulary detection. It set new zero‑shot records on COCO, LVIS, etc., showing strong recognition of unseen objects.
Edge (on‑device real‑time): Revolutionized edge deployment, running on NVIDIA Orin NX at 10 FPS – a key enabler for embodied AI, autonomous driving, and other latency‑sensitive scenarios.
Grounding DINO Accepted as ECCV 2024 Top Paper
The original paper "Grounding DINO: Marrying DINO with Grounded Pre‑training for Open‑Set Object Detection" was honored as a top paper at ECCV 2024, receiving high academic recognition.
From DINO to DINO‑X: A Unified Model for "Recognizing Everything" (November)
Building on the Grounding DINO 1.5 architecture, the team released DINO‑X – a unified model moving toward "recognizing everything". It introduces a "universal object prompt" that allows users to detect almost any object in an image without any prompts, especially excelling at rare long‑tail objects. To enable this, the team built the Grounding‑100M dataset with over 100 million high‑quality samples.
Deployment First: Rapid Ecosystem Expansion
Thanks to these breakthroughs, Grounding DINO quickly permeated various industries in 2024:
AI Companions & Consumer Electronics: The "AI Nailong" companion robot, powered by Grounding DINO, can interact with children in home childcare scenarios, demonstrating productization potential.
Smart Agriculture: Researchers combined Grounding DINO with the SAM segmentation model to achieve efficient zero‑shot automatic annotation in grape cluster detection, significantly reducing data preparation costs.
Autonomous Driving & Security: The edge‑deployable Grounding DINO 1.5 Edge shows great promise for real‑time object detection around vehicles and live analysis of surveillance feeds.
E‑commerce & Content Moderation: The Pro model's strong multimodal semantic understanding can be used for automatic product image tagging and social media image moderation, improving search and recommendation systems.
What Researchers Are Focusing On: New Directions for CLIP Variants
While Grounding DINO and its derivatives excel at localizing objects in images, researchers are also working on giving CLIP‑like models precise "visual grounding" abilities:
Injecting Localization into CLIP: Classic CLIP is great at global matching but weak at understanding object relationships. Approaches such as Grounding CLIP, RegionSpot, and Q‑GroundCAM enhance CLIP's localization accuracy at pixel, region, and quantization levels.
Continued Evolution of GLIP: As a parallel technical route to Grounding DINO, GLIP (Grounded Language‑Image Pre‑training) continues to evolve. A variant, CT‑GLIP, has even been applied to 3D CT medical images, enabling fully automatic organ‑level and disease‑region alignment.
Looking Ahead: More Accurate, Faster, More General
The 2024 progress of Grounding DINO and Grounding CLIP clearly outlines the future of vision AI:
Stronger Generalization: Models are moving toward truly "open‑world" recognition – able to identify and locate any unseen object, even without user prompts.
Faster Inference: Lightweight models and on‑device deployment are clear trends, enabling AI to operate in latency‑critical physical‑world interactions (robotics, autonomous driving, drones).
Deeper Scene Integration: With models like DINO‑X, we are moving toward unified foundations – one model handling detection, segmentation, pose estimation, object description, and other perception/understanding tasks simultaneously, laying the perceptual groundwork for advanced embodied AI and AGI.
