Browse State-of-the-Art › Video Grounding
Video Grounding
56 papers with code · 2 benchmarks · 10 datasets archive 2025-07-28
Video grounding is the task of linking spoken language descriptions to specific video segments. In video grounding, the model is given a video and a natural language description, such as a sentence or a caption, and its goal is to identify the specific segment of the video that corresponds to the description. This can involve tasks such as localizing the objects or actions mentioned in the description within the video, or associating a specific time interval with the description.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| QVHighlights (7 rows) | InternVideo2-6B | InternVideo2: Scaling Foundation Models for Multimodal Video Understanding | code | — | Compare |
| MAD (2 rows) | DeCafNet | DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in... | code | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
10 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
2 subtasks in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 114 · with a code link: 56 · where Syntology ran a sample: 23 (21 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (23 of 114 tagged: 21 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 114, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
23 Mar 2022 3 repositories listedFinding relevant moments and highlights in videos according to natural language queries is a natural and highly valuable common need in the current video content explosion era.
-
22 Mar 2024 2 repositories listedWe introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue.
-
3 Jan 2024 2 repositories listed Syntology official (archive's flag): 15 ran · 22 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 1 honoured, 1 violated, 10 with no contract checked; 10 where Syntology's instrument failed) · 12 unverified (of 34 harvested samples) · 34 pointer-only (licence)The key of CG-STVG lies in two specially designed modules, including instance context generation (ICG), which focuses on discovering visual context information (in both appearance and motion) of the instance, and…
-
10 Sep 2021 2 repositories listedInstead, from a perspective on temporal grounding as a metric-learning problem, we present a Mutual Matching Network (MMN), to directly model the similarity between language queries and video moments in a joint…
-
2 Jun 2025 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 2 where Syntology's instrument failed) · 7 unverified (of 17 harvested samples) · 1 pointer-only (licence)In this work, we explore RLT as a post-training strategy to enhance the video-specific reasoning capabilities of MLLMs.
-
22 May 2025 1 repository listedThe approach taken by existing methods of dividing video into clips and processing each clip via a full-scale expert encoder is challenging to scale due to prohibitive computational costs of processing a large number of…
-
7 May 2025 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 10 unverified (of 17 harvested samples) · 5 pointer-only (licence)To address these limitations, we propose OSGNet, an Object-Shot enhanced Grounding Network for egocentric video.
-
17 Mar 2025 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 2 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 3 pointer-only (licence)We introduce TimeZero, a reasoning-guided LVLM designed for the temporal video grounding (TVG) task.
-
13 Mar 2025 1 repository listedEach sequence in BOSTVG, paired with a free-form textual query, encompasses a varying number of targets ranging from 1 to 10.
-
9 Mar 2025 1 repository listedTemporal localization in untrimmed videos, which aims to identify specific timestamps, is crucial for video understanding but remains challenging.
-
16 Feb 2025 1 repository listedCompared to existing methods leveraging zero-initialized queries, object queries in our TA-STVG, directly generated from a given video-text pair, naturally carry target-specific cues, making them adaptive and better…
-
14 Jan 2025 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 7 harvested samples)Furthermore, we propose ST-Align dataset with 4.
-
14 Jan 2025 1 repository listed Syntology official (archive's flag): 2 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)We introduce Tarsier2, a state-of-the-art large vision-language model (LVLM) designed for generating detailed and accurate video descriptions, while also exhibiting superior general video understanding capabilities.
-
VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning12 Jan 2025 1 repository listedDespite the advancements of Video Large Language Models (VideoLLMs) in various tasks, they struggle with fine-grained temporal understanding, such as Dense Video Captioning (DVC).
-
18 Dec 2024 1 repository listedCompositional generalization is the capability of a model to understand novel compositions composed of seen concepts.
-
27 Nov 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)We construct MMDuetIT, a video-text training dataset designed to adapt VideoLLMs to video-text duet interaction format.
-
21 Jul 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 2 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 9 pointer-only (licence)Through a feasibility study, we demonstrate that LLM encoders effectively refine inter-concept relations in multimodal embeddings, even without being trained on textual embeddings.
-
1 Jun 2024 1 repository listedVideos carry rich visual information including object description, action, interaction, etc., but the existing multimodal large language models (MLLMs) fell short in referential understanding scenarios such as…
-
2 Apr 2024 1 repository listed Syntology 13 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 9 where Syntology's instrument failed) · 7 unverified (of 20 harvested samples) · 16 pointer-only (licence)In this paper, we study the effect of cross-modal fusion on the scalability of video grounding models.
-
21 Mar 2024 1 repository listedInspired by the activity-silent and persistent activity mechanisms in human visual perception biology, we design a Unified Static and Dynamic Network (UniSDNet), to learn the semantic association between the video and…
-
15 Mar 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 6 pointer-only (licence)Video-text Large Language Models (video-text LLMs) have shown remarkable performance in answering questions and holding conversations on simple videos.
-
27 Dec 2023 1 repository listedIn the weakly supervised temporal video grounding study, previous methods use predetermined single Gaussian proposals which lack the ability to express diverse events described by the sentence query.
-
12 Dec 2023 1 repository listedVideo moment retrieval is a challenging task requiring fine-grained interactions between video and text modalities.
-
11 Dec 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 3 pointer-only (licence)Existing approaches to video understanding, mainly designed for short videos from a third-person perspective, are limited in their applicability in certain fields, such as robotics.
-
30 Nov 2023 1 repository listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified (of 11 harvested samples) · 11 pointer-only (licence)Large language models (LLMs) have shown remarkable text understanding capabilities, which have been extended as Video LLMs to handle video data for comprehending visual details.
-
28 Nov 2023 1 repository listed Syntology official (archive's flag): 9 ran · 9 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 12 harvested samples) · 6 pointer-only (licence)Video Moment Retrieval (MR) and Highlight Detection (HD) have attracted significant attention due to the growing demand for video analysis.
-
22 Nov 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 5 pointer-only (licence)Extending image-based Large Multimodal Models (LMMs) to videos is challenging due to the inherent complexity of video data.
-
26 Oct 2023 1 repository listedIn this paper, we propose DiffusionVG, a novel framework with diffusion models that formulates video grounding as a conditional generation task, where the target span is generated from Gaussian noise inputs and…
-
12 Sep 2023 1 repository listedCompared with the general video grounding task, MTVG focuses on meticulous actions and changes on the face.
-
4 Sep 2023 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 1 pointer-only (licence)We study visually grounded VideoQA in response to the emerging trends of utilizing pretraining techniques for video-language understanding.
-
14 Aug 2023 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 1 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 4 pointer-only (licence)
-
20 Jun 2023 1 repository listed
-
6 Apr 2023 1 repository listed
-
24 Mar 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample) · 1 pointer-only (licence)
-
9 Mar 2023 1 repository listed Syntology official (archive's flag): 14 ran · 14 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 5 where Syntology's instrument failed) · 2 unverified (of 16 harvested samples) · 4 pointer-only (licence)
-
26 Feb 2023 1 repository listed
-
16 Feb 2023 1 repository listed
-
1 Jan 2023 1 repository listed
-
16 Nov 2022 1 repository listed
-
22 Oct 2022 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
27 Sep 2022 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 9 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified (of 18 harvested samples)
-
26 Sep 2022 1 repository listed
-
22 Sep 2022 1 repository listed
-
1 Sep 2022 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 1 pointer-only (licence)
-
18 Apr 2022 1 repository listed
-
30 Mar 2022 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 3 harvested samples)
-
25 Jan 2022 1 repository listed
-
1 Dec 2021 1 repository listed
-
6 Jul 2021 1 repository listed
-
21 Jun 2021 1 repository listed
-
3 Apr 2021 1 repository listed
-
10 Nov 2020 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
7 Apr 2020 1 repository listed
-
19 Jan 2020 1 repository listed
-
21 Jan 2019 1 repository listed
-
VideoITG: Multimodal Video Understanding with Instructed Temporal Grounding17 Jul 2025 0 repositories listed
-
SAMA: Towards Multi-Turn Referential Grounded Video Chat with Large Language Models24 May 2025 0 repositories listed
-
Enhancing Weakly Supervised Video Grounding via Diverse Inference Strategies for Boundary and Prediction Selection29 Mar 2025 0 repositories listed
-
VideoGEM: Training-free Action Grounding in Videos26 Mar 2025 0 repositories listed
-
SpaceVLLM: Endowing Multimodal Large Language Model with Spatio-Temporal Video Grounding Capability18 Mar 2025 0 repositories listed
-
Contextual Self-paced Learning for Weakly Supervised Spatio-Temporal Video Grounding28 Jan 2025 0 repositories listed
-
STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding1 Jan 2025 0 repositories listed
-
Multi-Scale Contrastive Learning for Video Temporal Grounding10 Dec 2024 0 repositories listed
-
Video LLMs for Temporal Reasoning in Long Videos4 Dec 2024 0 repositories listed
-
Seq2Time: Sequential Knowledge Transfer for Video LLM Temporal Grounding25 Nov 2024 0 repositories listed
-
SimBase: A Simple Baseline for Temporal Video Grounding12 Nov 2024 0 repositories listed
-
SynopGround: A Large-Scale Dataset for Multi-Paragraph Video Grounding from TV Dramas and Synopses3 Aug 2024 0 repositories listed
-
Multi-sentence Video Grounding for Long Video Generation18 Jul 2024 0 repositories listed
-
Described Spatial-Temporal Video Detection8 Jul 2024 0 repositories listed
-
AutoTVG: A New Vision-language Pre-training Paradigm for Temporal Video Grounding11 Jun 2024 0 repositories listed
-
Simplify Implant Depth Prediction as Video Grounding: A Texture Perceive Implant Depth Prediction Network7 Jun 2024 0 repositories listed
-
Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition7 May 2024 0 repositories listed
-
SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Grounding1 Apr 2024 0 repositories listed
-
VideoGrounding-DINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding1 Jan 2024 0 repositories listed
-
Video-GroundingDINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding31 Dec 2023 0 repositories listed
-
LLM4VG: Large Language Models Evaluation for Video Grounding21 Dec 2023 0 repositories listed
-
Multi-Modal Domain Adaptation Across Video Scenes for Temporal Video Grounding21 Dec 2023 0 repositories listed
-
EtC: Temporal Boundary Expand then Clarify for Weakly Supervised Video Grounding with Multimodal Large Language Model5 Dec 2023 0 repositories listed
-
29 Aug 2023 0 repositories listed
-
ViGT: Proposal-free Video Grounding with Learnable Token in Transformer11 Aug 2023 0 repositories listed
-
G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theory26 Jul 2023 0 repositories listed
-
No-frills Temporal Video Grounding: Multi-Scale Neighboring Attention and Zoom-in Boundary Detection20 Jul 2023 0 repositories listed
-
Generation-Guided Multi-Level Unified Network for Video Grounding14 Mar 2023 0 repositories listed
-
Exploiting Auxiliary Caption for Video Grounding15 Jan 2023 0 repositories listed
-
Collaborative Static and Dynamic Vision-Language Streams for Spatio-Temporal Video Grounding1 Jan 2023 0 repositories listed
-
Hierarchical Semantic Correspondence Networks for Video Paragraph Grounding1 Jan 2023 0 repositories listed
-
Iterative Proposal Refinement for Weakly-Supervised Video Grounding1 Jan 2023 0 repositories listed
-
WINNER: Weakly-Supervised hIerarchical decompositioN and aligNment for Spatio-tEmporal Video gRounding1 Jan 2023 0 repositories listed
-
Language-free Training for Zero-shot Video Grounding24 Oct 2022 0 repositories listed
-
Graph2Vid: Flow graph to Video Grounding for Weakly-supervised Multi-Step Localization10 Oct 2022 0 repositories listed
-
On the Effects of Video Grounding on Language Models1 Oct 2022 0 repositories listed
-
Exploiting Feature Diversity for Make-up Temporal Video Grounding12 Aug 2022 0 repositories listed
-
6 Jul 2022 0 repositories listed
-
Team PKU-WICT-MIPL PIC Makeup Temporal Video Grounding Challenge 2022 Technical Report6 Jul 2022 0 repositories listed
-
Gaussian Kernel-based Cross Modal Network for Spatio-Temporal Video Grounding2 Jul 2022 0 repositories listed
-
Position-aware Location Regression Network for Temporal Video Grounding12 Apr 2022 0 repositories listed
-
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding15 Mar 2022 0 repositories listed
-
Multi-Scale Self-Contrastive Learning with Hard Negative Mining for Weakly-Supervised Query-based Video Grounding8 Mar 2022 0 repositories listed
-
Unsupervised Temporal Video Grounding with Deep Semantic Clustering14 Jan 2022 0 repositories listed
Syntology lines on 24 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections