Browse State-of-the-Art › Video-Text Retrieval
Video-Text Retrieval
59 papers with code · 1 benchmark · 6 datasets archive 2025-07-28
Video-Text retrieval requires understanding of both video and language together. Therefore it's different to video retrieval task.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| Test-of-Time (4 rows) | Video-LLAMA | Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for... | code | Syntology official (archive's flag): 7 ran · 18 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
6 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 111 · with a code link: 59 · where Syntology ran a sample: 25 (22 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (25 of 111 tagged: 22 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 111, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
3 Oct 2023 6 repositories listed Syntology official (archive's flag): 5 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 3 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 14 harvested samples) · 3 pointer-only (licence)We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
-
18 Apr 2021 5 repositories listed Syntology official: harvested, nothing ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 3 pointer-only (licence)In this paper, we propose a CLIP4Clip model to transfer the knowledge of the CLIP model to video-language retrieval in an end-to-end manner.
-
1 Apr 2021 5 repositories listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 3 pointer-only (licence)Our objective in this work is video-text retrieval - in particular a joint embedding that enables efficient text-to-video retrieval.
-
5 Jun 2023 4 repositories listed Syntology official (archive's flag): 7 ran · 18 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (of 25 harvested samples) · 9 pointer-only (licence)We present Video-LLaMA a multi-modal framework that empowers Large Language Models (LLMs) with the capability of understanding both visual and auditory content in the video.
-
1 Mar 2020 4 repositories listed Syntology 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples)To improve fine-grained video-text retrieval, we propose a Hierarchical Graph Reasoning (HGR) model, which decomposes video-text matching into global-to-local levels.
-
15 Jul 2022 3 repositories listed Syntology official: harvested, nothing ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 1 pointer-only (licence)However, cross-grained contrast, which is the contrast between coarse-grained representations and fine-grained representations, has rarely been explored in prior research.
-
24 May 2022 3 repositories listedLarge-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.
-
21 Dec 2023 2 repositories listed Syntology community repositories only · 2 ran (of which 2 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 2 samples that ran constructed an object rather than computing a result (of 2 harvested samples) · 2 pointer-only (licence)However, the progress in vision and vision-language foundation models, which are also critical elements of multi-modal AGI, has not kept pace with LLMs.
-
11 Dec 2023 2 repositories listedAdapting existing short video (5-30 seconds) grounding methods to this problem yields poor performance.
-
4 Dec 2023 2 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 1 pointer-only (licence)This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
-
13 Feb 2023 2 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 7 where Syntology's instrument failed) · 2 unverified (of 10 harvested samples) · 10 pointer-only (licence)Particularly, on the MSRVTT retrieval task, UniAdapter achieves 49.
-
3 Jun 2022 2 repositories listed Syntology official (archive's flag): 2 ran · 3 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 2 pointer-only (licence)Video-Language Pretraining (VLP), which aims to learn transferable representation to advance a wide range of video-text downstream tasks, has recently received increasing attention.
-
13 Jan 2022 2 repositories listed Syntology official (archive's flag): 2 ran · 13 ran (of which 8 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 4 where Syntology's instrument failed) · 11 unverified (of 24 harvested samples) · 6 pointer-only (licence)As an additional benefit, our method achieves competitive results with much shorter pre-training videos on single-modality downstream tasks, e. g., action recognition with linear evaluation.
-
9 Sep 2021 2 repositories listedIn this paper, we propose a multi-stream Corpus Alignment network with single gate Mixture-of-Experts (CAMoE) and a novel Dual Softmax Loss (DSL) to solve the two heterogeneity.
-
10 Jun 2025 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)The parameter-efficient adaptation of the image-text pretraining model CLIP for video-text retrieval is a prominent area of research.
-
29 May 2025 1 repository listedWe introduce grounded video tokenization, a paradigm that organizes tokens based on panoptic sub-object trajectories rather than fixed patches.
-
20 May 2025 1 repository listedTo address these limitations, we introduce LoVR, a benchmark specifically designed for long video-text retrieval.
-
9 Feb 2025 1 repository listedTo overcome these challenges, we introduce a specialized cognitive module, temporal working memory (TWM), which aims to enhance the temporal modeling capabilities of MFMs.
-
5 Feb 2025 1 repository listedVideo-text retrieval has been stuck in the information mismatch caused by personalized and inadequate textual descriptions of videos.
-
26 Dec 2024 1 repository listedWe further enhance the use of harder-negatives in model training, and benchmark a variety of video-text models on RTime.
-
29 Nov 2024 1 repository listedGrounding the instruction in the environment is a key step in solving language-guided goal-reaching reinforcement learning problems.
-
9 Oct 2024 1 repository listedBy replacing a single text query with a series of text proxies, TV-ProxyNet not only broadens the query scope but also achieves a more precise expansion.
-
31 Jul 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples) · 3 pointer-only (licence)In this paper, we introduce MovieSeq, a multimodal language model developed to address the wide range of challenges in understanding video contexts.
-
16 Jul 2024 1 repository listedIn this paper, we propose a novel Spatio-Temporal Graph Transformer module to uniformly learn spatial and temporal contexts for video-language alignment pre-training (dubbed STGT).
-
7 Jun 2024 1 repository listedThis method proves to be effective on two action datasets for the motion description retrieval task.
-
20 Mar 2024 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples)To tackle these issues, we propose training free token merging for lightweight video Transformer (vid-TLDR) that aims to enhance the efficiency of video Transformers by merging the background tokens without additional…
-
31 Jan 2024 1 repository listedWe then summarize this empirical study into the M2-RAAP recipe, where our technical contributions lie in 1) the data filtering and text re-writing pipeline resulting in 1M high-quality bilingual video-text pairs, 2) the…
-
19 Dec 2023 1 repository listedThen, in Context-aware Simulator Learning stage, we train a Content-aware Prompt Simulator under a simulated test scenarios to produce the corresponding CaDP.
-
30 Oct 2023 1 repository listedBuilding video-language foundation models is costly and difficult due to the redundant nature of video data and the lack of high-quality video-language datasets.
-
29 Oct 2023 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 3 honoured, 0 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 15 harvested samples) · 5 pointer-only (licence)TESTA can reduce the number of visual tokens by 75% and thus accelerate video encoding.
-
8 Oct 2023 1 repository listed
-
29 Sep 2023 1 repository listed Syntology official (archive's flag): 17 ran · 17 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 5 honoured, 1 violated, 3 with no contract checked; 8 where Syntology's instrument failed) · 2 unverified (of 19 harvested samples) · 8 pointer-only (licence)
-
18 Sep 2023 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 1 violated, 5 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples) · 4 pointer-only (licence)
-
28 Aug 2023 1 repository listed
-
22 Aug 2023 1 repository listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 4 honoured, 1 violated, 3 with no contract checked; 7 where Syntology's instrument failed) · 3 unverified (of 18 harvested samples) · 18 pointer-only (licence)
-
15 Aug 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result (of 2 harvested samples) · 2 pointer-only (licence)
-
12 Jun 2023 1 repository listed
-
18 Apr 2023 1 repository listed
-
22 Mar 2023 1 repository listed
-
22 Feb 2023 1 repository listed
-
19 Feb 2023 1 repository listed
-
26 Jan 2023 1 repository listed
-
19 Jan 2023 1 repository listed
-
5 Jan 2023 1 repository listed Syntology official (archive's flag): 9 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 0 honoured, 0 violated, 9 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 10 harvested samples)
-
24 Nov 2022 1 repository listed
-
19 Oct 2022 1 repository listed Syntology 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
17 Oct 2022 1 repository listed
-
14 Sep 2022 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
9 Jun 2022 1 repository listed
-
26 Apr 2022 1 repository listed
-
28 Mar 2022 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; the one sample that ran constructed an object rather than computing a result (of 2 harvested samples) · 2 pointer-only (licence)
-
2 Dec 2021 1 repository listed
-
26 Jul 2021 1 repository listed
-
21 Jun 2021 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
9 Mar 2021 1 repository listed
-
9 Nov 2020 1 repository listed
-
1 Nov 2020 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples)
-
11 Jun 2018 1 repository listed
-
Towards Understanding Camera Motions in Any Video21 Apr 2025 0 repositories listed
-
LV-MAE: Learning Long Video Representations through Masked-Embedding Autoencoders4 Apr 2025 0 repositories listed
-
Learning Audio-guided Video Representation with Gated Attention for Video-Text Retrieval3 Apr 2025 0 repositories listed
-
V²Dial: Unification of Video and Visual Dialog via Multimodal Experts3 Mar 2025 0 repositories listed
-
Rethinking Noisy Video-Text Retrieval via Relation-aware Alignment1 Jan 2025 0 repositories listed
-
V^2Dial: Unification of Video and Visual Dialog via Multimodal Experts1 Jan 2025 0 repositories listed
-
CaReBench: A Fine-Grained Benchmark for Video Captioning and Retrieval31 Dec 2024 0 repositories listed
-
Beyond Coarse-Grained Matching in Video-Text Retrieval16 Oct 2024 0 repositories listed
-
NAVERO: Unlocking Fine-Grained Semantics for Video-Language Compositionality18 Aug 2024 0 repositories listed
-
EA-VTR: Event-Aware Video-Text Retrieval10 Jul 2024 0 repositories listed
-
Multi-Scale Temporal Difference Transformer for Video-Text Retrieval23 Jun 2024 0 repositories listed
-
HENASY: Learning to Assemble Scene-Entities for Egocentric Video-Language Model1 Jun 2024 0 repositories listed
-
Uncertainty-aware sign language video retrieval with probability distribution modeling30 May 2024 0 repositories listed
-
An Empirical Study of Excitation and Aggregation Design Adaptions in CLIP4Clip for Video-Text Retrieval25 May 2024 0 repositories listed
-
RETTA: Retrieval-Enhanced Test-Time Adaptation for Zero-Shot Video Captioning11 May 2024 0 repositories listed
-
13 Apr 2024 0 repositories listed
-
HaVTR: Improving Video-Text Retrieval Through Augmentation Using Large Foundation Models7 Apr 2024 0 repositories listed
-
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval26 Feb 2024 0 repositories listed
-
Video Editing for Video Retrieval4 Feb 2024 0 repositories listed
-
Leveraging Generative Language Models for Weakly Supervised Sentence Component Analysis in Video-Language Joint Learning10 Dec 2023 0 repositories listed
-
23 Oct 2023 0 repositories listed
-
Uncertainty-Aware Alignment Network for Cross-Domain Video-Text Retrieval21 Sep 2023 0 repositories listed
-
Uncertainty-Aware Alignment Network for Cross-Domain Video-Text Retrieval21 Sep 2023 0 repositories listed
-
TaCA: Upgrading Your Visual Foundation Model with Task-agnostic Compatible Adapter22 Jun 2023 0 repositories listed
-
22 May 2023 0 repositories listed
-
Mask to reconstruct: Cooperative Semantics Completion for Video-text Retrieval13 May 2023 0 repositories listed
-
10 May 2023 0 repositories listed
-
Deep Learning for Video-Text Retrieval: a Review24 Feb 2023 0 repositories listed
-
Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval30 Jan 2023 0 repositories listed
-
Dual Alignment Unsupervised Domain Adaptation for Video-Text Retrieval1 Jan 2023 0 repositories listed
-
HiVLP: Hierarchical Interactive Video-Language Pre-Training1 Jan 2023 0 repositories listed
-
1 Jan 2023 0 repositories listed
-
2 Dec 2022 0 repositories listed
-
TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval28 Sep 2022 0 repositories listed
-
Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval28 Sep 2022 0 repositories listed
-
Text-Adaptive Multiple Visual Prototype Matching for Video-Text Retrieval27 Sep 2022 0 repositories listed
-
15 Sep 2022 0 repositories listed
-
Boosting Video-Text Retrieval with Explicit High-Level Semantics8 Aug 2022 0 repositories listed
-
11 Jul 2022 0 repositories listed
-
Generalizing Multimodal Pre-training into Multilingual via Language Acquisition29 May 2022 0 repositories listed
-
11 Mar 2022 0 repositories listed
Syntology lines on 26 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections