Datasets › MSRVTT-QA

MSRVTT-QA

archive 2025-07-28

The MSR-VTT-QA dataset is a benchmark for the task of Visual Question Answering (VQA) on the MSR-VTT (Microsoft Research Video to Text) dataset. The MSR-VTT-QA benchmark is used to evaluate models on their ability to answer questions based on these videos. It's part of the tasks that this dataset is used for, along with Video Retrieval, Video Captioning, Zero-Shot Video Question Answering, Zero-Shot Video Retrieval, and Text-to-Video Generation.

Benchmarks archive 2025-07-28

All 5 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 2 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): Zero-Shot Video Question Answer · MSRVTT-QA (2 judged, 0 placed), Video Question Answering · MSRVTT-QA (1 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

First row (archive order)PaperCode
Visual Question Answering (VQA) MSRVTT-QA VLAB Accuracy 0.496 VLAB: Enhancing Video Language Pre-training by Feature... — 34 Compare
Zero-Shot Video Question Answer MSRVTT-QA Flash-VStream Accuracy 72.4 Flash-VStream: Memory-Based Real-Time Understanding for... IVGSZ/Flash-VStream 30 Compare
Video Question Answering MSRVTT-QA Mirasol3B Accuracy 50.42 Mirasol3B: A Multimodal Autoregressive model for... — 14 Compare
Visual Question Answering MSRVTT-QA Aurora (ours, r=64) Aurora (ours, r=64) Test Accuracy 44.8 — — 4 Compare
Zero-Shot Learning MSRVTT-QA HiTeA Accuracy 21.7 HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training — 1 Compare

Papers archive 2025-07-28

papers with a benchmark row: 66 · with a code link: 57 · where Syntology ran a sample: 34 (29 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (34 of 66 with a benchmark row: 29 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 66 of 66 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 66. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
LLaVA-Mini: Efficient Image and Video Large Multimodal Models with One Vision Token 1 1 7 Jan 2025 official (archive's flag): 3 ran · 3 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (1 pointer-only for licence)
LinVT: Empower Your Image-level Large Language Model to Understand Videos 1 1 6 Dec 2024 official (archive's flag): 7 ran · 10 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 3 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (12 pointer-only for licence)
TS-LLaVA: Constructing Visual Tokens through Thumbnail-and-Sampling for Training-Free Video Large Language Models 1 1 17 Nov 2024 official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 1 violated, 0 with no contract checked; 7 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence)
PPLLaVA: Varied Video Sequence Understanding With Prompt Guidance 1 1 4 Nov 2024 official (archive's flag): 4 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 2 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (3 pointer-only for licence)
SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Models 1 1 22 Jul 2024 official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (6 pointer-only for licence)
Tarsier: Recipes for Training and Evaluating Large Video Description Models 1 1 30 Jun 2024 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding 1 1 13 Jun 2024 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (8 pointer-only for licence)
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams 1 1 12 Jun 2024 not harvested
PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning 1 1 25 Apr 2024 official: harvested, nothing ran · 0 ran · 2 unverified (2 pointer-only for licence)
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding 1 1 8 Apr 2024 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (1 pointer-only for licence)
MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens 2 1 4 Apr 2024 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified
ST-LLM: Large Language Models Are Effective Temporal Learners 1 1 30 Mar 2024 official (archive's flag): 6 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 1 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 4 unverified (7 pointer-only for licence)
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM 1 1 27 Mar 2024 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified
Elysium: Exploring Object-level Perception in Videos via MLLM 1 1 25 Mar 2024 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (8 pointer-only for licence)
vid-TLDR: Training Free Token merging for Light-weight Video Transformer 1 1 20 Mar 2024 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios 1 1 7 Mar 2024 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 2 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 5 unverified (2 pointer-only for licence)
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization 1 1 5 Feb 2024 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (5 pointer-only for licence)
Shot2Story20K: A New Benchmark for Comprehensive Understanding of Multi-shot Videos 1 1 16 Dec 2023 official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (7 pointer-only for licence)
Vista-LLaMA: Reliable Video Narrator via Equal Distance to Visual Tokens 0 1 12 Dec 2023 not harvested
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models 2 2 28 Nov 2023 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (1 pointer-only for licence)
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark 3 1 28 Nov 2023 official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (2 pointer-only for licence)
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection 6 1 16 Nov 2023 community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (2 pointer-only for licence)
Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding 4 1 14 Nov 2023 not harvested
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities 0 1 9 Nov 2023 not harvested
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning 1 2 27 Sep 2023 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
Open-vocabulary Video Question Answering: A New Benchmark for Evaluating the Generalizability of Video Question Answering Models 1 3 18 Aug 2023 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (8 pointer-only for licence)
OmniDataComposer: A Unified Data Structure for Multimodal Data Fusion and Infinite Data Generation 1 1 8 Aug 2023 not harvested
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding 1 1 31 Jul 2023 not harvested
Self-Adaptive Sampling for Efficient Video Question-Answering on Image--Text Models 2 3 9 Jul 2023 not harvested
Lightweight Recurrent Cross-modal Encoder for Video Question Answering 1 1 30 Jun 2023 not harvested
COSA: Concatenated Sample Pretrained Vision-Language Foundation Model 1 1 15 Jun 2023 not harvested
Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models 2 1 8 Jun 2023 not harvested
Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding 4 1 5 Jun 2023 official (archive's flag): 7 ran · 18 ran (of which 7 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 1 violated, 7 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (9 pointer-only for licence)
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset 2 1 29 May 2023 official (archive's flag): 12 ran · 35 ran (of which 4 constructed an object rather than computing a result; 25 with no instrument failure: 4 honoured, 1 violated, 20 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (8 pointer-only for licence)
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending 0 1 22 May 2023 not harvested
VideoChat: Chat-Centric Video Understanding 1 1 10 May 2023 not harvested
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model 3 1 28 Apr 2023 official: not harvested · 0 ran · 1 unverified (1 pointer-only for licence)
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset 1 1 17 Apr 2023 not harvested
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks 1 1 29 Mar 2023 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 3 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Unmasked Teacher: Towards Training-Efficient Video Foundation Models 1 1 28 Mar 2023 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning 4 2 25 Mar 2023 official (archive's flag): 1 ran · 12 ran (of which 7 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified
MuLTI: Efficient Video-and-Language Understanding with Text-Guided MultiWay-Sampler and Multiple Choice Modeling 0 1 10 Mar 2023 not harvested
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video 4 2 1 Feb 2023 community repositories only · 17 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 0 honoured, 0 violated, 8 with no contract checked; 9 where Syntology's instrument failed) · 2 unverified (2 pointer-only for licence)
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training 0 2 30 Dec 2022 not harvested
VideoCoCa: Video-Text Modeling with Zero-Shot Transfer from Contrastive Captioners 0 1 9 Dec 2022 not harvested
VindLU: A Recipe for Effective Video-and-Language Pretraining 1 1 9 Dec 2022 not harvested
InternVideo: General Video Foundation Models via Generative and Discriminative Learning 2 1 6 Dec 2022 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified
X²-VLM: All-In-One Pre-trained Model For Vision-Language Tasks 2 2 22 Nov 2022 official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (6 pointer-only for licence)
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations 4 2 21 Nov 2022 official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks 0 1 15 Sep 2022 not harvested
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling 1 1 4 Sep 2022 not harvested
Video Question Answering with Iterative Video-Text Co-Tokenization 0 1 1 Aug 2022 not harvested
Clover: Towards A Unified Video-Language Alignment and Fusion Model 1 1 16 Jul 2022 not harvested
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models 3 3 16 Jun 2022 official: harvested, nothing ran · 14 ran (of which 9 constructed an object rather than computing a result; 14 with no instrument failure: 1 honoured, 1 violated, 12 with no contract checked; 0 where Syntology's instrument failed) · 20 unverified (1 pointer-only for licence)
Revealing Single Frame Bias for Video-and-Language Learning 2 2 7 Jun 2022 community repositories only · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (2 pointer-only for licence)
Flamingo: a Visual Language Model for Few-Shot Learning 5 3 29 Apr 2022 18 ran (of which 6 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 1 violated, 10 with no contract checked; 7 where Syntology's instrument failed) · 6 unverified (8 pointer-only for licence)
All in One: Exploring Unified Video-Language Pre-training 1 1 14 Mar 2022 not harvested
Align and Prompt: Video-and-Language Pre-training with Entity Prompts 1 1 17 Dec 2021 not harvested
DualVGR: A Dual-Visual Graph Reasoning Unit for Video Question Answering 1 1 10 Jul 2021 not harvested
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling 1 1 11 Feb 2021 not harvested
Just Ask: Learning to Answer Questions from Millions of Narrated Videos 1 1 1 Dec 2020 not harvested
Noise Estimation Using Density Estimation for Self-Supervised Multimodal Learning 1 1 6 Mar 2020 not harvested
Hierarchical Conditional Relation Networks for Video Question Answering 1 1 25 Feb 2020 not harvested
Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering 1 1 8 Apr 2019 official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
Motion-Appearance Co-Memory Networks for Video Question Answering 0 1 29 Mar 2018 not harvested
TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answering 2 1 14 Apr 2017 not harvested

Dataset loaders archive 2025-07-28

No loader listed in the archive.

Tasks archive 2025-07-28

License archive 2025-07-28

No licence recorded in the archive. Absence here is not a statement about the dataset's terms.

Modalities archive 2025-07-28

No modality tagged.

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • MSRVTT-QA

1 variant name, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections