Browse State-of-the-Art › Dense Captioning
Dense Captioning
34 papers with code · 1 benchmark · 2 datasets archive 2025-07-28
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| Visual Genome (4 rows) | ControlCap | ControlCap: Controllable Region-level Captioning | code | Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
2 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 69 · with a code link: 34 · where Syntology ran a sample: 13 (13 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (13 of 69 tagged: 13 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 69 of 69, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
24 Jul 2023 5 repositories listed Syntology 6 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified (of 10 harvested samples) · 10 pointer-only (licence)Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs.
-
2 May 2017 4 repositories listedWe also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events.
-
20 Nov 2016 3 repositories listed Syntology 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 9 harvested samples)Recent progress on image captioning has made it possible to generate novel sentences describing images in natural language, but compressing an image into a single sentence can describe visual content in only coarse…
-
24 Sep 2024 2 repositories listedThe comic domain is rapidly advancing with the development of single- and multi-page analysis and synthesis models.
-
4 Dec 2023 2 repositories listed Syntology community repositories only · 10 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 1 violated, 4 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples) · 1 pointer-only (licence)This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
-
6 Jun 2025 1 repository listedApplied to the NuScenes dataset, we present STSnu, the first benchmark that evaluates the spatio-temporal reasoning capabilities of VLMs based on comprehensive 3D perception.
-
2 May 2025 1 repository listedWe propose TEMPURA (Temporal Event Masked Prediction and Understanding for Reasoning in Action), a two-stage training framework that enhances video temporal understanding.
-
29 Nov 2024 1 repository listedEnabling Large Language Models (LLMs) to understand the 3D physical world is an emerging yet challenging research direction.
-
7 Aug 2024 1 repository listedThe framework consists of two main parts: a Single-Shot PCI Estimation Network and a Dense Captioning Network.
-
8 Jul 2024 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 2 violated, 2 with no contract checked; 6 where Syntology's instrument failed) · 0 unverified (of 10 harvested samples) · 1 pointer-only (licence)3D Vision-Language Pre-training (3D-VLP) aims to provide a pre-train model which can bridge 3D scenes with natural language, which is an important technique for embodied intelligence.
-
14 Jun 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 2 pointer-only (licence)The state of an object reflects its current status or condition and is important for a robot's task planning and manipulation.
-
16 May 2024 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 9 where Syntology's instrument failed) · 4 unverified (of 17 harvested samples) · 17 pointer-only (licence)To facilitate the use of referent tokens in subsequent language modeling, we provide a large-scale, automatically curated grounded scene-text dataset with over 1 million phrase-to-region correspondences and introduce…
-
25 Apr 2024 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples) · 2 pointer-only (licence)PLLaVA achieves new state-of-the-art performance on modern benchmark datasets for both video question-answer and captioning tasks.
-
17 Apr 2024 1 repository listed3D Visual Grounding (3DVG) and 3D Dense Captioning (3DDC) are two crucial tasks in various 3D applications, which require both shared and complementary information in localization and visual-language relationships.
-
28 Mar 2024 1 repository listed Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 4 honoured, 1 violated, 1 with no contract checked; 6 where Syntology's instrument failed) · 4 unverified (of 16 harvested samples) · 16 pointer-only (licence)However, the exploration of 3D dense captioning in outdoor scenes is hindered by two major challenges: 1) the domain gap between indoor and outdoor scenes, such as dynamics and sparse visual inputs, makes it difficult…
-
31 Jan 2024 1 repository listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)The multimodal model is constrained to generate captions within a few sub-spaces containing the control words, which increases the opportunity of hitting less frequent captions, alleviating the caption degeneration…
-
1 Jan 2024 1 repository listedRecent progress in Large Multimodal Models (LMM) has opened up great possibilities for various applications in the field of human-machine interactions.
-
30 Nov 2023 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 2 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 6 harvested samples) · 5 pointer-only (licence)However, developing LMMs that can comprehend, reason, and plan in complex and diverse 3D environments remains a challenging topic, especially considering the demand for understanding permutation-invariant point cloud 3D…
-
6 Sep 2023 1 repository listedMoreover, we argue that object localization and description generation require different levels of scene understanding, which could be challenging for a shared set of queries to capture.
-
8 Aug 2023 1 repository listed Syntology 4 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 1 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples)3D vision-language grounding (3D-VL) is an emerging field that aims to connect the 3D physical world with natural language, which is crucial for achieving embodied intelligence.
-
19 May 2023 1 repository listedIn this paper, we propose a novel method called Joint QA and DC GEneration (JADE), which utilizes a pre-trained multimodal model and easily-crawled image-text pairs to automatically generate and filter large-scale VQA…
-
8 May 2023 1 repository listedIIITD-20K comprises of 20, 000 unique identities captured in the wild and provides a rich dataset for text-to-image ReID.
-
6 Jan 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)Compared with prior arts, our framework has several appealing advantages: 1) Without resorting to numerous hand-crafted components, our method is based on a full transformer encoder-decoder architecture with a learnable…
-
1 Jan 2023 1 repository listedThe current approaches for 3D visual reasoning are task-specific, and lack pre-training methods to learn generic representations that can transfer across various tasks.
-
1 Dec 2022 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples) · 8 pointer-only (licence)Specifically, GRiT consists of a visual encoder to extract image features, a foreground object extractor to localize objects, and a text decoder to generate open-set object descriptions.
-
22 Apr 2022 1 repository listedDense captioning in 3D point clouds is an emerging vision-and-language task involving object-level 3D scene understanding.
-
10 Mar 2022 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 3 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples) · 3 pointer-only (licence)3D dense captioning is a recently-proposed novel task, where point clouds contain more geometric information than the 2D counterpart.
-
2 Mar 2022 1 repository listedThus, a more faithful caption can be generated only using point clouds during the inference.
-
1 Nov 2021 1 repository listedSuch information is even more important for story visualization since its inputs have an explicit narrative structure that needs to be translated into an image sequence (or visual story).
-
21 Oct 2021 1 repository listedPrior work in this domain has shown that there is ample room for improvement in the generated image sequence in terms of visual quality, consistency and relevance.
-
21 Jun 2020 1 repository listed
-
28 Feb 2018 1 repository listed Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
21 Nov 2016 1 repository listed
-
24 Nov 2015 1 repository listed
-
Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMs5 Jun 2025 0 repositories listed
-
3D Spatial Understanding in MLLMs: Disambiguation and Evaluation9 Dec 2024 0 repositories listed
-
3D Scene Graph Guided Vision-Language Pre-training27 Nov 2024 0 repositories listed
-
Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Driving10 Sep 2024 0 repositories listed
-
xGen-VideoSyn-1: High-fidelity Text-to-Video Synthesis with Compressed Representations22 Aug 2024 0 repositories listed
-
14 Aug 2024 0 repositories listed
-
13 Aug 2024 0 repositories listed
-
1 Aug 2024 0 repositories listed
-
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions9 Jul 2024 0 repositories listed
-
14 Apr 2024 0 repositories listed
-
Entity6K: A Large Open-Domain Evaluation Dataset for Real-World Entity Recognition19 Mar 2024 0 repositories listed
-
FlexCap: Describe Anything in Images in Controllable Detail18 Mar 2024 0 repositories listed
-
18 Mar 2024 0 repositories listed
-
A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes12 Mar 2024 0 repositories listed
-
CapDet: Unifying Dense Captioning and Open-World Detection Pretraining4 Mar 2023 0 repositories listed
-
UniT3D: A Unified Transformer for 3D Dense Captioning and Visual Grounding1 Dec 2022 0 repositories listed
-
8 Oct 2022 0 repositories listed
-
SAVCHOI: Detecting Suspicious Activities using Dense Video Captioning with Human Object Interactions24 Jul 2022 0 repositories listed
-
CapOnImage: Context-driven Dense-Captioning on Image27 Apr 2022 0 repositories listed
-
Semantic-Aware Pretraining for Dense Video Captioning13 Apr 2022 0 repositories listed
-
Describing image focused in cognitive and visual details for visually impaired people: An approach to generating inclusive paragraphs10 Feb 2022 0 repositories listed
-
1 Jan 2022 0 repositories listed
-
2 Dec 2021 0 repositories listed
-
3 Dec 2020 0 repositories listed
-
Improving Diversity and Reducing Redundancy in Paragraph Captions19 Jul 2020 0 repositories listed
-
14 Jun 2020 0 repositories listed
-
Activitynet 2019 Task 3: Exploring Contexts for Dense Captioning Events in Videos11 Jul 2019 0 repositories listed
-
Dense Procedure Captioning in Narrated Instructional Videos1 Jul 2019 0 repositories listed
-
Trimmed Action Recognition, Dense-Captioning Events in Videos, and Spatio-temporal Action Localization with Focus on ActivityNet Challenge 201914 Jun 2019 0 repositories listed
-
2 Apr 2019 0 repositories listed
-
YH Technologies at ActivityNet Challenge 201829 Jun 2018 0 repositories listed
-
Best Vision Technologies Submission to ActivityNet Challenge 2018-Task: Dense-Captioning Events in Videos25 Jun 2018 0 repositories listed
-
RUC+CMU: System Report for Dense Captioning Events in Videos22 Jun 2018 0 repositories listed
-
Fooling Vision and Language Models Despite Localization and Attention Mechanism25 Sep 2017 0 repositories listed
-
Visually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation4 Jul 2017 0 repositories listed
Syntology lines on 15 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections