Datasets › AudioCaps

AudioCaps

Introduced by Chris Dongjoo Kim et al. in AudioCaps: Generating Captions for Audios in The Wild1 Jun 2019 archive 2025-07-28

AudioCaps is a dataset of sounds with event descriptions that was introduced for the task of audio captioning, with sounds sourced from the AudioSet dataset. Annotators were provided the audio tracks together with category hints (and with additional video hints if needed).

Source: Audio Retrieval with Natural Language Queries

Image source: https://audiocaps.github.io/

Benchmarks archive 2025-07-28

All 6 leaderboards whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.

Syntology 8 newer papers were judged to report on this dataset's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): Audio Generation · AudioCaps (5 judged, 0 placed), Audio captioning · AudioCaps (1 judged, 0 placed), Text to Audio Retrieval · AudioCaps (3 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).

Papers archive 2025-07-28

papers with a benchmark row: 43 · with a code link: 36 · where Syntology ran a sample: 19 (18 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology

Show: all papers with a benchmark rowonly where code ran (19 of 43 with a benchmark row: 18 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)

Page 1 of 1: papers 1 to 43 of 43 with a leaderboard row on this dataset's benchmarks, newest first by the archive's date (ties by slug; undated papers last).

The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset, not that list; the archive's count for this dataset is 279. The Syntology column is from Syntology's graph, stated per sample; it is not part of any archive number. A line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified” (C is a part of N, never taken away from it); the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. When the archive marks a repository official for the paper, the cell starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record.

PaperCodeResultsDateSamples run Syntology
LAVCap: LLM-based Audio-Visual Captioning using Optimal Transport 2 1 16 Jan 2025 not harvested
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization 1 2 30 Dec 2024 not harvested
ETTA: Elucidating the Design Space of Text-to-Audio Models 1 2 26 Dec 2024 not harvested
Enhancing Retrieval-Augmented Audio Captioning with Generation-Assisted Multimodal Querying and Progressive Learning 0 1 14 Oct 2024 not harvested
SLAM-AAC: Enhancing Audio Captioning with Paraphrasing Augmentation and CLAP-Refine through LLMs 1 1 12 Oct 2024 not harvested
EnCLAP++: Analyzing the EnCLAP Framework for Optimizing Automated Audio Captioning Performance 1 2 2 Sep 2024 not harvested
Stable Audio Open 1 1 19 Jul 2024 official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified
Taming Data and Transformers for Audio Generation 1 2 27 Jun 2024 not harvested
Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding 1 1 19 Jun 2024 official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified
Improving Text-To-Audio Models with Synthetic Captions 1 1 18 Jun 2024 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 1 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (4 pointer-only for licence)
Long-form music generation with latent diffusion 1 1 16 Apr 2024 not harvested
InternVideo2: Scaling Foundation Models for Multimodal Video Understanding 2 1 22 Mar 2024 not harvested
CLAPSep: Leveraging Contrastive Pre-trained Model for Multi-Modal Query-Conditioned Target Sound Extraction 1 1 27 Feb 2024 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified
Fast Timing-Conditioned Latent Audio Diffusion 2 1 7 Feb 2024 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities 1 2 2 Feb 2024 official: no sample here; runs from other or unrecorded repositories · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 3 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning 1 2 31 Jan 2024 not harvested
Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation 1 2 2 Jan 2024 official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (2 pointer-only for licence)
Audiobox: Unified Audio Generation with Natural Language Prompts 0 1 25 Dec 2023 not harvested
Zero-shot audio captioning with audio-language model guidance and audio context keywords 1 2 14 Nov 2023 official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (17 pointer-only for licence)
ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation 1 1 19 Sep 2023 official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 1 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (3 pointer-only for licence)
RECAP: Retrieval-Augmented Audio Captioning 1 1 18 Sep 2023 official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (8 pointer-only for licence)
Retrieval-Augmented Text-to-Audio Generation 0 1 14 Sep 2023 not harvested
Zero-Shot Audio Captioning via Audibility Guidance 0 1 7 Sep 2023 not harvested
Rethinking Transfer and Auxiliary Learning for Improving Audio Captioning Transformer 0 1 20 Aug 2023 not harvested
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining 2 2 10 Aug 2023 official (archive's flag): 8 ran · 16 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 4 honoured, 4 violated, 5 with no contract checked; 3 where Syntology's instrument failed) · 11 unverified (19 pointer-only for licence)
Make-An-Audio 2: Temporal-Enhanced Text-to-Audio Generation 1 1 29 May 2023 11 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 2 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 1 unverified (5 pointer-only for licence)
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset 2 2 29 May 2023 official (archive's flag): 12 ran · 35 ran (of which 4 constructed an object rather than computing a result; 25 with no instrument failure: 4 honoured, 1 violated, 20 with no contract checked; 10 where Syntology's instrument failed) · 7 unverified (8 pointer-only for licence)
Any-to-Any Generation via Composable Diffusion 2 1 19 May 2023 official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 5 unverified (1 pointer-only for licence)
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities 2 1 18 May 2023 official (archive's flag): 5 ran · 5 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (1 pointer-only for licence)
Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model 1 1 24 Apr 2023 not harvested
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset 1 2 17 Apr 2023 not harvested
Prefix tuning for automated audio captioning 1 1 30 Mar 2023 not harvested
Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models 1 1 30 Jan 2023 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models 4 1 29 Jan 2023 official (archive's flag): 6 ran · 16 ran (of which 1 constructed an object rather than computing a result; 11 with no instrument failure: 1 honoured, 1 violated, 9 with no contract checked; 5 where Syntology's instrument failed) · 5 unverified (12 pointer-only for licence)
Exploring Train and Test-Time Augmentations for Audio-Language Learning 0 2 31 Oct 2022 not harvested
AudioGen: Textually Guided Audio Generation 1 1 30 Sep 2022 not harvested
Diffsound: Discrete Diffusion Model for Text-to-sound Generation 1 1 20 Jul 2022 not harvested
Cross Modal Retrieval with Querybank Normalisation 1 1 23 Dec 2021 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified
Audio Retrieval with Natural Language Queries: A Benchmark Study 1 3 17 Dec 2021 not harvested
AUTOMATED AUDIO CAPTIONING BY FINE-TUNING BART WITH AUDIOSET TAGS 1 2 15 Nov 2021 not harvested
Audio Captioning Transformer 1 2 21 Jul 2021 not harvested
Audio Retrieval with Natural Language Queries 1 2 5 May 2021 official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (1 pointer-only for licence)
AudioCaps: Generating Captions for Audios in The Wild 0 1 1 Jun 2019 not harvested

Dataset loaders archive 2025-07-28

3 loaders as listed in the archive; links are outbound and not re-checked here.

Tasks archive 2025-07-28

License archive 2025-07-28

Unknown

Modalities archive 2025-07-28

Languages archive 2025-07-28

No language tagged.

Variants archive 2025-07-28

  • AudioCaps

1 variant name, as the archive lists them.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections