Browse State-of-the-Art › Audio Classification
Audio Classification
183 papers with code · 22 benchmarks · 43 datasets archive 2025-07-28
Audio Classification is a machine learning task that involves identifying and tagging audio signals into different classes or categories. The goal of audio classification is to enable machines to automatically recognize and distinguish between different types of audio, such as music, speech, and environmental sounds.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
25 leaderboard tables shown for this task, 22 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted. 10 shown of 25 until expanded.
Syntology 5 newer papers were judged to report on this task's benchmarks so far (Claude Sonnet 4.5, reading their tables: a model's judgement, not a result; 0 placed): ESC-50 (1 judged, 0 placed), SHD (1 judged, 0 placed), FSD50K (2 judged, 0 placed), Speech Commands (2 judged, 0 placed). Drawn from 8,886 of 18,637 newer papers checked so far (733 more had nothing to place; 41 could not be read).
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
43 datasets whose archive record lists this task, ordered by the archive's paper count. 30 shown of 43 until expanded.
Subtasks archive 2025-07-28
4 subtasks in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 361 · with a code link: 183 · where Syntology ran a sample: 46 (41 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (46 of 361 tagged: 41 with a run with no instrument failure, 5 where every run was a failure of Syntology's instrument)
Page 1 of 4: papers 1 to 100 of 361, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
29 Sep 2016 16 repositories listed Syntology 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Convolutional Neural Networks (CNNs) have proven very effective in image classification and show promise for audio.
-
4 Mar 2021 12 repositories listed Syntology official: harvested, nothing ran · 41 ran (of which 20 constructed an object rather than computing a result; 30 with no instrument failure: 0 honoured, 2 violated, 28 with no contract checked; 11 where Syntology's instrument failed) · 14 unverified (of 55 harvested samples) · 11 pointer-only (licence)The perception models used in deep learning on the other hand are designed for individual modalities, often relying on domain-specific assumptions such as the local grid structures exploited by virtually all existing…
-
3 Oct 2023 6 repositories listed Syntology official (archive's flag): 5 ran · 10 ran (of which 0 constructed an object rather than computing a result; 9 with no instrument failure: 3 honoured, 0 violated, 6 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 14 harvested samples) · 3 pointer-only (licence)We thus propose VIDAL-10M with Video, Infrared, Depth, Audio and their corresponding Language, naming as VIDAL-10M.
-
22 Apr 2021 5 repositories listed Syntology community repositories only · 5 ran (of which 4 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 8 harvested samples) · 8 pointer-only (licence)We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and…
-
5 Apr 2021 5 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)In the past decade, convolutional neural networks (CNNs) have been widely adopted as the main building block for end-to-end audio classification models, which aim to learn a direct mapping from audio spectrograms to…
-
6 Mar 2018 5 repositories listed Syntology 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 11 harvested samples)The objective of audio classification is to predict the presence or absence of audio events in an audio clip.
-
18 Dec 2022 4 repositories listed Syntology community repositories only · 16 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 18 harvested samples) · 4 pointer-only (licence)In the first iteration, we use random projection as the acoustic tokenizer to train an audio SSL model in a mask and label prediction manner.
-
13 Jul 2022 4 repositories listed Syntology official (archive's flag): 1 ran · 17 ran (of which 12 constructed an object rather than computing a result; 16 with no instrument failure: 1 honoured, 1 violated, 14 with no contract checked; 1 where Syntology's instrument failed) · 15 unverified (of 32 harvested samples) · 10 pointer-only (licence)Following the Transformer encoder-decoder design in MAE, our Audio-MAE first encodes audio spectrogram patches with a high masking ratio, feeding only the non-masked tokens through encoder layers.
-
26 Apr 2022 4 repositories listedSelf-supervised learning (SSL) learns knowledge from a large amount of unlabeled data, and then transfers the knowledge to a specific problem with a limited number of labeled data.
-
21 Jan 2021 4 repositories listed Syntology community repositories only · 14 ran (of which 1 constructed an object rather than computing a result; 7 with no instrument failure: 1 honoured, 2 violated, 4 with no contract checked; 7 where Syntology's instrument failed) · 12 unverified (of 26 harvested samples) · 2 pointer-only (licence)In this work we show that we can train a single learnable frontend that outperforms mel-filterbanks on a wide range of audio signals, including speech, music, audio events and animal sounds, providing a general-purpose…
-
15 Jun 2020 4 repositories listed Syntology official: no sample here; runs from other or unrecorded repositories · 4 ran (of which 1 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples)Because of the scale invariance, this modification only alters the effective step sizes without changing the effective update directions, thus enjoying the original convergence properties of GD optimizers.
-
25 Apr 2024 3 repositories listedMachine learning has the potential to revolutionize passive acoustic monitoring (PAM) for ecological assessments.
-
19 Oct 2021 3 repositories listed Syntology official (archive's flag): 1 ran · 13 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 6 where Syntology's instrument failed) · 3 unverified (of 16 harvested samples) · 1 pointer-only (licence)However, pure Transformer models tend to require more training data compared to CNNs, and the success of the AST relies on supervised pretraining that requires a large amount of labeled data and a complex training…
-
22 Jul 2020 3 repositories listedBesides, we show that even though we use the pretrained model weights for initialization, there is variance in performance in various output runs of the same model.
-
28 May 2019 3 repositories listed Syntology official (archive's flag): 11 ran · 12 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 0 honoured, 0 violated, 12 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 13 harvested samples)Modern deep neural networks are well known to be brittle in the face of unknown data instances and recognition of the latter remains a challenge.
-
9 Jul 2018 3 repositories listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions.
-
18 Feb 2016 3 repositories listed Syntology official: harvested, nothing ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples)Traditional convolutional layers extract features from patches of data by applying a non-linearity on an affine function of the input.
-
28 Mar 2025 2 repositories listedIn the second stage, it learns CLAP features using the audio features learned from the LLM-based embeddings.
-
4 Jun 2024 2 repositories listedHere, we explore a new representation, a general-purpose audio-language representation, that performs well in both ZS and transfer learning.
-
9 Apr 2024 2 repositories listed Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 1 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 7 harvested samples) · 7 pointer-only (licence)This study proposes Masked Modeling Duo (M2D), an improved masked prediction SSL, which learns by predicting representations of masked input signals that serve as training signals.
-
22 Mar 2024 2 repositories listedWe introduce InternVideo2, a new family of video foundation models (ViFM) that achieve the state-of-the-art results in video recognition, video-text tasks, and video-centric dialogue.
-
30 Nov 2023 2 repositories listedIn this work, we introduce Acoustic Prompt Tuning (APT), a new adapter extending LLMs and VLMs to the audio domain by injecting audio embeddings to the input of LLMs, namely soft prompting.
-
14 Nov 2023 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 7 pointer-only (licence)Recently, instruction-following audio-language models have received broad attention for audio interaction with humans.
-
25 Sep 2023 2 repositories listedDilated convolution with learnable spacings (DCLS) is a recent convolution method in which the positions of the kernel elements are learned throughout training by backpropagation.
-
12 Jul 2023 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples)With the advent of deep learning models, classification of important signals from these datasets has markedly improved.
-
7 Jun 2023 2 repositories listedIn order to tackle both clip-level and frame-level tasks, this paper proposes Audio Teacher-Student Transformer (ATST), with a clip-level version (named ATST-Clip) and a frame-level version (named ATST-Frame)…
-
31 May 2023 2 repositories listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)Compared to Perceiver IO, our model requires absolutely no modality-specific processing at inference time, and uses an order of magnitude fewer parameters at equivalent accuracy on ImageNet.
-
18 May 2023 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 1 pointer-only (licence)In this work, we explore a scalable way for building a general representation model toward unlimited modalities.
-
9 Dec 2022 2 repositories listedCan we leverage the audiovisual information already present in video to improve self-supervised representation learning?
-
9 Nov 2022 2 repositories listed Syntology community repositories only · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 1 pointer-only (licence)We provide models of different complexity levels, scaling from low-complexity models up to a new state-of-the-art performance of .
-
30 Mar 2022 2 repositories listed
-
13 Mar 2022 2 repositories listed
-
11 Oct 2021 2 repositories listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
16 Apr 2021 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
5 Mar 2021 2 repositories listed
-
19 Feb 2021 2 repositories listed
-
24 Aug 2020 2 repositories listed
-
28 May 2020 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples)
-
8 Mar 2020 2 repositories listed
-
12 Jun 2019 2 repositories listed
-
23 Jun 2025 1 repository listed
-
3 Jun 2025 1 repository listed
-
29 May 2025 1 repository listed
-
28 May 2025 1 repository listed
-
21 May 2025 1 repository listed
-
21 May 2025 1 repository listed
-
16 May 2025 1 repository listed
-
2 Apr 2025 1 repository listed
-
17 Feb 2025 1 repository listed
-
2 Feb 2025 1 repository listed
-
4 Nov 2024 1 repository listed
-
1 Nov 2024 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
4 Oct 2024 1 repository listed
-
2 Oct 2024 1 repository listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 2 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples)
-
27 Sep 2024 1 repository listed
-
13 Sep 2024 1 repository listed
-
22 Aug 2024 1 repository listed
-
17 Aug 2024 1 repository listed
-
20 Jul 2024 1 repository listed
-
11 Jul 2024 1 repository listed
-
4 Jul 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
23 Jun 2024 1 repository listed
-
12 Jun 2024 1 repository listed
-
11 Jun 2024 1 repository listed
-
10 Jun 2024 1 repository listed
-
5 Jun 2024 1 repository listed Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 4 where Syntology's instrument failed) · 6 unverified (of 17 harvested samples) · 6 pointer-only (licence)
-
3 Jun 2024 1 repository listed
-
20 May 2024 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 1 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 8 harvested samples)
-
14 May 2024 1 repository listed
-
13 May 2024 1 repository listed
-
13 May 2024 1 repository listed
-
2 May 2024 1 repository listed
-
29 Apr 2024 1 repository listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 2 violated, 7 with no contract checked; 5 where Syntology's instrument failed) · 3 unverified (of 18 harvested samples)
-
14 Apr 2024 1 repository listed
-
9 Apr 2024 1 repository listed
-
9 Apr 2024 1 repository listed
-
24 Mar 2024 1 repository listed
-
15 Mar 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
14 Mar 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 4 constructed an object rather than computing a result; 5 with no instrument failure: 1 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 7 unverified (of 13 harvested samples) · 2 pointer-only (licence)
-
14 Feb 2024 1 repository listed
-
2 Feb 2024 1 repository listed
-
10 Jan 2024 1 repository listed
-
7 Jan 2024 1 repository listed Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 1 honoured, 0 violated, 9 with no contract checked; 3 where Syntology's instrument failed) · 4 unverified (of 17 harvested samples) · 3 pointer-only (licence)
-
4 Jan 2024 1 repository listed
-
15 Dec 2023 1 repository listed
-
6 Dec 2023 1 repository listed
-
16 Nov 2023 1 repository listed
-
15 Nov 2023 1 repository listed
-
11 Nov 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 3 harvested samples) · 3 pointer-only (licence)
-
8 Nov 2023 1 repository listed
-
24 Oct 2023 1 repository listed
-
18 Oct 2023 1 repository listed
-
15 Sep 2023 1 repository listed
-
11 Sep 2023 1 repository listed
-
9 Sep 2023 1 repository listed
-
1 Aug 2023 1 repository listed
-
10 Jul 2023 1 repository listed
-
6 Jul 2023 1 repository listed
-
30 Jun 2023 1 repository listed
-
27 Jun 2023 1 repository listed
Syntology lines on 33 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections